Big Data’s Future: Bigger Than Just Vertical Applications

Why one popular prediction about the direction big data will take in 2013 is too limited

I’m not much for making predictions about trends in the upcoming year. Still, it sure is fun to see what others predict—and of course, to debunk some of the loopiest ideas about what the future may hold. For example, one of the most popular 2013 predictions out there right now is that the big data marketplace will be all about vertical applications in the coming year (read this article from VentureBeat for the full argument).

It’s important to remember the source of this prediction: the venture capital community. The big data platform space is already well served, so VCs can’t make much additional money there. It serves their interests to try and shift the conversation to an underserved space with greater possibilities for funding and generating a return. These circumstances don’t necessarily mean the prediction is wrong—but it is always a good idea to keep the possibility of other motivations in mind. Kind of like Oracle saying the answer to all of life’s challenges is Exadata… but I digress.

You may also be asking yourself another related question at this point: Don’t you, Tom, focus on big data infrastructure? Doesn’t that make you biased in this argument too? Well, I am partially guilty as charged. I do focus on big data infrastructure. But my platform also supports quite a few IBM vertical apps (and the number is growing), so I’ve got skin in the game on both sides here. I have no axe to grind either way.

I would argue that focusing solely on vertical applications for big data goes against what enterprises really need—and it contradicts the larger trend toward rationalization. Based on my work with customers, the fundamental flaw in the vertical-applications-only argument is that no enterprise zero-bases their infrastructure, which means these “new” applications are simply additive and therefore create additional support burden and user overload. Can they be useful? Sure. But are more standalone apps what the enterprise needs at this point? That seems dubious to me.

When the larger (and correct) IT drive is towards rationalization, holistic understanding, and Fit for Purpose Architectures, simply introducing more siloed applications and getting frustrated over the additional information silos that you create is counterproductive. In most cases, it’s better to use the capabilities of your big data platforms to improve existing solutions instead of simply replacing them. Rather than creating a new application environment, you can use your big data infrastructure to eliminate what doesn’t work or doesn’t fit into your existing core decision support systems. You can then ship data to those environments so they pick them up as “native.” This approach is much more user-friendly for business users—and it dramatically shortens deployment cycles, drives utilization, and often is the lowest-TCO path available.

To be clear, I’m not saying that new vertical apps don’t have a role to play. They can certainly simplify deployment and improve ease of use. But the claim that vertical apps are the future of the big data space is to fundamentally miss the larger enterprise IT picture.

Do you agree or disagree? Let me know in the comments.
 

 
Previous post

It's All About the Data

Next post

Premier Healthcare Alliance Chooses IBM PureData System to Ensure Better, Faster Patient Care

Tom Deutsch

Tom Deutsch (Twitter: @thomasdeutsch) serves as a Program Director in IBM’s Big Data Team. He played a formative role in the transition of Hadoop-based technology from IBM Research to IBM Software Group, and he continues to be involved with IBM Research Big Data activities and transition from Research to commercial products. Tom created the IBM BigInsights Hadoop based product, and then has spent several years helping customers with Apache Hadoop, BigInsights and Streams technologies identifying architecture fit, developing business strategies and managing early stage projects across more than 200 customer engagements. Tom has co-authored a Big Data book and multiple thought papers.

Prior to that, Tom worked in the Information Management in the CTO’s office. Tom worked with a team focused on emerging technology and helped customers adopt IBM’s innovative Enterprise Mashups and Cloud offerings. Tom came to IBM through the FileNet acquisition, where he had responsibility for FileNet’s flagship Content Management product and spearheaded FileNet product initiatives with other IBM software segments including the Lotus and InfoSphere segments.

With more than 20 years in the industry, and as a veteran of two startups, Deutsch is an expert on the technical, strategic and business information management issues facing the Enterprise today. Most of Tom’s work has been on emerging technologies and business challenges, and he brings a strong focus on the cross-functional work required to have early stage project succeed.

Deutsch earned a bachelor’s degree from Fordham University in New York and an MBA degree from the University of Maryland University College.