Big Data’s Future: Bigger Than Just Vertical Applications

I’m not much for making predictions about trends in the upcoming year. Still, it sure is fun to see what others predict—and of course, to debunk some of the loopiest ideas about what the future may hold. For example, one of the most popular 2013 predictions out there right now is that the big data marketplace will be all about vertical applications in the coming year (read this article from VentureBeat for the full argument).

It’s important to remember the source of this prediction: the venture capital community. The big data platform space is already well served, so VCs can’t make much additional money there. It serves their interests to try and shift the conversation to an underserved space with greater possibilities for funding and generating a return. These circumstances don’t necessarily mean the prediction is wrong—but it is always a good idea to keep the possibility of other motivations in mind. Kind of like Oracle saying the answer to all of life’s challenges is Exadata… but I digress.

You may also be asking yourself another related question at this point: Don’t you, Tom, focus on big data infrastructure? Doesn’t that make you biased in this argument too? Well, I am partially guilty as charged. I do focus on big data infrastructure. But my platform also supports quite a few IBM vertical apps (and the number is growing), so I’ve got skin in the game on both sides here. I have no axe to grind either way.

I would argue that focusing solely on vertical applications for big data goes against what enterprises really need—and it contradicts the larger trend toward rationalization. Based on my work with customers, the fundamental flaw in the vertical-applications-only argument is that no enterprise zero-bases their infrastructure, which means these “new” applications are simply additive and therefore create additional support burden and user overload. Can they be useful? Sure. But are more standalone apps what the enterprise needs at this point? That seems dubious to me.

When the larger (and correct) IT drive is towards rationalization, holistic understanding, and Fit for Purpose Architectures, simply introducing more siloed applications and getting frustrated over the additional information silos that you create is counterproductive. In most cases, it’s better to use the capabilities of your big data platforms to improve existing solutions instead of simply replacing them. Rather than creating a new application environment, you can use your big data infrastructure to eliminate what doesn’t work or doesn’t fit into your existing core decision support systems. You can then ship data to those environments so they pick them up as “native.” This approach is much more user-friendly for business users—and it dramatically shortens deployment cycles, drives utilization, and often is the lowest-TCO path available.

To be clear, I’m not saying that new vertical apps don’t have a role to play. They can certainly simplify deployment and improve ease of use. But the claim that vertical apps are the future of the big data space is to fundamentally miss the larger enterprise IT picture.

Do you agree or disagree? Let me know in the comments.