Big data analytics often drive bet-the-business applications. As such, the underlying platform must weather the extremes that their environments may throw at them. This endurance includes expected extremes in both the physical environments in which big data analytics platforms are deployed and in the resource requirements of the workloads that execute on these platforms.
In IBM Data magazine the week of December 8, 2014, three new articles focus on how data platforms prove their mettle by weathering the extremes without compromising their missions. Robustness, the ability to withstand turbulent conditions, must exist on various levels.
Richard Talbot and Kimberly Madia discuss how IBM’s big data analytics platforms can deliver the real-time analytics necessary for robust telecommunications network operations through natural disasters. The systems’ low-latency, high-throughput performance is pivotal in this regard, but so is their independently certified ability to operate in harsh environmental conditions. After all, the physical servers in a telecommunications carrier’s data centers can be vulnerable to the same disaster zone conditions that impact everything in their vicinity. Considering that network connections are a literal lifeline for communities in distress, the analytics infrastructure that drives their operations need to be hardened to the utmost.
Then Thomas Eunice provides guidance for identifying highly suitable big data platforms for extreme analytics workloads, in terms of their complexity, volume, and performance requirements. Eunice also highlights the pivotal importance of in-database execution to accelerate the parallel performance of complex analytics on various platforms, including Apache Hadoop, data warehouses, and analytics servers. And he calls out the fact that handling an extremely wide variety of complex workloads in parallel is no catastrophe if they’re deployed to optimally configure data platforms of the right types.
In addition, David Birmingham discusses the role of massively parallel processing (MPP) provided by IBM® PureData® System for Analytics, powered by IBM Netezza® technology, in addressing critical data scalability challenges. Birmingham references a recent proof of concept against competing platforms to describe how support for extreme heavy lifting needs to be architected into a big data platform from the get-go. He describes a for-want-of-a-nail scenario in which a seemingly trivial architectural issue proved catastrophic—for a competing solution provider in front of a potential customer. “When one of the test demonstrations crashed,” Birmingham writes, “the vendor discovered that it was because an internal variable was an integer instead of a big integer.” Birmingham points out that although this discovery may seem “geeky,” the significance is nevertheless profound. “An integer can hold only 2 billion possible values,” he says, “which is a fatal flaw in a product that must move or manage many billions of rows at a time.”
Thanks for reading and engaging. Also, please check out our latest NewsBytes and Upcoming Events for important information and opportunities to educate yourself on the power of data.
James Kobielus ((@jameskobielus)
Editor in Chief, IBM Data magazine
Big data analytics often drive bet-the-business applications. As such, the underlying platform must weather the extremes that their environments may throw at them. This endurance includes expected extremes in both the physical environments in which big-data analytics platforms are deployed and in the resource requirements of the workloads that execute on these platforms.