Big data analytics is gaining momentum as increasing numbers of companies realize they can leverage data and analytics to serve their client organizations and help improve their business performance. They recognize that deriving value from data is no longer a choice, but a business imperative in the emerging digital economy.1 And in many organizations, the analytics architect is turning out to be a key player in helping to extract insight from that data.
The evolution of using data to help deliver business insights and value has gone through a number of phases.2 The first phase focused on using data for reporting. The second phase began with the rise of big data. The commodity-driven MapReduce framework and its open source equivalent, Apache Hadoop, provided the blueprint for this phase. The third phase that is now emerging is characterized by significant innovations in technology, such as data discovery; interactive visualizations; and in-memory, predictive, and prescriptive analytics.
Businesses are leveraging these new technologies to implement bigger and more ambitious solutions than ever that analyze a wide variety of data, apply sophisticated models, and generate actionable insights. The insights can then be used to respond to events in real time. These new technologies, which are more complex than their previous, simpler, stand-alone counterparts, require additional planning, rigor, and coordination. That is, they require architecture—and an architect.
The IT architecture discipline emerged and developed over the past few decades as a way to manage the relationship between business strategy and technology—a relationship that has been a challenge for many organizations to balance. Helping to reduce risk and ensure that IT projects achieve their business objectives is an established discipline for linking strategy, requirements, and constraints with a viable execution plan. Its benefits have been demonstrated on individual projects and on an enterprise-wide scale.3
Analytics architecture refers to the applications, infrastructures, tools, and leading practices that enable access to and analysis of information to optimize business decisions and performance. Architectural methods and techniques are designed to deliver a robust, cost-effective platform that supports current requirements and that can evolve to support future requirements without costly rework and disruptions.
The success of big data analytics programs calls for many considerations such as culture, governance, and sponsorship. In addition to the analytics architect, it requires an analytics ecosystem that involves collaboration between a number of roles, including the data scientist, the data steward, and the chief analytics officer (CAO).4
Of these data-centric professions, the analytics architect leverages the established architecture discipline to help ensure that business strategies align with the powerful capabilities of analytics to achieve business objectives consistently and cost-effectively. Because the analytics architect requires analytical skills and a data-driven mind-set, the role is somewhat similar to that of the data scientist.5 However, the analytics architect leverages knowledge of the organization’s information, application, and infrastructure environment as well as the current technology landscape to design a holistic and optimized analytics platform.
The role is central to the optimal analytics lifecycle, in which data from an operational system feeds the tools used for discovery, analysis, and modeling. Insights from this analysis feed back to the operational system to provide input that supports decision making and business processes (see figure).
The analytics architect’s central role in enabling the full analytics lifecycle
Key responsibilities of the analytics architect include operationalizing analytics, mapping requirements to implementation, selecting technology, and evaluating nonfunctional attributes such as security, usability, and stability.
IBM, in collaboration with the Massachusetts Institute of Technology (MIT) Sloan Management Review, conducted a survey of nearly 3,000 executives, managers, and analysts.6 Survey results found that embedding insights to drive business decisions and actions is one of the key incentives to realizing the business value of analytics. Performing discovery, analysis, and reporting on a snapshot of data in a one-off, stand-alone fashion is valuable.
However, organizations can amplify this value enormously by using the insights gained to inform decision making. Doing so involves providing feeds of live operational data in a constantly flowing loop. To operationalize analytics, analytics architects close this data-to-insight-to-action loop, which requires deep understanding of the applications and integration infrastructure environment.
Big data analytics architecture often needs to accommodate many and sometimes conflicting requirements and constraints. Requirements come from diverse stakeholders, such as line-of-business users, data scientists, analysts, and administrators. For example, data scientists need tools that enable free-format and ad hoc discovery, while business analysts need tools for structured discovery and tools that automate complex tasks such as data preparation.
Integrating a modern big data analytics platform with an existing data warehouse is also an important requirement. Constraints arise from the inevitable limitations with respect to budgets, schedules, skills, and current environments. Analytics architects analyze, shape, and prioritize these requirements as well as ensure they are implemented within current constraints. The implementation should satisfy current requirements and support future needs without significant rework.
Analytics architects are responsible for selecting appropriate technologies from the many open source, commercial on-premises, and cloud-based offerings available. Integrating a new generation of tools within the existing environment is also crucial to help ensure access to accurate and current data. One example is IBM® SPSS® Modeler data mining and text analytics software, which allows business users and analysts to perform many tasks that were previously exclusive to data scientists with advanced statistical skills. Because technology in this domain is evolving rapidly, another important responsibility of analytics architects is ensuring that components can be replaced with well-suited alternatives that do not require any adjustment or downtime.
When selecting technologies and building the analytics platform, analytics architects need to consider not only the functional requirements, but also the nonfunctional attributes of platform quality such as security, usability, and stability. Analytics architects plan, design, and monitor these key characteristics of the analytics platform to help ensure that it complies with enterprise standards and that it performs adequately as additional analytics solutions are implemented.
To help ensure a successful big data analytics implementation program, organizations should cultivate analytics architects from their existing talent pools of solution architects with analytics skills, or data scientists with engineering and technical skills. Conversely, practitioners in the architecture profession should develop analytics skills to fill the demand for this vital role.
Please share any thoughts or questions in the comments.
1 Big Data Imperatives: Enterprise Big Data Warehouse, BI Implementations, and Analytics, by Soumendra Mohanty, Madhu Jagadeesh, and Harsha Srivatsa, Apress, June 2013.
2 ““Preparing for Analytics 3.0,” by Thomas H. Davenport, Wall Street Journal, CIO Journal blog, February 2013.
3 A Systemic Perspective to Managing Complexity with Enterprise Architecture, by Pallab Saha, IGI Global, September 2013.
4 “Going Beyond Data Science Toward an Analytics Ecosystem: Part 1,” by Ahmed Fattah, IBM Data magazine, March 2014.
5 “Deriving Innovation from a Data-Driven Mind-set: Part 1,” by Ahmed Fattah, IBM Data magazine, January 2014.
6 “Analytics: The New Path to Value,” by Steve LaValle, Michael Hopkins, Eric Lesser, Rebecca Shockley, and Nina Kruschwitz, IBM Institute for Business Value, IBM Global Business Services, Business Analytics and Optimization, in collaboration with MIT Sloan Management Review, October 2010.