Failure Is Not a Four-Letter Word

I have made this observation in a number of my talks, but a recent conversation with a customer in Europe resulted in a full hour just on this topic. It was an especially good back and forth, and I thought it was worth capturing on paper (in bits and bytes). This all started with me asking the customer a simple question: What was the last project you took on where failure was a good thing? The reaction I got was not an uncommon one—a look of disbelief, followed by the customer asking just exactly how jet-lagged I was.

The customer’s answer—and yours—should be “Why, just this week actually, and we’re planning some additional failures on an ongoing basis.” Now, I will admit that this is a loaded question, and it’s rare that I get an answer indicating that anything other than failure should be avoided at all costs. But it does get the conversation started…

To be clear, I am not talking about operational systems here, or operational BI and the sort of reporting that we have all become dependent upon. What I am talking about is the discipline of experimentation as a corporate asset. I believe that managers and decision-makers must embrace experimentation as a core capability—as core as our existing focus on excellence in enterprise IT architecture and data modeling. What I am suggesting is that organizations adopt experimentation as a strategic process, and along with that, a tolerance (and even encouragement) of failure as a way of advancing the business. This tolerance of experimentation is rooted in the understanding that failure is often the single best way to learn something. If you aren’t trying and failing at validating your analytic hypotheses, I’d say you’re not being aggressive enough in working with your data and trying to model things that were previously under-modeled, underutilized, or just not utilized at all.

Of course, a lot of smart people have already figured this out. But in traditional enterprises—and by traditional, I mean mainly companies outside of the Internet space—we’ve all worked extremely hard to make sure that our projects are all reliable and predictable efforts because it is the only way to cost-justify most of what we undertake. Spinning up a hundred terabytes worth of system to play with it is just not commonly done. It’s even less common to spin up an environment to explore mixed information types in an effort to bridge the structured and unstructured information flows in your business.

For the vast majority of what we spend our time on, that reluctance is a good thing. Collectively, there is a reason we’ve become so good at planning. But the very fact that we have all become masters of planning for success means that it offers decreasing returns, especially when applied over traditional databases. If everybody is doing it, you’re not getting a competitive advantage out of it. We are seeing a shift now in our top-performing customers—a shift to embrace experimentation and pursuit of things that, until now, were not practical or pragmatic. They are explicitly looking to experiment and innovate (and fail) as a necessary step to understand their business in a more granular way, or use data in ways that used to be too daunting from a scale point of view, or simply utilize information types that are hard to manage using conventional sorts of systems.

With the technologies we have in our toolkits today, it is possible to start small, grow big, combine things that have never been combined before, and programmatically express jobs in a way that are just not possible in a conventional query. In other words, experimentation has become practical. Big data technologies have lowered the cost of failure to a point where experimentation—even on large data sets—is not only possible, but actually pragmatic. It’s highly logical to spend some time experimenting before you start a large-scale project where the end result has to live for multiple years and/or requires significant human and capital resources.

There’s an obvious intersection on this topic and the First Big Data Project Selection series that has been rolled up here. It also raises questions on decentralized experimentation versus creating centers of excellence (COEs) and what I call a Shared Analytic Cloud where the data, methods, and technologies are considered a shared corporate asset rather than a divisional endeavor. We could also talk about the roles of the people doing the experimentation (notice I did not say data scientists only—more on this later). We’ll save those topics for future articles.

As always, keep those comments, emails, and letters coming. I’m keen to hear how you are choosing to pursue experimentations and failure (whatever you prefer) in your organization, and the project and cultural issues you have encountered.