Data-Driven Isn’t Dead

Every few years, data science and technical terms enter the business lexicon, only to get popularised, over-hyped, and then retired from popularity. Machine learning, Artificial Intelligence, and so many other technologies are following these patterns. Unfortunately, even the most essential ideas can fall victim to this cycle.

The latest victim: data-driven.

What does it mean to be data-driven?

It’s no wonder that people are starting to tire of Big Data and the term “data-driven”. These days, just about every company claims the data-driven mantle.

Not for no reason. Data has become critical to success in any market. Even your local corner deli is probably using data to make some business decisions. This reality has led to a predictable backlash. Surely not every so-called data-driven business is using data the same way.

So what does it meant to be data-driven? At its core, a data-driven company uses their data to make better decisions and create more optimal outcomes. They find patterns and insights in the massive amounts of data their company collects every day to achieve business objectives more efficiently and improve KPIs.

In short, a data-driven company leverages their data to drive optimal outcomes.

A deli that looks at the high number of orders of a particular speciality and decides to make it a regular menu item may not be completing sophisticated analyses. Still, they may meet the minimum “data-driven” threshold if that decision increases revenue or gets more people in the door.

Of course, it would still be a stretch to equate a one-off decision with a cultural shift. A true data-driven company does not just occasionally leverage their data. They change the whole company culture around decision-making, improving it through effective use of data. Many companies conflate the two, arguably harming the reputation of “data-driven”.

Is data-driven dangerous?

But the excessive application of the term isn’t enough reason enough to throw out data-driven. Data-driven decisions are still a critical competitive advantage.

Unfortunately, many people disagree— some of the loudest voices calling on us to eliminate data-driven from our vocabularies: data scientists.

I’ve heard all sorts of arguments from people in the data science community about why data-driven is no longer the right goal. Some say it’s better to be simply “data-informed”, “data-aware”, or “data-conscious”: that our guts, checked by data from making really terrible calls, should drive outcomes. I’ve even seen it suggested that data-driven decision-making is dangerous, as if humans may lose control of the data. While Big Data has its limitations, most dangers and biases in analyses ultimately come from us, not the data.

The real problem with “data-driven” companies

A few data scientists coming out against “data-driven” do have a compelling point, however. Data-driven is often inaccurate. When you consider how most companies use data, it’s hard to argue that it is really being used to drive the best decisions.

Far too often, a forecast is made, and then a decision is taken based on those assumptions to deal with the reality presented. For example, a store manager looking at a demand forecast that says his store will sell 10 blue men’s shirts next week will order 10 blue men’s shirts to meet that demand. Very little consideration is made as to whether selling those 10 blue men’s shirts is the best way to meet revenue or margin goals. Data is driving the decision, but perhaps the wrong decision in the context of overall business goals.

This flaw in predictive analytics is often marked as an inherent flaw in data-driven decisions. Because the forecast can only tell you what will likely happen (assuming no crisis occurs), not how to achieve a given goal regardless of circumstances, a data-driven mindset is often written off as less than.

This fails to acknowledge the true issue: a predictive, not prescriptive approach.

How do we drive outcomes with data?

A true data-driven decision leverages data to drive outcomes: it uses data to show you the best path to achieving your business objective. When I hear a data scientist say that data-informed decisions are better than data-driven decisions, I can only assume that they use predictive analytics to interpret that data. If so, they’re right.

At its core, predictive analytics are most useful as a decision support tool. They exist to inform and give you context for your decision. It must be a data-informed approach when you use them because you otherwise risk optimizing towards the wrong goal. It’s a mistake that compounds over time because a good data scientist will improve the algorithm over time, achieving a sub-optimal outcome increasingly efficiently!

To be data-driven, you must optimise to achieve the best possible outcome. That inherently requires a prescriptive approach. In this case, you define the objectives best for your company and then leverage the data to get you there, no matter what crises and challenges arise along the way. It’s the only way to drive outcomes with data and finally fulfil the promise of a data-driven mindset.

Making data-driven meaningful again

Once you reveal the hidden problem behind the data-driven debate, it’s clear that people aren’t mad about the word; they are deeply frustrated by the limits of their approach to data overall. Luckily, this has a straightforward solution.

The right analysis — a prescriptive analysis — makes it possible to meaningfully drive decisions with your data.

Approached this way, it becomes clear that defaulting back to data-informed (or even worse, simply data-aware) decisions is limiting. The impact grows as you leverage your data more effectively. Organizations have to take the harder but ultimately better approach to analytics: prescriptive.

Data-driven, prescriptive businesses and projects are set up for success. So don’t discount the value of your data by simply moving back to the days where data-informed seemed like enough. You can meaningfully transform into a data-driven company with the right analysis. Hopefully, if we do that, people will start loving the term “data-driven” again as much as I do now.