The 5 secret ingredients to solve for profit, rather than accuracy
Customers can be unpredictable. Hardly anything about them is forecastable correctly. Change approach maybe?
Prediction errors are everywhere. Over the past 15 years, I met and worked with over 1,000 managers to help them make better decisions every day. First, while at McKinsey as a management consultant; then, at Evo Pricing developing B2B technology products, as a researcher.
Throughout this experience, I felt the fundamental need to predict the future that we humans systematically share: to feel more in control.
Letting go of the urge to narrowly forecast can, however, yield extraordinary results!
Embracing the Rapid Response approach of Prescriptive Artificial Intelligence transforms performance and frees up wasted resources. For example, when applied to the supply chain.
Even a perfect sales forecast, by itself, could not lead to optimal profit-driven decisions.
If, for example, your perfect forecast of demand was = 50, would you want your supply chain to ship 50 pieces? In fact, I will show how the optimal shipment could be =0 or =100, depending on other factors.
1 exhibit to size the challenge
Fundamentally, forecasting sales is the process of making predictions of future sales based on past and present data.
This activity may suffer from the curse of granularity (or, more technically, the curse of dimensionality): at high levels of aggregation, like for example budgeting total sales, it is possible to make estimates; but down to lower levels of detail, things become hairy: there just is too much going on at the same time.
Daily managerial decisions, unfortunately, must happen exactly at those lower levels of granularity, where forecasting is blurred and inefficient. For example:
- What prices to charge customers?
- How much product to stock in the warehouse?
- Which customers to invest in for acquisition and/or renewal?
Like photography: from a distance, printed photos make sense. Look closer, and all you see are circles of confusion. Even the largest global leaders in their own sectors indeed struggle to forecast with low, single-digit error rates.
In 2021, this may seem surprising. However, it’s due to the intrinsic characteristic of the problem. Let me focus on Supply Chain specifically for ease of treatment.
The challenge of Supply Chain forecasting
When going too granular, it is not possible to make accurate predictions. If you cannot even know how your own spouse may behave in certain circumstances, how could you ever predict the response of a total stranger? The analogy is relatively direct.
Supply Chain Managers, whose decisions require granular forecasting, create physical and economical waste.
Some companies tweak measurements to cover up the underlying inefficiency of forecasting, for example, by reporting broad aggregates, averages, or worse. Ouch! Of course, this solves nothing.
The broader art & science of forecasting sales is flawed. An urge to predict the future has been human nature for centuries after all: hard to change!
Our planet suffers from Supply Chain forecast errors
There are over $2 trillion of inventory in the United States alone — $2,040 billion. $1.43 of inventory for every $1 in yearly sales: 43% more inventory today, just right now, than all the product sold throughout this entire year.
Faced with the choice of producing more or risking to sell less, any manager would easily pick the first option, produce more and sell more. But what happens to our planet?
We waste huge resources every day.
According to the United Nations, the waste is enormous. How big?
- The world produces 50 million tonnes of electrical waste a year, weighing more than all the commercial airliners ever made. Only 20% is recycled.
- The world wastes 1.3 billion tons of food, one-third of the total, each year (source: United Nations).
- Each European wastes 179kg of food per year, according to the Swedish Institute for Food and Biotechnology.
While 45% of children under 5 years of age die because of undernutrition (source: World Health Organization), our supply chains waste one-third of the food produced every single day.
Can we better match supply and demand?
If somehow we could prevent the gaps opened up by forecasting errors, the benefits to the planet, the economy and ultimately everyone would be huge.
Developing Prescriptive solutions for success
To avoid forecasting errors, solve for impact, not accuracy; let data and machines find their own answers.
Where prediction is impossible, focus on Rapid Response.
Action/Reaction/Feedback theory focuses on cause-effect Action/Reaction — with maybe a footnote somewhere saying that Feedback is important. Since feedback can be unpredictable, classical theory tends to ignore or under-study it. Easier to make a career studying predictable cause-effect links.
A Supply Chain system based on feedback, a prescriptive system, is instead geared towards continuous learning & discovery: a feedback-first machine.
To build a supply chain learning machine, let us cook a yummy recipe using 5 not-so-secret ingredients carefully.
Ingredient #1: target effectiveness, not accuracy
This is a profound cultural shift for many people whose entire job description was to build accurate systems.
At least in the traditional definition of predictive systems, accuracy is measured as observed versus predicted demand, which is less relevant to prescriptive systems. A prescriptive system is literally a specific type of learning machine that efficiently targets optimal results for its objective. If you set an inaccurate objective by asking the wrong question, then you will get optimally wrong results.
Not just wrong, but very efficiently wrong, even!
Here are some examples of how traditional accuracy can be inefficient business-wise:
- Business outcomes are measured in $. Imagine to predict you will sell 10, and then sell 20 — which means you will need to replenish faster. This would be a 100% inaccurate prediction: error of 10 on a baseline of 10. But it is also a 100% business gain! Why a penalty to a 100% gain? In principle, it’s understandable, but business-wise it is wrong. It makes no sense really when you think about it.
- Cost of risk. Not all errors have the same cost. The previous example shows one case of understock (sell more than expected); what if instead, you end up overstock? Imagine you have two products A and B, normally a fast-seller and a slow-seller, selling respectively 100 and 10 units per week. You run a promo and predict 300 sales of each, but they only sell 200 each. Both forecast errors are 50%, but the accuracy measure tells you nothing about the risk of having left 100 extra pieces of A (just 1 week of regular sales) versus B (10 weeks of sales!). Poor metric.
- Customers substitute across products. Measuring granular accuracy assumes that sales of different products are independent, which is incorrect. For example, you have two products, A and B, and estimate to sell 10 of A and 10 of B. Then you have the same 50% average error (MAPE) for the scenarios where the actual is 5&15 (substitution) or 5&5 (same proportion but less overall). In the first case, the aggregate estimate of 20 was 100% right, but in the other case, it’s 50% wrong. Accuracy tells you nothing valuable about substitution.
- Probability versus reality. Expected sales are a probability distribution a priori, and there is no good way to assess the actual probability distributions from observed outcomes a posteriori, without adding noise and distortion. You start with a curve (expectation), end up with a number (sales); how to tally the two against each other?
- Demand versus sales. You can only directly observe sales, not the underlying demand. Therefore even at a conceptual level, you cannot measure demand accuracy, which would be the correct metric. To do so, you would need to reverse engineer the adjustments to sales required to approximate demand, but by doing so, you would be effectively creating your own data, and the methodology would not be reliable.
All these points are examples suggesting much the same thing: accuracy is just not such a relevant business metric overall, and specifically, it should never be used to build or measure a prescriptive system.
Rather than improving accuracy, change perspective: overcome structural limits by targeting profits directly.
The beauty of learning systems is that they directly target any business outcome you set. Profit is more relevant to business than accuracy.
Ingredient #2: merge demand planning with logistics
If you are like me: at this point, not convinced yet! Where is the data? Where is the evidence?
Feeding forecasts into logistics systems destroys value. The magic “secret” recipe: blend costs and revenues together!
Unfortunately, managers in companies today are suboptimally split into silos. One of my clients explained just yesterday how their forecast feeds the order system that in turn feeds the shipment system. Three steps away removed from the customer and the market.
Each step breaks up the problem into manageable sub-chunks that may make sense from the management point of view but dissipate information and, therefore, ultimately create waste. Destroying value from the shareholder’s point of view.
Here is why, in numbers. Using a simplified but relevant example.
Let us imagine you are the supply chain manager of a grocery retailer selling bananas. You need to decide how many bananas to ship today to your store. Similar logic would apply to any type of product or service and any time horizon, but let’s keep things simple for the argument’s sake.
Let us imagine that your true expected Demand function for bananas is a classical normal distribution, with mean/mode/average demand of 50 and standard deviation of 10. Any other probability distribution could be used, or even just implied, but let us keep things very simple for this example.
Traditional systems think in terms of point estimates; you may need to think a little bit to see the probability distribution example.
Now, if someone asked you what is the forecast demand of bananas, and assuming you want to keep your job, you would likely reply with the least wrong, or the most accurate, value of 50.
You would expect actual sales to have an error in the region of ±10 (assuming 1 standard deviation just for simplicity), which is a 20% error rate, but, hey, that is actually a pretty good estimate if you go by the opening benchmark above! So all is good. Or not?
Your error measure would compare sales ex-post with expected demand ex-ante, which is conceptually wrong as they are different. But, let us accept this fault: demand is not directly observable, only sales are. Therefore no traditional way could be used to directly measure your estimate of demand. OK, so let’s move forward.
However, there is a bigger problem.
Entrenched in the forecast/order/shipment process, a huge systematic usage error occurs every day, even with perfect forecasting.
If I were the logistics manager receiving your estimated forecast of 50, let us assume I fulfill all of it, so I pass along your forecast pieces efficiently to the store. What happens then?
Every shipment effectively trades-off a guaranteed cost today for a likely profit tomorrow.
- Today, I spend money directly for guaranteed costs in inventory, logistics, finance, shrinkage (the product lost in transit) and overhead, without even considering the opportunity cost of not keeping the stock in the warehouse.
- Tomorrow, I will generate potential profit from units sold out of those we had shipped, but also additional cost from waste and returns where product is left unsold.
Depending on the exact values of costs and profit, optimal shipment can be entirely different from forecast, even if zero-error.
For example, assuming a total cost of 40 for every unit shipped, an expected gross profit of 60 (assuming list price of 100 and product cost of 40) for every unit sold, and standard deviation of 35, then the curve of expected profit based on the number of shipments N is as follows:
However, if instead of assuming a total cost of 10 for every unit shipped, and the original standard deviation of 10, then the curve of expected profit based on the number of shipments N changes as follows:
Note: full workings in the appendix.
In this example, by shipping 50 units, you would GUARANTEE a systematic profit mistake, even with a correct forecast. Direct negative impact on profits. By cutting out all valuable demand information between forecasting and logistics, using a single point estimate as demand input for your separate logistics system, you will have systematically destroyed value.
The logistics system would not be able to make any informed assessment of optimal shipment, let alone of more complex cross-product estimates like substitution. It will just try to fill the point estimate of demand.
Traditional forecasting is not aligned with the perspective of customers, and therefore destroys business value by virtue of its own design.
Interestingly, the simplified but realistic expected profit curve above shows a strong skew. Since the expected profit is much higher on the right-hand of the optimal solution, managers instinctively tend to over-ship, creating systematic waste (see the section on our planet above) rather than risking missing out on valuable sales.
Ever heard of safety stocks systematically added on top of estimates?
You do not trust your own forecast, so you keep a margin of safety. So you keep wasting your own stock and miss out on sales opportunities.
Ingredient #3: embrace risk as opportunity
Depending on the standard deviation of demand i.e. how volatile and unpredictable the market is, as well as on logistic costs, the optimal shipment could be anything from 0 to 100+, forecast demand still 50.
In the modern world, demand tends to become more and more unpredictable after all, making traditional forecasting even more obsolete. So this is a common scenario. But risk can also be an upside opportunity!
Risk is technically the demand variance: what happens when demand becomes more volatile and less predictable, even with the same forecast of 50.
In this example, instead of a standard deviation of 10, now 5x more volatile, therefore also standard deviation is now 50.
In this case, the optimal shipment would not be 60 anymore, but 98. Broadly 2x the forecast of 50. By using traditional forecasting, you would introduce a huge systematic profit error capping the shipment to 50.
And you would be left none the wiser because you would keep shipping 50 and being close to 100% accurate by selling all 50 pieces.
On average, sell 50. Which sometimes means not out-of-stock, making the detection of missed opportunity tricky. And so repeating the same mistake over and over, until the demand function changes — for example, a competitor gets your missed sales of bananas and thank you for the new customers gifted to them.
Business opportunity gone forever, all while being nearly 100% accurate.
Better to be rich or right?
Ingredient #4: Shipment Coverage = Frequency + Lead-time + Risk + Constraints
Depending on cost and variance, the optimal shipments can be anywhere from 0 to 100, even while forecast demand is always 50.
This simplified example assumes you are going to ship bananas today for delivery tomorrow to the store. In other scenarios, frequency of shipment may be lower, lead time of shipment (the time it takes between order and receipt) longer. Predictive systems would forecast sales forward, making the error even larger.
A prescriptive system, on the other hand, takes those parameters as input: the longer the lead time and the slower the shipment frequency, the greater uncertainty in demand, and the greater would be the systematic error of traditional predictive systems and processes.
What about constraints?
For example, the bottom left of the table above could show higher numbers than 99 had I not had the warehouse constraint (and the sneaky appetite to eat 1 banana myself!). Constraints have a simple impact: limiting the freedom of prescriptive systems to operate.
Every constraint can only worsen the expected profit, but never improve it. So the fewer, the better.
Why, for example, imposing minimum shelf quantities? Or minimum shipment volumes when shipment profit is already being maximized? No need, really.
More details in my other Medium piece: https://towardsdatascience.com/supply-chain-optimization-worth-it-or-not-20ae4c6e635
Ingredient #5: get inspired by Rapid Response
How to implement a rapid response system?
Prescriptive analytics is the new frontier of management effectiveness, yet they open up significant questions:
- How to determine the Demand function?
- How to make sense of system recommendations?
- How to validate results if accuracy is not the right metric?
Every day I see prescriptive systems routinely over-perform traditional predictive systems by 20–30% greater inventory efficiency: more sales with less inventory. Sometimes less can be more.
At the same time, I feel the cognitive dissonance. Real quotes from managers I work with:
- We have been working for years to improve accuracy by 1 or 2 points. How could a 20–30% improvement even be possible? [it’s actually often a day 1 improvement]
- After A/B testing, how can long-term improvement be measured vis-a-vis other variable business factors? [by choosing the right target KPIs]
- Will we have the right data to estimate demand perfectly? [existing data is typically OK when enriched with external data sources; and the more, the merrier, whereas perfect is not possible anyways]
Ultimately prescriptive systems can only be measured by testing them, which is a self-recursive loop: they can only work if one believes in them as the solution to otherwise unsolvable challenges.
Examples by analogy
Embracing all the complex points above may be difficult initially. So let us be inspired by analogy to other Rapid Response systems:
Analogy: Customer demand & the weather
Weather forecasting actually is: rapid observation — pattern recognition — update. Since the problem is notoriously hard, like customer demand, scientists rely on advanced techniques to gather big-data and adjust estimates in real-time.
Exactly the same steps required towards a Rapid Response Supply Chain.
Analogy: Customer demand & the stock market
The stock market is not so much a forecasting engine but a Rapid Response system. In the long-term, it approximates the estimated value of companies’ equity (in aggregate, you can also forecast customer demand), but, on a daily basis, it is the most efficient engine to rapidly incorporate new information as soon as it comes along.
Stock prices are determined in real-time by the balance of supply and demand. A continuous discovery engine that can rather efficiently allocate trillions of actual dollars every day. Or, at least, no one could come up with an even more efficient mechanism yet!
Why not adopt the same approach to match physical supply and demand, to allocate trillions of inventory dollars in everyday supply chain decisions?
Respond rapidly, the money will follow
Start from the business objective, and unlock tremendous business value every day. In doing so, help our planet better than many sustainability programmes ever could.
Rapid Response guarantees that any error self-corrects, so what is the risk?
To build a Rapid Response system, you need two ingredients:
- Machine learning to estimate expected demand, built on as large a data set as possible
- Profit optimization to prescript decisions, using classical operations research techniques but at a large scale.
In a previous post, I advocated for the salmon strategy: start from the impact you want, then work backwards. Relatively logical once seen this way.
Like the Chinese proverb:
A Journey of a Thousand Miles Begins with a Single Step
The accuracy-driven traditional predictive process of forecast-order-shipment is broken. Embracing the Power of Prescriptive is the logical first step to finally match supply with demand in our world of ever-growing complexity.
Happy rapid responding!
Full workings in the appendix, also including the downloadable illustrative source file for a detailed review of the examples.
About the author
Fabrizio Fantini is the brain behind Evo. His 2009 PhD in Applied Mathematics, proving how simple algorithms can outperform even the most expensive commercial airline pricing software, is the basis for the core scientific research behind our solutions. He holds an MBA from Harvard Business School and has previously worked for 10 years at McKinsey & Company.
He is thrilled to help clients create value and loves creating powerful but simple to use solutions. His ideal software has no user manual but enables users to stand on the shoulders of giants.