How to measure forecast accuracy: MAPE pitfalls, WAPE, MASE and bias

"Our forecast is 85% accurate" is one of the least useful sentences in inventory planning. Accurate by which measure? At what level — product, category, total? Over which horizon? Compared with what?

Accuracy metrics are not interchangeable. Each one answers a different question, and the most popular one, MAPE, answers it badly for a typical Shopify catalogue. This guide works through the main metrics with small, checkable numbers.

Key takeaways

• MAPE overweights slow sellers and breaks entirely when actual sales are zero. Avoid it for SKU-level inventory decisions.
• WAPE weights errors by volume, so it reflects the units — and money — you actually got wrong.
• Bias tells you the direction of error. Two forecasts with the same WAPE can have completely different bias, and bias is what builds excess or causes repeat stockouts.
• MASE compares your forecast with a naive "same as last period" forecast. Below 1 means you beat it.
• Measure at the lag that matters: the forecast you made one lead time ago, not last week's.

The errors behind every metric

Every metric starts from the same building block, the error for one product in one period:

error = forecast − actual

We'll use this sign convention throughout: a positive error means you forecast too high. The metrics differ in how they add these errors up.

MAPE and why it misleads

Mean absolute percentage error averages the percentage error of each item:

MAPE = average of |forecast − actual| ÷ actual

Here are five products for one month:

Product E can't be included at all — you can't divide by zero. Across A to D:

MAPE = (10% + 15% + 25% + 150%) ÷ 4 = 200% ÷ 4 = 50%

A 50% MAPE sounds dreadful. But look at where it comes from: product D, which was 6 units out. Being 6 units wrong on a product that sold 4 doesn't matter much to the business. Being 50 units wrong on product A does, and it contributes only 10%.

MAPE has other problems too. It treats over- and under-forecasting differently (an under-forecast can never be more than 100% wrong; an over-forecast can be any size), so methods tuned to minimise MAPE tend to forecast low. For inventory, forecasting low means stockouts.

WAPE: errors weighted by volume

Weighted absolute percentage error adds up the absolute errors and divides by total actual sales:

WAPE = Σ |forecast − actual| ÷ Σ actual

For the same five products:

Σ |error|  = 50 + 30 + 10 + 6 + 2 = 98
Σ actual   = 500 + 200 + 40 + 4 + 0 = 744
WAPE       = 98 ÷ 744 = 13.2%

Product E is included without trouble, and product A — where most of the units are — dominates, as it should. WAPE is the metric we'd recommend as your headline accuracy number. If you want it weighted by money rather than units, multiply each product's errors and actuals by unit cost first.

Note that WAPE depends on the level you calculate it at. Errors cancel out when you add products together, so a category-level WAPE will look better than the average product-level WAPE. Calculate it at the level you buy — the variant — and then add up.

Bias: the direction of error

Bias measures whether errors lean one way:

bias = Σ (forecast − actual) ÷ Σ actual

For the five products, total forecast is 450 + 230 + 50 + 10 + 2 = 742, against actual sales of 744:

bias = (742 − 744) ÷ 744 = −0.3%

Near zero. But bias computed across products hides a lot, because over-forecasts on one product cancel under-forecasts on another. Bias is most useful per product, over time. Consider two forecasts for the same product over six weeks:

Forecast X's absolute errors are 10, 20, 0, 15, 5 and 10, which add up to 60. Forecast Y's are 10, 10, 10, 10, 10 and 10, also 60. Both have the same WAPE:

WAPE = 60 ÷ 600 = 10%

But their bias is completely different:

bias X = (660 − 600) ÷ 600 = +10%
bias Y = (600 − 600) ÷ 600 = 0%

Forecast Y's errors cancel out over time; safety stock handles them. Forecast X is always high, so every order it drives is about 10% too big, and the excess accumulates on your shelves. Fix bias before you chase a lower WAPE. Bias is usually a process problem — an optimistic override, a promotion counted as normal demand, a trend that was never damped — rather than a modelling one.

MASE: did you beat doing nothing?

Mean absolute scaled error asks a different question: is this forecast better than the simplest possible one? It divides your forecast's mean absolute error by the mean absolute error of a naive forecast ("next week will be the same as this week") over the product's history.

Here's a product's weekly history:

The naive forecast's errors are the week-to-week changes: |24 − 20| = 4, |18 − 24| = 6, |22 − 18| = 4, |26 − 22| = 4 and |20 − 26| = 6. Their average:

naive MAE = (4 + 6 + 4 + 4 + 6) ÷ 5 = 24 ÷ 5 = 4.8

Now your forecast for the next four weeks is 22 per week, and actual sales come in at 25, 19, 23 and 21. Errors of 3, 3, 1 and 1:

forecast MAE = (3 + 3 + 1 + 1) ÷ 4 = 2
MASE = 2 ÷ 4.8 = 0.42

A MASE below 1 means you beat the naive benchmark; above 1 means you'd have done better copying last week. For seasonal products, scale by a seasonal naive forecast (same week last year) instead.

MASE has two big advantages for a catalogue. It is scale-free, so you can compare a 5-unit product with a 500-unit one. And it never divides by an individual zero sale, so it works for slow sellers. That makes it a good metric for choosing between models for each product, whereas WAPE is better for reporting accuracy across the business. Stokrio's demand forecasting uses MASE to pick each product's model in backtests and reports realised accuracy and bias of past forecasts against what actually sold.

Measure at the right lag

A forecast made last week for this week is easy. The forecast that drove your purchase order was made one lead time ago. If your supplier takes eight weeks, the accuracy that matters for inventory is the forecast you made eight weeks before each week of sales.

Keep every forecast you generate, with the date you generated it. Then for each week, compare actual sales with the forecast made one lead time earlier. This is less flattering than one-step-ahead accuracy, and much more honest.

Exclude what wasn't demand

Don't let the measurement punish the forecast for things it couldn't know:

• Stockout periods. If you were out of stock, actual sales understate demand, so the forecast will look too high when it may have been right. Exclude those periods or compare against estimated demand.
• Unplanned wholesale orders. Measure retail demand separately.

But do include planned promotions, if the promotion was in the plan. If the forecast should have known about it, it should be judged on it.

Did your overrides help?

If people adjust the statistical forecast — and they should, when they know something the history doesn't — measure the statistical forecast and the final plan separately. If the final plan's WAPE is lower, the overrides added value. If it's higher, or the plan's bias is consistently positive, the overrides are costing you money. This comparison is often called forecast value added, and it's the fastest way to have a productive conversation about who changes the forecast and why.

What to put on the dashboard

1. WAPE by month, at variant level, weighted by cost, at your lead-time lag.
2. Bias by month and per product, with the ten most biased products listed.
3. MASE per product, to spot products where the forecast loses to naive and needs attention.

Skip MAPE. If someone insists, show it alongside WAPE and let the difference start the conversation.