Judge a pharmacy forecast by the worst week of winter
Retail forecasting teams report average accuracy across thousands of products. For a pharmacy chain, the measure that matters is whether essential health products are on the shelf in the few weeks when everyone needs them at once, and boards should ask for that measure every autumn.
For most of my career, forecast accuracy has been reported as an average: weighted absolute percentage error or a close relative, calculated across every product and store and compared with last year and with the plan. I have produced that figure for a research firm, a supplier and two loyalty businesses since 2003. I have come to think that for a pharmacy chain it says almost nothing about what matters. My position is that a pharmacy's forecasts for essential health products should be judged on availability in the worst weeks of the respiratory and allergy seasons, and that the board should see that measure every autumn, before the season begins.
Average accuracy hides the weeks that matter
Mirebrook's forecasting system produces about 41 million store-item forecasts each week across 648 stores. In fiscal 2025 its weighted error across the front store was about 27%, which compares respectably with anything I saw at Hedgecombe or Vellacott. That figure is dominated by cosmetics, snacks, household products and seasonal beauty, where a missed forecast means a markdown or a lost sale and little more.
The products that worry me are a small share of that volume: children's fever medicine, oral rehydration salts, saline nasal sprays, glucose test strips, basic pain relief and, in May, allergy tablets. Their demand is calm for most of the year and then rises sharply and simultaneously across a region when an illness arrives. An annual average rewards a model for predicting the calm weeks well. A parent whose child has a fever in the third week of December has no interest in how well we forecast August.
The shortage of children's fever and pain medicine in late 2022 made this plain to me. Supply was the binding constraint, and no forecast could have created stock that manufacturers did not have. Yet our forecasts also failed in ways we could have controlled. They had learnt, from years of loyalty-linked sales, that demand followed promotions and member offers, and they were slow to recognise demand that rose without either. When we reviewed the period afterwards, our forecasts for essential products had been least accurate in exactly the eight weeks when they mattered most, while our headline accuracy for the year had improved.
A second problem lies in the data. About four-fifths of our front-store sales carry a member card, and it is tempting to build forecasts on member data because it is richer. Members are not all customers. During the shortage, non-member purchases of children's medicine rose faster than member purchases, because parents were visiting stores they did not normally use. A forecast trained mainly on member behaviour underestimated them, and the stores serving them ran out first.
An essentials list with an allocation rule
Since 2023 we have treated 212 essential health products as a separate forecasting problem. Their forecasts are judged on the share of store-days in the eight highest-demand weeks of each season when the product was on the shelf, which we call peak-week availability, and we expect 95% for most of the list. The models for these products use all store sales whether or not a member card was presented, together with weather, school calendars and the respiratory illness reporting that provinces publish. They are retrained before each season rather than continuously, so that a calm summer does not teach them to expect a calm winter.
We also agreed an allocation rule in advance. When supply falls below forecast demand, stock is sent to stores in proportion to their demand before the shortage, adjusted weekly for local illness reporting, with a limit per purchase that pharmacists can vary with a recorded reason. Member value plays no part in it. The executive committee approved the rule in 2023 so that nobody has to invent one during a crisis, under pressure from whichever store, owner or customer group is loudest.
Peak-week availability across the essentials list was 91% in the winter of 2023–24 and 97% in 2024–25. It was not free. We now hold about C$7.5 million of additional inventory on these products at the seasonal peak, and after a mild spring we wrote off C$1.2 million of expired stock. Our inventory team argued, reasonably, that the money would earn more in fast-moving beauty lines. The executive committee decided it should not be spent there, and the board was told why.
What the board should see each autumn
The boards of most retailers see forecasting, if at all, inside a discussion of inventory or working capital. I think the board of any retailer that sells essential health products should receive a short paper each autumn covering four things: the products the company treats as essential and the reasons; peak-week availability for those products over the last two seasons; the allocation rule that will apply if supply fails; and the inventory the company has decided to hold against the coming season, with its cost.
That paper runs to a page and a half. It turns forecasting from a technical subject into a decision a board can own: how much money to hold in stock against a hard winter, and who receives the product when there is not enough. Those are not questions for data scientists. They are questions about what kind of pharmacy a company intends to be, and a board is entitled to answer them before the season rather than read about the answer afterwards.
I do not claim that average accuracy is worthless. It remains the right measure for most of our assortment, and our planners rely on it every week. Nor do I claim that our essentials list is complete; we added nine products after the 2024–25 season and removed four. My claim is narrower, and I think harder to dismiss. A forecast is a decision about where scarce stock will be when people need it, and the quality of that decision is visible only in the weeks when demand is highest and supply is thinnest. Measuring those weeks separately costs a modest amount of analysis and some inventory. Not measuring them is a choice too. A pharmacy that reports only average accuracy has decided, without saying so, to measure itself on the weeks when nobody is ill.