Judge a replatforming by how quickly each step can be undone
Boards are usually offered a choice between a bold replatforming and timid patching. The better question is whether each step of the change can be reversed before the next peak, and what it costs to keep that option open.
In September 2020, six months after I joined Halvorne, our board received a well-argued proposal to replace our commerce platform in one programme. The website, checkout, promotions and order management all ran on a single application first built in 2009, extended by dozens of teams and owned in full by none of them. On Boxing Day 2019 it had failed for more than three hours at the busiest moment of our year. The proposal priced a hosted replacement at A$88 million over two years, with a single cutover before Black Friday 2022.
I argued against it, and the board, after some hesitation, agreed with me. Peers have since asked whether my objection was to the vendor, the consultants or the money. It was to none of them. It was a judgement about reversibility. I would make the same judgement again, with some changes, and I think it applies well beyond our company.
Why the single cutover keeps winning the board paper
The single cutover has real advantages, and the people who propose it are not naive. It promises one design, one vendor relationship and one date after which the old system stops costing money. It avoids an awkward middle period in which two systems must agree on prices, stock and orders. It gives a chief customer officer a new website to launch rather than a series of invisible improvements. And it fits the way boards approve capital: a business case, a budget, a date and a benefit.
What it hides is the shape of the risk. In a single cutover, most of the uncertainty is resolved on one day, under the heaviest load the new system has yet carried, with no practical way back. I watched this happen as a developer at an English catalogue retailer in 2010. The new platform went live seven weeks before Christmas, order capture failed on and off for eleven days, and the old system could not be restored because its data had already been migrated and changed. The programme was not badly run. It was simply designed so that its largest risk arrived at its least recoverable moment.
Retail makes the problem sharper than most industries, because our calendar is fixed and unforgiving. An electronics retailer earns a large share of its online profit between Black Friday and the first week of January. A cutover that slips past October is usually deferred to February, and one that goes wrong in November becomes a profit warning.
A step is small enough when it can be reversed in an hour
The alternative is usually called incremental modernisation, and the phrase undersells it. We put a routing layer in front of the old application and moved one capability at a time behind it: product search first, then product content, the cart, checkout, order management and, finally, promotions. Each new service ran beside the old one. For each release we sent a small share of traffic down the new path, compared results, raised the share and kept the ability to send everything back to the old path within an hour.
That hour became our working definition of a safe step. If a change could not be reversed within an hour, it was too large, and the team had to split it further or build the way back before shipping. Some changes, such as a new data model for orders, cannot be reversed by switching a route, so we ran them in parallel for weeks and reconciled every order between the two systems before trusting the new one. Over 34 months we made 31 such releases. None was scheduled between 1 November and the first week of January.
Reversibility changed the board's conversations as much as the engineering. Instead of approving A$88 million against a date, the board approved a first tranche of A$22 million and a set of tests for releasing the next. Each tranche report showed which capabilities had moved, how the new path had performed at peak and what we had learnt about the old system's hidden behaviour. Directors could have stopped the programme at any tranche boundary with something useful in hand. They never did, but the option had value, and they knew it was there.
Paying honestly for two systems at once
The fair objection to this approach is cost, and its advocates often understate it. For almost three years we paid to run the old application and the new services together, to keep data synchronised between them and to staff teams who understood both. Dual running cost us about A$11 million against my original estimate of A$9 million. The whole programme cost A$71 million, less than the single-cutover proposal, but the comparison flatters us less than it seems: our programme took longer, and its visible changes arrived later.
There were other costs. Our chief customer officer gave up a complete website redesign for Black Friday 2022; customers saw the new design arrive in pieces through 2023. Some engineers found the work of moving behaviour piece by piece less satisfying than building something new, and two of our best left for greenfield work. And our riskiest component, the promotions engine, was left until last because it was the most entangled. It caused our only significant peak incident of the programme: four minutes on Boxing Day 2023 in which an expired promotion displayed the wrong price, leaving 2,300 orders that we honoured at a cost of about A$190,000. Leaving the hardest piece to the end felt careful at the time. In practice it meant our least-rehearsed change carried our most valuable day.
When is a single cutover right? When the old system is unsupported and cannot be secured, when its data has no clean seams along which to divide it, or when the business is small enough for the whole change to fit inside one quiet season. Those cases exist, and I would not argue a board out of them. I would simply ask any board considering a single cutover to require a written answer to one question: if this goes wrong on the busiest day of our year, how do we get back, and how long will that take? If the answer is that we cannot, the board is being asked to approve a bet, and it should know the size of the stake.
Our platform is not finished and never will be. The test I apply to each new piece of work is the one I applied in 2020: can we undo this before it matters? It is a modest question. In my experience it has saved more money than any architecture diagram I have drawn.