Rank a utility's systems by what stops when they fail, not by the network they sit on

I think electric utilities protect their systems in the wrong order. We spend most of our security effort on the control systems that the federal reliability standards cover, which is right, and much less on the business systems outside that boundary, which is a mistake whenever those business systems are what would make us stop. My position is that a utility should rank every application by what stops if it is unavailable for three days, for whom and at what cost, and should protect the systems at the top of that ranking to the standard of its control centre, whichever network they happen to sit on.

The boundary compliance draws

The NERC CIP standards are mandatory for the cyber systems that monitor and control the bulk power system, including control centres, transmission substations and generating plants, with requirements scaled to each system's impact on the grid. They have made the industry's operational technology considerably harder to attack than it was twenty years ago, and I do not want to be read as dismissing them. Our operations and compliance teams maintain that programme carefully, and it is audited.

The standards define what must be protected for reliability. They do not, and are not meant to, tell a utility which business systems it cannot operate without. Those systems sit on the corporate side: the applications that submit offers and schedules to the regional market, nominate natural gas to power plants, account for energy and settle with customers and, for a cooperative like ours that runs services for its members, manage outages and send bills. A utility can be fully compliant and still have its trading desk, its gas schedulers and its storm room depending on the same email-connected workstations and directory as its accounts payable clerks.

In May 2021 a ransomware attack reached the business systems of a US fuel pipeline company. The company stopped the pipeline as a precaution for several days, and fuel shortages followed in parts of the south-east. I do not know the full reasoning behind that decision and will not speculate about it. The lesson I drew is general: an attacker does not need to touch the controls that move a product to stop the product moving. It is enough to leave the operator unsure that it can schedule, measure or bill what it moves.

Ninety-one applications and one question

I joined Merrowind two months after that attack, and our board's audit and risk committee asked whether something similar could stop us. I could not answer, so my first task was an inventory. We listed ninety-one business applications and put one question to their owners, in conversation rather than by survey: if this system and its data were unavailable for three days, what could you not do?

Most answers were reassuring. Payroll can run late, purchasing can use paper and the website can be down. Thirteen answers were not reassuring at all. Without the market systems we could not submit day-ahead offers or schedules, leaving our plants and our members' load exposed to the market without our instructions. Without gas nominations two plants could run short of fuel after the next nomination deadline. Without energy accounting we could not settle with our members or check the market's settlements. Later, an ice storm in February 2023 added a fourteenth: without the shared outage management system, fourteen member cooperatives would restore power by telephone and paper maps.

These systems shared the corporate directory, the corporate backups and, in places, administrators' credentials with everything else. Nobody could tell me how long it would take to restore them, because nobody had tried.

We moved them into a separate enclave between 2022 and September 2023, adding the outage system in 2024. It has its own identity directory and administrators, hardware security keys, no email or web browsing on its workstations, backups held offline and procedures for running each function by telephone and spreadsheet for 72 hours. It cost US$8.7 million in capital and costs US$1.4 million a year to run, and we deferred a new data platform by a year to pay for it. Our vice president of energy markets objected to traders losing remote access from their own devices, and our chief financial officer would have preferred more insurance. Both positions were reasonable, and the decision was our chief executive's.

The number to bring to the board

The most useful product of the work was not the enclave but a number: how long it takes to bring the fourteen systems back from offline copies when everything else is assumed lost. In a planned test in November 2024 the answer was nineteen hours. The trading desk worked from the manual procedures for most of that time, and the test exposed two gaps: a market certificate stored on a corporate laptop and a gas nomination contact list eighteen months out of date.

I now give our audit and risk committee that restore time alongside the compliance position, and I would encourage any utility board to ask for its equivalent. A framework rating tells directors how a programme compares with a standard. A tested restore time tells them how long the company would be unable to do something specific, which is the question they will face if an attack succeeds. Listed companies now operate under federal disclosure requirements covering material cyber incidents and the board's oversight of cyber risk. A measured number gives that oversight something solid to rest on.

Ranking by consequence has costs. It moves budget from systems that staff can see into systems few people notice. It irritates people whose tools become less convenient. It has to be revisited whenever a system is bought, so we now require a classification before purchase rather than after. And it can be wrong. My first version left out the outage system because I thought of it as a member service rather than a Merrowind operation, and members' line crews corrected me during the ice storm.

The question I would put to any utility board is the one we put to our own staff. List the business systems whose loss for three days would stop you scheduling, fuelling, settling or restoring power. For each, ask who administers it, where its backups are kept and when it was last restored from them. If the list is short and the answers recent, the boundary is in the right place. If not, a compliance map has drawn it for you.