A recovery exercise can be competently organised and still answer a question that is too narrow. The participants arrive, the scenario unfolds and the component returns within the time expected. The report is prepared. Somewhere beyond that report, a customer still cannot complete the task that made the service important. I am interested in that distance between an exercised component and an operational result. Closing it requires more than adding another measure. It requires agreement about whose work, authority and experience belong inside the test.
In banking, many apparently simple actions depend on several linked activities. An instruction must be received, understood, authorised where necessary, processed and reconciled. The person waiting also needs a usable account of what has happened. A disruption may interrupt any of those steps. If the exercise ends as soon as one platform is restored, it can leave the subsequent obligations unexamined. That does not make the technical test worthless. It means the bank should describe it accurately and decide what further evidence is needed before making a broader recovery claim.
Choose an end condition that matters outside the exercise room
My starting question is what the relevant person needs to finish. That may be a customer action, a staff action on the customer’s behalf or a controlled referral to somebody with appropriate authority. The completion condition should be specific enough that participants can recognise it. It should also identify the circumstances in which completion is not yet possible. Otherwise the exercise encourages people to improvise their own definition of success after they have seen the result, often using the part of the process their own department can most easily demonstrate.
There is a legitimate objection here: no exercise can reproduce the full complexity of a live bank. Resources are limited, and expanding every test indefinitely would make the programme unmanageable. I agree. The answer is not to pretend that each rehearsal covers everything. It is to state the scope and connect complementary tests deliberately. A component test can establish one part of the evidence. An operating rehearsal can examine how that part connects with people, records and decisions. Management should be able to see both the connection and the remaining gap.
The people who handle the affected work need a voice in defining the scenario. They may know that a nominally available alternative depends on a colleague who is absent from the test, a permission that cannot be obtained quickly or a reconciliation process that starts only later. Those observations are not excuses to avoid testing. They help the exercise resemble a meaningful operating question. Specialists must also remain responsible for their disciplines. Inviting operations into the discussion does not authorise the COO to decide whether a technical or security condition has been satisfied.
In our 2021 exercise, twelve scenarios produced eight results within the agreed conditions and four outside them. The programme had been completed, but that was not the result the bank needed to understand. The four exceptions concerned unresolved operating conditions after part of the service had returned. Keeping them visible required additional conversations and AUD 780,000 of follow-up work. It also required us to acknowledge that some participants had entered the exercise with different assumptions about where recovery ended.
Let the failed condition retain its name
Organisations are often better at recording corrective activity than at preserving the precise question that the activity is meant to answer. A finding becomes a workstream, the workstream acquires milestones and the milestones eventually turn green. By then, the original failure may be difficult to reconstruct. I want the account to retain a plain description of what could not be done, under which conditions and by whom. That description should remain attached to the corrective work until the relevant capability has been examined again.
This is not an argument for treating every failure as permanent. It is an argument for distinguishing remediation from verification. A revised procedure, a new support arrangement or additional training may be necessary, but its existence is not the same as evidence that the original condition has been resolved. A proportionate retest helps establish what changed. If the scenario itself changes, that should also be explained. Otherwise a successful second exercise may reflect an easier question rather than an improved capacity to answer the first one.
The reporting challenge is to remain clear without producing an unmanageable catalogue. Senior management and directors need to understand material limitations, the choices made and the resources required. They do not need every operational instruction reproduced in a board paper. A concise account can still preserve the important distinctions: demonstrated, not demonstrated, outside scope and awaiting further evidence. Those categories should direct attention to decisions. They should not become an elaborate language for avoiding responsibility or an invitation to classify an inconvenient result out of view.
I also want the cost reported. Follow-up work competes with other uses of people and money. If management chooses to postpone it, that is a decision with an operating consequence, not simply a scheduling detail. Conversely, expenditure alone is not proof of improvement. In our exercise, the AUD 780,000 allocation described resources committed to correction. It did not justify saying that all twelve conditions had been met. Keeping money, activity and demonstrated capability separate made the account less convenient and more useful.
Recovery continues after the interruption
When the immediate disruption ends, attention naturally moves towards returning to normal. That is precisely when unfinished work can lose its visibility. Instructions may have accumulated, manual records may require reconciliation and customers may still be acting on earlier messages. Restoring capacity can help, but it does not itself establish which obligations remain or how they should be ordered. The recovery plan needs an account of this later stage, including the people and authority required to close it responsibly.
I would not use an unqualified instruction to clear the queue as a substitute for that account. Some work can proceed through the normal controls; some needs additional review; some should not be repeated because the original instruction may already have been completed elsewhere. The bank needs to understand those differences before equating speed with recovery. Customer communication must also reflect what is known. A broad message that service has returned can be misleading if a particular action remains unavailable or an earlier request still needs attention.
The standard I apply to myself is modest but demanding: describe the capability the exercise actually demonstrated, preserve what it did not, and connect the next decision to that evidence. I do not expect a rehearsal to predict every future disruption. I do expect it to improve our knowledge of how the bank would respond. The most valuable result may therefore be a well-understood failure that changes preparation. A completed exercise has administrative value; an examined customer task gives that exercise an operating purpose.