Validate the capital return as carefully as the pricing model

A property and casualty insurer in Canada spends a great deal of effort validating the models that set its prices and reserves. It spends much less on the chain of systems, mapping tables and spreadsheets that turns its claims and policy records into the figures it reports to OSFI, including the inputs to its minimum capital test, the MCT. My position is that this chain should be governed as a model: inventoried, owned, changed under control and independently challenged. The judgements embedded in it are as consequential as those in a pricing model, and they are often made by people who do not know they are making them.

I came to this view through an error I should have caught earlier. In 2022, my first year as Haverlock's chief data officer, my team was tracing data lineage for our IFRS 17 implementation when we found that a claims-system migration in 2019 had mapped claims under home-based business endorsements into our personal property line instead of commercial property. The amounts were modest: about C$46 million of unpaid claims across eight quarters. The effect on our MCT ratio was about two points on a ratio above 200 per cent. Nobody's solvency was in doubt.

But the error had passed through every control we had. The return reconciled to the general ledger, because the ledger used the same mapping. The actuarial work drew on the same data. Our auditors tested totals, and the totals were right. What was wrong was a classification inside the totals, and classification is exactly what the capital framework is sensitive to, because the risk factors applied to unpaid claims and premium liabilities differ by line of business.

The return nobody treats as a model

Most insurers would say their regulatory returns come from a reporting process, not a model. I understand the distinction. A model estimates something uncertain; a return reports something that has happened. Yet between the claims system and the return sit dozens of decisions that are estimates in all but name. Which line does a home-based business claim belong to? How is a package policy for a small business split across lines? Which manual adjustments are carried forward each quarter, and why? When a source system is replaced, who confirms that the new mapping means what the old one meant?

Each of those decisions is an assumption. In a pricing model, an assumption of that consequence would have a named owner, a documented rationale, a change log and an independent reviewer. In most regulatory reporting chains I have seen, it has a spreadsheet tab and the memory of the analyst who built it.

OSFI's guidance on model risk management takes a broad view of what a model is, and its revision reaches further across a federally regulated insurer's activities, including models built with machine learning. I would not claim the guideline requires any insurer to treat its reporting chain as a model; that is a matter for each insurer and its supervisor. I would rather an insurer erred towards including it than spent effort arguing it out.

Mapping tables are assumptions

After the 2022 discovery we argued internally about what to do. Our chief financial officer's first preference was to correct the mapping prospectively and note the change in the next filing, on the grounds that the amounts were immaterial. I argued that we should tell our OSFI relationship manager before the next filing, resubmit the two most recent quarterly returns and take the root cause to the audit committee. The chief executive and the audit committee chair agreed. The work took nine weeks of finance and data time in the middle of our IFRS 17 parallel run and delayed our customer data platform by six months.

We then identified 184 critical data elements feeding the capital return and the annual regulatory return. Each was given a business owner in finance, actuarial or claims, a documented lineage from source system to return, and an automated reconciliation comparing the element at each step. Mapping tables now sit under change control: a change needs the owner's approval and a test against the prior quarter's data before it can be used. Manual adjustments carried forward for more than two quarters are reported to the controller and the appointed actuary with an explanation.

Treating the chain as a model also meant independent challenge. Once a year a team outside the reporting chain, drawn from our model validation function, reviews a sample of the critical elements from end to end, as it would review a pricing model. The first review found two further mapping questions, both small, and one manual adjustment that had been carried for eleven quarters by an analyst who had since left the company.

What an audit committee should ask for

Audit committees of insurers receive assurance on the financial statements and, increasingly, on IFRS 17 processes. Few receive a direct report on the data behind the regulatory returns. I now report to ours twice a year on four measures: the share of critical data elements with a documented and tested lineage; the number of mapping changes and whether each was approved before use; the count and age of manual adjustments; and any reconciliation breaks above a threshold, with how each was resolved.

The measures have improved. Quarter-end reconciliation that took seventeen working days in 2022 took seven in 2025, and our external auditor's 2025 management letter carried no data finding on the returns for the first time in four years. I mention these results cautiously, because the purpose is not a better-looking report. It is a committee that knows where the judgements in its returns are made, and by whom.

There is a cost, and some colleagues still think it excessive. The lineage programme cost about C$3.8 million over two years and takes roughly six people to maintain. Finance teams resent change control on tables they used to edit freely. I understand that, and I would rather they resented the control than discovered, as I did, that a migration nobody questioned had been quietly reclassifying claims for more than two years.

My suggestion to an audit committee chair is modest. Ask management to name the ten judgements in the regulatory returns most likely to be wrong, who owns each one and when each was last checked by someone who did not make it. If the answers come easily, the chain is governed. If they do not, the committee is relying on a model nobody has validated.