Case I
Keeping the author of a claim letter visible
In February 2025 we tested a proposed claims-correspondence assistant against 1,800 closed property files. The commercial proposition assumed that routine letters could be released with little intervention. Reviewers classified 1,440 drafts as having the correct supporting references, 216 as using an outdated endorsement and 144 as applying a reference from the wrong jurisdiction. Those categories covered the test set; a correct reference was not itself a complete assessment of a letter's insurance judgement.
The sponsor wanted to retain the original automation ambition. Claims colleagues were concerned that fluent wording would make a wrong reference harder to notice. I recommended a narrower release: the assistant could prepare a draft, but an authorised colleague had to inspect its source material and approve the communication. We accepted a USD4.6 million programme with a smaller initial scope, retained the review effort and moved automated dispatch out of the approved release. That was a material reduction in the promised convenience.
My team separated source retrieval, proposed wording and the release action. Claims and legal colleagues defined what the reviewer needed to see. We also recorded the reason for a substantive rewrite, so that the pilot would not report every approved final letter as an untouched machine success. The distinction made review slower at first, and some users asked for the extra record to be removed. We kept it while the evidence was immature.
In the subsequent supervised pilot, eighty-three of nine hundred letters were rewritten before release. Median preparation time for that cohort fell from twenty-two minutes to fourteen, excluding the later claim decision and any further investigation. Every letter still had a human approver. We did not extrapolate the observed rewrite rate to all products or describe the tool as an independent decision-maker.
I would involve experienced reviewers in the initial product brief next time. We spent too long debating whether people would remain in the process before specifying what a useful review actually required.