The headline counted the easy majority
The evaluation contained 5,000 orders. Responsible reviewers had identified forty requiring special commercial documentation; 4,960 were routine. Our candidate labelled every order routine. It therefore achieved 99.2% overall accuracy while missing all forty special cases. The arithmetic was correct, and the result did not justify the assistance we were considering. We held the model before deployment and incurred ₹44 lakh in assessment and rework.
The high score came from the composition of the reviewed population. Routine cases dominated it, so a rule that always chose the routine label was usually correct in a simple count. That did not make the rule useful for distinguishing the uncommon work. The score rewarded agreement on the majority without showing, by itself, that the intended special-documentation cases had not been found at all.
This was a useful example to discuss outside the modelling team because no elaborate technical explanation was needed to see the issue. If a proposed assistant never identifies the class of work that prompted its introduction, a high overall percentage cannot settle its value. The important question is what the business expects the assistance to distinguish and how the evaluation describes performance on that distinction.
The case was a predeployment review. It does not claim that customers were harmed or that a legal requirement was missed in live operations. The labels concerned a defined commercial-documentation review, and responsible people retained decisions about actual orders. Keeping that setting clear prevents the example from becoming a more dramatic story than the evidence supports. The lesson concerns evaluation and authority, not an invented operational incident.
I also did not conclude that accuracy should never be reported. It describes a legitimate aggregate property when its calculation and population are understood. The problem was allowing that property to stand for readiness to perform a particular business task. A score can remain part of the account while losing the authority to decide the whole question. That is a more precise response than replacing enthusiasm for one measure with a blanket rejection of it.
The intended assistance determines what errors matter
We needed to show the different kinds of decisions in the reviewed sample. The all-routine rule correctly labelled 4,960 routine cases and missed forty special cases. That breakdown made the limitation visible immediately. It also opened the more important conversation about the intended role of the model and the work retained by human reviewers. The system was not being given independent authority to determine every document an order required.
An evaluation should reflect that role. A tool used to prioritise review may need a different account from one proposed to make a final decision. The relative consequences of overlooking a case and asking someone to examine an unnecessary case also matter. Those consequences are business and specialist questions as well as analytical ones. A data team should help make them explicit rather than assume the most convenient aggregate score has already represented them.
The forty cases were the uncommon group in this defined evaluation. That does not mean the same proportion would occur in every future period or customer mix. Changes in the population could alter the meaning of an overall score even if the rule behaved similarly. The reviewed evidence therefore needed to retain its scope. A deployment recommendation would require attention to the intended setting, rather than treating one sample percentage as a permanent property of the business.
We also needed to avoid making a single replacement measure the new answer to every question. A class-aware summary can reveal something hidden by overall accuracy, but it cannot by itself settle workflow fit, reviewer capacity, appropriate authority or the quality of the underlying labels. The point was to build an intelligible evaluation around the decision, not to find another number that could take the place of judgement.
I wanted commercial owners involved because they understood why a case required different attention. Their expertise helped define the task the model was being asked to assist. Technical colleagues could then examine whether the candidate and its evaluation corresponded to that task. Neither group should have to surrender its questions to the other's vocabulary. A useful review lets them see the same proposed work from their respective responsibilities.
Holding the candidate was a decision, not a completed benefit
The model was held, and assessment and rework cost ₹44 lakh. Those are the outcomes this account supports. I do not claim a realised productivity gain, avoided customer loss or improved documentation performance. Preventing an unsupported deployment can be a sensible decision without being assigned a fictional financial benefit. The later value of a revised approach would need evidence from its own development and review.
The hold also needed a clear explanation to the people who had worked on the candidate. A high score had encouraged confidence, and simply announcing that the metric was wrong would have been inaccurate. The metric was calculated correctly. The recommendation attached to it was too broad. Distinguishing those points made the discussion about the work and the decision rather than about whether the team's effort had been dismissed.
I retained the all-routine rule as a useful comparison because it exposed how little the headline alone established. A more sophisticated candidate should be assessed against the intended task, not admired because its machinery is more complex. The comparison did not tell us which alternative would be suitable. It clarified why the present evidence was insufficient and what the next evaluation needed to make visible.
For an executive reader, the practical habit is to ask what a successful score could still be failing to do. Which cases are uncommon? What does the proposed assistant need to distinguish? Who retains authority when the output is uncertain or wrong? These questions are not a demand that directors design models. They are a way to understand whether management's evidence corresponds to the business role being proposed for automation.
The number 99.2% remains in my account because removing it would make the case less instructive. It was true within the evaluation and misleading as a complete argument for the intended assistance. Data leadership has to be comfortable with that combination. The task is to explain the gap clearly enough that colleagues can make a responsible next decision, while preserving the cost, the work still to do and the authority of the people whose judgement the tool is meant to support.