Chief Data Officer / India
What changed when the attractive result was questioned
Decisions & consequences
01Withdrawing the renewal score that borrowed a later invoice
In 2022 we wanted to estimate renewal thirty days before a rental contract ended. A retrospective evaluation correctly classified 182 of 200 contracts, or 91%. I approved that result for a management discussion before asking when every input became available. The review then found a renewal invoice created after the intended prediction date among the inputs. The model was using knowledge from the future of the decision it was supposed to support.
The commercial sponsor wanted to retain the score as evidence that the underlying approach was promising. I agreed that the project might still be worth investigating, but the 91% claim could not stand as an account of usable early prediction. We withdrew it and allocated ₹41 lakh to rebuilding the evaluation around inputs available at the stated decision date. The work also required me to explain why my approval had been premature rather than locate the problem solely in the analyst's implementation.
A separate prospective shadow cohort of 200 contracts produced 154 correct classifications, or 77%. These were different contracts under a different evaluation design. We did not describe the difference as a measured fourteen-percentage-point deterioration caused by removing one field. Nor did 77% establish that the model outperformed a suitable baseline or created commercial value. The consequences of false positives and false negatives, and the action a prediction would trigger, still needed assessment.
The choice cost us a strong headline and delayed use of the model. It did not automatically alter hire eligibility, prices or customer treatment. Qualified commercial and operating owners retained those decisions. I would now specify the prediction date and input availability before approving an evaluation plan, then keep the baseline and error consequences beside the result. A score should be evidence about a decision the company can actually make, not a demonstration of how well a retrospective file can explain an outcome it already contains.
Refusing to rank depot demand from completed hires alone
In 2023 an expansion discussion compared two selected depots over the same twenty-eight days. DepotA completed 120 hires and depot B completed 80. The first had cycles available throughout the period. The second had availability on sixteen days and no available cycles on twelve. A proposal treated the higher completed count as evidence thatA was the stronger market and should receive the next allocation.
Harish challenged what the completed transactions could tell us about the twelve unavailable days. The records described what customers managed to hire, not every request they might have made if equipment had been present. I declined both the simple ranking and an alternative calculation that dividedB's eighty hires by sixteen available days and multiplied by twenty-eight. That extrapolation assumed the unobserved days would resemble the observed ones without establishing why.
We commissioned ₹32 lakh of observation work connecting appropriately collected request evidence with availability and deferred the expansion conclusion. Operating and financial owners retained the eventual equipment and investment decisions. The research was not permission to collect every customer movement or infer individual behaviour from unrelated records. Its purpose was to answer the defined depot question through a suitable account of requests and service conditions.
The immediate outcome was a decision to obtain better evidence, not a demonstrated increase in demand, utilisation or revenue. We did not reconstruct a missed-hire total or claim thatB would outperformAafter restocking. The commercial cost was postponing an allocation that colleagues wanted to settle quickly. I would now ask what the service made possible before interpreting the completed transaction file. Availability does not explain every difference between depots, but leaving it out lets a supply constraint masquerade as a complete account of the market.
Keeping a four-point reminder result proportionate to the trial
In 2024 we tested a booking reminder with 400 eligible adult business accounts. Two hundred were randomly assigned the reminder offer and two hundred the usual process. Within the same eight-week outcome window, forty-two offered accounts and thirty-four usual-process accounts completed renewals. The observed rates were 21% and 17%, a four-percentage-point difference. The trial cost ₹18 lakh.
The commercial team saw a practical reason to expand the reminder programme. The direction was encouraging, and waiting also has a cost. I supported further evaluation but did not treat the modest result as settled evidence of an effect across the full customer population. Random assignment strengthened the comparison; it did not eliminate sampling uncertainty, questions about execution or the need to examine whether one account's treatment affected another. A larger renewal count also did not establish net commercial benefit after the intervention's costs.
We retained the original allocation and outcome window in the report and chose another prespecified trial before a full rollout. The next evaluation would be designed around the decision and the uncertainty that mattered, rather than continued until a favourable number appeared. We did not retroactively select a responsive subgroup and present it as the original result. Customer permissions and programme terms remained with qualified owners.
The outcome was a proportionate next commitment, with a positive observed signal and its limits both visible. No universal uplift, profit result or precise causal certainty was claimed. I would now agree the possible next actions before seeing the trial result, including what the company would do with a modest difference. That makes it easier to avoid two equally unhelpful responses: declaring success because the number is positive, or dismissing useful evidence because it does not settle every future decision.
Finding the customer inside a report with no name column
In 2025 we considered a community market report built from thirty-six depot-week cells. Direct customer names had been removed. A review found that fourteen cells were dominated by a single business, while twenty-two were not. A recipient familiar with the local market might connect a dominated cell with a customer even without a name in the file. The absence of that field did not settle the disclosure question.
The commercial sponsor wanted the detailed report because local patterns would make it more useful. I accepted that a coarser account would lose some value, but declined to treat deletion of names as sufficient approval for release. We allocated ₹24 lakh to reviewing the reporting design with qualified privacy, legal and commercial owners. Their review considered the purpose, the recipients, the composition of groups and outside knowledge that could make a row more revealing than it looked internally.
The chosen direction was a coarser report subject to the responsible owners' review and agreed conditions. We did not invent a universal minimum group size, claim that the twenty-two other cells were automatically safe or assert that every resulting aggregate was anonymous. Nor does this fictional case describe customer identities actually being disclosed outside the company. It records a design review before the proposed detailed release.
The cost was reduced local detail and additional review expense. We claimed no measured revenue benefit or elimination of all disclosure risk. I would now include the likely recipient's knowledge when defining a report, rather than inspect only the fields exported. Useful market reporting requires a judgement about what the combined account reveals. A missing name column can be one part of that work, but it cannot stand in for the assessment of the actual use and context.