Keep the evidence attached to the result

A new external research product was planned around eight source arrangements and coverage of 8,400 companies. The internal prototype could display the source material, and the commercial draft treated that availability as evidence that the same material could appear in client-facing outputs. I asked the product team to describe the actual distribution, including downloadable excerpts and reports carrying the client's branding. That description differed from the internal use for which some sources had been obtained.

Legal colleagues reviewed the arrangements against the proposed product. Three sources did not support that distribution under the terms then in place. We could seek amended terms, redesign the output or release a smaller product. The commercial sponsor preferred to launch with the broad coverage figure and resolve the detail during onboarding. I recommended against that sequence because the unresolved permission affected what we were offering, not merely an internal administrative step. The legal judgement remained with qualified colleagues; I made its data and product consequences explicit.

We spent ₹24 lakh revising the source-to-output controls and delayed launch by five weeks. The first release used five sources and covered 6,100 companies. Two prospective clients chose to wait because the narrower coverage did not meet their needs. We did not count those prospects as retained revenue or imply that the remaining companies were inherently better evidence. The product was simply smaller and aligned with the approved use of its sources.

The remaining arrangements stayed under separate commercial discussion. No future permission was assumed in the published scope. I learnt to require the distribution description before a coverage commitment is made, because a broad internal prototype can otherwise create an expectation the company is not yet able to fulfil. The change was not a claim that access to information is always restricted in the same way; it was a decision based on the specific arrangements and output under review.

We used a single text label to identify a defined type of business-risk statement in company research. A review sample of 240 passages produced disagreement on 74 between two experienced reviewers. The first proposed remedy was more training on the existing label. I asked the research lead to examine the disagreements before assuming that one reviewer had simply applied the instructions poorly. Many passages contained several voices: a company's own statement, a quotation from another party and an analyst's explanation of what remained uncertain.

The single category did not clearly say which of those forms counted. One reviewer marked the presence of the words; another required the company itself to make the statement. Both could point to examples in the old guidance. We separated source attribution from the substantive research category and added an unresolved route for passages whose meaning could not be established from the available material. The research team owned the revised definitions and examples. My team rebuilt the annotation fields and evaluation process around them.

The change cost ₹16 lakh and delayed the planned dataset refresh by four weeks. We revisited 1,600 previously labelled passages using the revised method rather than mechanically translating old labels into the new structure. In a fresh 240-passage evaluation, disagreement remained on 29; those cases received adjudication or stayed unresolved. The improvement did not establish that the classification was universally correct. It showed that the revised question was more consistently understood within the reviewed sample.

Product colleagues had to explain a more complicated output, and some clients preferred the simplicity of one flag. We retained the distinction because simplicity would otherwise hide whose statement the label described. The lesson was to inspect the question before trying to improve an agreement score. An unresolved category can be valuable when it preserves uncertainty that a forced binary decision would conceal.

A draft sector study reported 14% revenue growth using 420 businesses that remained active and had comparable records in the latest period. The original study population had contained 500. The remaining eighty included businesses that had closed, been acquired, changed reporting or lacked an available source. The calculation for the 420 could be correct while the proposed headline about the original sector group remained misleading. I asked the research team to make the population change explicit before publication.

The editorial team wanted to retain the headline with a technical note. I thought that would leave most readers with a conclusion broader than the evidence. We withdrew the draft and rebuilt the population record, documenting the reason for each exclusion where it could be established. We did not assign zero revenue to every missing business or assume that all closures reflected the same economic event. Those shortcuts would have produced a different but equally unsupported measure.

The review cost ₹13 lakh and postponed the study by three weeks. The final publication described the 14% figure as a comparison of continuing businesses with comparable evidence, alongside a separate account of the original 500 and the eighty outside that comparison. It did not replace the headline with an invented whole-population growth rate. Clients could see the limitation and decide whether that narrower comparison was relevant to their question. One planned promotional claim about sector-wide resilience was removed.

The commercial cost was a less striking launch and additional explanation in client discussions. The benefit was not a new estimate of performance but a more faithful account of the population studied. I learnt to keep membership changes visible from the beginning of a longitudinal dataset. A comparison should not quietly redefine its subject simply because continuing businesses are the easiest ones to measure in the latest period.