The model indicated a 32% energy saving under specified assumptions. A later six-week site observation showed an 18% reduction against the selected earlier period. It was tempting to tell a simple story: the model had been optimistic, the site result was lower and the company should publish the more cautious figure. That account would still have been unsound. Loads, weather and operating hours differed between the periods, and the observation did not by itself establish what the intervention had caused. The problem was the basis of comparison, not merely the size of the percentage.
A draft portfolio presentation nevertheless used the modelled 32% as a broadly repeatable customer result. The commercial team saw a compelling illustration of the technology's value. Technical owners saw conditions that had disappeared from the headline. I withdrew the broad claim and authorised further measurement work. At the point recorded in the case, there was no new validated saving to substitute. The unfinished result was part of the account.
Identify what each percentage describes
A model can help examine a possible change under stated conditions. Its usefulness depends on the purpose, inputs, assumptions and evidence supporting that use. A site observation concerns what was measured during a particular period. The two can inform each other, but they do not become equivalent because both are expressed as percentages. Before asking whether the result is encouraging, I want to know the question each number is capable of answering.
The 32% figure belonged to the demonstration's operating assumptions. The 18% described a reduction against a selected earlier period under different site conditions. We had not established that the intervention alone accounted for that difference. A more careful account would examine the comparison basis, relevant changes in operation and measurement limitations. Qualified engineers and client owners retained the actual equipment decisions. The CEO's role was to ensure that commercial material did not outrun the evidence, not to replace that professional work with an executive interpretation.
The distinction matters because a precise-looking number can travel farther than the conditions attached to it. A demonstration becomes a slide, a slide becomes an account proposal and the proposal becomes a remembered expectation. Each person may believe the earlier stage established more than it did. By the time someone asks for the basis of the claim, a modelled possibility may be treated as a measured customer outcome. That is an organisational problem as well as a technical one.
I do not regard every modelled benefit as unsuitable for discussion. Prospective clients need to understand why a technology may be worth examining. The claim should state its status and intended use, with enough context to avoid implying an achieved result. Nor should a site observation be dismissed merely because it cannot establish causation alone. It may identify useful questions and inform the next stage. The discipline is to preserve what the evidence does and does not establish, rather than demand that every number be either conclusive or worthless.
Do not repair a comparison by choosing the smaller number
Some colleagues suggested replacing 32% with 18% in the presentation. That would have looked restrained and might have reduced the immediate disagreement. It would not have resolved the mismatched conditions. A lower number can be as unsupported as a higher one if it is assigned a meaning the evidence does not justify. I wanted the team to understand that the review was not a negotiation over how much optimism the company could safely retain.
We removed the broad portfolio claim rather than revise it through that shortcut. The decision made the commercial account less immediate. Sales colleagues lost a memorable benefit figure and had to explain a more conditional proposition. I recognised that cost. An evidence standard has practical consequences when competitors may present simpler claims or a buyer wants a quick comparison. Those pressures do not supply the missing basis, but management should acknowledge them instead of pretending the decision creates no commercial difficulty.
The account also needed to avoid implying that the service had necessarily failed. We had a question about the strength and scope of the claim, not a complete technical conclusion about every use of the model. The response was to investigate properly and describe the current state accurately. A profile should not turn that into either a triumphant energy-saving story or a general confession that the technology did not work. Both would say more than the recorded evidence allowed.
The same care applies to responsibility. Our firm provides engineering technology and analysis services. It does not acquire a client's equipment-approval authority because its model is persuasive. The technical and client owners need to examine the relevant evidence and operating context. Commercial material should help the buyer understand the service and its limits, not appear to offer an alternative route around those decisions. A CEO can insist on that boundary without pretending to settle each engineering question personally.
Fund the evidence before repeating the result
We authorised ₹46 lakh for a measurement programme and a revised baseline protocol. The work would distinguish the model assumptions, observed conditions, measurement limits and the conclusion the comparison was intended to support. The money was an approved expense for further evidence, not a quantified benefit or a completed validation outcome. We did not claim a new saving simply because the programme had been funded.
I wanted the evidence plan to have a clear decision purpose. More measurement is not automatically better if the organisation has not said what uncertainty it needs to resolve. Technical owners should define a defensible approach for the intended use, while management understands what the result could change in the offer or investment. The board can then assess why the additional work is worthwhile and what remains conditional while it is carried out.
For directors, the practical question is whether a benefit claim has changed status as it has moved through the business. Was it a modelled possibility, a bounded demonstration, an observed association or a conclusion supported for a specific use? What conditions were retained, and which were lost? Directors do not need to reconstruct every model. They should be able to see whether the claim presented to customers and investors has a basis appropriate to the meaning it carries.
The experience changed the timing of my questions. I now want the measurement account considered alongside the first benefit projection, before a memorable percentage becomes central to the commercial story. That does not remove uncertainty or eliminate the need for judgement. It makes the uncertainty visible at the point where the company chooses what to promise. A smaller headline is not a substitute for that work. The responsible result may be no general headline yet, with a clear explanation of the evidence still being developed.