Skip to website
Zoya QelzuriChief Marketing Officer

The journal · Learning evidence

A better practice score still leaves a workplace question

The mistake in our campaign was not the arithmetic. It was the journey from a bounded exercise result to a promise about work that we had not observed.

I once approved a headline about 40% fewer mistakes because the calculation behind it was straightforward. Eighty learners had completed a defined customer-service scenario, and their average errors fell from 12 to 7.2 across the reviewed exercise. The difference was 4.8, and 4.8 divided by 12 was 40%. The figures were consistent. The commercial statement was still too broad because it allowed the reader to treat a practice result as evidence of fewer mistakes in the workplace.

That distinction matters in a learning-software business. The product provides an environment for practice, but the customer's purpose usually extends beyond that environment. A manager wants someone to handle a difficult conversation differently, make a better decision or apply a method under real conditions. Marketing naturally wants to connect the product with that purpose. The question is whether the evidence can support the connection we ask the buyer to make.

The observation and the promise were about different settings

Our review had no comparison group and no workplace follow-up. It could describe what happened to the observed scores inside the defined scenario. It could not separate the influence of repeated exposure, familiarity with the exercise, facilitation or other factors. Nor could it show how the same people behaved when the circumstances and pressures of their jobs returned. Those were not minor qualifications to an otherwise complete workplace claim. They were parts of the question we had not investigated.

Meera Vaqsumi, who heads learning evaluation, asked me to identify the outcome named by the headline. My first answer referred back to the calculation. That was revealing: I was defending the precision of the percentage instead of the meaning of the sentence. A percentage can be exact while the noun beside it remains too broad. The task was to decide whether we meant errors in this exercise or mistakes in the work the customer cared about.

I withdrew the broad statement and revised 14 assets at ₹18 lakh cost. We kept the practice observation with its population and setting. We did not claim that the learners had failed to learn, that the product had no value or that a comparison group would necessarily have produced a different conclusion. The evidence did not justify those negative statements either. It supported a useful but narrower account, and that was the account we could publish.

A limit can be useful without becoming the whole sales conversation

Colleagues worried that a qualification would make every presentation sound defensive. I understood that concern. A buyer needs a comprehensible proposition, not an exhausting list of everything the supplier has never measured. But the answer cannot be to let a short sentence carry a stronger meaning and hope that the appendix corrects it. The main statement should describe the actual result. Supporting material can then explain the setting and what another customer would need to evaluate.

This approach can make the implementation conversation more concrete. If the observed result concerns practice, the customer can ask how it will create opportunities to apply the method afterwards. Who will observe the relevant work? What will managers do when the new behaviour conflicts with an established process? What period would provide a meaningful opportunity to use the learning? Those questions do not belong entirely to the software supplier, but the supplier can avoid hiding them behind an outcome it has not demonstrated.

I also resist turning this correction into a new universal marketing claim. We did not establish that more qualified wording increases conversion or that every buyer prefers a longer explanation. The known outcome was 14 revised assets and ₹18 lakh correction cost. Sales colleagues had to replace material already in use. The decision was justified by the need to describe the evidence accurately, not by an invented later commercial reward that would make the trade-off disappear.

Responsibility remains with the person who publishes the account

It would have been convenient to call the problem a missing technical caveat. That would place the burden on the evaluator and leave marketing as a neutral editor of someone else's evidence. I do not think that is sufficient. I had approved the headline, and the headline's likely meaning was part of my responsibility. A specialist can explain the limits of a review, but the commercial team chooses how those limits survive selection, design and repetition.

The approval process now asks for two short statements before the headline is settled. One describes what was observed, including the population and setting. The other describes what we want the reader to conclude. If they refer to different outcomes, we examine the bridge between them instead of assuming that a compelling presentation supplies it. This is a practical editorial test, not a substitute for research design or a promise that every claim will become easy to resolve.

The same exercise helps when a result is genuinely strong. Clear scope does not diminish valid evidence. It lets a prospective customer understand why the result might be relevant and which differences in its own setting matter. A claim that tries to cover every possible application can become less useful precisely because it removes the conditions a serious buyer needs to inspect. The point is to preserve meaning, not to make confidence sound hesitant.

A director does not need to review each scenario score to ask how the enterprise approves its major outcome claims. The board can examine whether management distinguishes activity, learning and workplace application, and whether the commercial promise has grown as it passes through channels. It can ask who owns the underlying evidence and how unresolved conditions affect contracts, customer expectations and investment. Those questions concern the business's obligations rather than the wording of every campaign.

I would also want the account to show corrections without hiding the earlier judgement. In this case, I approved the broader promise and later changed it. That history matters because a description of a good process alone would miss the decision that made the process necessary. The case gives a board or colleague something more useful than a claim that I have always been careful: it shows what I reconsidered, what the correction cost and which question remained unanswered.

A better practice score can be an encouraging starting point. It may justify further investigation, a more focused customer conversation or a revised programme. It does not need to be dismissed merely because it cannot yet support every hoped-for outcome. The discipline is to let the result do the work it can do, then name the next question honestly. For our campaign, the next question concerned work beyond the simulation. Keeping that question open made the story less complete, but more faithful to the evidence we actually had.