Skip to website
Farhan QelvaniChief Data Officer · Essays & perspective
← All essays

Reader research and commercial inference5 min read

The readers who answered are not everyone we hoped to understand

A subscription survey supplied useful opinions from 450 respondents. It could not supply the missing preferences of another 450 invitees or a sales forecast for the whole shop.

The most persuasive number in our subscription research was sixty-two per cent. Of 450 adult customers who completed the survey, 279 preferred the proposed monthly curated-book package. Colleagues could picture the service, the audience and the first launch. The number seemed to give those ideas a scale. The difficulty was that the study had invited 900 people from a defined recent-purchaser list, and the commercial proposal wanted to apply the result to a much wider customer population.

The 279 preferences were real responses within our fictional study. They were not a mistake to be corrected away. They also did not answer for the 450 invitees who had not responded, still less for shoppers outside the invitation frame. We needed to preserve what the research had learned without allowing the clearest percentage to acquire a different denominator as it moved into the launch paper. That is often where a useful study becomes an unsupported forecast.

## Follow the population through the argument

The first population was the one the business wanted to understand: potential subscribers among a broad set of customers. The second was the invitation frame: recent adult purchasers whose records made them eligible for this research. The third was the group that completed the survey. Those populations overlapped, but they were not interchangeable. Each step could change the mix of people being represented, and the final recommendation needed to account for those changes rather than describe them as an administrative detail.

Nonresponse is particularly easy to interpret in a way that suits the preferred conclusion. An enthusiastic team may imagine silent invitees as interested people who were simply busy. A sceptical team may treat silence as rejection. Neither interpretation follows from the absence of an answer alone. Reading habits, time, access, interest in surveys and many other circumstances could affect participation. We did not have evidence sufficient to assign a subscription preference to every person who had remained silent.

The commercial team made a stronger and more useful point: volunteers might resemble the early audience for a curated service. That was plausible. People willing to discuss books may also be more willing to try a reading subscription. But a plausible description of a possible early audience does not establish how large that audience is or how many people would pay under an actual offer. The survey could help define a next inquiry into that hypothesis without being treated as the completed answer.

We also needed to distinguish preference in a research setting from purchase. The question presented a proposed package; it did not create a paid commitment. People may like an idea and still make a different decision when price, timing and other spending compete for attention. This is not a reason to distrust every stated preference. It is a reason to name it accurately and connect it to the decision it can support, rather than silently promote it into an order forecast.

## Use adjustment without pretending it creates missing knowledge

We could compare respondents and invitees on available purchase-frequency information. That helped us see one way the groups differed. It did not tell us whether they were equivalent in enthusiasm for curated selections, appetite for a monthly obligation or other relevant characteristics we had not measured. An adjustment can improve a comparison under stated assumptions, but it cannot make those assumptions true merely by producing a polished weighted percentage.

I was therefore unwilling to describe a purchase-frequency adjustment as a complete remedy for nonresponse. The issue was not that weighting is always inappropriate. It was that our proposed commercial inference required more confidence than this limited adjustment could justify. The report retained the descriptive respondent finding and explained what remained unknown. That allowed colleagues to use the reasons people gave for liking the package while avoiding a population adoption estimate that looked more settled than it was.

The ₹21 lakh research programme had nevertheless produced value. Interviews and responses described desired kinds of selection, concerns about receiving unwanted books and reasons a regular service might appeal. Those findings could shape a narrower follow-up. I did not want the absence of a representative forecast to make the entire study appear useless. A study can be informative about motives and questions while remaining unsuitable for estimating a market size. Recognising that distinction helps an organisation learn without inventing certainty to justify the expense.

Management held the ₹84 lakh broad launch proposal while considering a separately approved next step. That was a deferral of a proposed commitment, not a realised saving. The offer might later be revised and launched on a better basis, or the business might choose a different route. The ₹21 lakh already spent remained in the account. Describing a held proposal as money saved would have created a second unsupported inference while trying to correct the first one.

## Commission the decision before commissioning the questionnaire

My own error was allowing the intended use of the study to remain too loose at the beginning. Colleagues wanted to learn whether a subscription could be attractive, but the eventual paper wanted an adoption forecast for a substantial launch. Those are related questions with different evidential requirements. If we had specified the decision and its required confidence earlier, we could have designed the invitation, recruitment and follow-up with that purpose in view, or agreed from the outset that this was exploratory work.

This does not mean every research programme must become a large, slow exercise. A small inquiry can be exactly right for deciding which questions deserve more attention. A commercial team may also choose to run a bounded pilot under uncertainty. The essential discipline is to state what the evidence supports and what risk management is choosing to take beyond it. The decision can then be assessed honestly rather than presented as something the respondents have already authorised on behalf of everyone else.

For directors, I would make the inference chain visible. Who was invited, who answered, what were they asked to choose and which business population appears in the forecast? Where does the argument move from observation to assumption? These questions can be asked without turning the board into a survey-design committee. They help directors examine whether management has understood the uncertainty attached to a proposed commitment and whether the next step is proportionate to it.

I remain interested in the 279 people who preferred the package. Their reasons are part of the story, and dismissing them would be as unhelpful as extrapolating them without care. The lesson is to let each piece of evidence retain the population and action it actually describes. Respondents can inform a better service, nonrespondents can remain unknown, and management can still make a considered next decision. Research becomes more useful when it carries those boundaries clearly, instead of borrowing the absent customer's voice to make the launch paper sound complete.