A table of seventy-two completed ratings looks like a useful body of evidence. It is more substantial than an anecdote, and each row can be a genuine record. Yet the number of rows does not tell us which explanations the study can distinguish. In our fictional sensory demonstration, twelve panellists each assessed six products in exactly the same sequence. The arrangement tied each product to its position. An apparent product difference could not be separated from that position simply by presenting the table more confidently.
This was a product decision as well as a research question. The team had collected the observations and wanted a comparison ready for a demonstration. Asking to postpone the ranking meant additional expense, an altered timetable and a more difficult conversation about why completed work was insufficient for the intended explanation. I want to preserve those costs in a leadership story. It is easy to endorse careful study design in principle; the more revealing judgement is what happens when the design question arrives after the team believes collection has finished.
## The sequence hidden behind a finished table
The source records were not false because the study arrangement was limited. Each person had encountered and rated the specified products. The problem lay in the proposed inference. Product one was always first, product two always second and so on. Whatever effects might be associated with position were entangled with product in this exercise. We did not have to claim that fatigue or carry-over had caused a particular score to recognise the difficulty. The design had left the explanations tied together.
A common response is to ask whether more participants would make the ranking dependable. More observations can be useful for many purposes, but repeating the same arrangement does not create a comparison that the arrangement lacks. If every new panellist follows the same fixed sequence, product and position remain linked. The data leader should make that limitation visible before discussing the apparent strength of a larger total. Otherwise a request for better evidence can become a request for more of the same unresolved evidence.
Another tempting response is to estimate an order correction from the table. That proposal needs its own justification. The records do not suddenly contain independent variation that the study failed to provide. A model may introduce assumptions, and there may be situations in which specialists can defend an analytical approach. What concerned me was a product presenting an adjusted ranking as though it followed directly from the observations. The user needed to know which part was observed and which part depended on additional reasoning.
Peter Qezvomi asked us to reopen the study question. The commercial team understandably wanted the ranking because it was the visible outcome promised by the demonstration. I asked them to distinguish the customer's practical interest in comparing products from our particular claim to have made a defensible comparison. The first remained legitimate. It did not settle the second. That distinction made the delay easier to explain without dismissing either the customer need or the work colleagues had already performed.
## Designing the next observation
We supported a revised study at ₹24 lakh and moved the demonstration by seven weeks. Its schedule placed each of the six products in each of the six positions twice across twelve panellists. The total remained seventy-two ratings. The improvement concerned the arrangement of those observations, not an increase in their headline count. Recording that distinction helped the approval discussion focus on what the additional expenditure was intended to change. It also prevented the new study from being described merely as a larger confirmation exercise.
Position balance was the specified improvement. It was not a claim that all possible influences had been removed. The relationship between preceding and following samples, the conditions of a session, participant differences and the research question still required appropriate attention. I wanted the product team to understand why a precise statement of the improvement was stronger than a sweeping statement that the design was now unbiased. The latter would have repeated the original habit of asking one useful feature to stand for the whole evidential judgement.
The revised ratings were collected and retained separately from the original exercise. Combining them into a larger undifferentiated total would have concealed the reason for repeating the work. A reader needed to know which observations came from the fixed sequence and which came from the revised schedule. The scientific team could then assess the appropriate treatment for its purpose. Our software responsibility was to preserve the distinction and make it accessible where an analysis was selected or explained.
I also kept the intended population bounded. Twelve panellists in an invented demonstration did not establish a general market-preference result. Even a well-arranged exercise answers a question about specified observations under specified conditions. The software can help a researcher describe those conditions, but it cannot make the receiving commercial claim broader simply because a chart is easy to read. The discipline is to connect the observation, analysis and proposed use each time rather than granting every later use the authority of the first review.
## What the software company owes the decision
The product requirements changed before another ranking was published. Sequence had to remain part of the record, and the planned arrangement had to be available for review before collection. These requirements concerned the point at which a future weakness could be recognised. They did not amount to software certifying every study. A customer might still choose an unsuitable arrangement or use an analysis beyond its scope. Our task was to avoid a product design that hid the relevant choice or made a complete table appear sufficient by itself.
There was a resource question behind that task. Making study context visible requires work in design, data representation, documentation and support. The benefit cannot always be expressed as fewer clicks. A meaningful interface may ask a user to resolve a question that an earlier interface silently ignored. I want that friction discussed honestly. It can be justified when it prevents the product from offering a conclusion before the experiment has earned it, but the team should still make the question understandable and proportionate to the user's work.
At board level, I would explain the decision through the promise the enterprise was making. Management had a demonstration timetable, a customer question and a set of observations. It chose further funded work because the existing arrangement did not separate the intended explanation. The board could examine that reasoning, the cost and the revised scope without taking over the research design. It should also see whether the company learned something about its product requirements, rather than treating the additional study as an isolated inconvenience.
The sequence belongs in the evidence because it helps explain what an observation means. Losing it can leave an apparently complete dataset unable to support the question readers expect it to answer. My contribution as a CDO is to connect that methodological distinction with a concrete product and investment decision. In this case, the outcome was a delayed demonstration, a separately recorded revised study and an explicit cost. Those are useful results to discuss without inventing a market victory or pretending that the next ranking needs no further scientific judgement.