Twenty-seven completions do not erase three unfinished tasks

The most convenient summary of a product improvement is often the least complete one. Thirty venue staff attempted representative booking tasks using a keyboard. Nine completed them and twenty-one were blocked. After a funded programme of interaction changes, twenty-seven completed the same tasks and three still needed assistance. The second result gave us a reason to recognise progress. It also gave us three reasons to keep working. I did not want the improvement to become a way of making the unfinished experience disappear.

The exercise concerned a defined activity in a scheduling product. It was not a formal accessibility certification, and it did not represent every person, disability, assistive technology or device. That qualification is central to understanding the work. A team can act seriously on a limited study without claiming that it has answered every question. The problem begins when the limit is removed from the story and a useful result is presented as a universal property of the product.

## Watch the task before counting its outcome

A completion count tells us whether a participant reached the agreed endpoint. It does not explain how they got there or why another participant stopped. Researchers observed focus order, the identification of controls and the sequence of actions required to create or amend a booking. Some barriers were easy to reproduce once somebody had seen them. Others depended on what a participant expected the next action to be. The observations gave design and engineering something more useful than a general request to make the interface easier.

I wanted us to preserve the task account alongside the count. If a person reached the endpoint only after repeated uncertainty, that experience still deserved attention. If somebody stopped because the product offered no clear next step, a satisfaction question could not repair the missing interaction. Research should help the team understand the relationship between behaviour and outcome. It should not simply produce a number that appears to settle the discussion about whether the product is good enough.

The participants were venue staff doing work relevant to their jobs. That made the tasks meaningful, but it did not make the group a complete representation of the market. We needed to describe the conditions plainly: who took part, what they attempted and what counted as completion. Those details make a result more useful to a future team deciding whether a new change should be tested in the same way or requires a different examination.

We allocated ₹44 lakh to the interaction programme. That decision reflected both the observed barriers and the importance of booking work to customers. Product and engineering agreed the sequence of changes. I did not want a long list of cosmetic adjustments to stand in for addressing the moments that prevented progress. The aim was a usable path through the task, including a comprehensible focus sequence and controls that staff could identify without relying on a pointer.

## Treat repeat testing as a particular kind of evidence

The same thirty participants returned for the repeat exercise. Twenty-seven completed and three required help. Reusing participants helped us inspect whether the observed problems had been addressed, but it introduced familiarity. People had already encountered the task and the product. Their later performance could not automatically tell us how a first-time user would behave. I included that condition when reporting the improvement, rather than let the result imply that we had tested a wholly new group.

That does not make repeat testing worthless. It answers a useful question about a change under known conditions. The task is to explain which question it answers. A new-participant study, a wider range of assistive technology or a different booking situation could add evidence, but those would be further pieces of work. They should be planned for a reason, not mentioned vaguely as if their results were already included in the current account.

The three participants who still needed assistance also required a more precise description than a failure label. We needed to understand where they stopped, what help was given and whether the difficulty reflected a remaining interaction problem or another condition requiring investigation. A count should lead us back to the experience. If we use it only to classify people into successful and unsuccessful groups, we risk treating the users as the variable to explain away rather than examining the product they encountered.

I resist the temptation to combine all encouraging observations into a single access claim. A smoother focus order, more recognisable controls and higher completion in one exercise are related findings, but they do not prove everything about the product. Nor should the existence of unresolved work prevent us from acknowledging a demonstrated improvement. The useful account holds both: what changed under the tested conditions and what remains uncertain or difficult.

## Give the unresolved work an owner

An unresolved finding can disappear without anyone deciding to ignore it. The project closes, the delivery team moves to another priority and the three cases become a note in a research report that few people reopen. I want the next action named while the result is still being discussed. Somebody should be responsible for understanding the remaining barriers, agreeing what to change and returning with an account of the next examination.

This is a leadership responsibility as much as a research practice. A team will notice which parts of a result receive attention. If senior colleagues celebrate only the improved completion count, the remaining cases become awkward details that slow the meeting. If the discussion asks what those cases teach us and what happens next, the team has permission to bring incomplete evidence forward. That makes future decisions more informed, including decisions where the result is less flattering.

Commercial language needs the same discipline. A customer may hear a broad statement about accessibility as an assurance concerning their particular staff and obligations. Our study did not establish that assurance. We could describe the keyboard work, the changes and the observed outcomes, then explain what further assessment might be needed for the customer's situation. Precise language is not a retreat from ambition. It lets a buyer understand what the product evidence actually covers.

The enduring lesson for me is that improvement and completion are different claims. We made booking tasks more usable for the tested group and retained responsibility for the difficulties still observed. That is a stronger basis for product development than either declaring victory too early or dismissing progress because it was incomplete. The next useful question belongs to the people who still needed assistance: what did the product ask them to understand or do that our current design had not made possible?