Chief Technology Officer / India
The missing conditions changed what we could responsibly release
- Engineering employees
- 58
- Four heads; nine contractors counted separately
- Annual engineering operating expense
- ₹14.6 crore
- Separate from ₹4.8 crore approved change investment
- Software and engineering leadership
- 30 years
- Implementation, delivery, architecture and insurer CTO responsibility
Decisions & consequences
012019: the thirty-minute run did not describe the working day
I had accepted a release recommendation supported by a thirty-minute performance run. The response behaviour met the agreed short-run criteria, and the delivery team wanted to preserve the launch date. Before release, a longer exercise sustained a synthetic workload at forty requests per second for eight hours. Backend process memory rose from 1.4 to 4.9 gigabytes. The load generator’s resource use remained stable, and investigation traced retained application objects in a request-handling path. The result could not be explained away as measurement growth on the test machine.
The infrastructure lead proposed allocating more memory so the programme could proceed. I rejected that as the complete response because it did not establish how the retained objects would behave over a longer period. We assigned ₹54 lakh to a ten-week correction and review programme. The launch moved by two weeks within that programme’s final stage. Meera Qelzavri’s quality account retained the original result beside the corrected version, with the same workload pattern and environment assumptions recorded for comparison. We did not turn the exercise into a claim about real customer incidents; it was a pre-release finding under constructed conditions.
After the change, three eight-hour repetitions held backend memory within a 1.4 to 1.8 gigabyte band and met the selected response criteria. Those observations supported the revised recommendation for the defined workload. They did not establish that every duration, data mix or future feature would behave the same way. The release account therefore kept the duration and workload limits visible, with operating monitoring assigned separately. We claimed neither an avoided-loss amount nor an infrastructure saving from the correction.
My initial decision had given a short demonstration more weight than it deserved. Changing it required accepting delay and expense while explaining why extra capacity alone would not answer the question. I learned to ask what period of the service’s life a test describes, and to be precise about the remaining period rather than let a successful opening half-hour stand for an entire operating day.
2023: a new status value broke an older consumer contract
A proposed service change introduced an additional status value that our current application understood. The producer-side tests passed, but a review of forty-eight synthetic client-contract scenarios found eleven in which an older consumer rejected or mishandled the unfamiliar value. The other thirty-seven passed the defined checks. This was not a finding that eleven real customers had suffered a failed policy transaction. It showed that the proposed interface behaviour did not fit every consumer contract included in the release proposition.
The delivery lead suggested mapping the new state onto an existing successful status for older clients. I rejected that shortcut because compatibility could not be achieved by changing the business meaning of the response. We allocated ₹68 lakh and twelve weeks to an explicit version and transition arrangement. Operating owners defined how the states should be understood, while engineering implemented the corresponding contract behaviour. The quality team repeated the consumer-side fixtures rather than rely on the current producer’s own successful demonstration. Client migration and release authority remained separate from the fact that a new interface version existed.
Nine of the eleven failed routes passed after correction. Forty-six of the original forty-eight scenarios therefore passed, with two still unresolved. We withheld those two routes from the release scope and preserved their existing supported behaviour. The result did not justify a statement that all clients were compatible, nor did it allow us to infer how untested client versions would respond. The programme cost was recorded without inventing revenue or a quantified loss avoided by the change.
The engineering decision was to treat compatibility as a relationship with consumers, not a property asserted solely by the service producing the response. A new value can be perfectly valid in one model and still create a commitment that another application cannot interpret. I wanted the recommendation to show that difference, including the cases we had not yet earned permission to move.
2025: new encryption succeeded while older specimens remained unreadable
An encryption-key rotation proposal demonstrated successful new writes and reads. We then examined seventy-two synthetic encrypted specimens spanning four key generations that the authorised historical-access route still needed. Sixty were readable through the proposed process; twelve were not. Investigation identified missing or incorrectly resolved key-version references in the retrieval path. The exercise did not establish that cryptographic material was compromised or that real policy records had been lost. It exposed a gap between the new-write demonstration and the promised ability to read authorised history.
The programme team wanted to complete the rotation and resolve the older cases afterwards. I required the historical-access limitation to be part of the release decision. Security and information owners retained responsibility for protected key handling, applicable retention and authorised disposal. Engineering received ₹82 lakh for a fourteen-week programme to repair version references and test the intended retrieval route. We did not copy raw keys into application logs or propose indefinite retention as a universal solution. The design account had to preserve both restricted access and the evidence needed for legitimate historical use.
Eight of the twelve failed specimens became readable under the corrected route. Sixty-eight of the seventy-two therefore passed, while four remained unresolved and outside the affected rollout. Protected versions still required by the authorised process remained under the existing control arrangements pending the responsible owners’ decision. The report did not call the entire historical set ready, and it did not convert synthetic test results into a claim about production recovery. No standard algorithm, rotation frequency or legal retention period was prescribed by this fictional programme.
The lesson was about the scope of evidence. A successful new encryption cycle can demonstrate the current path while leaving older dependencies unexamined. My recommendation had to include those dependencies and the professional authority over them, even when doing so made the release account less tidy. Engineering leadership means showing what still needs to remain usable, not allowing a new technical milestone to erase the earlier service obligation.