Count the surprised claimants, not the callers

Most insurance marketing departments, including mine until 2024, judge an advertisement by what happens in the hours after it runs: calls, applications, cost per application and, in a careful department, cost per policy issued. I now think those measures, used alone, train an insurer to oversell. My position is that the most important measure of insurance advertising is taken later, at the claim, and that it should be a simple one: the share of claimants who find that we paid less than they expected. A board should see that figure as often as it sees acquisition cost.

I am not arguing that response measures are useless. A direct insurer that ignored them would waste money within a quarter. I am arguing that they are incomplete in one particular direction. Every incentive they create points towards a more attractive promise, and nothing in them records the cost of a promise the policy does not keep.

What a response rate cannot see

I learnt direct response in an Osaka agency in the late 1990s, where we counted telephone calls in fifteen-minute windows after each television spot. The lessons were consistent. A lower monthly premium in larger figures raised calls. A phrase suggesting that almost anyone could join raised calls among older viewers. Conditions shown along the bottom of the screen for a few seconds made almost no difference to calls, which is precisely why they were shown that way. None of us thought we were deceiving anyone. The conditions were present, legally correct and approved.

The same pattern appeared at Tomoshibi Life twenty years later, with better tools. In 2021 our relaxed-underwriting medical plan for applicants aged 50 to 85 was advertised with the line that most people could join after three health questions. The plan halved hospital benefits in the first policy year and cost about 1.7 times our standard plan at the same age. Response was excellent. Complaints from families who received half of what they had expected were rising, and our analysis later suggested that many applicants in their fifties and sixties would have qualified for the cheaper standard plan. Our response dashboard showed none of this, and could not have. The harm arrived months after the advertisement had been declared a success and its budget renewed.

That is the structural problem. When a household appliance is oversold, returns and reviews correct the seller within weeks. In insurance the moment of truth may come years after purchase, and a customer who discovers the gap rarely knows which advertisement formed the belief. The feedback that disciplines advertising in most consumer markets barely exists in ours. An insurer has to build it deliberately, or it will keep refining the promise.

Asking claimants one question

Since April 2024, every claimant whose medical or cancer claim we pay receives a short follow-up after the money arrives, by telephone or post as they prefer. It begins with one question: was the amount more than, about the same as, or less than you expected? Those who answer less are asked what they had expected and where they think the expectation came from. We record the channel through which each policy was bought and the advertising running in that channel at the time.

In the first six months, 14% of responding claimants said we had paid less than expected. The share was 21% for policies bought after a television advertisement, 9% for those bought through comparison tables that set out benefits line by line, and 11% for policies bought online. The common reasons were the first-year reduction on the relaxed plan, confusion between a hospital stay and outpatient treatment, and a belief that a cancer diagnosis benefit would be paid again on recurrence, when our policy allows a second payment only after two years. Each reason could be traced to a phrase, or an omission, in material we had approved.

The measure has plain weaknesses. Most policies have no claim in a given year, so the answers come from a minority of customers and lag the advertising by months or years. Disappointed claimants are more likely to reply. Memory is unreliable, and a customer may credit our advertising with a belief that came from a neighbour or a magazine. We offset some of this with an explain-back sample at application, in which new customers describe in their own words what their policy pays, and we compare the two results. I would still rather have a noisy measure aimed at the right question than a precise one aimed at the wrong one.

The changes it has produced are specific. Television advertising now states the first-year reduction aloud, not only on screen. Our medical plan pages explain the difference between hospital and outpatient cover with a worked example placed above the premium table. Cancer advertising states the two-year interval. By the first half of 2026 the share of claimants reporting a smaller payment than expected had fallen to 9%, and to 13% for policies bought after television. It has cost response. Our television cost per policy has risen by about a quarter since 2021, and I have had to defend that rise in every budget round.

Putting the figure in front of the board

A measure that lives inside the marketing department will lose every argument with acquisition cost, because acquisition cost appears in the budget and claimant surprise does not. Since the 2025 financial year our board has received both figures on the same page of my quarterly report, by channel, alongside the advertising that was running when the affected policies were sold. Our audit and supervisory board members receive the detail twice a year.

The effect on decisions has been larger than I expected. When our direct sales team proposed reviving the line that most people could join, the board paper showed the claimant figure for the period when that line had last run. The proposal was withdrawn before the meeting. Response data alone, which favoured the line, would have made that conversation much harder.

Some colleagues say this is a compliance task rather than a marketing one. Our compliance team reviews every advertisement against Japan's rules on misleading representations and the industry association's display guidelines, and does so well. But an advertisement can satisfy every rule and still leave a reasonable viewer with the wrong belief. Only the person who decides what is said, at what size and for how long can correct that, and in most insurers that person is the chief marketing officer.

I would make one request of any insurer's board. Ask the marketing function to show, beside its cost per policy, how many claimants last year were surprised by what they received and which advertising they had bought after. If nobody can answer, the insurer is judging its most consequential public statements by the reaction of people who have not yet needed them.