Why Simulated Shoppers Get the Average Right and the Aisle Wrong

Ask a simulated shopper why they chose the cheaper bottle and the model gives you a coherent, plausible answer. Now ask why they chose it at 7:40 on a Tuesday, eight minutes before a train, with a child holding the basket. You get the same answer.

That flatness is the tell. The failure is not bias in the moral sense. It is an absence.

What a persona actually specifies

Strip a synthetic respondent down and you find a prompt. The prompt says: 34, urban, household income below $60,000, two children, buys personal care monthly. Kantar’s published tests did exactly this — demographic markers copied from a real 5,000-person survey so the synthetic panel matched the human one on paper.

Notice what the list contains. Person attributes: age, income, education. Object attributes: the product, its price, its category. Both are stable. Both are readable from a database.

Notice what it does not contain. Nobody is in the aisle. Nothing is on sale next to it. Nobody is waiting in the car.

Russell Belk made this argument in 1975, in a paper marketers cite and then ignore: behaviour is best explained by the person, the object, and the situation, and models that omit the third leave substantial variance unaccounted for. The situation is not background noise. It is a variable.

Why the mean survives and the variance dies

Here is the consequence, and it is precise enough to predict which tests will pass.

When a model has a persona but no situation, it cannot generate the situational spread that real people produce. So it falls back on the most probable answer for that demographic — the modal response. Averaged across a thousand simulated respondents, that modal answer lands close to the human mean. The distribution, though, collapses.

Kantar’s fifty-repeat experiment is the clean demonstration. Repeat one persona prompt fifty times and GPT-4’s answers turned stereotypical, drifting markedly more positive than the fifty human responses they were matched against. And the failure was not uniform: on price, a property of the object and therefore stated in the prompt, synthetic answers tracked humans reasonably well. On emotional questions — where the answer depends on circumstances the prompt never described — the gap widened sharply.

The same signature shows up in subgroup cuts. Among respondents earning under $125,000, the synthetic panel understated how many considered price extremely important. STRAT7’s 2025 holdout test found the same shape: headline agreement within two or three points, willingness-to-pay figures inflated by 16 percent, and synthetic respondents contradicting their own price ordering 68 percent of the time.

What this tells you to do

The diagnosis gives you the fix, and the fix is not a better model.

If the missing variable is situation, then stop asking synthetic respondents questions whose answers depend on one. Trade-offs stated in the prompt — price ladders, pack architecture, feature bundles — are exactly where the method holds, because the object carries the context. Emotional resonance, in-store behaviour, and anything that changes month to month are exactly where it fails, because the context lives outside the prompt.

That is also why Bain’s framing works and the replacement framing does not. Synthetic customers layered on top of real research delivered comparable insight in half the time at a third of the cost — with humans describing the situations the model cannot imagine.

Write the situation into the prompt and you buy a little fidelity. Measure it and admit it and you get the real thing.