The Economics Behind Why a Cheaper Interview Makes Research Less True
Every research budget contains an unexamined exchange rate. On one side sits the cost of asking a person a question. On the other sits the value of the answer you did not expect. Synthetic customers make the first number collapse and leave the second number exactly where it was, and almost nobody notices that the two are not on the same curve.
Here is the arithmetic that gets missed.
The marginal interview is not the average interview
A 604-person human panel produced more than fifty distinct labels for a single open-ended question. One hundred and twenty simulated respondents, given the same question, produced about twelve. The headline findings matched. The information did not.
Sixty independent interviews buy you roughly sixty new chances to hear something you did not anticipate. The sixtieth interview is worth almost as much as the first, because the thing you are chasing is a rare answer, and rare answers arrive on their own schedule. That is why a thirty-person qualitative study can outperform a thousand-person one: it is not sampling error the researcher is hunting, it is surprise.
Synthetic respondents do not work this way. Once the model has settled on the dominant answer, respondent number twenty-five and respondent number ninety return the same sentence in different fonts. Among product managers, 44% of real humans converged on the dominant term. Among simulated ones, 93% did. The panel was not more decisive. It was more compressed.
Why volume still reads as rigor
The incentive is doing the work here, not any individual’s judgment. Procurement compares quotes per interview, and the synthetic quote wins by three orders of magnitude. A deck reporting n=400 reads as more thorough than a deck reporting n=40, and the auditorium does not ask whether the 400 were independent.
This is where the cost advantage quietly converts into a blind spot. The 2025 GRIT Insights Practice Report found 67% of suppliers already embed generative AI straight into client deliverables, while 40% of researchers name data quality as their top barrier. Both numbers can be true at once, and together they describe a machine that is scaling the report faster than it is scaling the evidence.
The part that costs nothing to fix
The cheap version is not useless. It is mispriced. Interview-grounded twins tracked real people’s survey answers at 85% of the reliability humans show with themselves two weeks later, and replicated four of five classic experiments with effect sizes correlating at 0.98. Grounded personas are genuine instruments. The problem is that they were never sold as instruments. They were sold as volume.
So spend the savings on the thing the simulation cannot give you. Keep the eleven interviews that produce twelve labels. Then ask a real panel of forty whether the rare answer travels.
The takeaway
Research value lives in the tail, and the tail is a fixed cost. Simulation moves every other cost toward zero while leaving that one intact — which makes it dangerous precisely in the hands of teams optimizing for price per response. Interview the simulation to find the questions. Interview a human to find the answer.
Summary
- The sixtieth human interview still carries near-full information value; the sixtieth synthetic one usually does not.
- Volume reads as rigor to a room that never audits independence, which is why compression goes undetected.
- Interview-grounded simulations are real instruments with strong replication; the failure is selling them as sample size rather than as hypothesis tools.
- Spend the saved budget on validating the rare answer, not on adding synthetic respondents.
