When Optimizing for AI Becomes Manipulating the Answer

Search marketing spent twenty-five years arguing about a line nobody could quite see. Then it spent two years convincing itself the line had moved.

It hasn’t moved. The surface underneath it has.

That distinction matters more than it sounds, because almost every questionable decision in generative engine optimization right now gets justified with a version of the same sentence: SEO had gray areas too. True. But those gray areas sat in a different room, with different stakes and a much larger audience for the fallout. What follows is an attempt to say where the line actually sits, what the evidence says about who is crossing it, and why the honest operator is currently the one absorbing the cost.

The old argument, wearing new clothes

The white-hat versus black-hat debate was never really about tactics. It was about who bears the cost of your visibility. Publishing a genuinely useful page and formatting it for discovery shifts the cost onto you — the research, the editing, the fact-checking. Buying links shifts it onto the algorithm and, eventually, onto the reader who lands on something thinner than the ranking promised. Twenty years of enforcement, punctuated by the occasional deindexing bloodbath, produced a rough working consensus: optimize for relevance and quality and you are fine; deceive the system or the reader and you are not.

Google eventually wrote that consensus into policy. In March 2024 it shipped three new spam categories in a single core update — scaled content abuse, site reputation abuse and expired domain abuse — and reported afterward that low-quality, unoriginal content in its results had fallen by roughly 45%, ahead of the 40% it had projected. The update took 45 days to roll out, the longest on record at the time. Enforcement was swift and occasionally indiscriminate: thousands of sites lost visibility within a week, and not all of them deserved it.

That history is why the GEO conversation keeps collapsing into analogy. We have seen this before. It worked out. But the analogy smuggles in an assumption that no longer holds.

What actually changed: ten links became one sentence

In classic search you competed for a position in a list. The user scanned, compared, clicked two or three things, and formed a view. Your manipulation, if you ran one, bought you a slot — one voice among many, in a format that at least implied the existence of alternatives.

Generative engines collapsed that. The user asks and gets an answer. Not a list, not a set of options with sources underneath. A paragraph, in confident prose, that reads as settled.

The behavioral evidence is blunt. Pew Research Center tracked the browsing of 900 US adults across 68,879 searches in March 2025. Roughly 18% of those searches produced an AI summary. When a summary appeared, users clicked a traditional result on 8% of visits; without one, they clicked 15% of the time. Clicking a citation inside the summary happened on 1% of visits. And 26% of sessions ended on the search page itself, against 16% when no summary appeared.

Read those numbers as a market structure rather than a curiosity. The citation — the one affordance that lets a reader check you — is the least-used element on the page. The claim is the product. The source is decoration.

The citation layer is broken at the source

Now put that beside what the engines actually do with sources. The Tow Center for Digital Journalism tested eight AI search tools with 1,600 queries built from real article excerpts — 20 publishers, 10 articles each, 8 tools. Excerpts were chosen so that pasting them into Google returned the original within the first three results.

More than 60% of responses identified the wrong article, publisher or URL. Perplexity was wrong on 37% of queries; Grok 3 on 94%. ChatGPT misidentified 134 of 200 articles, hedged just 15 times, and never once declined to answer. Gemini and Grok 3 produced more fabricated or broken URLs than working ones — 154 of Grok 3’s 200 citations led to error pages.

So the reader is skipping verification on a surface where verification fails more often than it succeeds. That is the structural fact underneath the ethics question. Manipulation is not new. Manipulation into a channel that both sounds authoritative and resists checking is.

The line, stated plainly

Here is the test I keep returning to, and it is the one the industry keeps trying to talk its way around: does your optimization make you a better answer, or a more visible one at the reader’s expense?

Everything legitimate passes it, including things that feel vaguely cynical. Restructuring a page so a model can extract a clean, self-contained paragraph? That is craft. Adding a statistic you actually measured, with the year and the sample? That is evidence. Naming a real expert and quoting them accurately? That is exactly what the citation economy is supposed to reward. You are not gaming the engine; you are doing the work the engine was built to find.

Everything illegitimate fails it the same way: it transfers cost to someone who did not consent. The reader gets a claim dressed as consensus. The competitor loses an answer they earned. The engine’s credibility absorbs the damage and pays for it in trust it cannot rebuild on demand.

Three questions that sort most cases

The single test is easy to state and easy to rationalize around, so it helps to keep three sharper ones in reserve.

The disclosure test. Would you be comfortable if the engine printed your source list next to the answer — every page you planted, every domain you bought, every “independent” review you wrote? If not, the activity is manipulation, regardless of what it is called internally.

The universalization test. If every competitor ran the same play, would the answer still be useful to the reader? This one kills most gray-zone defenses, because most of them scale into noise. One brand seeding three editorial mentions is a nudge. Fifty brands seeding three each is a retrieval pool where the loudest claim wins and accuracy is uncorrelated with position.

The epistemic test. Does the activity change what is true, or only how clearly the truth is presented? Structure, freshness signals, entity clarity, fluency — none of these change facts. Fabricated statistics, invented experts and purchased consensus do. The line between those two lists is not a matter of degree.

The manipulation playbook, with receipts

For a long time the manipulation risk in GEO was argued theoretically. It is not theoretical anymore.

Lasso Security Research ran an experiment that should be required reading for anyone selling AI visibility. They built one ordinary-looking site about gluten-free cooking, planted a single false health claim in it, and applied seventeen standard GEO techniques to that same claim — one at a time, then in combinations. Same page. Same claim. Same model. Only the presentation changed. Five production models, 5,525 runs.

Baseline: zero citation, zero promotion. Asked directly, without retrieved context, all five models refused to recommend the substance. The guardrails worked.

Then the researchers wrapped the claim in editorial endorsement: three short articles, in the voice of plausible niche publications, each citing the site and repeating the claim. Llama-4-Maverick promoted it in 92% of runs. GPT-4o-mini in 78%. DeepSeek-R1 in 26%. Grok and Claude Haiku held at zero. Add a ranked-list format and those two susceptible models went to 98% and 84%.

The mechanism deserves naming precisely, because it is the part that gets lost in the models-are-getting-safer narrative. The models were not fooled by a trick. They did what synthesis is supposed to do: they weighed agreement across sources. The attack manufactured the agreement. Manufactured consensus is not a jailbreak; it is a supply-chain problem.

And here is the finding that should worry anyone who thinks citations make this auditable. In 487 runs, a model promoted the claim while the attacker’s page was sitting in its context window — and never cited it. The claim got attributed to the planted editorials instead. Across all runs that produced a detectable signal, 48.2% promoted the claim without citing the original page, against 48.0% that cited without promoting.

The audit trail points at the wrong door. That is qualitatively worse than being cited by a bad source, because a bad source at least exists to be found.

The techniques that still work are the honest ones

Here is where the argument usually turns moral, and where it does not need to.

The original GEO paper — Aggarwal and colleagues, published at KDD 2024, the source of the term itself — tested nine content modifications across roughly 10,000 queries on a generative engine benchmark. The three strongest were quotation addition, statistics addition and source citation. The one that scored below doing nothing was keyword stuffing.

Sit with that. The academic foundation of this entire discipline says the reliable ways to earn a citation are the ways that involve making a checkable claim — a real figure, a real quote, a real reference. Every technique that took work and added verifiable substance outperformed the technique that took no work and added only repetition. The honest path is not a moral preference here. It is the measured one.

Why detection hasn’t caught up

I would like to report that the defense side has closed the gap. The research says otherwise.

GEO-Bench standardized the comparison across attack families and found something inconvenient: black-box content rewriting matches or exceeds gradient-based attacks on rank promotion, produces more fluent text, and evades both keyword-based and perplexity-based detectors on some domains. Whether an attacker has model access does not predict how strong the attack is. Effectiveness and stealth trade off, but the frontier sits further out than defenders assumed.

Counter-GEO-Bench, accepted at EMNLP 2026, tested three off-the-shelf defenses against information-distorting rewrites across 247 human-verified queries. The best reduced attack success by 5.7% relative. Granite Guardian’s reduction was not statistically significant. The reason is almost embarrassingly simple: safety classifiers are trained to catch policy violations, and a fabricated statistic in confident prose violates no policy. It reads as information.

The paper’s own purpose-built baseline, a lightweight guard, cut success by 47.6% relative with near-zero loss in answer quality. So the problem is tractable. It is just not solved by anything currently deployed.

Which produces the worst possible interim equilibrium: manipulation is cheap, effective and largely undetected, while the honest operator’s advantage — being genuinely the best answer — is rewarded only to the extent the engine can tell the difference. Today it often cannot.

The pragmatic case, which is stronger than the moral one

Consumers are not naive about this, and the survey data is unusually consistent across three independent studies from 2026.

Adobe surveyed 1,011 US consumers who use assistants to shop. 49% said they had abandoned a purchase after the AI’s description did not match the brand’s own site. 59% said they lose trust in a brand when an AI states something incorrect about it — even when the brand did not write the answer. Asked who is to blame, 38% pointed at the platform and only 13% at the brand. The blame is diffuse. The lost sale is not.

Kinsta’s poll of 1,008 US consumers found 52.1% trust traditional search results more than AI answers, 90.7% are worried about AI-generated misinformation, and only 18.8% trust an AI product recommendation without further research. WordPress VIP’s 2,000-respondent survey found 42% rank AI answers without clear attribution as the thing they trust least online — below airline fees and confusing legal print — while 86% say they always or sometimes go looking for the original source.

Notice the shape of that. Trust in AI answers is not collapsing. It is relocating — away from the answer and toward the source behind it. The 1% citation click rate and the 86% intent to find the original are not a contradiction. They describe a reader who has learned to treat the answer as a lead rather than a verdict.

Which means the commercial value of being cited is contingent on being checkable. If the citation layer becomes known as unreliable — and manufacturing consensus is precisely how that happens — the visibility you bought depreciates with the channel’s credibility. You are not extracting from a commons. You are borrowing against it.

The correction cycle, and how long it takes

Platforms do eventually correct. The record is consistent enough to be useful.

Google’s helpful-content system shipped in 2022. The real correction landed in March 2024 as a core update plus three new spam policies, and took 45 days to roll out. The FTC’s rule on fake reviews and testimonials followed a similar arc: proposed in 2023, finalized in August 2024, effective that October, covering AI-generated reviews, company-controlled “independent” review sites, review suppression and purchased social signals. Cloudflare’s default blocking of training and agent crawlers on ad-supported pages, announced on 1 July 2026 and taking effect on 15 September, is the most recent — and the most consequential, because it changes the default rather than the option.

Two to three years, every time. And each correction has been a blunt instrument: site-wide classification, category-wide rulemaking, network-level defaults. The pattern is not “platforms will catch you.” It is “platforms will catch the whole category, and the honest operators will absorb part of the blast.”

That is the real argument for ethics here, and it is not moral at all. It is about which side of the correction you want to be standing on.

A workable standard

Five commitments, in descending order of how often they get skipped.

Optimize for the question, not the query string. The moment your content strategy becomes “what will the model quote,” you have started optimizing for the intermediary instead of the reader. It is also the strategy that the GEO paper’s own results show underperforms.

Treat accuracy as non-negotiable, and ignorance as no defense. The FTC rule carries a should-have-known standard: an influx of similarly worded, generic praise is a warning sign you are expected to notice. “The model wrote it” is not a shield.

Disclose the method. Not because a regulator is watching today, but because the correction cycle above guarantees someone eventually will, and a paper trail is a liability in a way that a published method is not.

Earn corroboration; do not manufacture it. This is the cleanest red line the research offers, because manufactured endorsement is the single most effective lever in the entire playbook. If your visibility plan involves third parties agreeing with you who do not actually agree with you, you are not in a gray zone. You are in the 92%.

Model the downstream human. 32% of US adults used an AI tool for health information or advice in the past year, and 69% of those users say they trust it a great deal or a fair amount. Those who use it trust it more than those who do not. Whatever you optimize, someone will make a decision on it, and the distance between “the model said it” and “someone acted on it” is where this stops being a marketing question.

The question nobody wants to answer

There is an uncomfortable corollary hiding in all of this.

If being cited reliably requires manufacturing the appearance of consensus, then the honest operator is structurally disadvantaged until detection improves. Telling them to be good is not a strategy; it is a request to lose slowly. That is not an ethics problem. It is a market-design problem, and it needs a market-design answer: signals that reward verifiable sourcing, penalties aimed specifically at manufactured corroboration, and platforms willing to pay for the difference.

Some of that is starting to appear. Cloudflare’s pay-per-use model compensates publishers when their content surfaces in answers rather than merely when it is fetched. Nearly four in ten of the world’s most popular landing pages now block AI crawlers outright — 12.7% by naming a specific agent, another 27.2% through a blanket wildcard rule. And the block-to-allow ratios show operators have thought carefully: GPTBot faces 5:1 against, CCBot 12:1, while OAI-SearchBot — the crawler behind ChatGPT’s web search — faces just 2:1 and is the most tolerated agent in the dataset. Training crawlers build a competing product. Search agents send traffic. Sites are drawing that distinction without waiting for a standard.

The infrastructure layer is being renegotiated. The content layer has not been.

What this comes down to

GEO reopened the oldest argument in search marketing on a surface where one manipulated answer can reach a reader who will check nothing. That is not a reason to abandon optimization; it is a reason to be precise about what you are optimizing.

The line has not moved. It is still the same question: does this make me a better answer, or just a louder one? And the evidence is unusually kind to the honest side of that question. The techniques that make you better — real statistics, real quotations, real sourcing, clean structure — are the ones the KDD benchmark says still work. The techniques that make you louder are the ones with a two-year shelf life and a category-wide correction waiting at the end of it.

The brands still being cited in 2029 will, almost by definition, be the ones whose claims survive being checked. That is not a moral prediction. It is just what happens to content that has to stand next to its own sources.