Learning From What You Shipped, Not Just What You Planned

Most teams do not skip the retrospective because they are lazy. They skip it because the last one they attended turned into a trial, and nobody wants to be the defendant twice.

The campaign ships. The numbers land. Somebody writes a deck with three green arrows and one apologetic yellow one, and everyone moves to the next launch. Six weeks later a colleague proposes the exact same audience strategy that flopped in March, and nobody in the room remembers that it flopped — or why.

That is not a knowledge problem. It is a memory problem with a cultural cause. And it is costing you more than the campaign did.

Why the retrospective is the first thing cut

Every team I have spoken to has the same calendar. The pre-launch reviews are sacred. The mid-flight optimisations are sacred. The post-campaign review is a nice-to-have that gets rescheduled until it evaporates.

Two forces drive it.

The deadline reflex

The next campaign is already late. The retrospective competes with work that has a stakeholder attached to it, and reflection never has a stakeholder. So it loses. This is not a character flaw, it is an incentive structure — and incentives win.

The blame reflex

Here is where it gets uncomfortable. Ask a room of marketers why they dread the post-mortem and the honest answer is usually some version of: because someone is going to end up holding the bag.

Amy Edmondson, the Harvard Business School professor who has spent three decades studying how organisations handle failure, once put a question to executives: of all the failures in your organisation, how many are genuinely blameworthy? Their answers landed in the single digits — roughly 2 to 5 percent. Then she asked how many are treated as blameworthy. After a pause, or a laugh, the answers came back at 70 to 90 percent.

Read that gap again. The overwhelming majority of what your organisation treats as misconduct is, by its own leaders’ assessment, ordinary organisational life: unclear processes, tasks that are too hard to execute reliably, uncertainty nobody could have resolved in advance, experiments that tested a hypothesis and found it false.

The consequences compound quietly. Failure that cannot be discussed cannot be diagnosed. A team that expects punishment for candour will give you a clean timeline and a vague root cause, which is the most expensive document in marketing.

What skipping actually costs

Two losses, and they are asymmetric.

The mistake you pay for twice

The first time a campaign fails, you pay for it in media spend. The second time — when a new hire, or a new agency, or a reorg, walks the same idea into the same wall — you pay again, plus the cost of not knowing you had already paid.

Tucker and Edmondson documented the mechanism with unusual precision in a study of hospital nursing units. Over 239 hours of observation across nine hospitals, they recorded 194 process failures. The nurses were excellent — experienced, well-trained, dedicated. And they almost always responded with what the researchers called first-order problem solving: they worked around the fault and kept the patient safe.

Walk to another unit and borrow the linen. Improvise the missing supply. Chase down the absent information. Every one of those workarounds is a small act of heroism, and every one of them leaves the underlying system exactly as broken as it was. The problem returns. The workaround returns. Nobody learns.

Second-order problem solving — actually fixing the system — required three things the units did not have: time, psychological safety to escalate, and managers who were present enough to act on it. Absent those, the loop is stable and self-reinforcing: workarounds produce exhaustion, exhaustion produces burnout, burnout erodes the willingness to raise problems, and the problems keep coming.

Your marketing team is that nursing unit. The workaround is the “we just know to book the media a week early” folklore. The system never learns, and the hero gets a bonus for coping.

The win you cannot repeat

The second loss is the one nobody counts, and it is the bigger one.

A campaign that outperforms is not automatically a repeatable asset. It produced a result. Whether you can produce that result again depends entirely on whether you captured why it worked — which element carried the lift, which assumption turned out to be true, which audience segment did the work that the targeting brief credited to the creative.

Without that capture, your best campaign of the year becomes a story you tell at an offsite. The award-winning cases in the IPA Effectiveness Databank — more than 1,600 of them since 1980 — are valuable precisely because someone forced the causal story into writing. Most in-house wins never get that treatment.

What a good retro actually is

Four properties. Miss any one and the ritual decays.

Blameless is not accountability-free

This is the most misunderstood idea in the whole practice, so let’s kill the myth.

Blameless means the review does not indict a person for having done the wrong thing with the information available at the time. It does not mean nobody owns anything. In Google’s formulation, a blameless post-mortem identifies contributing causes without indicting individuals or teams — and then demands a written action plan, with every item assigned to a name and a date.

Blameless reviews are stricter, not softer. It is trivially easy to close a review by declaring “human error” and moving on. It takes real work to explain why a competent person, holding the information they held, made the choice that made sense to them in the moment.

John Allspaw made this argument at Etsy in 2012 and it has not improved with age. He described the cycle that blame creates: a person acts, the person is punished, trust between the frontline and management drops, people go quiet about details, management loses visibility, latent conditions become invisible, errors get more likely, and the cycle restarts. He called the alternative a just culture — one that balances safety and accountability instead of trading them against each other.

His summary line is the best one-liner in the literature: less-skilled companies name, blame, and shame.

Hypothesis first, opinion second

A retrospective without a written hypothesis is a taste test.

If the brief said “we believe a shorter hook will lift completion rate on cold traffic,” then the review has something to grade. If the brief said “drive awareness and engagement,” then the review will produce adjectives.

The discipline is to write the hypotheses down before the campaign ships — one line each: what we believe, how we would know, what result would prove us wrong. Edmondson’s work on intelligent failure makes the same point from the other direction: failures at the frontier are valuable precisely because they were designed to produce information. That only holds if you specified in advance what information you were buying.

System conditions, not personalities

“The creative was weak” is not a finding. It is a verdict.

The usable version is a question about conditions: what in the brief, the approval chain, the review timeline, the asset library, or the measurement setup made a weak creative the most likely outcome? Fix the condition, and the next twelve campaigns inherit the fix. Blame the person, and you have replaced one creative director with another who faces the same conditions.

Output: a changed assumption

Here is the test I would apply to any retrospective: did a belief change?

Not “did we document what happened.” Not “did we all feel heard.” Did some assumption that shaped the campaign get revised, retired, or confirmed strongly enough that it now enters the next brief as a constraint?

If the answer is no, you held a meeting. Meetings are not learning.

Where the loop leaks

Now the uncomfortable part: even teams that do run reviews often fail the conversion step.

Two independent 2026 datasets, read together, describe the same failure from opposite ends. Marketing Week’s Language of Effectiveness survey — 636 brand-side marketers — found that 71.1 percent see a disconnect between how marketing defines success and how the CEO and CFO define it. Slightly under half, 47.9 percent, say their organisation has no mechanism at all for measuring creative impact.

GrowthLoop and Ascend2, surveying more than 300 marketers and data leaders, found that 58 percent spend a moderate or significant amount of time on experimentation, yet only 20 percent report high impact from it. Worse: 77 percent say their “winning” tests fail once scaled. And only 23 percent can reliably connect a marketing action to a business outcome.

Sit with that 77 percent for a moment. Three-quarters of teams are running tests, declaring winners, and discovering later that the win was noise, or context-specific, or measured against a metric that does not move the business. That is not a testing problem. That is a review problem: nobody went back to ask why the winner stopped winning.

The Effie UK analysis of its award winners points at the same remedy from the success side. Winners used continuous tracking at a rate of 73 percent, against 33 percent of the general entry pool. The effective teams were not more creative. They were more disciplined about closing the loop between what happened and what they believed.

The three thirds

The US Army’s after-action review is the most-copied and most-botched learning ritual in modern management. Peter Senge called it arguably the most successful organisational learning method ever devised, and then observed that almost every corporate attempt to import it has failed — because people reduce a living practice to a sterile technique.

The reason is structural, and it is the single most useful thing in this article.

Army leaders described the AAR as three thirds: preparation, the meeting itself, and — the one everybody cuts — follow-through. Researchers who studied corporate transplants found the predictable failure modes. AARs conducted ad hoc, as a response to a specific problem rather than as a routine part of how work is done, almost never improved future performance. When the practice was reframed as knowledge capture rather than actionable insight, candour dropped and the sessions became box-ticking. And when participants were expected to produce recommendations for unspecified others to implement, “no one was accountable for action, so recommendations gathered dust.”

First third: preparation

Before the campaign ships, write down the hypotheses. Book the review. Name the person who will own each action item — while the campaign is still running, not after.

The calendar is the mechanism. A review that is not scheduled before launch is a review that will be scheduled against a launch.

Middle third: the meeting

Ninety minutes is enough. Walk the hypotheses line by line. For each: what did we expect, what happened, what in the system made the difference. Refer to people by role, not name — “the on-call media buyer,” not “Dan.” That single convention does more for candour than any speech about culture.

Then stop. Do not let the meeting become a status readout. Do not let it become a performance review with the word “retro” taped over the door.

Last third: follow-through

Every action item gets an owner and a date, and lands in the same tracker as product work — not in a shared drive folder named Learnings 2026.

Etsy’s rule was blunt: post-mortem action items trump whatever else the engineer is working on, including shipping product. Google’s version is a twice-yearly FixIt week where people close the oldest open items. Both are the same insight wearing different clothes — the follow-through is the review. Everything before it is preparation for the follow-through.

One more line from Google’s SRE book deserves to be printed on a wall: an unreviewed post-mortem might as well never have existed. A document nobody read is not institutional memory. It is storage.

Why it compounds

Here is the argument for why this is worth the hour.

The hit rate moves

A team with a working learning library is not smarter in any general sense. It simply stops re-buying the same information. It knows which audience segments have repeatedly failed to convert, which creative mechanic has never survived a scale test, which agency brief structure reliably produces weak work.

That knowledge does not make the next campaign brilliant. It removes a category of avoidable error, which is a far more reliable source of improvement than trying to be brilliant.

This is the logic of the experience curve, with one honest correction. Bruce Henderson’s 1968 formulation — costs falling 20 to 30 percent for every doubling of cumulative output — has been tested extensively since, and the empirical distribution is far wider and messier than the original claim. Progress ratios cluster around 70 to 80 percent, but the variance is enormous, and in plenty of firms learning does not happen at all.

Which is exactly the point. Learning curves are not automatic. They are produced by attention. Firms that never capture what they learned are the ones sitting at the flat end of the distribution, paying full price for every mistake, forever.

The memory does not resign

Marketing has a specific and well-documented memory problem: the people holding the context leave.

Spencer Stuart’s annual tracking puts average CMO tenure at S&P 500 companies at 4.1 years, against 5.0 years for the C-suite overall. In consumer brands it drops to 3.5 years. And 31 percent of S&P 500 companies have no CMO at all, meaning the marketing memory sits somewhere else on the org chart, or nowhere.

Four years is roughly eight to twelve campaign cycles. If the causal learning from those cycles lives in one person’s head, the organisation’s institutional memory has a four-year half-life and the next leader will rebuild it from scratch, at full tuition.

A logged, searchable, hypothesis-anchored learning record is the only version of that memory that survives a departure. It is also the only version that can be handed over during a two-week notice period.

The “why” is the part that transfers

The campaign itself does not transfer. The audience is different next quarter, the platform has changed its algorithm, the cultural moment has moved.

What transfers is the mechanism: why a particular message structure worked on a particular kind of scepticism. Why a discount framed as information outperformed a discount framed as generosity. Why a short hook lost to a long one when the audience was already problem-aware.

Shallow reviews record what happened. The campaign was up 14 percent in click-through. Deep reviews record why, and the why is the only part that survives contact with a different campaign.

Risks, honestly

I would not trust a piece that only listed the upside.

Retro as theatre

The most common failure is the review that happens, feels fine, and changes nothing. Everyone says something went well, everyone says something could improve, nobody’s belief changes, and the deck is filed.

The countermeasure is the output test: name the assumption that changed. If the room cannot, the meeting was theatre and you should cancel the next one rather than pretend.

The sugarcoating inverse

The opposite failure is the review that is so committed to psychological safety that it becomes dishonest. “Everything was great” is not a safe review; it is a useless one. Safety without candour produces the same information loss as blame, just more politely.

Edmondson’s framing is useful here: some failures are genuinely praiseworthy — reasonable actions under genuine uncertainty, experiments designed to learn — and some are not. A review that cannot distinguish the two is not being kind. It is declining to do the job.

Over-learning from one data point

One campaign is a sample of one. A single review can install a false lesson — a creative that worked because of a trending audio, a channel that looked efficient because of a seasonality artefact.

The learning only compounds when reviews accumulate into patterns. Three reviews is a hint. Eight is a signal. The discipline is to log single observations as observations, not as rules, and promote them only when the pattern holds.

The time cost is real

A genuine review costs ninety minutes plus prep plus the follow-through. That is a real cost, and it will always lose an argument against shipping.

Which is why it cannot be defended by argument. It has to be defended by the calendar: pre-scheduled, mandatory, and with the same status as a media plan review. The teams that protect it treat it as infrastructure, not as reflection.

A template you can run next week

Keep it short enough that people will actually do it.

Note what is absent: a slide for “what went well” as a feelings exercise, and a slide for “lessons learned” as a summary. Both are where reviews go to die.

What to do on Monday

1. Put the retro on the calendar at campaign kickoff, not at campaign close. Ninety minutes, with the same attendance expectation as the launch review.
2. Force the hypotheses into the brief. If the brief cannot state what would prove it wrong, the campaign is not testable and the review will be a taste test.
3. Assign the action items before the campaign ends. Every item gets a human and a date, and lives in the same tracker as everything else that matters.
4. Change the language. Swap “why did you” for “what made that look like the right call.” Swap “who missed it” for “what would have made it visible.” Swap “human error” for “the process allowed it.” These are small edits and they change what people are willing to say.
5. Publish the log where the next team will trip over it. A learning library nobody can search is a filing cabinet.
6. Model it from the top. The most senior person in the room asking a blame-free question in the first five minutes does more than any policy document. If the CMO opens with a mistake of her own, the room recalibrates instantly.

The bottom line

Campaigns are perishable. They decay the moment they stop running: the audience moves, the platform changes, the moment passes.

The learning is the only part that appreciates. And it appreciates only if someone extracts it, writes it down, gives it an owner, and forces it into the next brief.

Every skipped retrospective is a mistake you have elected to make twice, and a win you have elected not to repeat. That is not a culture problem in the abstract. It is a line item, and it is on the wrong side of the ledger.

The campaign is a cost. The learning is the asset. Collect it, or pay for it again next time.