Which AI metrics should an agency show its clients? Not the ones most agencies are proudest of. Seats licensed, agents deployed and assets generated describe your effort. Clients pay for outcomes: what each piece of work costs, how fast it arrives, how often it is wrong and what it does for profit. Measure those, and your AI story becomes a commercial one.
The pressure to switch is now coming from the contract. According to Campaign US, a WFA and Agency Mania Solutions study of 69 multinationals with $147bn in combined spend found that labour-based agency remuneration has fallen from 54% in 2011 to 17% in 2026. Some 58% of brands expect more performance-based pay. Only 20% have changed their model because of AI so far, but 61% plan to.
Why are clients asking what your AI actually delivers?
Because time is collapsing as a proxy for value. If a report that took a day now takes an hour, billing the day looks odd and billing the hour looks like a pay cut. The WFA study cites AI as making time an ever weaker measure of what an agency is worth.
Yet the shift is stuck halfway. Adweek reported this month that brands are slow to adopt outcome-based remuneration, even as AI erodes FTE-based agency revenue. Horizon Media Holdings president Bob Lord called FTE billing the "old system" and questioned whether it can be broken "without some kind of external force".
That stalemate is an opening for independents. If the old model needs an external force, an agency that turns up with credible outcome numbers can be that force, rather than waiting for procurement to impose one. But only if the numbers are the right ones.
What counts as a vanity metric in agency AI?
One Singapore agency founder who spent 18 months trying to make his roughly 20-person agency AI-native put it on a single slide at an AI adoption session this week: what we counted versus what matters. As the diagram shows, the left-hand column is the material that fills an internal all-hands deck: AI users, tasks automated, agents deployed, assets generated. The right-hand column is tied to business outcomes: cost per output, time to output, quality/error rate and profit impact.
His point was blunt. The customer does not care whether you used AI. They care whether the output is good. Nobody renews a retainer because the agency has 40 active seats.
Here is one way to translate each vanity number into something a client can feel.
| What we counted | Why it flatters | What matters instead | The client question it answers |
|---|---|---|---|
| AI users | Measures licences, not useful work | Cost per output | "Are we paying less for each report, edit or asset?" |
| Tasks automated | Counts attempts and ignores the rework | Time to output | "Do we get the work sooner?" |
| Agents deployed | Rewards building over results | Quality / error rate | "Can we trust what lands in our inbox?" |
| Assets generated | Volume is cheap, and most of it is never used | Profit impact | "Did this move sales, margin or efficiency?" |
The pairings are ours, not a law of nature. The principle holds either way: every number you report should survive the client asking "so what?"
Why is "good enough" AI output so expensive?
The most useful idea in the talk was about acceptance rates. Say an AI workflow drafts ten client replies, captions or report summaries and six are accepted as they are. Most teams celebrate the six and quietly fix the other four by hand. The founder's argument: build a closed loop that feeds every correction back into the system, so six becomes seven, then eight, then nine.
The danger is the opposite. Output that works 60–70% of the time gets waved through as "good enough", and mediocrity becomes the house standard. Nobody decided to lower the bar. It just drifted.
In agency terms, that drift looks like a social team spending Friday afternoon rewriting AI captions that saved them nothing, or a CRM shop whose segmentation is right most of the time and embarrassing the rest. The time saved on paper is eaten by editing, and the client sees the misses, not the average.
Acceptance rate is a quality metric you can track from day one. Rising acceptance means the loop is working. Flat acceptance means you have bought a subscription, not a capability.
Why do agencies keep reporting the wrong AI numbers?
Value Perception Theory explains it simply: people judge value by the benefit they experience relative to what they give up, not by what the seller put in. Inputs are visible to the agency and invisible to the client. "Agents deployed" is an input. A report that arrives two days earlier and needs no corrections is a benefit.
Agencies default to input metrics because they are easy to count and they make the team look busy. The holding companies have trained the market to talk about proprietary platforms in the same way. But a client does not feel your stack. They feel turnaround, accuracy and whether the work moved their numbers.
It is also why time-based billing is struggling. Hours are an input too.
How do you build an AI scorecard clients will pay for?
Start small: one workflow, one client, one quarter. Then expand.
- Pick a workflow the client already feels. Monthly reporting, competitor audits, asset resizing or campaign QA. If the client would not notice it improving, it is the wrong place to start.
- Measure the baseline honestly. Hours, cost per output and corrections per delivery before any AI is involved. Without a baseline, every improvement is an anecdote.
- Set the target in outcome terms. "Report delivered in one day instead of three, with zero client corrections" beats "use AI for reporting".
- Count the full cost. Include the time people spend checking and editing AI output. A workflow that saves four hours and adds four hours of fixing has saved nothing.
- Track the acceptance rate weekly. Log what was accepted, what was edited and why. Feed the edits back into prompts, templates and examples so the rate climbs.
- Report in the client's language. Cost per output, time to output, error rate and, where you can link it, the effect on sales or margin. Leave seat counts in the internal deck.
- Take the evidence into the commercial conversation. Two quarters of outcome data give you the basis for an output- or performance-linked fee, the direction the WFA study says most brands expect.
Where does creative fit into outcome metrics?
Creative people worry, fairly, that outcome metrics reduce ideas to spreadsheets. They do not have to. The aim is to close the gap between a creative concept and a measurable result before the media money goes out, not to let the spreadsheet write the idea.
That means bringing evidence early: which audience tensions the work answers, how competitors in the category are messaging and how likely a creative is to earn the click. On the Defy platform, creative predictions are validated against real ad performance, so the outcome conversation with the client starts from evidence rather than taste. Taste still decides what gets made. Evidence decides what gets defended.
The same logic applies to delivery. SAMY, a global influencer agency, adopted Defy Agents to streamline performance reporting and cut three days of work to one. That is a time-to-output number a client can feel, which is the whole point.
