AI Pricing: Why Cost-Per-Outcome Beats Tokens

MIT found that 95% of enterprise AI pilots show no P&L impact. Here's the cost-per-outcome formula we use to catch that in week one, not quarter three.

Key takeaways

• MIT's NANDA study found 95% of enterprise generative AI pilots show no measurable P&L impact (MIT NANDA, 2025).
• Token price hides the real cost: retries, over-fetched context, and human review time. Cost-Per-Outcome (CPO) counts all of it.
• A model that spends 7.4x more on tokens per intent can be 83% cheaper per intent once human escalation is counted.
• The EU AI Act's high-risk deadline moved. Chatbot transparency (Article 50) has applied since August 2, 2026. Annex III high-risk obligations were deferred to December 2, 2027.
• You can build a working CPO baseline for one narrow workflow in a week.

Someone on your team is going to ask what the AI budget actually bought. Maybe your CEO already has. If you don't have a number ready, that's not a communication problem. That's the actual problem.

I've watched this play out from both sides now: five businesses, and thirteen years of watching teams either adopt change or quietly wait it out. The pattern repeats. Someone negotiates a 10% discount on tokens in January. Nobody's measuring retry rates. By Q2, spend is up, reliability is flat, and the discount is long gone.

What actually breaks a cost-per-outcome model?

Production AI rarely fails because the model is weak. It fails because nothing around the model is instrumented: no evaluation pipeline, no drift monitoring, no one person who owns quality. MIT's Project NANDA reviewed 300+ enterprise AI initiatives between January and June 2025. It found that 95% delivered no measurable P&L impact, despite $30 to $40 billion in enterprise GenAI spend (MIT NANDA, "The GenAI Divide," 2025). S&P Global's 2025 Voice of the Enterprise survey tells a similar story. It found 42% of companies abandoned most of their AI initiatives that year, up from 17% in 2024. The average org scrapped 46% of proofs of concept before they ever reached production (S&P Global Market Intelligence, 2025).

Neither report blames the models. Both point at the same gap: nobody built the infrastructure to know if the thing was working. Your moat was never model access. It's evaluation discipline, and almost nobody has it.

Here's the part that surprises people: a model priced 50% cheaper per token is often the more expensive choice in production. Cheaper models frequently need more correction loops to format a JSON object correctly, or generate more tokens to say the same thing. We call that the verbosity tax. It erases the unit-price advantage before you've even measured anything else.

Why does token pricing hide the real cost?

Token price tells you what the raw material costs. It doesn't tell you anything about the finished product. Manufacturing has a name for this: the hidden factory, the rework and scrap that never hits the P&L but quietly eats the margin. AI operations have their own version. Retry rates. Context over-fetching, paying to ingest 10,000 tokens when 500 would've answered the question. Agent loops nobody scoped.

Here is what "cost" includes beyond the API invoice:

• Context waste: This is often the biggest silent killer. A 2,000-token system prompt repeated across 100,000 daily requests costs you 200M tokens a day in static text alone. Without caching or optimization, you are paying to process the same text millions of times.
• Reliability overhead: As usage scales, timeouts increase. A single retry-heavy endpoint running at a 15% failure rate inflates that feature's bill by roughly 15%: calls you paid for that never delivered value.
• Escalation: This is the budget killer. When the AI fails, the user gives up and creates a support ticket, and a person picks it up.
• Engineering overhead: The hours your team spends investigating cost spikes, debugging prompt regressions, and building internal dashboards that are outdated by next quarter.

Want a live example? In June 2026, Salesforce agreed to acquire Intercom (by then renamed Fin) for roughly $3.6 billion, and closed the deal that September (Salesforce, 2026). Fin's own pricing page lists $0.99 per resolution, billed only once per conversation no matter how many actions the agent takes, genuinely outcome-based (Fin, "Pricing," 2026). Salesforce's own Agentforce lists at $2 per conversation, or Flex Credits at $500 per 100,000 credits (Salesforce, "Agentforce Pricing," 2026). Every cost example on that page carries a footnote: it leaves out Data 360 (formerly Data Cloud) credits and other consumption services. The per-conversation number on the pricing page isn't the number that hits your budget. That's the hidden factory, printed in the footnotes.

So stop asking which model is cheapest per token. Ask which stack delivers the lowest cost per accepted outcome at the quality bar you actually need.

The cost-per-outcome formula

Cost-Per-Outcome ties spend to results, not infrastructure activity:

CPO = Total Workflow Cost / Number of Accepted Outcomes

Cost-Per-Outcome (CPO) divides total workflow cost, model tokens, retrieval, orchestration overhead, and human review time by the number of accepted outcomes. Most pricing conversations skip the review-time line. It's usually the one that decides the answer: at a 20% escalation rate, human review cost alone can outweigh every infrastructure cost combined. CPO treats the review gate as part of the outcome, not overhead subtracted from it.

Every approval gate we build is there because the human is still in charge. That's not a compliance checkbox. It's the actual point.

How can the "expensive" model be 83% cheaper?

Here's the formula with numbers in it. Take one support chatbot and two ways to build it.

Scenario A, the "efficient" chatbot. You optimize for tokens and choose a lightweight, mini-tier model at $0.15 per 1M input tokens. The model struggles with context, requiring an average of 5 turns per resolution, and 30% of requests escalate to a human. Hidden labor cost: 30% × $15.00 (Tier 1 support: 10 minutes at $90/hr loaded cost) = $4.50.

Scenario B, the "expensive" agent. You optimize for success and choose a premium model at $10.00 per 1M input tokens. It resolves in 1 turn, and only 5% escalate. Hidden labor cost: 5% × $15.00 = $0.75.

Both per-token prices are illustrative list prices.

Cost per intent is the average spend every time a user interacts, regardless of the outcome. Cost per success divides that by the success rate, so it shows what you pay per solved problem. On either measure, the "expensive" model is at least 83% cheaper for the business ($0.76 per intent vs. $4.50; $0.80 per success vs. $6.43, which is 88%).

If you only looked at your cloud invoice, Scenario A looks like a win. If you look at the P&L, Scenario A is a disaster. The labor cost of escalation dwarfs the savings on compute.

Self-hosting doesn't get you out of this. A common objection: "We run our own GPUs, so token costs don't apply to us." They do, arguably more so. Your cost per token is your GPU lease cost divided by usage: $2/hr × 730 hours = $1,460 a month for one A100, idle or not. If the self-hosted model fails to resolve the user's problem, you are burning GPU time for zero value, plus incurring the exact $15 escalation cost. If it resolves fewer requests than the API model, the labor costs will likely wipe out your GPU savings. The full comparison sits in the post on AI unit economics.

When is cost per token still the right number?

We aren't suggesting you throw cost per token away entirely. It remains a vital metric for specific infrastructure tasks:

• Infrastructure monitoring: Detecting runaway loops or DDoS-style token consumption.
• Contract negotiations: Calculating volume to commit to Provisioned Throughput Units (PTUs) or reserved capacity discounts.
• Model benchmarking: Comparing raw efficiency between model architectures in controlled experiments.

Where does the cost data come from?

You can't calculate CPO from a monthly invoice. Mature cost management requires tracking spend across five dimensions simultaneously. Infrastructure proxies such as LiteLLM can split spend by key, user, team, model and request tag, but they can't tell whether a call produced an accepted outcome. That join happens at the application layer.

1. 01
   User. Which end-users generate the most tokens? Are your power users profitable, or just expensive?
2. 02
   Team. Which internal team owns the spend? Engineering, Product, or Customer Success?
3. 03
   Feature. This is the dimension most teams skip. When you can see that Feature A costs €8K/month and generates €50K in revenue, while Feature B costs €6K/month and generates €2K, the optimization path becomes obvious.
4. 04
   Model. Which model is being called, and is it the right one for the task?
5. 05
   Prompt version. Which version is deployed, and how does its cost compare to the previous one?

The difference between "tracking" and "attribution" looks like this:

• Before: "We spent €18K on OpenAI last month."
• After: "Our support chatbot costs €7.2K/month on a frontier model. 60% of that is one prompt template that could run on Claude Haiku at a tenth of the cost or less. Switching saves us €3.9K/month."

Getting from before to after takes a three-step instrumentation pipeline:

1. 01
   Event telemetry. Use OpenTelemetry (OTel) to capture metadata for LLM calls. Tag every call with request_id, session_id, model, and feature_tag.
2. 02
   Outcome labeling. The critical missing link. Tag each session with success=True/False, from implicit signals (the user did not click "Contact Support") and explicit ones (the user clicked "Thumbs Up", or code execution returned exit code 0).
3. 03
   Cost join. Link your billing data to your telemetry via request_id, then aggregate by feature and outcome.

For the billing side, the FinOps Foundation's FOCUS specification provides a standardized schema for normalizing billing data across providers, which you extend with your own AI dimensions. Tools such as LangSmith, Portkey and Arize have built-in cost-tracking features that bridge this gap. Alternatively, you can build a custom pipeline feeding OTel data into your analytics database.

Two rules for whichever you pick. Collect asynchronously, logging after the response, so measurement adds zero latency to the hot path. And stay provider-agnostic, tracking every provider through a single interface.

A 30-day plan, mapped to how we actually build

You can't optimize a number you haven't measured yet. This is roughly the sequence we run on a new build. It maps cleanly onto the four thinking skills Anthropic calls AI fluency: Delegation, Description, Discernment, and Diligence.

1. 01
   Week 1, Delegation: instrument and baseline the workflow before touching anything.
2. 02
   Week 2, Discernment: build the evaluation gate that decides what to trust.
3. 03
   Week 3, Delegation, refined: route and cache against the baseline you now have.
4. 04
   Week 4, Diligence: shadow mode, then ship only if CPO actually improved.

Week 1: Delegation

Instrument and baseline first. Pick one narrow, high-value workflow. Log token usage by team and step, tagged with the dimensions above. Find out if you're currently paying €0.50 or €5.00 per successful transaction, because right now you don't know, and neither did we before we measured our own builds.

Week 2: Discernment

Build the evaluation gate. Set real acceptance criteria for quality, latency, and error tolerance. Add CI checks. Use a frontier-grade model to score the outputs of your faster, cheaper one. This is the human-in-the-loop layer, made concrete instead of theoretical.

Week 3: Delegation, refined

Route and cache. Turn on semantic caching for repeat queries, which cuts LLM spend roughly in line with your cache hit rate. Send simple requests to cheap models, hard ones to reasoning models. Most teams have never audited their prompts for token efficiency either; verbose system prompts and redundant context injection are common culprits. Measure the delta against your Week 1 baseline, not against a vendor's benchmark.

Week 4: Diligence

Shadow mode, then ship. Replay real traffic through the optimized stack before it touches a customer. Ship only if quality holds and CPO actually improved. If it didn't, you now know that before your customer does, not after.

Because LLMs are stochastic, your metrics will fluctuate. A prompt might succeed 95% of the time today and 92% of the time tomorrow, so averages alone can mislead. With only 50 samples, a cost per success of $0.76 might carry a range of ±$0.20, too noisy to make decisions. Rule of thumb: require at least 100 sessions per feature before acting on the number.

Keep watching after you ship. Only 11% of enterprises can forecast their AI spend within ±10%, down from 15% a year earlier (Mavvrik and Benchmarkit, "State of AI Cost Governance," 2026). One reason we see often is prompt changes that silently increase token consumption. You need to know the cost impact of a new prompt before the end of the billing cycle.

I ran a version of this loop for a full year with no client and no brief, just synthetic data and sandboxes, pushing to see where a stack actually broke. The pattern held every time: the teams that measured first were the only ones who could tell you, with a number, whether the "optimization" helped.

What changed with the EU AI Act deadline?

Short answer: part of it moved, part of it didn't, and the difference matters more than the headlines suggest. The Digital Omnibus on AI was published in the EU's Official Journal on July 24, 2026 (Regulation (EU) 2026/1744). It defers Annex III high-risk obligations, the use-based category most workflows fall under, from August 2, 2026 to December 2, 2027. Article 50 transparency obligations, covering chatbot disclosure, took effect on August 2, 2026, on the original schedule and were unaffected by the delay. The only Article 50 relief is a grace period until December 2, 2026 for marking AI-generated output from systems already on the market. For most teams building AI workflows now, Article 50 alone is reason enough to build governance in from day one. The same logging that supports a CPO baseline maps directly onto Article 12 record-keeping requirements, regardless of which deadline applies. The architecture traps behind those obligations are in the EU AI Act post.

This is also, honestly, good news for the CPO argument. You don't need a looming deadline to make the case. Enterprise buyers are already asking versions of the same three questions during governance review, and a CPO dashboard changes the quality of the answer:

"We track CPO variances and auto-rollback past a 15% spike" is a real answer. "We monitor it" isn't.

What does this actually cost to build?

A working Cost-Per-Outcome baseline takes about a week to build for one narrow workflow: instrument logging, pick the outcome, and measure current cost per accepted result. If you want to find out what one of your workflows really costs per outcome, book a scoping call.

Frequently asked questions

What is cost-per-outcome (CPO) pricing?

CPO divides total workflow cost, including model tokens, retrieval, orchestration, and human review time, by the number of accepted outcomes. It measures what a business result actually costs, not what a unit of infrastructure activity costs.

Why does a cheaper AI model sometimes cost more in production?

A model priced lower per token often needs more retries to format outputs correctly or generates more tokens to convey the same answer. That "verbosity tax" can erase the unit-price advantage once you count the full workflow cost.

Does the EU AI Act still take effect on August 2, 2026?

Partly. Article 50 transparency obligations for chatbots have applied since August 2, 2026. Annex III high-risk obligations were deferred to December 2, 2027 by the Digital Omnibus, published in the Official Journal on July 24, 2026.

What happens to the human review step in a CPO model?

It stays, and it should. Human review is often the highest single cost in a CPO calculation, but removing it is what produces the silent failures behind MIT's 95% no-impact finding. Treat it as the gate that protects the outcome, not overhead to cut.