Falling Token Prices Won't Save the Budget You Now Own
By Izzy Aly, Co-Founder, Systems and Architecture
AI spend grew more than 20x. If you inherited the AI mandate, here's why visibility, not price, is the fix.
Your CEO said, "We need to be doing AI," looked at you, and walked off. That was the whole brief. Now you're the one who has to explain why the AI line item tripled last quarter, and "the tokens got cheaper" is not an answer the board will accept.
You've watched this movie before. The bill for the last one is $44.5 billion a year.
A decade ago, cloud infrastructure was supposed to save everyone money. Pay for what you use. Scale down when you don't. No more CapEx hardware rotting in a closet. Then the bill arrived. And kept arriving. Cloud waste alone was projected to hit $44.5 billion in 2025, with enterprises taking an average of 31 days to identify and eliminate it (Harness, FinOps in Focus, February 2025), and 94% of organisations reporting avoidable cloud spend (HashiCorp/Forrester, 2022).
AI is running the same pattern, faster. An agentic workflow can spawn 50 inference calls per user request. Thirty-one days is an eternity when costs compound that fast.
Key takeaways
- Token prices fell 92% in 17 months (GPT-4 to GPT-4o), yet enterprise GenAI spend grew more than 20x, from $1.7B in 2023 to $37B in 2025 (Menlo Ventures, Dec 2025).
- Agentic workflows and hidden "reasoning tokens" are the multipliers: reasoning-token use per org rose roughly 320x in 12 months (OpenAI, The State of Enterprise AI, Dec 2025).
- The fix isn't waiting for prices to stabilise. It's tagging spend by feature before the mandate becomes a crisis.
Why doesn't cheaper AI mean cheaper bills?
Because usage is expanding faster than unit prices are falling, and nobody's tracking the multiplier. GPT-4 launched at $30 per million input tokens in March 2023. GPT-4 Turbo dropped that to $10. GPT-4o opened at $5 and settled at $2.50 by August 2024, a 92% reduction in 17 months. Andreessen Horowitz calls the broader trend "LLMflation": roughly a 1,000x reduction in inference cost per unit of capability over three years. Epoch AI (March 2025) puts the annual decline at 9x to 900x depending on the task, and 40x a year to match GPT-4's performance on PhD-level science questions.
Meanwhile, total enterprise GenAI spend went from $1.7 billion in 2023 to $37 billion in 2025, more than a 20x increase (Menlo Ventures, State of Generative AI in the Enterprise, December 2025). OpenAI's own compute footprint tracked the same curve, growing from 0.2 GW in 2023 to about 1.9 GW in 2025, according to its CFO. Prices fell 1,000x. Spend grew more than 20x anyway. That gap is the entire problem, and it's a usage problem, not a pricing problem.
Jevons paradox: the 160-year-old warning behind your bill
In 1865, economist William Stanley Jevons noticed that more fuel-efficient steam engines led to more total coal burned, because cheap energy made more uses worth it. Microsoft CEO Satya Nadella named the same effect in January 2025: "Jevons paradox strikes again! As AI gets more efficient and accessible, we will see its use skyrocket." Three specific mechanisms are doing that right now.
The agentic multiplier
Cheaper inference didn't just make chat cheaper. It changed how software gets built. A single "fix this bug" request to a coding agent isn't one API call; it's a loop: plan, draft, test, error, refine, verify. One user action can spawn 5, 10, or 50 inference calls. OpenAI's own usage data shows average reasoning-token consumption per organisation rose roughly 320x in the 12 months to late 2025 (OpenAI), which is agentic architecture reshaping usage patterns, not a fluke.
What are hidden reasoning tokens, and why do they cost more?
Reasoning tokens are the internal "thinking" steps reasoning models like OpenAI's gpt-5.6-sol generate before producing a visible answer, and they're billed as output tokens, which cost roughly 5-6x the input rate on current price lists. A prompt might show 100 input tokens and 50 visible output tokens on the invoice, with 5,000 hidden reasoning tokens behind it that you never see. Cost stopped tracking answer length. It now tracks problem complexity, invisibly.
Why does EU compliance add a real cost multiplier?
If you operate in Europe, the cheapest model isn't the one with the lowest token price; it's the one that doesn't trigger regulatory exposure. Under the EU AI Act, high-risk use cases like HR screening or credit scoring carry real compliance costs. Here's the reframe most teams miss: that governance layer isn't a separate cost bolted onto your AI spend. Built correctly, it's the same infrastructure that gives you the visibility this whole article is about. A human review gate that catches a runaway 50-call agent loop is doing compliance and cost control in the same motion.
What's actually costing you AI budget visibility?
Four missing decision-making skills, not one generic tooling gap, explain almost every AI budget surprise. Most "control your AI costs" advice reads like a checklist; sorting the failure modes by which skill is actually missing tells you what to fix first.
| Missing discipline | What's actually happening | The fix |
|---|---|---|
| Diligence | Visibility fragments as usage scales. Prompt chains, RAG stacks, and fine-tuning jobs pile up under one API key, so spend can't be attributed to a feature or team. | Audit hooks and data-flow documentation exist before anyone asks for them, not after. |
| Discernment | Your FinOps tooling can't see prompts. Most platforms were built for compute and storage, not prompt-driven workloads, and can't tell you that a poorly written prompt costs 10x more. | A human judgement call on whether a feature or model swap is worth the cost. No dashboard makes this decision for you. |
| Description | "Cheap" enables waste at scale. At $30 per million tokens, optimising a prompt paid for itself immediately. At $0.15 per million, an unoptimised 1,000-token prompt run 10 million times still costs $1,500 a month instead of $150 for a tight 100-token one. | Writing a tight, tested prompt once volume is high, the same discipline expensive tokens used to force by default. |
| Delegation | The optimisation window is closing. Deciding which tasks AI should own gets harder once you've got dozens of features, three model providers, and habits already built around unoptimised prompts. | Decide now, while it's cheap to decide, which tasks stay human and which get automated. |
In most AI workflows, the Diligence gap shows up first: nobody can say what a single "agent run" actually costs until someone tags it. That's not a tooling gap. It's a five-minute decision nobody made yet. Cloud taught the same lesson once already: teams that built FinOps discipline early saved far more than the ones who scrambled during a budget crunch.
Governance is a hedge, not a bet
The real question isn't whether this gets expensive. It's when, and that depends on a variable you don't control: where token prices go next.
- Bull case: prices keep falling, Jevons Paradox kicks in harder, agents run wilder, total spend keeps rising.
- Bear case: subsidies end, energy constraints bite, unit prices rise, your unit economics get worse either way.
Both scenarios end in the same place: rising total spend. The only move that works in both is building cost visibility and governance now, so you can throttle volume in a cheap-price world and optimise unit cost in an expensive one. This is exactly what a governed AI implementation should give you by default, not as an add-on: a human stays in the loop on every high-cost decision, nothing about how a workflow spends money is a black box, you measure the baseline before anything gets touched, and someone stays accountable after launch instead of disappearing after kickoff.
Is this actually your problem yet?
Maybe not, and it's worth saying so plainly. If you're pre-product-market-fit, buying an enterprise FinOps platform is premature; you have bigger things to prove first. But "not yet" isn't the same as "never," and the same 31-day lag Harness measured in enterprises (Harness, 2025) applies just as much to a two-person team as it does to an enterprise FinOps function. The gap doesn't need enterprise scale to hurt; it just needs enough time to compound unnoticed.
So if your AI spend is anywhere near €1,000 a month, don't ignore this entirely. Start tagging API calls now, before there's anything worth optimising. Building the habit of tracking cost-per-feature costs nothing today. Retrofitting it later, once you've got millions of unlabelled logs and no record of which feature generated which cost, is expensive, slow, and usually happens right after the bill that finally got someone's attention.
What can you do about AI costs this week?
You don't need a six-month programme. Four things, in order:
- Tag every API call with which feature triggered it and which team owns it. This is the foundation everything else depends on.
- Calculate cost per core action: what it actually costs to generate one customer response or run one agent workflow end to end. Without this number, model selection is a guess.
- Set one alert. Pick your biggest AI cost centre, set a daily threshold, and make sure a person gets notified if it's crossed. This alone would've prevented a good share of the cloud cost horror stories of the last decade.
- Map your data sensitivity. If GDPR or the EU AI Act applies, know which prompts touch personal or high-risk data before you pick a model or a region. A legal mistake costs more than any amount of token optimisation saves.
Frequently asked questions
Why did enterprise AI spending grow more than 20x if token prices dropped 1,000x?
Usage expanded faster than prices fell. Enterprise GenAI spend rose from $1.7B in 2023 to $37B in 2025 (Menlo Ventures, Dec 2025), more than a 20x increase, while the cost of GPT-3-level inference fell roughly 1,000x between 2021 and 2024 (a16z) and GPT-4-level prices fell about 40x a year (Epoch AI).
What is the Jevons Paradox, and why does it apply to AI costs?
Jevons Paradox, named for economist William Stanley Jevons in 1865, describes how efficiency gains increase total consumption rather than reducing it, because cheaper access opens up more use cases. Microsoft CEO Satya Nadella cited it directly in January 2025 to explain rising AI usage as prices fall.
What are hidden reasoning tokens, and why are they expensive?
Reasoning tokens are internal "thinking" steps that reasoning models like OpenAI's gpt-5.6-sol generate before producing a visible answer. They're billed as output tokens, roughly 5-6x more expensive than input tokens, and can run into the thousands per query without appearing in the visible response.
What's the first practical step to controlling AI spend?
Tag every API call with the feature and team that triggered it. This single step, done before volume scales, is what makes every other cost-control measure, from model selection to budget alerts, possible later.
The pattern is clear
Cloud waste is a projected $44.5 billion problem despite a mature FinOps industry. AI is on the same trajectory, moving faster. The people who instrument their AI spend and governance before it's urgent are the ones who actually capture the value of falling token prices. The ones who wait will be writing the same "how we cut our AI costs by 40%" post in 2027 that's been written about cloud for the last five years.
The only question is which one you want to be: the person who got ahead of it, or the person explaining to the board why last quarter's AI bill ran 3x the forecast.
- Topics
- Share