Book a call (opens Google Calendar)
Book a call (opens Google Calendar) hey@pronovahq.com

The 95% Accuracy Trap: Why Multi-Step AI Agents Fail

By Izzy Aly, Co-Founder, Systems and Architecture

A 95% per-step accuracy means your 10-step AI agent fails 40% of the time. The maths behind cascading errors, and how to fix agent reliability.

That impressive 95% accuracy? It means your 10-step workflow succeeds only 60% of the time. Here's the maths nobody shows you in the keynote.

The agent demo was flawless. Ten steps, perfectly choreographed. The document was ingested, parsed, validated, cross-referenced, and submitted to the ERP system without a hiccup.

Then you deployed it to production.

Three weeks later, your finance team is drowning in failed transactions. Customer support is handling complaints about invoices processed with incorrect amounts. And somewhere in your billing dashboard, a number is climbing faster than it should.

This isn't just bad luck. It is a predictable mathematical certainty. Gartner predicts that more than 40% of agentic AI projects will be cancelled by the end of 2027, due to escalating costs, unclear business value or inadequate risk controls (Gartner, June 2025). Behind many of those cancellations sits a factor the press release doesn't name: the 95% Accuracy Trap.

The Maths That Breaks Agent Workflows

Here's the problem: when you chain probabilistic steps together, reliability doesn't scale linearly. It decays exponentially.

If each step in your agent workflow has a 95% accuracy rate, and you need 10 steps to complete the task, the probability of the entire workflow succeeding is:

0.95^10 = 59.9%

Your sophisticated orchestration system just became a coin flip.

Steps in Workflow95% Per-Step97% Per-Step99% Per-Step
577.4%85.9%95.1%
1059.9%73.7%90.4%
1546.3%63.3%86.0%
2035.8%54.4%81.8%
How whole-workflow success decays as steps are chained Line chart of the chance that a whole agent workflow succeeds, computed as per-step accuracy to the power of the number of steps. At 95% per step: 77.4% at 5 steps, 59.9% at 10, 46.3% at 15 and 35.8% at 20. At 97% per step: 85.9%, 73.7%, 63.3% and 54.4%. At 99% per step: 95.1%, 90.4%, 86.0% and 81.8%. A reference line marks 50%, a coin flip; the 95% line drops below it between 10 and 15 steps. The 10-step point at 95%, 59.9%, is called out. This assumes independent failures. CHANCE THE WHOLE WORKFLOW SUCCEEDS, BY PER-STEP ACCURACY 100% 75% 50% 25% coin flip 59.9% at 10 steps 99% per step 97% per step 95% per step 5101520 steps in the workflow Per-step accuracy to the power of the step count. Assumes independent failures.
How whole-workflow success decays as steps are chained Line chart of the chance that a whole agent workflow succeeds, computed as per-step accuracy to the power of the number of steps. At 95% per step: 77.4% at 5 steps, 59.9% at 10, 46.3% at 15 and 35.8% at 20. At 97% per step: 85.9%, 73.7%, 63.3% and 54.4%. At 99% per step: 95.1%, 90.4%, 86.0% and 81.8%. A reference line marks 50%, a coin flip; the 95% line drops below it between 10 and 15 steps. The 10-step point at 95%, 59.9%, is called out. This assumes independent failures. WORKFLOW SUCCESS, BY STEP ACCURACY 100% 75% 50% 25% coin flip 59.9% at 10 99% 97% 95% 5101520 Steps in the workflow. Success = per-step accuracy to the power of the step count. Assumes independent failures.
Fig. 1 — Whole-workflow success is per-step accuracy to the power of the step count, assuming independent failures. At 95% per step, a 10-step run succeeds 59.9% of the time.

At 95% per-step accuracy, which many teams would celebrate for a single LLM call, you drop below a coin flip at 15 steps. Even if you push that to 99% per step, a 20-step workflow still yields only 81.8% overall success.

Why the Real Numbers Are Worse

Crucially, this maths assumes independent failures. In reality, errors often cascade; a mistake at step 3 corrupts the context for steps 4 through 10. This correlation makes the real-world failure rates significantly higher than the table predicts.

The key insight: a 4-percentage-point improvement in per-step accuracy (from 95% to 99%) takes your 10-step success rate from 60% to 90%. This is why per-step monitoring isn't a nice-to-have. It's the highest-leverage investment you can make in agent reliability.

The Hallucination Cascade: When Errors Compound

The maths above assumes a binary pass/fail grading scale. Reality is more complex.

Research analysing LLM agent failure trajectories across benchmark tasks (such as ALFWorld and WebShop) traces task failures to cascading errors, in which a single root-cause mistake propagates through every downstream decision (Zhu et al., "Where LLM Agents Fail and How They Can Learn From Failures", 2025).

Here's how it plays out:

  • Step 3: The agent misclassifies a "Pro Forma Invoice" as a "Standard Invoice." Not fatal on its own.
  • Step 7: Because the agent believes it's processing a standard invoice, it looks for a Purchase Order number that doesn't exist.
  • Step 8: To resolve the missing PO, the agent hallucinates a PO number based on patterns from its training data.
  • Step 10: The agent successfully submits a valid-looking but fraudulent transaction to your ERP system.

The workflow is technically completed. All steps returned 200 OK. No exceptions were thrown. But the agent confidently executed a logical failure by submitting fraudulent data that appeared valid to downstream systems.

The hallucination cascade: one wrong step, four steps of wrong data The invoice example as a step flow. Step 3: the agent misclassifies a Pro Forma Invoice as a Standard Invoice. Step 7: it looks for a Purchase Order number that doesn't exist. Step 8: it hallucinates a PO number from patterns in its training data. Step 10: it submits a valid-looking but fraudulent transaction to the ERP. Every step returns 200 OK, but the data is wrong from step 3 on. No exception is thrown, so the failure stays quiet until a human auditor catches the discrepancy weeks later. ONE MISCLASSIFICATION CARRIES THROUGH THE RUN 3 7 8 10 Misclassifies Pro Forma read as Standard Invoice Looks for a PO a PO number that doesn't exist Hallucinates a PO number from training patterns Submits to ERP a valid-looking, fraudulent entry 200 OK200 OK200 OK200 OK Data wrong from step 3 on every later step builds on it Quiet failure: no exception is thrown and the ERP accepts it. A human auditor catches the discrepancy weeks later.
The hallucination cascade: one wrong step, four steps of wrong data The invoice example as a step flow. Step 3: the agent misclassifies a Pro Forma Invoice as a Standard Invoice. Step 7: it looks for a Purchase Order number that doesn't exist. Step 8: it hallucinates a PO number from patterns in its training data. Step 10: it submits a valid-looking but fraudulent transaction to the ERP. Every step returns 200 OK, but the data is wrong from step 3 on. No exception is thrown, so the failure stays quiet until a human auditor catches the discrepancy weeks later. ONE ERROR CARRIES THROUGH 3 7 8 10 Misclassifies Pro Forma read as Standard Looks for a PO a PO number that doesn't exist Hallucinates a PO number from training patterns Submits to ERP a valid-looking, fraudulent entry 200 OK200 OK200 OK200 OK Data wrong from step 3 on No exception is thrown. A human auditor catches the discrepancy weeks later.
Fig. 2 — One misclassification at step 3 carries through steps 7, 8 and 10. Every step returns 200 OK, so nothing flags the failure until an auditor finds it.

This is the most dangerous risk in agent deployment: quiet failures. Without step-level visibility, these errors are undetectable until a human auditor catches the discrepancy weeks later.

Why Traditional Monitoring Fails

If you come from a DevOps background, your instinct is to monitor agents like microservices: uptime, latency, error rates, and request tracing. This approach fails for three reasons:

  1. Agents fail silently. A traditional service either returns a valid response or throws an error. An agent step can return a perfectly formatted, confidently stated, completely wrong answer. Your HTTP status codes are all 200, but your agent just hallucinated a compliance requirement that doesn't exist.
  2. Non-determinism makes reproduction impossible. The same input to the same agent can produce different outputs depending on model temperature and inference randomness. You can't replay a failure by simply rerunning the request; you need the full trace captured at the moment of execution.
  3. Failures are distributed across time. In traditional software, a bug manifests when the code is broken. In an agent system, a prompt drift at step 2 might not manifest as a visible failure until step 8. Without step-level quality scoring, you're debugging a 10-variable equation with one data point: the final output.

The $47,000 Wake-Up Call

The reliability problem is bad. The economic problem is worse.

In an incident one of its engineers wrote up for Towards AI, a multi-agent system of four LangChain agents ran up a $47,000 API bill in production. Two of its agents got stuck in an infinite conversation loop and talked to each other for 11 days before anyone noticed (Towards AI, October 2025).

Nothing stopped the loop and nothing flagged the spend: the weekly API cost climbed from $127 to $18,400 while the team read the rise as usage growth and believed it was 'just running smoothly'. It was recursive agent-to-agent calls consuming tokens in a loop.

This isn't an edge case. It is the predictable result of deploying autonomous systems without economic guardrails. Unlike a human employee who stops when confused, agents retry by default. If stuck in an unsatisfiable condition, your token meter keeps spinning.

When This Problem Hits Hardest

The 95% accuracy trap bites hardest in specific scenarios:

  • Complex workflows (10+ steps): The maths is unforgiving. Every additional step compounds the failure probability.
  • Autonomous decision-making: If the agent can take actions without human checkpoints, errors propagate unchecked.
  • Retry-heavy architectures: Aggressive retry strategies common in many agent frameworks can amplify both failure cascades and runaway costs if not paired with circuit breakers.
  • Distributed context: When the "state" exists across prompt history, scratchpad reasoning, and tool outputs, debugging becomes forensic reconstruction.

This doesn't mean agents are broken. Single-step classification, summarisation, or Q&A tasks with human review work fine at 95% accuracy. The trap bites when you automate multi-step decisions without checkpoints. The question isn't "Should we use agents?" It's "Where do we need step-level validation?"

The EU Compliance Dimension

If your agent makes decisions in a high-risk area such as recruitment or credit, the EU AI Act requires automatic logging (Article 12) and human oversight (Article 14), and since the Digital Omnibus those Annex III obligations apply from 2 December 2027 (Regulation (EU) 2026/1744). An agent workflow without audit trails is a compliance failure waiting to happen.

How to Beat the 95% Trap

The error-compounding problem isn't an argument against agents. It's an argument for observability. Here's what breaks the trap:

  1. Distributed Tracing with Step-Level Granularity: Each agent execution requires a full trace that captures the input, prompt, raw completion, tool calls, and output. When a 10-step workflow fails, you walk backward to find where the chain diverged.
  2. Step-Level Quality Scoring: Tracing tells you what happened. Quality scoring tells you if it was good. Each step should have automated evaluators that score factual accuracy and format compliance to catch errors before they cascade.
  3. Circuit Breakers: Monitor each step's failure rate and automatically halt execution when it crosses a threshold. This prevents a single degrading step from corrupting the entire pipeline or wasting tokens on a doomed task.
  4. Cost Caps and Budget Guardrails:Implement per-step token limits, per-execution budget ceilings, and daily aggregate caps. When a limit is hit, alert and terminate gracefully to prevent runaway billing.
Four guardrails on a multi-step agent run An agent run of steps 1 to n, with four guardrails. One, distributed tracing runs under every step and captures the input, prompt, raw completion, tool calls and output. Two, a quality score after each step checks factual accuracy and format compliance. Three, a circuit breaker halts the run when a step's failure rate crosses a threshold. Four, cost caps at three levels: a token limit per step, a budget ceiling per execution and a daily aggregate cap. On any breach, the system alerts and terminates gracefully. FOUR GUARDRAILS ON A MULTI-STEP AGENT RUN Agent run Step 1Step 2Step 3Step n 01 Tracing Input, prompt, raw completion, tool calls, output 02 Quality score After each step: factual accuracy, format compliance 03 Circuit breaker A step's failure rate crosses the threshold Halt the run 04 Cost caps Per stepPer executionDaily token limitbudget ceilingaggregate cap On any breach: alert, then terminate gracefully.
Four guardrails on a multi-step agent run An agent run of steps 1 to n, with four guardrails. One, distributed tracing runs under every step and captures the input, prompt, raw completion, tool calls and output. Two, a quality score after each step checks factual accuracy and format compliance. Three, a circuit breaker halts the run when a step's failure rate crosses a threshold. Four, cost caps at three levels: a token limit per step, a budget ceiling per execution and a daily aggregate cap. On any breach, the system alerts and terminates gracefully. FOUR GUARDRAILS ON ONE RUN 123n 01 Tracing full trace, every step 02 Quality score each step 03 Circuit breaker halts the run 04 Cost caps per step · per execution · daily aggregate On any breach: alert, then terminate gracefully.
Fig. 3 — Where the four fixes sit on one run: tracing under every step, a score after each step, a breaker on failure rate and cost caps at three levels.

Reaching 99% per step isn't magic. It's better prompt engineering, structured outputs with schema validation, and automated evals that catch drift before it compounds.

Observability as the Foundation

Most teams discover the observability gap the hard way: after the first production incident. The smarter play is to build visibility before you scale.

Visibility drives improvement. Tracing enables debugging. Auditing ensures compliance.

Push per-step accuracy from 95% to 99% through better prompts and validation, and your 10-step workflow goes from 60% to 90% success. Add circuit breakers, quality gates, and cost caps, and you catch the remaining 10% before it hits users or your invoice.

A 10-step agent workflow has 10 potential failure points, 10 prompts that could drift, and 10 decision points that need audit trails. Without observability, you're flying blind. And the maths guarantees you'll crash.

Topics
Author
Izzy Aly · Co-Founder, Systems and Architecture
Share

Keep reading

All posts →