What Actually Breaks: Field Notes on Enterprise AI Agents
Reliable AI agents fail on plumbing, not intelligence. The real failure modes, the reliability stack that fixes them, and a 10-item production checklist.
Note: Statistics and figures reflect data available as of August 2026. Verify for latest figures.
Take an agent that gets every individual step right 95% of the time. That's a genuinely good agent. Now give it a twenty-step workflow: pull the ticket, look up the customer, check entitlement, query three systems, reconcile the answers, draft a response, post it.
0.95 to the twentieth power is 0.358. Your excellent agent finishes the whole job correctly 36% of the time.
Nobody put that number in the demo. The demo was one happy path, run by the person who built it, on data they chose. It worked. Then it hit production and started failing in ways that had nothing to do with how smart the model was.
I've watched enough of these deployments to be blunt: the gap between an agent demo and an agent in production is reliability engineering, not model capability. The failure modes are known, mostly boring, and long solved in every other branch of distributed systems. We keep pretending they don't apply because there's a language model in the loop.
The cliff is real, and so is the other side of it
I covered the why of stalled agent pilots in the pilot-to-production piece. This is the how.
The headline number everyone quotes is Gartner's: more than 40% of agentic AI projects will be canceled by end-2027, driven by escalating costs, unclear business value, or inadequate risk controls. Read that list again. Not one of those causes is "the model wasn't good enough."
But the cynical read is also wrong. Agents work at scale for companies that treated them as an engineering problem. Salesforce reported 2.4 billion Agentic Work Units across Agentforce and Slack in its Q4 FY26 results, up 57% quarter-on-quarter. Microsoft said on its Q4 FY26 call that Agent 365, its agent control plane, had nearly 40 million agents registered across tens of thousands of companies within roughly two months of launch. ServiceNow crossed $1 billion in AI annual contract value, per its 2026 earnings commentary.
So the technology ships. The distribution of outcomes is brutally bimodal, and what separates the halves is unglamorous engineering discipline.
A taxonomy of what actually breaks
There's a useful academic version of this. "Beyond the Leaderboard," a July 2026 synthesis by Albayaydh, Zhao and Flechais, reviewed 27 papers across 19 benchmarks and grouped agent failures into six clusters: tool invocation errors, planning failures, long-horizon degradation, multi-agent coordination failures, safety and security failures, and measurement validity problems. Its sharpest line is that strong performance on individual sub-tasks does not reliably translate into end-to-end success.
My field version collapses to four.
1. Tool-call errors and silent API failures
The model calls your API with a malformed parameter. Or the right parameter and a stale ID. Or correctly, and the API returns a 200 with an empty payload because an upstream cache expired.
The model doesn't know any of this went wrong. It reads whatever came back, treats it as ground truth, and confidently continues. Silent failure is worse than a crash: a crash gets logged and paged, while a silent failure gets summarized into a plausible-sounding answer and emailed to a customer.
2. Context and memory drift
On a long task, the agent's working context fills with its own intermediate output. Twenty tool calls in, the original instruction is buried under a pile of JSON the agent generated, and it starts optimizing for the wrong objective. It isn't hallucinating exactly. It has lost the plot the way a tired person does at hour nine.
3. Compounding error rates
This is the 36% from the opening. Toby Ord's paper "Is there a half-life for the success rates of AI agents?" models agents as having a roughly constant hazard rate of failure per minute of equivalent human task time, producing exponentially declining success as tasks get longer. Longer tasks contain more subtasks, and failing any one fails the whole thing.
METR's time-horizon research makes the same point empirically, and the underappreciated detail is the gap between reliability thresholds. In METR's original work, 80% time horizons ran roughly 5x shorter than 50% horizons: Claude 3.7 Sonnet managed about 59 minutes of human-equivalent task at 50% success, but only about 15 minutes at 80%. Frontier horizons have climbed into the hours at 50% by METR's May 2026 update, but the shape holds. The distance between "works half the time" and "works reliably" is most of the problem.
Which is why the per-step number matters. At 95% per step, a 20-step chain lands at 36%. At 99%, it hits 82%. At 99.9%, it's 98%. You don't get to production by making the agent smarter. You get there by adding nines to each step.
4. Permission over-scope and nobody owning it
Gravitee's State of AI Agent Security 2026, a survey of 750 senior technology leaders across the UK and US in April 2026, found 54% of organizations had already suffered a security incident related to AI agents in the preceding twelve months. Over-privileged access, agents granted broader system reach than their function required, was the study's most consistently reported failure.
The number that actually stopped me, though: only 7.2% had a named individual with formal accountability for AI agent behaviour, and only about a fifth said all their agents were fully secured and governed before going live.
That's the real failure mode. An agent with production credentials and no owner isn't an AI problem. It's an unattended cron job with a budget and an imagination.
The reliability stack
None of the fixes are exotic. They're the same patterns we'd apply to any unreliable distributed dependency, which is exactly what a model is.
Retries and idempotent tools. Every tool should be safe to call twice. Give writes an idempotency key, so "create ticket" returns the existing ticket rather than opening a duplicate. Then retry aggressively, because a retried transient failure is the cheapest nine you'll ever buy. And make failures loud: a tool returning an empty result must say so in a form the model cannot mistake for success.
Checkpointing and approval gates. Break long chains into segments with durable state between them, so a failure at step 14 resumes from step 13 instead of restarting. Then gate anything irreversible or expensive: refunds, external emails, production writes, anything touching money or a customer. The gate isn't weakness. It's how you take a 36% workflow and make its worst outcome "a human said no" instead of "we refunded 4,000 people."
Evals as regression tests. This is the discipline gap I see most often. Teams have a test suite for their code and nothing but vibes for their agent. Build a fixed scenario set with known-good outcomes, run it on every prompt, model and tool change, and gate deploys on it. A model version bump is a dependency upgrade. Treat it like one.
Observability and tracing. Log every tool call: inputs, outputs, latency, retries, token spend, and the decision behind it. When an agent does something bizarre at 3am, you need the trace, not the final answer. Per-step rates are also the only way to find which step is dragging your chain math down, and it's never the one you'd guess.
Protocols and plumbing
A large share of agent fragility is integration fragility, which is why the standards actually matter.
MCP has effectively won the tool-access layer. When Anthropic donated it to the Linux Foundation's new Agentic AI Foundation in December 2025, it cited more than 10,000 active public MCP servers and 97 million-plus monthly SDK downloads across Python and TypeScript, with adoption spanning ChatGPT, Gemini, Microsoft Copilot, Cursor and VS Code. The foundation was co-founded with Block and OpenAI, backed by Google, Microsoft, AWS, Cloudflare and Bloomberg. I unpacked what MCP is in the USB-C piece; the short version is that it turns N-times-M custom integrations into N-plus-M.
On the agent-to-agent side, A2A passed its one-year mark in April 2026 with more than 150 supporting organizations, integrations across Google, Microsoft and AWS platforms, and a v1.0 stable specification adding multi-protocol support, enterprise multi-tenancy and modernized security flows, per the Linux Foundation.
The reliability argument for both is simple: a standard interface is a testable interface. Bespoke glue code between every agent and every system is where undocumented behaviour and unversioned breakage live. Standards don't make your agent smarter. They shrink the surface where it can silently break.
The fair caveat: neither protocol solves semantics. Both move bytes reliably; neither guarantees the agent on the other end read your intent correctly. A 2026 arXiv analysis of governance gaps in MCP, A2A and ACP makes exactly this point, that these protocols can't currently express delegated authority limits or accountability chains. Plumbing is necessary, not sufficient.
Governance before scale, not after
Four things, and none of them require a committee.
Scoped credentials. Each agent gets its own identity with least-privilege access to only the systems its job requires. Not a shared service account. Not the developer's token. Given that over-privileging topped the Gravitee survey, this is the highest-leverage control on the list.
Audit trails. Immutable, queryable, retained. "Which agent changed this record, on whose behalf, and why" must be answerable in minutes.
Kill switches. One control that halts a class of agents immediately, tested before you need it. Microsoft building an entire control plane and registering 40 million agents into it is the market admitting fleet-level control is a requirement, not a nice-to-have.
A named owner. Per agent. On the org chart. See the 7.2% figure above.
On regulation, the timeline moved and you should know how. The EU's Digital Omnibus was endorsed by Parliament on 16 June 2026, approved by Council on 29 June, and entered into force on 27 July 2026. It defers high-risk obligations for standalone Annex III systems to 2 December 2027, and to 2 August 2028 for AI embedded in regulated products. But Article 50 transparency obligations largely held their 2 August 2026 date, so if your agent talks to EU users or generates synthetic media, nothing moved for you (details here). The deferral bought engineering time, not a pass.
The counterargument: "just wait for smarter models"
This pushback isn't stupid. METR's data genuinely shows frontier time horizons roughly doubling every seven months, with recent updates suggesting a faster pace. If that continues, a chunk of today's per-step error rate will evaporate on its own, and some scaffolding we're building now is throwaway. I concede that partially: capability growth will fix some of categories one and two.
But look at what actually kills these projects. Gartner's cited causes are cost, unclear value and inadequate risk controls. Gravitee's top finding is over-privileging, its most damning that almost nobody has a named owner. A smarter model does not scope your credentials, give your agent an owner, write your audit trail, tell you what the workflow is worth, or fix the upstream API that returns 200 on failure.
There's also a compounding trap in waiting. Even at 99.9% per step you're only at 98% across twenty steps, and the tasks we hand agents grow in ambition at least as fast as the models improve. The chain lengthens as the steps get better.
Waiting is a strategy for the capability half of the problem. It's no strategy at all for the ownership half.
The production-readiness checklist
Copy this. It's ten questions, and if you can't answer all of them with a yes, you have a demo.
- Named owner. One person, on the org chart, accountable for this agent's production behaviour?
- Scoped identity. Its own credential with least-privilege access, not a shared or human account?
- Idempotent tools. Every tool safe to invoke twice without duplicating an effect?
- Loud failures. Every tool returns an unambiguous error the model can't mistake for a valid empty result?
- Eval suite. A fixed scenario set with known-good outcomes gating every prompt, model and tool change?
- Measured per-step reliability. You know your per-step success rate and chain length, and have multiplied them out?
- Checkpoints. A failed long-running task resumes from its last good state instead of restarting?
- Approval gates. Every irreversible, financial or customer-facing action passes a human or hard policy check?
- Full-trace observability. Every tool call logged with inputs, outputs, latency, retries and cost, any run reconstructable?
- Tested kill switch. You can stop this agent and every agent like it in under a minute, and have actually tried?
Items 1, 9 and 10 are usually the last ones anybody builds, and they're the three that decide whether your first serious incident is a bad afternoon or a board conversation.
Agent reliability isn't a frontier research problem. It's the operational discipline we already apply to databases, payment gateways and deploy pipelines, pointed at a component that happens to be probabilistic. The teams shipping agents that work aren't the ones with the best model access. They're the ones who got bored enough to add the nines.
Related Reading
Enjoying this article?
Get posts like this in your inbox. No spam, unsubscribe anytime.
