Agent Systems · 21-c07
26 June 2026 · Production notes
An agent does not answer once and fall silent like a chatbot. It carries state, calls tools, hands work off to other agents, and talks to the outside world across long sessions. That is exactly why it breaks differently: it loops forever, spams tools, drifts off goal, lets its information go stale, or quietly leaves the job half done while reporting 'success'. This post walks through the five failure modes you meet most in production, and how to detect and prevent each one.
- Axis
- breaking in production
- Question
- where does it break?
- Lever
- budget + state
- Risk
- silent failure
Concepts in this post
- infinite loop
- tool spam
- goal drift
- stale state
- silent partial failure
- budget
- termination
Earlier posts defined an agent through goal, state, tools, loops, and termination, and flagged budgets (step/token/tool limits) and state management as the foundations of reliability. This post returns to those foundations, but in reverse: when the pieces don't hold, in what shapes does the system break?
These five modes are not an academic list; they are concrete faults that come back as a bill, an outage, or a wrong decision. For each one we answer the same four questions: what it looks like (symptom), why it happens (root cause), how you detect it, and how you mitigate it. Most of the mitigations are not a new model; they are a budget, a guardrail, and a reviewable state put in the right place.
Why agents break differently
A plain chatbot produces one answer and stops; if it errs, the error lives in that single visible answer. An agent lives inside a loop: it observes the environment, decides, acts through a tool, reads the result, and starts over. Because that loop carries state, mutates the world, and runs long, it opens new failure surfaces.
The Microsoft AI Red Team's failure taxonomy for agentic systems says exactly this: agents create or amplify the failure surface because they persist state, call tools, delegate to other agents, and consume untrusted content across long sessions. A small deviation in one step carries into later steps and compounds.
That is why the five modes here are linked. Infinite loops and tool spam are resource-exhaustion faults; goal drift and stale state are the quiet slipping of reasoning; and silent partial failure is the shared consequence of all of them: the job ends up half done, but the system doesn't say so.
Each turn updates state; a deviation in one turn carries into the next.
five faults riding on the loop
infinite loop
the loop never ends
tool spam
tools get flooded
goal drift
drifts off goal
stale state
information goes stale
silent partial failure
half done, says 'success'
Infinite loops
Infinite loop: the agent repeats the same step without making new progress and never reaches its stop condition (termination). The MAST study names two distinct faults here: 'step repetition' (needless re-doing of an already completed step) and 'unaware of termination conditions'.
Why it happens: the agent doesn't measure progress. Without asking 'did that last step move me closer to the goal?', it just produces the next move. Sometimes the root cause is adversarial: the Breaking Agents study shows prompt injection can trap an agent into repeating a command until maximum iterations are reached. The reported result is striking: a 15.3% baseline malfunction rate rises to 59.4% under attack.
How you detect it: look at the signature of consecutive steps. Same tool, same arguments, same observation in a row means no progress. How you mitigate it: a single hard limit is not enough. An iteration budget (how many turns), a semantic progress check (did new information arrive), and loop detection (are the last N steps a repeat) must work together. The shared point in MAST and Breaking Agents: an agent reviewing itself (self-reflection) is not enough to catch this mode.
search("error code 500")search("error code 500")search("error code 500")search("error code 500")search("error code 500")search("error code 500")Tool spam
Tool spam: the agent floods tool calls without doing real work. It is a close cousin of the infinite loop, but the emphasis differs: here the problem is not just repetition but volume. Every tool call means latency, money, and load on an external system. The Microsoft taxonomy groups this under denial of service and resource-exhaustion effects.
Why it happens: a tool call has no cost to the agent, so there is no 'measure first, call second' reflex either. A badly designed tool, an underspecified task, or contradictory observations push the agent into a 'let me try once more' loop. Breaking Agents shows this amplifies in multi-agent settings: a single manipulated agent can drag downstream agents into needless calls and wasted resources.
How you detect it: track the tool-call rate (calls per unit time or per step) and the error rate; a sudden spike heralds spam. How you mitigate it: put a budget on tool calls too. A per-step and per-session call cap, exponential backoff on repeated failures, and a tool-selection discipline that avoids calling unless truly needed. In the animation below, calls pile up toward a cap; once the cap is full, the system rejects the next call.
search() acceptedfetch() acceptedsearch() acceptedfetch() acceptedsearch() acceptedfetch() cap full · rejectedsearch() cap full · rejectedGoal drift
Goal drift: the agent slowly moves away from the goal it was originally given. It doesn't make one wrong decision; across long context, under competing pressures and accumulated history, the goal vector shifts. Arike et al.'s goal-drift report defines it cleanly: an agent's tendency to deviate from its assigned goal over time, especially under long context and conflicting objectives.
Why it happens: the report finds that as context grows, the model becomes more susceptible to pattern-matching behaviour. The agent starts conforming to the latest pattern in context instead of following the original instruction. The Microsoft taxonomy's 'goal hijacking' and 'session context contamination' are the security side of this: an external input quietly redirects the objective while the agent keeps looking productive.
The good news is in the report too: the best scaffolded Claude 3.5 Sonnet maintained near-perfect goal adherence beyond 100,000 tokens in the hardest setting, yet all evaluated models showed some drift. How you detect it: measure the alignment of intermediate steps to the goal, not just the final answer. How you mitigate it: strong goal elicitation (re-state the goal periodically), bound the context, and treat external content as untrusted. Below, the original goal is a fixed target; the agent's effective objective drifts away from it as context grows.
a periodic reminder pulls alignment back
Stale state
Stale state: the agent assumes its information is still valid, while the world has moved on. It has two faces. The first is lost progress: without persistence, an interruption wipes out the work the agent did. The second is the opposite: over-trusted durable state carries stale or contaminated information across runs.
LangGraph's persistence docs describe the cure for the first face: a checkpointer holds thread-scoped graph-state snapshots, while a store keeps long-lived data, so an agent can continue a conversation, resume after interruption, and recover from failure. But the same persistence creates the second face: wrong or stale information written once can leak into later runs. AgentDojo measures this through state: tools read and mutate environment state, and correctness is judged against pre/post state, not message content.
How you detect it: compare the age of the snapshot the agent relies on (when it was read) with the world's actual value; if the two diverge, the state is stale. How you mitigate it: give state an explicit model. Timestamp snapshots, re-read before critical decisions, attach provenance (where and when did this come from) to durable memory, and keep writes reviewable. Below, the agent takes a snapshot; while the counter shows the world changing, the snapshot freezes in place.
now
read at: t0
re-read before a critical decision
Silent partial failure
Silent partial failure: part of the job fails quietly, but the agent reports the result as 'success'. This is the most insidious mode, because the final output looks plausible; the fault is in an intermediate step. MAST names this 'premature termination' and 'no/incomplete verification': the system thinks it is done before it is, or never verifies the output at all.
Why it happens: the agent often focuses only on producing the final message, not on verifying that intermediate steps actually ran. ToolEmu therefore takes the object of evaluation to be the full trajectory (the sequence of steps), not the final answer; in their reported evaluation even the safest agent produced failures in 23.9% of test cases. OpenAI's agent-eval docs bring the same logic to production: a trace captures an end-to-end record of model calls, tool calls, guardrails, and handoffs for one run.
How you detect it: look at the trace, not the final text. A wrong tool choice, a skipped handoff, a violated guardrail, or a half-finished branch is only visible when you inspect the sequence of steps. How you mitigate it: capture traces, grade the trajectory with graders (automatic scorers), and once you understand 'good' behaviour, turn it into repeatable datasets and eval runs. In the pipeline below, two branches succeed and one silently drops; the aggregator still says 'success', until branch-level verification catches it.
branch 1
fetch data
ok
branch 2
transform
silently dropped
branch 3
write
ok
Control backbone: budget + state
These five modes look scattered, but they hang on a single control backbone: budget and state. A budget bounds the resource-consuming modes (infinite loops and tool spam); visibility into state and the trajectory catches the modes where reasoning quietly slips (goal drift, stale state, silent partial failure).
One defensive layer is not enough. The shared lesson of Breaking Agents and MAST: an agent reviewing itself cannot find these faults alone; you need an external layer that watches the behaviour. ToolEmu, AgentDojo, and the OpenAI eval docs make that layer concrete: grade the trajectory (not just the final answer), look at state mutations, treat external content as untrusted, and catch regressions with regular eval runs.
The practical summary is below: each failure mode with its symptom, detection signal, and mitigation in one table. None of them needs a new model; all are solved by a budget, a guardrail, and a reviewable state put in the right place.
Sources
A good agent knows within what bounds to use the freedom it was given. Those bounds are not one bright idea; they are the sum of a budget in the right place, a guardrail that watches from outside, and a reviewable state.
So before an agent ships, don't only ask 'can it give the right answer'; ask 'when it goes wrong, who notices, how, and when'. Silent failure is the most expensive failure.
- S1Microsoft AI Red Team, Taxonomy of Failure Modes in Agentic AI Systems v2.0, 2026
- S2Cemri et al., Why Do Multi-Agent LLM Systems Fail?, NeurIPS 2025 (arXiv:2503.13657)
- S3Zhang et al., Breaking Agents: Compromising Autonomous LLM Agents Through Malfunction Amplification, 2024 (arXiv:2407.20859)
- S4Ruan et al., Identifying the Risks of LM Agents with an LM-Emulated Sandbox (ToolEmu), ICLR 2024 (arXiv:2309.15817)
- S5Debenedetti et al., AgentDojo: Evaluating Prompt Injection Attacks and Defenses for LLM Agents, NeurIPS 2024 (arXiv:2406.13352)
- S6Arike et al., Evaluating Goal Drift in Language Model Agents, 2025 (arXiv:2505.02709)
- S7LangChain, LangGraph Persistence (docs, accessed 2026-06-26)
- S8OpenAI, Evaluate Agent Workflows (docs, accessed 2026-06-26)
