Operational Resilience for Agents — Failure Modes and Recovery

COMPEL Specialization — AITE-ATS: Agentic AI Systems Architect Expert Article 16 of 40

Thesis. A web service fails when it goes down or when it returns wrong answers. An agent fails in those ways too, but it also fails in ways web services cannot: it loops without progress, it reasons into circles, it mis-interprets tool output, it carries a corrupted memory into the next session, it cascades refusals through a multi-agent graph until the entire workflow halts. Classical resilience patterns — retries, timeouts, circuit breakers, bulkheads — still apply, but they must be adapted to the agentic context, and new patterns are needed for the agentic-specific failures. This article catalogs eight agent failure modes, specifies three circuit-breaker patterns adapted for agents, and walks the recovery workflow from detection through post-mortem.

Eight agent failure modes

Failure 1 — Infinite loop / no-progress

The agent’s loop iterates without converging. Same thought repeated; same tool called with slight variations; no state change. Budget burns; wall-clock elapses; tool quotas consume. The Article 4 failure mode for ReAct, but all loop patterns can exhibit it.

Detection: step counter, no-progress detector (hash of recent thoughts/actions), wall-clock watchdog.

Mitigation: max_steps cap, budget cap (Article 9), restart policy, kill-switch escalation.

Failure 2 — Tool timeout cascade

One slow tool call delays the loop; the loop retries; retries compound. In multi-agent systems, a slow worker delays all agents in a synchronous chain. Classic distributed-systems failure with agentic amplification.

Detection: per-tool latency alerting, cascade-depth monitoring.

Mitigation: per-tool timeouts set lower than the session’s overall budget, asynchronous tool calls where possible, circuit breakers per tool.

Failure 3 — Model refusal cascade

The model refuses a legitimate request (safety filter too strict, prompt looks adversarial). The agent retries with slight variations; each retry burns tokens. Eventually the agent concludes the task is impossible and halts — but the task was possible, the model was mis-aligned with the operational profile.

Detection: refusal-rate anomaly per agent or per task class, trace analysis of refusal reasons.

Mitigation: refusal-classification diagnostics (is this a safety refusal, an over-cautious refusal, or a legitimate capability limit?), operational-safety prompts that clarify context, fallback to a different model or path, human handoff.

Failure 4 — Context-window exhaustion

Multi-step loops accumulate context. Eventually the context hits the model’s window limit. The model either truncates (losing the beginning, often the system prompt), errors out, or reduces quality. Classical OOM analog, but with silent quality degradation that is harder to detect.

Detection: context-size monitoring per loop iteration, OOM-style alerting at 80% of window.

Mitigation: summarization at configurable thresholds, sliding-window context policies, externalized state (Article 15 replayability stores full history outside the context), model upgrade to larger context if justified.

Failure 5 — Memory corruption

Memory write with bad data (hallucination, poisoning, bug) persists. Future sessions retrieve the bad memory as authoritative. The failure compounds session-over-session until detected.

Detection: memory-integrity checks (provenance present, classification valid), poisoning red-team batteries (Article 17), user-complaint signal correlated to memory contents.

Mitigation: write gating (Article 7), TTL on low-trust memories, rollback to a prior memory snapshot, tombstone corrupted entries with blocked-retrieval flag.

Failure 6 — Partial side effect

An action began but did not complete. Payment initiated but not recorded. Email sent but not logged. Database row inserted but audit trail not written. Agent interprets the state as “failed” and retries, producing double effect.

Detection: post-action reconciliation, idempotency-key collision detection, external-system state polling.

Mitigation: idempotency keys on every side-effecting tool (Article 5), compensation actions (sagas), reconciliation workflows, HITL escalation for ambiguous states.

Failure 7 — Downstream system overload

Agent-driven volume overwhelms downstream — the CRM can’t keep up with the agent’s rate, the payment processor rate-limits, the email server queues. Classical rate-limiting failure, but the agent may not understand backpressure signals.

Detection: downstream latency and error monitoring, agent-caused volume attribution.

Mitigation: rate limits at agent’s tool layer, circuit breaker on downstream errors, backpressure-aware retry policy, queuing with replay.

Failure 8 — Coordination deadlock or livelock (multi-agent)

Agents wait for each other indefinitely, or pass work back and forth without progress. Article 12 failure modes.

Detection: per-message age, cycle detection in delegation graph, task-age alerting.

Mitigation: timeouts on inter-agent waits, cycle-breaking policy (break after N bounces), escalation to supervisor, fallback to single-agent mode.