AI Agent Reliability: State, Tool Calls, Tracing and Recovery

An AI agent is an LLM that takes multiple steps, calls tools and maintains state across those steps. Reliability for an agent is harder than for a single LLM call because the failure surface is larger: each step can fail, each tool call can error, the state can be lost and the model can loop. A reliable agent is one that completes the task correctly, or fails in a way that can be recovered — not one that loops forever, silently corrupts state or reports success when a tool failed. The LangChain State of Agent Engineering survey found that agent reliability in production is the top concern among practitioners, ahead of cost and latency.

The reliability properties of an agent are: durability (state survives crashes and restarts), tool-call safety (tool calls are validated, retried and circuit-broken), loop control (the agent cannot run indefinitely), human oversight (irreversible actions require approval), observability (every step is traced) and recoverability (a failed workflow resumes from the last checkpoint, not from scratch). Each property is a design decision, not a default — an agent built without these properties will fail in production in ways that are hard to diagnose and harder to fix.

The distinction from single-call reliability is the state and loop dimensions. A single LLM call is stateless — if it fails, you retry the same call. An agent is stateful — if it fails at step 5 of 10, you cannot retry from step 1 without re-doing the first 4 steps, some of which may have had side effects (a tool that sent an email, a tool that wrote a database record). Durable state with checkpoints is the mechanism that makes recovery possible: the agent's state is persisted after each step, and a failed workflow resumes from the last checkpoint with the state intact.

Tool calls are the most common agent failure point. A tool call fails when the tool API is unavailable, when the tool input is malformed, when the tool returns an error, or when the tool returns data the model does not expect. Each failure mode has a different fix, and the diagnostic discipline is to log the tool call — the input, the output, the error and the model's response to it — so the team can see what happened. Without tool-call traces, the team sees only 'the agent failed' and cannot distinguish a tool outage from a model reasoning error.

The most dangerous tool-call failure is the silent contradiction: the tool returns an error, but the model reports success to the user. This happens when the model does not parse the tool's error response correctly, or when the model 'hallucinates' a success because its prior expectation was that the tool would succeed. The signal is a mismatch between the tool's response and the model's claim: the tool returned a 500 error, but the model told the user 'the record has been updated'. The fix is to validate the tool's response before the model processes it — check the status code, parse the error, and pass a structured error to the model that explicitly says 'the tool failed, do not claim success'.

Tool input validation prevents a class of failures before the tool is called. The model generates a tool call with arguments, but the arguments can be wrong — a missing required field, a wrong type, a value outside the valid range. If the tool is called with invalid input, it errors or behaves unexpectedly. The fix is to validate the tool input against a schema before the call: if the model generates `{ 'email': 'not-an-email' }`, the validation catches it and returns an error to the model before the tool is called. This turns a runtime tool error into a schema validation error, which the model can often correct on a retry.

Agent state is the information the agent needs to resume: the task, the steps completed, the tool calls made and their results, the model's reasoning and any intermediate outputs. Without durable state, a crash or restart loses everything and the agent starts from scratch — re-doing steps, re-calling tools and potentially repeating irreversible side effects. Durable state means the state is persisted to a store that survives process crashes: a database, a key-value store or a file system. The state is written after each step (a checkpoint), so a failed workflow resumes from the last checkpoint with all prior state intact.

The checkpoint design has two decisions: what to store and when to store it. What to store: the full state — task, steps, tool calls, results, reasoning — so the resumed agent has the same context as the original. When to store: after each step, and especially before irreversible actions. A checkpoint before an irreversible action means that if the action fails or the process crashes after the action, the agent can resume knowing the action was taken, rather than repeating it. The checkpoint must be atomic — the state is either fully written or not written at all — so a crash during the checkpoint does not produce a partially written, inconsistent state.

The storage choice affects recovery time and durability. An in-memory state store (the default if no store is configured) is fast but lost on crash. A database-backed store (PostgreSQL, DynamoDB, Redis with persistence) is durable and supports querying across agent runs. A file-based store (JSON files on disk) is simple but does not scale to concurrent agents. The tradeoff is latency vs durability: in-memory is fastest but unreliable; database-backed adds write latency but survives crashes. For production agents that take irreversible actions, durability is not optional — the cost of a lost checkpoint (re-doing irreversible actions) is higher than the cost of a database write per step.

Retries handle transient tool failures — a network blip, a rate limit, a temporary unavailability. The retry policy defines how many times to retry, the backoff between retries and when to give up. Exponential backoff with jitter is the standard pattern: wait 1 second, then 2, then 4, with a random jitter to avoid thundering herds when multiple agents retry simultaneously. The retry count must be bounded — unbounded retries are a resource leak and a cost driver. The timeout defines how long to wait for a single tool call before considering it failed; a tool call that hangs indefinitely blocks the agent and consumes resources.

Idempotency is the property that repeating a tool call has the same effect as calling it once. A tool that charges a card is not idempotent — calling it twice charges the card twice. A tool that reads a record is idempotent — calling it twice returns the same result. Retries are safe only for idempotent tools; retrying a non-idempotent tool can cause duplicate side effects. The fix for non-idempotent tools is an idempotency key: a unique identifier for the intended action, sent with the tool call, so the tool can deduplicate — if the same key arrives twice, the tool returns the result of the first call rather than executing again. If the tool does not support idempotency keys, the agent must track completed actions in its state and not retry actions that have already been executed.

The circuit breaker pattern stops calling a tool that is consistently failing. After a threshold of consecutive failures (e.g., 5), the circuit breaker opens and the agent stops calling the tool for a cooldown period (e.g., 60 seconds). During the cooldown, the agent falls back to a default response, an alternative tool or a human escalation. Without a circuit breaker, the agent retries a failing tool on every step, wasting time and resources and never making progress. The circuit breaker is reset after the cooldown, or when a health check shows the tool has recovered. The circuit breaker state is part of the agent's state and should be persisted in the checkpoint.

Human approval is the control that prevents an agent from taking irreversible or high-impact actions without oversight. The principle is simple: any action that cannot be undone — sending an email, charging a card, deleting a record, modifying a production system — requires a human to approve it before the agent executes it. The agent prepares the action, presents it to a human approver, and waits. If the human approves, the action executes; if the human rejects, the agent takes an alternative path or stops. The approval is logged with the action, the approver and the timestamp, creating an audit trail.

The approval gate has operational implications that must be designed for. The agent is now a long-running, asynchronous process — it waits for a human, which can take minutes, hours or days. The state must be durable (the agent survives the wait), the wait must be bounded (a timeout after which the agent escalates or cancels), and the approval interface must be usable (the approver sees the action, the context and the consequences clearly). A poorly designed approval gate is worse than no gate — the approver rubber-stamps because the interface does not give them enough information to make a real judgement, or the approvals queue up and block all agents.

The approval policy defines which actions require approval, who can approve and what the timeout is. The policy is a design decision based on the risk of the action and the cost of the delay. A low-risk action (sending an internal notification) may not require approval; a high-risk action (modifying a production database) always does. The approver is the person with authority over the action — the on-call engineer for a production change, the manager for a financial action. The timeout defines what happens if no one approves — the agent escalates to a fallback, cancels the task or retries with a different approach. The policy must be written down and reviewed, not improvised per incident.

A recursion limit is the maximum number of steps an agent can take before it is forced to stop. Without a recursion limit, an agent that cannot find a path to its goal loops indefinitely — calling the same tool, getting the same result, reasoning the same way, forever. Each iteration consumes tokens (cost), blocks resources (capacity) and produces no progress (value). The recursion limit is the safety net that turns an infinite loop into a bounded failure: the agent stops at the limit, the state is preserved at the last checkpoint, and the team can diagnose why the agent could not find a path.

The limit is set based on the expected task complexity — a simple task (retrieve and summarise) may need 5 steps; a complex task (research, call multiple tools, synthesise) may need 20. The limit should be generous enough to allow the agent to complete normal tasks but tight enough to catch runaway loops quickly. The signal that the limit is too low is that legitimate tasks are being cut off; the signal that it is too high is that runaway loops consume significant resources before being stopped. The limit is tuned over time based on the distribution of step counts for completed tasks.

Runaway detection goes beyond the step count. An agent that makes the same tool call with the same arguments more than once without a state change is in a loop — the recursion limit will eventually catch it, but the detection can be faster. The fix is to track tool-call signatures in the agent's state and detect repetition: if the same tool call is made N times with the same arguments and the state has not changed, the agent is in a loop and should be stopped. This is a complement to the recursion limit, not a replacement — the recursion limit is the hard cap, the repetition detection is the early warning.

Agent tracing is the practice of recording every step the agent takes: the reasoning, the tool calls, the tool inputs and outputs, the state transitions and the final result. A trace is the evidence that explains why the agent did what it did — without it, the team sees only the final output and cannot diagnose a failure. The trace should be structured and queryable, not a flat log: each step is a record with the step number, the action, the input, the output and the state, so the team can search for patterns (e.g., 'show me all runs where the agent called the same tool more than 3 times'). OpenTelemetry provides the instrumentation standard for distributed tracing and can be applied to agent workflows.

Agent evaluation is the practice of scoring the agent's behaviour against a quality rubric, not just the final output. Did the agent take the right steps? Did it call the right tools? Did it use the tool outputs correctly? Did it stop when it should have? An evaluation rubric for agents includes: task completion (did the agent achieve the goal), tool-call accuracy (did the agent call the right tools with the right inputs), step efficiency (did the agent take unnecessary steps), and safety (did the agent avoid irreversible actions without approval). The rubric is applied to traces, not just outputs — the trace is the evidence that the rubric scores.

The connection between tracing and evaluation is the regression suite. Production traces that show failures become evaluation cases for the next release: the query, the expected behaviour and the assertion that the agent does not repeat the failure. The suite is run on every change to the agent — a new tool, a new prompt, a new model — and catches regressions before they reach production. The suite should cover known failure cases (the runs that have failed before) plus a sample of normal runs (to detect regressions in runs that have not failed). The OpenTelemetry traces from production feed the evaluation suite, closing the loop between production and development.

Recovery is the process of getting a failed agent workflow back on track. The worst recovery is to restart from the beginning — the agent re-does completed steps, re-calls tools and potentially repeats irreversible side effects (sending the same email twice, charging the same card twice). The best recovery is to resume from the last checkpoint — the agent skips completed steps, re-executes only the failed step and continues. The difference is durable state: without checkpoints, restart is the only option; with checkpoints, resume is possible.

The recovery workflow starts with detecting the failure — the agent errored, timed out, hit the recursion limit or was cancelled. The next step is to inspect the checkpoint: what state was the agent in, what step was it executing, what was the error. The recovery decision is then based on the failure type: a transient failure (tool outage) triggers a checkpoint recovery — resume from the last checkpoint when the tool recovers. An unrecoverable failure (the task is no longer achievable) triggers a human-assisted recovery — pause and ask a human to correct the state or approve a new path. A tool or model unavailability triggers a fallback recovery — switch to a degraded mode that can complete the task with lower quality.

The recovery is verified by re-running the evaluation on the recovered workflow. Did the agent complete the task? Did it avoid repeating side effects? Did it stay within the quality and service SLOs? A recovery that completes the task but exhausts the quality error budget (because the fallback has lower quality) is not a full recovery — it is a partial recovery that the team must review. The recovery process is tested with game days — inject a failure (kill the process, take down a tool, inject a tool error) and practice the recovery. A team that has never tested recovery discovers the gaps during a real incident, when the cost is highest.