AI Reliability Engineering: SLOs, Failure Modes and Recovery

Reliability for a deterministic software system means the system does what it is specified to do — a request returns the correct response within the latency SLO. Reliability for a probabilistic AI system is different: the "correct answer" is a distribution, not a constant. The same input can produce different outputs at different temperatures, and whether an output is "good" depends on a quality judgement (accuracy, faithfulness, relevance, safety) that a deterministic system does not require. AI reliability engineering extends SRE practice with quality SLOs that measure whether the output is good enough, not just whether the service is up and fast.

The distinction matters because the most common AI failure — quality degradation — does not trigger an error. The system returns a 200 OK response with a plausible-sounding but wrong answer. No exception, no latency spike, no saturation. Service SLOs (latency, availability) show green; the system is "reliable" by traditional measures and failing by AI measures. Google's SRE practice establishes that 100% is never the right reliability target — but for AI systems, the target must include a quality dimension or the team operates blind to the dominant failure mode. Quality is the production barrier that traditional reliability does not measure — a system that returns answers fast but returns wrong answers is not reliable in the sense that matters.

Quality SLOs are the AI extension to service SLOs. A service SLO says "99.9% of requests return within 2 seconds"; a quality SLO says "95% of classifications match the human gold standard" or "90% of RAG answers are faithful to the retrieved context." A system can meet its service SLO and fail its quality SLO — the responses are fast and available, but wrong. AI reliability engineering requires both: service SLOs that measure availability and latency, and quality SLOs that measure output quality. Without quality SLOs, the error budget does not account for the failures that matter most.

Service SLIs measure the system as a service: latency (how long requests take), availability (what fraction succeed), throughput (how many requests per second), and saturation (how loaded the system is). These are the four golden signals from SRE practice, and they apply to AI systems as they do to any service. For AI workloads, service SLIs must be decomposed per layer: model serving latency, retrieval latency, agent loop time, and tool-call latency are separate latency signals — a single number hides the layer that is slow.

Quality SLIs measure the output, not the service. For ML systems: accuracy, precision, recall, F1 against a labeled gold standard. For LLM and RAG systems: faithfulness (does the answer stick to the retrieved context), relevance (is the answer useful for the query), safety (does the answer avoid harmful content), and bias (does the system discriminate across protected groups). Quality SLIs are harder to measure than service SLIs because they require a quality judgement — a labeled dataset, a human evaluator, or an LLM-as-judge rubric — not just a timer. They are also harder to alert on because they are collected periodically, not per request.

Business SLIs connect the system to the outcome the organisation cares about: conversion rate (does the AI feature increase conversion), user satisfaction (CSAT, retention), cost per outcome (how much compute per successful task), and task completion (does the agent finish the task the user asked for). Business SLIs are the ultimate measure of reliability — a system that meets its service and quality SLOs but does not move the business metric is not reliable in the sense that matters. Business SLIs are the slowest to collect and the hardest to attribute, but they are the ones that survive budget conversations.

An SLO has three components: an SLI (what you measure), a target (what fraction of events must meet the SLI) and a time window (the period over which the target applies). For AI systems, the SLO must include a quality dimension. A service SLO of 99.9% successful requests over 30 days defines reliability for the service; a quality SLO of 95% faithful answers over the same window defines reliability for the output. Both have error budgets — the allowed failure rate — and both must be tracked. Defining SLOs after production means operating without a reliability target; Google's SRE practice is that SLOs should be defined before production.

The error budget is the inverse of the SLO target: if the SLO is 99.9%, the allowed bad-event ratio is 1 − 0.999 = 0.001 (0.1% of events may fail). The error budget aligns incentives between product development and reliability: as long as the budget has room, new releases can be pushed; when the budget is exhausted, releases are halted and resources go to testing and reliability. The burn rate measures how fast the budget is being consumed: burn rate = observed bad ratio ÷ allowed bad ratio. A burn rate of 1.0 means the system is burning budget at exactly the rate the SLO allows; above 1.0, the budget is being consumed too fast; below 1.0, the system has headroom.

The equivalent time-based burn rate accounts for where you are in the window: equivalent burn rate = (budget fraction consumed) ÷ (window fraction elapsed). If you have consumed 60% of the budget in 30% of the window, the equivalent burn rate is 0.6 ÷ 0.3 = 2.0 — you are burning twice as fast as allowed and will exhaust the budget before the window ends. The planner below lets you input an SLO target, time window, total events, bad events and elapsed days to calculate the burn rate and budget status. Benchmarking against your system's actual error distribution is required for production SLO setting — the planner is a calculation tool, not a guarantee of compliance.

AI failure modes cluster into four layers: data, model, application and dependency. Data failure modes include distribution shift (the production data no longer matches the training distribution), missing features (an expected input is absent), and schema changes (a feature is renamed or retyped). These failures degrade the model without raising an error — the model still returns a prediction, but the prediction is wrong because the input has drifted. Data failures are the hardest to detect because they do not trigger an exception; they require drift monitoring with a baseline and threshold.

Model failure modes include quality degradation (the model's accuracy drops over time as the world changes), drift (the model's output distribution shifts), and bias amplification (the model's errors concentrate on a protected group). These failures are detected by periodic evaluation against a held-out gold standard, not by per-request monitoring. Application failure modes include prompt injection (an adversarial input manipulates the model), tool-call errors (an agent calls a tool that fails or returns unexpected data), and context overflow (the input exceeds the model's context window). These are the failures that emerge at the boundary between the model and the application that wraps it.

Dependency failure modes include provider outage (the LLM API is down), rate limiting (the provider throttles requests), and capacity exhaustion (the serving cluster cannot handle the load). These are traditional reliability failures — the system is unavailable because a dependency is unavailable — but they interact with AI-specific patterns: a provider outage triggers a fallback to a smaller local model, which has lower quality, which can exhaust the quality error budget even though the service SLO is met via the fallback. The failure-mode table below maps each layer to its failure, signal, cause, containment and recovery.

Resilience patterns reduce the blast radius of a failure — the number of users and requests affected. Fallback and graceful degradation: when the primary model or provider fails, the system falls back to a smaller, cheaper or cached response rather than returning an error. A RAG system that cannot reach the LLM provider falls back to returning the retrieved documents with a summary generated by a smaller local model — degraded quality, but the user gets something useful. The tradeoff is that the fallback has lower quality, which consumes the quality error budget; the team must decide whether degraded quality within budget is acceptable.

Circuit breakers stop calling a failing dependency after a threshold of failures, preventing cascading failures. An agent that calls an external API trips a circuit breaker after three consecutive failures, stops calling the API for a cooldown period, and returns a default response or falls back to a cached result. Without a circuit breaker, the agent retries indefinitely, exhausting resources and increasing latency for every user. Retry and timeout with idempotency handle transient failures — a tool call that times out is retried with exponential backoff, but only if the operation is idempotent (repeating it does not cause harm). Non-idempotent operations (writing a record, charging a card) must not be retried blindly.

Bulkheads and queue isolation partition resources so a failure in one part does not consume all resources. A serving cluster with separate request queues for real-time and batch traffic can shed batch traffic when real-time traffic spikes, protecting the latency SLO for real-time users. Without bulkheads, a spike in batch traffic can starve real-time requests, causing a latency SLO breach for all users. These patterns are general reliability patterns from SRE practice, but they have AI-specific applications: circuit breakers around tool calls, bulkheads around agent loops, fallbacks around LLM providers, retry around retrieval with idempotent retrieval queries.

Error budgets govern release velocity. As long as the budget has room, new releases can be pushed; when the budget is exhausted, releases are halted and resources go to testing and reliability. The thresholds translate burn rate into action: a burn rate below 1.0 means the system has headroom and releases can proceed at normal velocity. A burn rate between 1.0 and 2.0 means the system is approaching the limit — releases should slow, canary windows should lengthen, and the team should investigate before pushing more changes. A burn rate above 2.0 means the budget is being consumed too fast — releases should stop, and resources should go to finding and fixing the cause.

Quality regression thresholds add a second axis. A release that passes the service SLO check but regresses quality by more than a defined threshold (e.g., accuracy drops 2 percentage points from the previous release) should be slowed or stopped even if the service SLO is green. This prevents the team from shipping releases that meet the service SLO while silently degrading output quality — the failure mode that service SLOs alone cannot catch. The quality threshold is set during release planning and enforced by the evaluation gate: a release that fails the quality regression threshold does not promote.

The freeze decision — who decides and how — must be defined before it is needed. A release freeze is a high-stakes decision: it slows feature delivery, which creates pressure to override it. The decision should be owned by a named role (the incident commander, the service owner, the SRE on call) with a defined trigger (burn rate above threshold, quality regression above threshold, customer-impacting incident). The decision and its rationale are recorded so the team can review whether the threshold was right and adjust it. A freeze that is never triggered means the thresholds are too loose; a freeze that triggers constantly means they are too tight.

Recovery testing for AI systems means exercising the fallback, rollback and resilience patterns before an incident forces you to. The most effective format is a game day: a scheduled exercise where the team injects a failure (provider outage, model degradation, tool-call failure) into a staging or shadow environment and practices the response. Game days surface gaps that are invisible in normal operation — the fallback model has lower quality than expected, the rollback takes longer than the error budget allows, the circuit breaker trips too early or too late. A team that has never run a game day discovers these gaps during a real incident, when the cost is highest.

Fallback validation answers: does the degraded mode actually work? A fallback that returns a cached response from six months ago may be technically available but practically useless — the data is stale, the model has changed, and the user gets a wrong answer. Fallback validation tests the fallback end-to-end: trigger the fallback, measure the quality of the fallback response, and check whether the quality error budget can absorb the degraded period. If the fallback exhausts the quality budget faster than the primary, the fallback is not a reliability improvement — it is a liability.

Rollback testing answers: can you revert within the error budget? A rollback that takes 30 minutes to propagate is useless if the error budget allows 5 minutes of bad events. Rollback testing measures the time from rollback trigger to full traffic on the previous version, including the time to detect the failure, decide to roll back, execute the rollback and verify the previous version is healthy. The process below defines the recovery testing steps from game day through evidence collection.

Operational reliability is demonstrated by three evidence types: SLO compliance record, incident history with mean time to recovery, and recovery test evidence. The SLO compliance record shows whether the system met its service and quality SLOs over the reporting period — the percentage of time within SLO, the error budget consumed, and the burn rate trend. A system that claims to be reliable without an SLO compliance record is claiming without evidence.

Incident history with MTTR shows how the system behaves when it fails — how often incidents occur, how long they last, and what the team learned from each. The SRE Book's blameless postmortem culture is the practice: each incident produces a postmortem that identifies the root cause, the contributing factors and the action items, without blaming individuals. The MTTR trend over time shows whether the team is getting better at recovery — a flat or worsening MTTR means the recovery patterns are not improving. Incident history also shows whether the failure modes match the failure classification: if incidents cluster in a layer the classification did not predict, the classification needs updating.

Recovery test evidence — the output of game days and rollback tests — proves the resilience patterns work. A fallback that has never been tested is a hypothesis, not a reliability control. The evidence is the game day report: the scenario, the response, the metrics, the gaps and the action items. Together, these three evidence types answer the question a hiring manager, an auditor or an incident reviewer asks: can you prove this system is reliable, and can you prove you can recover when it is not?