PaperlayerView exact revision on arXiv

Source-linked AI summary

Compound AI System Reliability: A Failure Taxonomy and Resilience Pattern Catalog from 150 Production Incidents

Rudrendu Kumar Paul, Sourav Nandy

arXiv:2610.02503v1cs.SEcs.AIcs.DCcs.LG

TL;DR

Compound AI systems can produce undetected quality failures when components are individually healthy but their interactions break down. The paper analyzes production incidents to build a boundary-based failure taxonomy and evaluates resilience patterns; using three or more patterns reduces mean MTTR by 71% against unstructured monitoring.

  • Problem

    A production RAG pipeline’s answer quality fell from 0.82 to 0.59 F1 for eleven days before detection, despite component health checks passing.

    Source: §1 · p. 1

  • Method

    The authors derive 23 failure modes from 150 production incidents and evaluate resilience patterns through controlled fault-injection experiments.

    Source: §1 · p. 2

  • Results

    71% reduction in mean MTTR: systems using three or more patterns averaged 8.4 minutes versus 28.7 minutes for unstructured monitoring only.

    Source: §5.1 · p. 4

  • Takeaways & Limitations

    The taxonomy and pattern catalog are released as a practitioner resource for reliability engineering in compound AI systems.

    Source: Impact Statement · p. 5

    The effectiveness measurements come from a single six-component architecture under controlled conditions and are indicative, not universal guarantees.

    Source: §6 · p. 4

Abstract

from arXiv · show

Deploying compound AI systems reliably and safely requires understanding failure modes that emerge at component boundaries, not within individual models. Cascading errors propagate across component boundaries, silent quality degradation evades standard monitoring, and coordination failures yield incorrect collective behavior from individually correct parts. We analyze 150 production incident reports from open-source compound AI projects and anonymized enterprise deployments to construct a taxonomy of 23 failure modes organized into five categories: retrieval failures, generation failures, tool failures, orchestration failures, and integration failures. For each category, we propose resilience patterns with measured effectiveness from controlled fault injection experiments. Circuit breakers reduce cascade propagation by 89%, output quality gates catch 73% of silent degradation before user impact, and component isolation reduces blast radius by 64%. Systems implementing three or more resilience patterns from our catalog reduce mean-time-to-recovery (MTTR) by 71% compared to unstructured monitoring baselines. We release the incident taxonomy and pattern catalog as a practitioner resource.

1. Introduction

Compound AI reliability breaks down at component boundaries, where individually healthy parts can produce incorrect system outputs. The paper addresses this gap with an incident-derived taxonomy and empirically evaluated resilience patterns.

1. Introduction

  • An index rebuild shifted embedding normalization, leaving 40% irrelevant documents in retrieval results while the generator produced confident wrong answers.The pipeline’s answer quality fell from 0.82 to 0.59 F1 before anyone noticed.Source: §1 · p. 1
  • Failures can remain invisible to component-level monitoring when internal health checks pass but semantic relationships between components break down.The introduction identifies this gap between component correctness and system correctness as the defining reliability challenge.Source: §1 · p. 1
  • Prior work examines individual failure dimensions, but the paper identifies a gap in unified coverage of the compound-system stack paired with validated resilience patterns.The cited studies include Vinay (2025), Rabanser et al. (2026), and Cemri et al. (2025).Source: §1 · p. 1
  • The paper derives 23 failure modes across five component-boundary categories from 150 production incidents and evaluates resilience patterns through controlled fault injection.The incidents span retrieval, generation, tool use, orchestration, and integration boundaries.Source: §1 · p. 2
  • The authors frame boundary reliability as a safety concern because silent failures and uncontrolled cascades can harm users unable to verify or contest outputs.The introduction points to high-stakes public-sector, healthcare, and social-infrastructure settings.Source: §1 · p. 2

2. Related work

Prior failure analyses address model, multi-agent, and infrastructure levels, but leave gaps in coverage of compound-system boundaries. This work bridges those levels with a full-stack taxonomy and fault-injection-validated resilience patterns.

2. Related work

  • Model-level studies characterize single-model deployments; Vinay (2025) catalogs LLM application failures, while Yu et al. (2026) attribute 45.7% of analyzed failures to environment and infrastructure faults.Both studies do not address propagation across compound-system component boundaries.Source: §2 · p. 2
  • Multi-agent studies identify collaboration failures, cascades, and framework defects, but do not cover the full compound stack; Cemri et al. (2025) reports κ = 0.88 and Xie et al. (2026) raises defensive success from 0.32 to 0.89.Zhu et al. (2026) finds API misuse and documentation desynchronization dominate framework-level defects.Source: §2 · p. 2
  • An analysis of 156 high-severity production incidents in AI inference services found that 60% stemmed from inference-engine failures, with timeouts accounting for roughly 40% of that category.Source: §2 · p. 2
  • Chen et al. (2025b) finds that compound-pipeline model selection requires joint optimization because local component optima do not yield global system optima.Chen et al. (2025a) surveys the compound AI systems landscape end-to-end.Source: §2 · p. 2
  • The paper bridges these levels by analyzing failures across boundaries from retrieval through orchestration and pairing its taxonomy with fault-injection-validated resilience patterns.Source: §2 · p. 2

3. Methodology

The methodology combines a documented production-incident corpus with independent coding and a controlled fault-injection testbed. It uses the corpus to construct the taxonomy and the testbed to evaluate resilience patterns.

3.1. Incident corpus

  • The corpus contains 150 incidents: 97 from public reports and issue trackers across 12 open-source projects, and 53 anonymized enterprise incidents collected from January 2024 to December 2025.The enterprise incidents came from production RAG, agent, and pipeline deployments.Source: §3.1 · p. 2
  • Included incidents involved at least two interacting AI/ML components, documented root causes, and user-visible impact or an on-call page.External failures were excluded unless the system architecture amplified them.Source: §3.1 · p. 2

3.2. Taxonomy construction

  • Two researchers independently coded each incident for its failing boundary, propagation, detection, detection time, and root cause, then merged codes into 23 boundary-based failure modes.The reported intercoder agreement reached Cohen’s κ = 0.84 after two rounds.Source: §3.2 · p. 2
  • The taxonomy table presents five categories, 23 modes, and their frequencies in 150 incidents, showing the top three modes per category.Source: Table 1 · p. 3

3.3. Fault injection experiments

  • The six-component testbed covers a query router, retriever, reranker, generator, tool executor, and orchestrator, with 23 fault scenarios spanning all five categories.Scenarios include embedding-model substitution, incremental latency, and JSON-serialization precision corruption; measured metrics include cascade depth, detection time, blast radius, and MTTR.Source: §3.3 · p. 3
  • Each scenario ran 100 trials with and without each candidate pattern, and the authors report means with 95% confidence intervals.The testbed, scripts, and raw trial logs are available in the practitioner resource to support reproducibility.Source: §3.3 · p. 3

4. Failure taxonomy

The taxonomy organizes 23 failure modes by component boundary and shows how failures manifest across the compound-system stack. Silent degradation, cascades, and semantically wrong but valid outputs recur across categories.

4. Failure taxonomy

  • Retrieval failures are the largest category at 25.3% of incidents; index drift alone accounts for 10.7%, and irrelevant retrieval can yield fluent answers for nine days before detection.In the described incident, cosine similarity fell from 0.78 to 0.41 after an embedding-model mismatch, while the retriever continued returning results.Source: §4 · p. 3
  • Generation failures account for 20.7% of incidents, and retrieval failures have a 2.3x multiplier on downstream generation errors in the corpus.The authors link this coupling to generators synthesizing answers from degraded or incoherent retrieval context.Source: §4 · p. 3
  • Tool failures account for 19.3% of incidents; in 8 of 10 timeout incidents, one slow tool reduced system-wide throughput by 60–80% within minutes.The orchestrator can hold execution context open, consuming memory and blocking requests.Source: §4 · p. 3
  • Orchestration failures account for 18.7% of incidents, with one infinite retry loop costing $2,400 over a weekend when malformed tool JSON was not excluded from retries.The retry logic also lacked exponential backoff.Source: §4 · p. 3
  • Integration failures account for 16.0% of incidents and can arise at type boundaries, where JSON round-trip precision loss changed a threshold from 0.85 to 0.8500000000000001 and silently dropped 23% of results.Source: §4 · p. 3

4.1. Cross-cutting patterns

  • Silent degradation affects 51% of incidents and takes a mean 4.2 days to detect, compared with 12 minutes for crash failures.These systems can produce wrong outputs while passing health checks.Source: §4.1 · p. 3
  • Cascade amplification, where one component’s failure amplifies through downstream components, occurred in 34% of incidents; semantically wrong but structurally valid outputs occurred in 22%; 11% exhibited two or more cross-cutting patterns simultaneously.Semantic errors passing validation are invisible to schema validation and health checks.Source: §4.1 · pp. 3–4

5. Resilience patterns

The catalog evaluates five resilience patterns for compound AI systems, from boundary-level checks to resource isolation. Combining three or more patterns reduces mean time to recovery, while each individual pattern addresses a different failure mechanism.

5. Resilience patterns

  • 89%: circuit breakers reduce cascade depth from a mean of 3.8 components to 0.4 components by returning structured fallbacks when error rates exceed a threshold.The adaptation for AI systems monitors semantic quality, not only HTTP status codes.Source: §5 · p. 4
  • 73%: output quality gates catch silent degradation incidents before they reach users by checking retrieval relevance and output consistency.The retrieval gate uses a calibrated mean cosine-similarity threshold of τ = 0.62 in the experiments.Source: §5 · p. 4
  • 64%: component isolation reduces blast radius from 78% of concurrent requests to 28% by assigning independent resource pools to components.A tool executor can then exhaust its rate-limit budget without starving the retriever or generator.Source: §5 · p. 4
  • 81%: semantic validators catch semantic errors that pass schema validation by checking output meaning against expected domains or the original query.They check meaning rather than structure alone.Source: §5 · p. 4
  • Typed interfaces with runtime validation eliminate 92% of taxonomy integration failures by replacing loose JSON passing with explicit boundary contracts.The implementation used Pydantic models.Source: §5 · p. 4

5.1. Combined effectiveness

  • 71%: systems using three or more patterns achieve mean MTTR of 8.4 minutes (± 2.1), versus 28.7 minutes (± 5.3) with unstructured monitoring only.The combination matters because circuit breakers contain cascades, quality gates detect silent degradation, and component isolation limits blast radius; no single pattern addresses more than 40% of failure modes.Source: §5.1 · p. 4

6. Discussion and limitations

The authors identify limitations in the incident corpus, the MTTR comparison baseline, and the scope of fault-injection testing. They also note that organizational failures are outside the taxonomy and that resilience patterns introduce runtime costs.

6. Discussion and limitations

  • The enterprise incident corpus contains 53 incidents from a small number of organizations.Source: §6 · p. 4
  • The 71% MTTR reduction is measured against unstructured monitoring only; standard retry-with-exponential-backoff remains an unevaluated comparison baseline.The authors describe the existing baseline as naive and leave the stronger comparison for future work.Source: §6 · p. 4
  • Fault-injection effectiveness was measured on one six-component architecture under controlled conditions, not in live production.Systems with 20+ components, nested agent hierarchies, or human-in-the-loop stages are outside the testbed; real-world effectiveness depends on system-specific factors.Source: §6 · pp. 4–5
  • The taxonomy covers technical failure modes, while organizational failures such as insufficient on-call staffing, missing runbooks, and poor incident communication remain out of scope.Source: §6 · p. 5
  • Resilience patterns incur latency and resource costs, including 120 ms median per-request overhead for output quality gates and 18% higher peak memory for component isolation.For latency-sensitive deployments, the authors recommend asynchronous quality gates that log degradation for alerting rather than block responses.Source: §6 · p. 5

7. Conclusion

The conclusion presents a taxonomy of compound-system failures and a fault-injection-validated catalog of resilience patterns. It reports a 71% MTTR reduction from combining patterns and releases the taxonomy as a practitioner resource, while describing the catalog as a starting point rather than a complete solution.

7. Conclusion

  • The paper classifies 23 compound-system failure modes from 150 production incidents by component interaction boundaries.Source: §7 · p. 5
  • Combining circuit breakers, quality gates, and component isolation reduces MTTR by 71% in the reported validation.The authors state that failure surfaces grow faster than linearly as compound systems become more complex, and characterize the patterns as a starting point, not a complete solution.Source: §7 · p. 5
  • The taxonomy is released as a structured JSON schema for community extension and integration with incident-management workflows.Source: §7 · p. 5

Impact Statement

The catalog is a practitioner resource for teams treating component-boundary reliability as a safety requirement in high-stakes domains. Combining three or more patterns reduces MTTR by 71% without model changes or infrastructure overhaul.

Impact Statement

  • 71% lower mean-time-to-recovery comes from combining three or more patterns, without model changes or infrastructure overhaul.Source: Impact Statement · p. 5
  • Silent failures and uncontrolled cascades disproportionately affect users unable to verify or contest machine-generated outputs.Source: Impact Statement · p. 5
  • The taxonomy and resilience catalog are defensive practitioner resources for reliability-conscious teams, and the paper identifies no dual-use risks.Source: Impact Statement · p. 5

Related papers

Loading 2610.02503v1…