When Passing Tests Hides Vulnerabilities: An Empirical Study of Silent Failures in Agentic Systems

1. Introduction

• A patch can pass functional validation (L1) while retaining vulnerabilities detectable by static security analysis (L2) or exploitability assessment (L3).Such failures evade standard quality gates and may reach production systems.Source: §1 · p. 1
• Test-passing evaluation and existing failure taxonomies overlook security flaws without obvious failures, while security-aware evaluation has not characterized multi-agent repair pipelines.Prior taxonomies focus on visible signals such as crashes or task non-completion.Source: §1 · pp. 1–2
• The research questions examine silent-failure types and prevalence, propagation across agent roles and architectures, and distributions across vulnerability contexts, severity, and exploitability.The study aims to support targeted verification rather than treating silent failures as a homogeneous defect class.Source: §1 · p. 2
• Across seven agent frameworks and two security-focused datasets, the study analyzes 1,030 execution traces and identifies 170 confirmed silent failures.Its taxonomy centers on security-compliance decoupling from unit-test passing.Source: §1 · p. 2
• The proposed taxonomy models failure type, originator role, injection stage, and code location, while propagation analysis traces omissions across pipeline stages and compares single- and multi-agent systems.The taxonomy explicitly models failures that evade standard detection mechanisms.Source: §1 · p. 2
• The authors publicly release execution traces, generated patches, verification results, and taxonomy codes to support access, replication, and validation.The release is described as the study dataset.Source: §1 · p. 2

2. Research Methodology

• The methodology combines experimental setup, multilevel verification, manual review, and qualitative coding with reliability procedures to identify, verify, and analyze silent failures.The section presents these components as the basis for constructing and validating the failure taxonomy.Source: §2 · p. 2

2.1. Research Design Overview

• The two-phase study supports research on failure categorization, propagation mechanisms, and vulnerability susceptibility while enabling consistent comparison across seven frameworks.Figure 1 presents the research methodology overview.Source: §2.1 · p. 2, Figure 1 · p. 3
• Across three iterations, researchers sampled tasks, ran all seven frameworks, applied L0–L3 verification, manually reviewed flagged candidates, and qualitatively coded confirmed cases.Observations from each iteration informed the next iteration’s sampling and analytical focus.Source: §2.1 · p. 3

2.2. Dataset Selection

• Dataset selection required security relevance, CWE annotations, Python-only tasks for uniform execution across frameworks, and public availability, producing a combined pool of 1,236 tasks.Source: §2.2 · p. 3
• SecurityEval contributed 121 retained Python tasks from 130 prompts spanning 69 CWE categories, for initial pattern discovery and cross-framework comparison.Each task provides an insecure code context and asks the agent to generate a secure version.Source: §2.2 · p. 3
• CVEfixes contributed 1,115 Python single-file fixes with CWE annotations, capturing real-world repairs that often require adding new security logic.The dataset contains 5,365 vulnerability fixes linking CVE records to code changes in open-source projects.Source: §2.2 · p. 3

2.3. Agent Framework Selection

• Frameworks had to be relevant to code repair, open source, and released between 2023 and 2026; seven were selected to represent distinct coordination patterns.Table 2 overviews the selected frameworks and their coordination architectures.Source: §2.3 · pp. 3–4, Table 2 · p. 4

2.4.1. Model Configuration and Hardware

• All frameworks used GPT-4o-mini at temperature 0.2 and standardized task content across framework-specific prompt formats, with dataset-specific code and repair instructions.Experiments ran on a 5 GB NVIDIA A100 MIG slice with 8 CPU cores, 17.5 GB RAM, 512 GB storage, and a 600-second timeout; no extra security prompts were added unless implied by the task.Source: §2.4.1 · p. 4

2.4.2. Multi-Level Verification Framework

• L3 scores combined input sources, dangerous sinks, target-CWE matches, retained vulnerability patterns, and safe-practice signals; scores of 15 or higher triggered manual review.The threshold and signal weights were calibrated on Iteration 1 candidates and served as a screening sensitivity parameter, not a final classification boundary.Source: §2.4.2 · pp. 4–5
• Patches passing L0 syntax and L1 functional checks were silent-failure candidates if L2 or L3 later raised security concerns.L2 used Bandit and mapped findings to CWE identifiers, flagging any HIGH-severity finding; L3 estimated exploitability from risky source-sink patterns.Source: §2.4.2 · pp. 4–5

2.4.3. Manual Review and Confirmation Criteria

• The first author reviewed flagged patches against findings and task descriptions using a codebook, while codes were consolidated and earlier judgments updated across iterations.Another author independently recoded 20% of the corpus as part of the reliability assessment.Source: §2.4.3 · pp. 5–6
• Reviewers classified each flagged patch as True Positive, Partial, or False Positive, retaining both True Positive and Partial cases as confirmed failures.Partial cases involved an attempted but incomplete mitigation with a plausible exploit path remaining; they accounted for 39 of 170 confirmed failures.Source: §2.4.3 · p. 6

2.5.1. Data Collection

• 170 silent failures were confirmed from 1,030 valid traces after multi-level screening yielded 252 candidates.The team purposively sampled 150 tasks in three rounds of 50, targeting progressive coverage of failure patterns rather than statistical representativeness.Source: §2.5.1 · p. 6

2.5.2. Coding Procedure

• The authors used three-stage thematic coding of individual execution traces, moving from open labels to axial dimensions and selective refinement around security-compliance decoupling.They compared new cases with earlier cases and considered coding complete when no new codes emerged in two consecutive batches of 50 traces.Source: §2.5.2 · pp. 6–7

2.5.3. Reliability Measures

• κ=0.71 for composite D0×D1 agreement indicated substantial inter-rater agreement; D0 triage and D1 category agreement were κ=0.63 and κ=0.66, respectively.Authors resolved disagreements through discussion and refined the codebook before applying it to the full corpus.Source: §2.5.3 · p. 8

3.1. Silent Failure (RQ1)

• Omission accounted for 82/170 (48.2%) confirmed silent failures, Introduction for 52/170 (30.6%), and Inadequacy for 36/170 (21.2%), across ten open codes.Cases were assigned to the dominant failure mechanism, even when they exhibited multiple security symptoms.Source: §3.1 · p. 8
• Insecure API Preserved was the most frequent code (42, 24.7%), while Weak Mitigation had the highest mean L3 risk (17.7) among codes occurring in more than one case.Inadequacy describes attempted defenses that remain partial, bypassable, or based on an incorrect security primitive.Source: §3.1 · p. 9

3.2. Propagation Patterns Across Agent Architectures (RQ2)

• Introduction failures were more common in multi-agent systems (41/115, 35.7%) than single-agent systems (11/55, 20.0%), while Inadequacy was more common in single-agent systems (16/55, 29.1%).Omission shares were nearly identical across architectures (54/115, 47.0% in multi-agent and 28/55, 50.9% in single-agent systems). In reviewed multi-agent cases, Reviewers rarely flagged deficiencies; recurring patterns included planning-stage and design-originated omissions and design-originated introductions.Source: §3.2 · pp. 11–12, Table 7 · p. 12
• On 26 tasks, three or more frameworks produced failures sharing the same confirmed CWE, accounting for 62.9% (107/170) of cases.Four tasks had matching-CWE failures from six of seven frameworks; the study cautions that shared model, prompt, and task factors prevent attributing convergence to architecture alone.Source: §3.2 · p. 12

3.3. Distribution Across Vulnerability Types, Code Locations, and Risk Levels (RQ3)

• API-substitution vulnerability families were predominantly associated with Omission, logic-intensive families with Inadequacy, and vulnerabilities such as information disclosure and open redirect with Introduction.Omission dominated weak cryptography (8/9, 88.9%), deserialization (7/8, 87.5%), XXE (7/8, 87.5%), and command injection (5/6, 83.3%).Source: §3.3 · pp. 12–14, Figure 6 · p. 13
• Inadequacy had the highest mean L3 risk (17.7), versus Omission (10.7) and Introduction (8.0), while Introduction was mostly medium severity (46/52, 88.5%).Input handling, configuration, and API calls together accounted for over three-quarters of failures; Inadequacy was concentrated in input handling (33/36), Introduction in configuration and business logic, and Omission in API calls and input handling.Source: Table 9 · p. 13, §3.3 · p. 13

4.1. Security-Compliance Decoupling as a Distinct Failure Phenomenon (RQ1)

• 78/170 (45.9%) confirmed failures were Insecure API Preserved (42) or Weak Mitigation (36), showing two recurring mechanisms dominate.Insecure API Preserved leaves a dangerous existing call in place; Weak Mitigation applies an incomplete, bypassable defense and has the highest mean exploitability score among recurring codes (L3 = 17.7).Source: §4.1 · p. 14
• Omission, Inadequacy, and Introduction reflect different reasoning gaps and call for mode-specific checks rather than one uniform mitigation.Suggested measures include safe-API substitution for Omission, defense-completeness checks for Inadequacy, and reviewing all generated artifacts for Introduction; other codes point to deployment context, secure defaults, and inferred controls.Source: §4.1 · pp. 14–15

4.2. Why Current Assurance Pipelines Miss Silent Failures (RQ2)

• Every confirmed case would count as a successful repair under test-passing benchmarks, although functional checks do not assess the security deficiencies found.Deficiencies often arise before review and pass through later roles without security-focused re-evaluation, supporting security-oriented benchmarks and upstream checks.Source: §4.2 · pp. 15–16
• 107/170 failures occurred in convergent cases, with 26 tasks triggering comparable failures across at least three frameworks.The frameworks varied in architecture, but the study could not isolate model effects from shared prompts, tasks, datasets, or vulnerability patterns; multi-agent systems showed more Introduction failures and single-agent systems more Inadequacy failures.Source: §4.2 · pp. 15–16

4.3. Structured Failure Patterns as a Basis for Targeted Verification (RQ3)

• Vulnerability family and code location were associated with failure category: API-substitution vulnerabilities with Omission, logic-intensive ones with Inadequacy, and repair-introduced ones with Introduction.Omission clustered in API calls, Inadequacy in input-handling logic, and Introduction in configuration artifacts.Source: §4.3 · p. 16
• 22/170 confirmed cases were critical severity, and multi-agent pipelines could introduce weaknesses in configuration or auxiliary artifacts beyond the prompted patch scope.Such artifacts may escape functional tests and security review because they are not the prompt’s explicit target and roles may not inspect every output artifact.Source: §4.3 · p. 16
• The authors recommend routing security checks by vulnerability family and location, scanning every generated artifact, and independently verifying agent patches before trusting them.They note that how the set of at-risk artifacts scales with pipeline complexity remains unquantified.Source: §4.3 · pp. 16–17

5.1. Construct Validity

• The operational definition covers only patches passing syntactic and functional checks but flagged by security analysis or exploitability assessment, not all security or software-quality failures.With only 131 true positives, all three axial categories and seven open codes remained represented; Deployment Context Risk (9 cases), Ambiguous Repair Intent (1 case), and Vulnerable Test Code (1 case) appeared only among Partial cases. Omission versus Inadequacy depended on whether any mitigation attempt was observable, with boundary cases resolved through coder discussion.Source: §5.1 · p. 17

5.2. Internal Validity

• Bandit false negatives, exploitability-score settings, and qualitative trace assignments may affect the confirmed corpus and reported originator and propagation-pattern distributions.Manual review excluded 82 false-positive candidates, but the corpus remains a conservative estimate of failures detectable by the framework; candidates above the fixed threshold were reviewed, and scores were applied uniformly across frameworks. Trace evidence, predefined rules, comparisons with earlier cases, and consensus discussions were used to address ambiguous role and pattern assignments.Source: §5.2 · p. 17

5.3. External Validity

• Both datasets are Python-focused, and the seven frameworks may not represent proprietary systems or repository-scale repair across languages.Because every framework used GPT-4o-mini, observed cross-framework convergence cannot be separated from possible model, prompt, benchmark, task, or vulnerability-pattern effects; the source remains open, motivating replication with other foundation models.Source: §5.3 · pp. 17–18

5.4. Conclusion Validity

• The descriptive findings are corpus-dependent, so small differences should not be treated as stable rankings or causal or population-level conclusions.Uniform verification and review, mutually exclusive dominant categories, and consistent coding support the conclusions; the authors report raw counts with percentages and frame results as patterns in the corpus.Source: §5.4 · p. 18

5.5. Reliability

• A second author independently coded 35 sampled cases, yielding composite Cohen’s κ of 0.71, while stochastic LLM inference prevents guaranteed trace reproduction.The composite required agreement on review label and axial category, with per-dimension κ values of 0.63 and 0.66; the verification pipeline itself is deterministic for the same input.Source: §5.5 · p. 18

6.1. Benchmark-Based Evaluation of LLM Agents

• Benchmark evaluations often treat passing unit tests as success, although test-passing patches may preserve vulnerable behavior and these metrics reveal little about failure mechanisms.Broader work on agent limitations has examined issues such as misalignment and coordination, but remains primarily qualitative with limited empirical analysis of specific failure patterns.Source: §6.1 · p. 18

6.2. Failure Analysis in Multi-Agent Systems

• Multi-agent failure taxonomies and attribution methods focus on observable errors, leaving failures that appear successful yet retain vulnerabilities outside their scope.Attribution tools such as Who&When and AgenTracer rely on an observable failure signal to locate fault-inducing steps.Source: §6.2 · pp. 18–19

6.3. APR and the Plausible-versus-Correct Distinction

• APR distinguishes plausible test-passing patches from correct ones, and prior work reports that neural and LLM repairs may miss vulnerability types or remain unreliable on real-world cases.Prior evaluations primarily assess individual patches rather than how deficiencies arise and propagate across roles in a multi-agent pipeline.Source: §6.3 · p. 19

6.4. Security Verification of Generated Repairs

• Security test suites, hybrid verification, and artifact-level adversarial checks assess generated repairs, but deficiencies can survive these checks and their origins within agent roles remain unmodeled.Prior work reports fixes that pass configuration checks without resolving security issues and patches that remain exploitable under simple attacks.Source: §6.4 · pp. 19–20

6.5. Conclusive Summary

• Representative prior work does not jointly address agent-role attribution, failure propagation, security verification, and silent failures in test-passing repairs.The study examines how vulnerabilities are produced within agent roles, propagate through pipeline stages, and evade review.Source: Table 10 · p. 19, §6.5 · p. 20

7. Conclusion

• Across 1,030 valid traces from seven frameworks and two datasets, 170 confirmed silent failures support a taxonomy of Omission, Inadequacy, and Introduction.The study characterizes these failures as security-compliance decoupling rather than ordinary functional defects alone.Source: §7 · p. 20
• Test-passage and LLM-based review pipelines were insufficient, while similar insecure solutions recurred across frameworks amid shared experimental factors and architecture-specific failure profiles.The shared factors noted include the base model, prompts, tasks, and vulnerability patterns.Source: §7 · p. 20
• Recurring patterns across vulnerability families, code locations, and generated artifact types motivate targeted verification covering the full repair scope rather than general post hoc inspection.Source: §7 · p. 20
• Research evaluations should assess both functional validation and security-semantic correctness, control for base-model and orchestration effects, and practice should verify all generated artifacts with tool-backed security checks.The practice recommendation extends verification beyond the primary patched file and augments reviewer roles.Source: §7 · p. 20
• Future work should replicate findings across models, languages, and tool-supported designs, test targeted verification in deployment, and trace where security considerations weaken or disappear during repair.The proposed direction aims to move assurance toward earlier intervention before patches are accepted.Source: §7 · p. 20

CRediT authorship contribution statement

• Wenji Bai contributed to Conceptualization, Methodology, Software, Formal analysis, Investigation, Data curation, Writing - original draft, and Visualization.Source: CRediT authorship contribution statement · p. 20
• Muhammad Waseem contributed to Conceptualization, Methodology, Validation, Review, Editing, Supervision, and Project administration.Source: CRediT authorship contribution statement · p. 20
• Zeeshan Rasheed contributed to Formal analysis and Investigation.Source: CRediT authorship contribution statement · p. 20
• Jaakko Peltonen contributed to Validation, Review, Editing, and Supervision.Source: CRediT authorship contribution statement · p. 20
• Pekka Abrahamsson contributed Resources, Supervision, Review, and Editing.Source: CRediT authorship contribution statement · p. 20

Data availability

• The dataset link is provided in reference.Source: Data availability · p. 21

• ChatGPT was used to refine grammar, improve sentence structure, and resolve formatting issues; the authors reviewed and edited the content and retained full responsibility.Source: Declaration of AI Assistance · p. 21