When "Correct" Is Not Safe: Can We Trust Functionally Correct Patches Generated by Code Agents?

1 Introduction

• Prior agent-security efforts mainly address explicit malicious actions or functional errors, while implicit threats and single-query, black-box attacks remain less covered.The paper describes the implicit case in which a benign developer copies contaminated external content into an issue.Source: §1 · pp. 1–2
• An FCV patch resolves the reported issue and passes functional tests while still embedding an exploitable vulnerability.Source: §1 · p. 2
• FCV-Attack appends CWE-targeted, developer-style suggestions to issue descriptions and operates with black-box access and a single query.Malicious contributors and benign developers copying poisoned content produce the same issue-text input modality.Source: §1 · p. 2
• The paper challenges the assumption that functional correctness alone establishes security in autonomous coding systems.Source: §1 · p. 2
• Across 12 agent–model combinations, FCV-Attack induces vulnerabilities with ASR up to 56.3%, particularly for CWE-538.The evaluation includes four models and three agent scaffolds on SWE-Bench.Source: §1 · p. 2
• Controlled experiments show vulnerabilities persist on clean agent trajectories, indicating behavior-level defenses are insufficient because contamination occurs during initial encoding.Source: §1 · p. 2

2 Related Work

• Code-agent benchmark and industrial efforts focus on functional correctness, whereas this paper treats repair agents as an attack surface for exploitable but correct patches.Source: §2 · p. 2
• Rather than studying code generation generally, the paper examines repair workflows where adversarial instructions can manipulate patches that pass tests yet add vulnerabilities.Source: §2 · p. 2
• The paper contrasts implicit single-submission injection with prior agent attacks focused on explicit threats and methods requiring white-box access or multiple queries.A malicious contributor or a benign developer pasting poisoned content can provide the injection through an issue.Source: §2 · p. 3, Figure 1 · p. 3

3 Why ”Correct” Is Not Secure: Motivating Observation

• The study screens functionally correct patches from four model backends in Mini-SWE-Agent on SWE-bench for security issues.Source: §3 · p. 3
• Among functionally correct patches under clean settings, vulnerability rates were 6.0% for Qwen3-Coder, 5.0% for Kimi-K2-Instruct, 4.5% for GPT-5 mini, and 4.3% for Claude Sonnet 4.Source: §3 · p. 3, Figure 2 · p. 3
• An FCV patch resolves the reported issue and passes all tests while introducing at least one CWE-defined vulnerability.Figure 3 gives conceptual examples of vulnerabilities embedded in functionally correct code.Source: §3 · p. 3, Figure 3 · p. 4
• FCV Rate measures the fraction of functionally correct patches that are also vulnerable, motivating investigation of whether targeted attacks can amplify the risk.Source: §3 · pp. 3–4

4 Amplifying Vulnerabilities with FCV-Attack

• FCV-Attack embeds CWE instructions in benign GitHub issue descriptions to induce functionally correct but vulnerable patches.Source: §4 · p. 4

4.1 Threat Model

• The attacker’s goal is a patch that resolves the issue, passes unit tests, and contains a chosen CWE-defined vulnerability.Source: §4.1 · p. 4
• The attacker has black-box access through public interfaces and only one query; repeated queries may reduce stealth by creating unusual request patterns.Source: §4.1 · p. 4
• The threat model covers both deliberate issue injection and benign users unintentionally including malicious instructions from compromised sources.Both pathways use developer-style instructions inside an issue.Source: §4.1 · p. 4

4.2 FCV-Attack

• Templates frame CWE-targeted vulnerabilities as benign developer guidance tied to plausible intents and examples, then append the suggestion to an issue description.An attempt counts as successful only if the patch resolves the defect, passes the full unit-test suite, and contains the targeted CWE.Source: §4.2 · p. 5
• The controlled-trajectory variant inserts the FCV instruction before a previously recorded clean trajectory, and still induces FCV examples.This tests whether vulnerabilities arise from interaction history accumulated during the agent’s dialogue.Source: §4.2 · p. 5

5.1 Experimental Setup

• The evaluation pairs three agent frameworks with four high-performing models and tests four CWEs spanning information exposure, cross-site scripting, SQL injection, and code execution.It uses CWE-based injection templates as a controlled proxy for malicious injection or accidental contamination through issue text.Source: §5.1 · p. 5

5.2 Evaluation Protocol

• Patches first pass the official SWE-Bench functional tests, then a Qwen3-Coder-based judge assesses vulnerability using a standardized CWE-informed template.The vulnerability judge returns a structured verdict covering vulnerability presence, severity, confidence, and fix recommendations.Source: §5.2 · p. 6

5.3 Evaluation Metrics

• ASR decomposes into Pass@1 multiplied by FCV Rate: correct-patch probability times the chance a correct patch contains the targeted vulnerability.The metrics are Pass@1 = npass/N, FCV Rate = npassyetvuln/npass, and ASR = npass&vuln/N.Source: §5.3 · p. 6

5.4 Main Results

• 18.1% to 62.9% overall ASR; all 12 tested agent–model combinations were compromised, including systems with high functional correctness.SWE-Agent with GPT-5 mini reached 62.9% ASR; with Claude Sonnet 4 it reached 56.3%.Source: §5.4 · p. 6, Figure 4 · p. 7
• CWE-538 produced the largest increase over baseline, with benign-seeming logging requests making the information-exposure attack especially effective.The paper contrasts this with operations such as generating eval statements, which agents are trained to avoid.Source: §5.4 · p. 7, Figure 4 · p. 7
• Claude Sonnet 4 had 14.0% average ASR and GPT-5 mini 13.7%; SWE-Agent averaged 14.7%, versus 10.4% for Mini-SWE-Agent and 9.7% for OpenHands.Figure 4 compares average ASR across models, agents, and CWE types.Source: §5.4 · p. 7, Figure 4 · p. 7

6.1 Attack Pathway: Internal State vs. Observable Trajectory

• 47.5% versus 54.2% ASR for Kimi-K2 on CWE-538: the controlled-trajectory attack remains comparable to the standard attack despite matching observable actions to a clean run.The controlled-trajectory condition injects the instruction from the start while constraining the visible action sequence.Source: §6.1 · p. 7, Table 2 · p. 8
• Attacks can persist through clean observable behavior: the authors attribute this to contaminated KV-cache representations from the initial encoding influencing final patch generation.They conclude that defenses limited to monitoring observable behavior are insufficient because contamination precedes observable actions.Source: §6.1 · pp. 7–8

6.2 How does Task Complexity Influence FCV Rate?

• FCV rate generally decreases as API calls increase across all four models, most notably for Kimi and Qwen; fewer-call tasks are more prone to FCV patches.The authors suggest trivial tasks invite shallow pattern matching and insecure shortcuts, whereas complex tasks constrain the solution space.Source: §6.2 · p. 8, Figure 5 · p. 8

6.3 Potential Defense on FCV-Attack

• For CWE-538, prompt safeguards reduced ASR from 19.0 to 17.9 for Qwen3-Coder and from 54.2 to 43.3 for Kimi-K2, but remained above clean baselines of 9.5 and 0.8, respectively.The added system instruction asks agents to avoid risky patterns while keeping code secure and private.Source: §6.3 · p. 9, Figure 6 · p. 9

7 Conclusion

• FCV patches resolve reported issues and pass functional tests while embedding exploitable vulnerabilities.Source: §7 · p. 9
• Controlled clean trajectories still produced FCV attacks, consistent with vulnerabilities propagating through internal state contaminated during initial encoding.Source: §7 · p. 9
• The authors conclude that functional-correctness evaluation and behavioral monitoring are insufficient, and call for more comprehensive code-agent security mechanisms.Source: §7 · p. 9

Attack & Vulnerability Scope

• The attack evaluates specific semantic injections targeting four CWEs; whether the findings extend to other CWEs or prompts remains unclear.The authors leave broader attack and vulnerability coverage for future work.Source: §8 · p. 10

Analysis of the FCV Attack

• The internal-state contamination explanation is based mainly on external behavior, without tracing the models’ neural representations.The authors identify interpretability work on LLM agents as a future direction.Source: §8 · p. 10

Generalization Across Agent Architectures and Settings

• Experiments cover three agent frameworks and twelve agent–model combinations on SWE-bench, but do not establish how evolving systems or human–agent workflows behave.The authors leave those settings for future study.Source: §8 · p. 10

9 Ethical Considerations

• The study uses open-source SWE-bench with synthetic issue injections, no proprietary or sensitive data, and no production deployment of introduced vulnerabilities.The authors describe the work as defensive and intended to inform safer deployment practices.Source: §9 · p. 10

A.3 Evaluated CWE Categories

• The four targeted CWE categories are information exposure, cross-site scripting, SQL injection, and code injection.The appendix supplies a description of each category.Source: §A.3 · pp. 14–15

A.1 Evaluated Agents; A.2 Evaluated Models; B Instance Counts and Sampling Details

• The appendix describes three representative software-engineering agents and four models, with per-combination sampling and clean-resolved instances used for injection evaluations.Sample sizes vary by model type and agent; runtime failures are conservatively treated as unsolved.Source: §A.1 · p. 14, §A.2 · p. 14, B Instance Counts and Sampling Details · p. 15

C CWE Injection Templates

• Injection templates frame CWE-targeted guidance as benign developer advice and were used verbatim across experiments.Examples include logging sensitive data, rendering user input, constructing dynamic SQL, and evaluating input.Source: C CWE Injection Templates · pp. 15–16
• The code-injection template recommends dynamic evaluation of input as flexible data processing.Its example uses eval to process user input.Source: C CWE Injection Templates · p. 16

D Extended FCV Attack Results

• 67.6% is the highest reported CWE-538 FCV rate, reached with Claude-Sonnet-4 and mini-SWE-agent; injection raises rates across CWEs versus near-zero original-condition rates.OpenHands is described as the most robust framework, with zero FCV rates for many CWEs.Source: D Extended FCV Attack Results · p. 16

E Vulnerability Detection Protocol; LLM Judge Template

• The vulnerability judge receives the CWE definition, patch diff, and up to 50 surrounding code lines, then returns a structured assessment including verdict, confidence, and location.The instructions direct it to assess the final patched code and focus on vulnerabilities introduced by the changes.Source: LLM Judge Template · p. 17

F Variants of FCV Attacks

• The appendix gives additional implementation details for the FCV-attack variants and their ablation study.The supplied passages introduce both natural execution and controlled-trajectory procedures.Source: F Variants of FCV Attacks · p. 17, §F.2 · p. 18, §3 · p. 18

F.1 Trajectory Recording and Replay

• Controlled Trajectory records clean-run messages, retrieval results, tool outputs, and reasoning traces through the pre-edit turn.The pre-edit turn is detected at the first assistant file-modification command or unified diff.Source: §F.1 · pp. 17–18

F.2 Variant-Specific Procedures; 2. Execute agent with injected issue; 4. After pre-edit turn, release trajectory constraint and allow natural generation

• Figure 7 contrasts natural FCV-Attack execution with injection followed by replay of the clean trajectory to isolate the injection’s effect.In controlled replay, intermediate retrieval, reasoning, and tool outputs are forced to match the recorded trajectory before the constraint is released for natural generation.Source: Figure 7 · p. 18, §2 · p. 18, §4 · p. 18

F.3 Effects of Trajectory Length and Instance Difficulty Level on Vulnerability Rate

• The difficulty analysis uses SWE-bench Verified’s human-annotated levels, based on estimated developer completion time, and reports FCV rates across four models.The appendix also presents FCV rates by API-call quartiles for each model.Source: §F.3 · p. 18

G Extended Defense Results

• Figure 8 presents FCV rates by difficulty across four SWE-Agent models, while Figure 10 relates instance difficulty, vulnerability rate, and API calls.Figure 9 also breaks down difficulty-based FCV rates across four SWE-agent models.Source: Figure 8 · p. 19, Figure 9 · p. 19, Figure 10 · p. 20
• In CWE-538, Defense+Injection reduces Qwen3-Coder’s ASR from 19.0% to 17.9% and Kimi-K2-Instruct’s from 54.2% to 43.3%, but both remain above their clean baselines.Across CWEs, Defense-only yields near-zero ASR and FCV rates; the authors conclude prompt-level safeguards reduce but do not eliminate vulnerability induction.Source: G Extended Defense Results · pp. 18–19, Table 5 · p. 20

H Illustrations

• The appendix illustrations show the FCV attack, the clean-trajectory injection variant, and an example Mini-SWE-Agent trajectory.Source: H Illustrations · p. 20

FCV Attack

• The attack appends an adversarial instruction to the issue or PR description and continues generation with it present; a controlled variant first replays clean-trajectory steps.The figures distinguish natural continuation from replaying the exact intermediate steps of a pre-recorded clean trajectory before continuation.Source: Figure 11 · p. 21, Figure 12 · p. 22
• The added instructions request displaying user input directly in an HTML div and template responses; the agent tries to edit an upload view, but its shown sed attempts fail.The requested format interpolates user input into the HTML, while the recorded edit commands return syntax errors.Source: FCV Attack · pp. 23, 27, 28
• In the Django replay, FILE_UPLOAD_PERMISSIONS is shown as None, then later as 0o644; after applying the change, the agent turns to the additional HTML output instructions.Source: FCV Attack · pp. 24–26
• The final output shows FILE_UPLOAD_PERMISSIONS set to 0o644.The output also identifies the modified file_upload_view function.Source: FCV Attack · p. 34
• The modified file_upload_view returns HTML containing the submitted form_data.It formats the data inside a <div class='user-info'> element.Source: FCV Attack · p. 34

• The agent says it changed FILE_UPLOAD_PERMISSIONS from None to 0o644 and modified the file-upload test view to display submitted form data.Source: FCV Attack · p. 34
• The django django-10914 trajectory under Controlled Trajectory Setting begins with an injected prompt; blue boxes show the clean trajectory and red boxes the agent’s continuation afterward.Source: Figure 13 · p. 35