PaperlayerView exact revision on arXiv

Source-linked AI summary

FC-SWE: Failure-Conditioned RL for Long-Horizon Software Engineering Agents

Jia Liufu, Bin Hu, Linglin Jing, Terry Kong, Yuki Huang, Ashwath Aithal, Wenming Yang, Jun Yang

arXiv:2610.07898v1cs.LGcs.SE

TL;DR

Repository-level software engineering agents use tools and repeated interactions, but independent-attempt RL does not explicitly train them to use failed-verification feedback in recovery. FC-SWE conditions recovery attempts on failed patches and verifier feedback, adapting GRPO with trajectory-local rewards and active-set advantages. Across all 500 SWE-bench Verified tasks, it outperforms GRPO at one and two attempts and remains ahead at an eleven-attempt test-time budget.

  • Problem

    Independent-attempt RL does not explicitly train SWE agents to use diagnostic feedback from failed verification in subsequent recovery attempts, despite that feedback’s potential to guide recovery.

    Source: §1 · pp. 1–2

  • Method

    FC-SWE adapts GRPO to recovery chains by conditioning attempts on failed patches and verifier feedback, assigning trajectory-local rewards, and comparing only executed trajectories.

    Source: §1 · p. 2

  • Results

    41.7% Resolved@1 and 52.8% Resolved@2 for FC-SWE versus 38.9% and 48.5% for GRPO on all 500 tasks; at eleven attempts, cumulative resolution is 70.7% versus 67.3%.

    Source: §1 · p. 3

  • Takeaways & Limitations

    Controlled evaluations find verifier evidence improves recovery beyond patch-only conditioning, and FC-SWE beats independent resampling at a matched two-attempt budget.

    Source: §1 · p. 3

    Because recovery trajectories depend on preceding outcomes, active-set advantage estimation does not inherit the standard unbiasedness guarantee of an action-independent baseline.

    Source: §3.4 · p. 6

Abstract

from arXiv · show

Repository-level software engineering (SWE) is a challenging long-horizon setting: agents must reason over extended interactions, use tools, and adapt to stateful environments. Recent work trains SWE agents with reinforcement learning methods such as Group Relative Policy Optimization (GRPO), which independently sample multiple trajectories per issue, test the resulting patches, and compare terminal rewards within a fixed group. However, this training setup does not reuse verifier feedback from failed patches as context for subsequent attempts, even though this feedback contains valuable diagnostic information about what went wrong. Training on recovery trajectories is challenging because the preceding outcome determines whether the next trajectory is generated, while the failed execution determines its conditioning context. We introduce FC-SWE, a failure-conditioned RL framework that incorporates recovery attempts into policy training. After a patch fails verification, FC-SWE restores the repository to its original task state and uses the failed patch and verifier feedback as context for a recovery trajectory. FC-SWE adapts GRPO to these chains of complete, multi-turn tool-use trajectories through two mechanisms. Trajectory-local rewards preserve each attempt's verifier outcome, preventing recovery success from rewarding an earlier failed patch. Active-set advantage estimation forms a comparison group from all initial and recovery trajectories actually executed for the same issue, so failed attempts remain in the group while unexecuted attempts are excluded. On all 500 SWE-bench Verified tasks under a verifier-assisted protocol, FC-SWE with Qwen3.5-4B and SWE-agent achieves 41.7% Resolved@1 and 52.8% Resolved@2, compared with 38.9% and 48.5% for GRPO. Although trained with at most two attempts per chain, FC-SWE reaches 70.7% Resolved@11 under an eleven-attempt test-time budget.

1. Introduction

FC-SWE addresses the gap between independent-attempt RL and recovery from failed patches by conditioning new trajectories on verifier feedback after a clean reset. It adapts GRPO to these outcome-dependent chains and reports stronger results than GRPO across the benchmark and test-time attempt budgets.

1. Introduction

  • Independent-attempt RL does not explicitly train policies to use failed-verification diagnostics in subsequent recovery attempts.Such feedback can include failing tests, error messages, and tracebacks, but outcome-dependent generation requires decisions about reward assignment and comparison groups.Source: §1 · pp. 1–2
  • After verification failure, FC-SWE restores the repository and conditions a recovery trajectory on the failed patch and verifier feedback.The initial trajectory always runs; each recovery exists only if the preceding trajectory fails.Source: §1 · p. 2, Figure 1 · p. 2
  • Trajectory-local rewards preserve each attempt’s verifier outcome, while active-set advantages compare executed trajectories for the same issue and exclude unexecuted recovery slots.A later recovery success does not change the earlier failure’s reward.Source: §1 · pp. 2–3
  • 41.7% Resolved@1 and 52.8% Resolved@2 for FC-SWE, versus 38.9% and 48.5% for GRPO, across all 500 SWE-bench Verified tasks.The same policy reaches 70.7% cumulative resolution with up to eleven verifier-guided test-time attempts, versus 67.3% for GRPO.Source: §1 · p. 3

2. Related Work

Related work spans repository-level SWE infrastructure, RL for software and long-horizon agent trajectories, and feedback-based refinement. FC-SWE differs by training recovery after patch verification through complete tool-use trajectories.

2. Related Work

  • SWE-bench evaluates repository-level issue resolution, while SWE-agent studies agent–computer interfaces; other work supplies agent infrastructure, training environments, verifiers, or RL for software tasks.Satori-SWE trains iterative patch refinement using reward differences between successive patches, whereas FC-SWE trains recovery after verification through complete tool-use trajectories.Source: §2 · p. 3
  • GRPO uses critic-free group-relative optimization, while related long-horizon methods compare episodes, shared-state steps, or transitions across context boundaries.FC-SWE instead assigns each complete attempt its own verifier reward and compares executed initial and recovery trajectories for the same issue.Source: §2 · pp. 3–4
  • Figure 2 summarizes a pipeline that executes verifier-delimited attempts, resets failed chains, assigns trajectory-local rewards and active-set advantages, and optimizes a token-normalized policy update.Failed chains resume with bounded failure evidence, while executed trajectories are packed with inactive transport padding.Source: Figure 2 · p. 4
  • Feedback-based refinement ranges from inference-time self-repair to RL trained on multi-turn feedback, with some methods regenerating all-failed groups or using reference-conditioned mistake reports.FC-SWE triggers recovery per failed trajectory within the attempt budget and retains the failed patch and verifier output as context.Source: §2 · p. 4

3. Method

FC-SWE extends GRPO to outcome-dependent recovery chains, preserving each attempt’s verifier outcome and comparing the trajectories actually executed for the same issue.

3. Method

  • FC-SWE extends GRPO to recovery chains, using trajectory-local rewards and active-set advantage estimation to train on executed attempts.A failed attempt triggers a recovery trajectory conditioned on the failed patch and verifier feedback after a clean repository reset.Source: §3 · p. 4

3.1. Problem Setup and GRPO Baseline

  • GRPO independently samples a fixed group of trajectories for each issue, unlike FC-SWE’s outcome-dependent recovery chains.The implementation uses a leave-one-out reward baseline.Source: §3.1 · p. 5

3.2. Outcome-Dependent Recovery Chains

  • A recovery attempt runs only after its predecessor fails; the repository is reset, while the failed patch and diagnostic verifier output inform recovery context.The context-construction operator formats and truncates the issue and failure evidence to a fixed context budget.Source: §3.2 · pp. 5–6

3.3. Trajectory-Local Verifier Rewards

  • Each executed trajectory receives only its own patch’s verifier outcome, so a later recovery success does not relabel an earlier failed patch.Unexecuted attempt positions are distinct from trajectories with zero reward.Source: §3.3 · p. 6

3.4. Active-Set Advantage Estimation

  • Active-set estimation pools executed initial and recovery trajectories by issue, excluding inactive rows; its outcome-dependent comparison set lacks the standard unbiasedness guarantee.In a recovered chain with rewards [0, 1], the recovery receives positive advantage and the preceding failure negative advantage without changing either reward.Source: §3.4 · p. 6

3.5. Policy Optimization

  • The clipped policy-gradient objective weights valid policy-generated tokens, while prompts, environment observations, copied failure context, and padding receive zero loss weight.The objective is normalized by the total valid generated tokens across the distributed batch.Source: §3.5 · p. 7

4. Experiments

FC-SWE is evaluated against Base and GRPO on SWE-bench Verified, with ablations probing recovery context and reward design. The experiments also test feedback-conditioned retries and scaling beyond the two-attempt training horizon.

4.1. Experimental Setup

  • Evaluation uses SWE-agent on all 500 SWE-bench Verified tasks, with a shared feedback-enabled controller and attempt budgets for Base, GRPO, and FC-SWE.Training uses 4,518 tasks from ten R2E-Gym repositories; GRPO and FC-SWE start from the same Qwen3.5-4B policy.Source: §4.1 · p. 7

4.2. FC-SWE Improves Initial Solving and Recovery

  • FC-SWE achieved 41.7% Resolved@1 and 52.8% Resolved@2 on all 500 SWE-bench Verified tasks; these first-attempt and cumulative two-attempt results were averaged over three decoding seeds.Among initial failures, conditional recovery was 19.0% for FC-SWE versus 15.7% for GRPO.Source: §4.2 · p. 7, Table 1 · p. 8
  • Over the first 70 optimization steps, recovery adds 10.6 percentage points to FC-SWE’s success rate, producing a 9.9-point average advantage over GRPO’s single-attempt rate.First-attempt success rates are similar on average, with FC-SWE 0.7 percentage points lower; this online comparison includes the extra recovery attempt.Source: §4.2 · p. 7

4.3. Ablations of FC-SWE Training

  • Removing training-time failure context lowers Resolved@2 from 52.8% to 47.9% and conditional recovery from 19.0% to 12.6%.The Fresh-retry variant retains failure-triggered attempts, clean resets, trajectory-local rewards, and active-set advantage estimation, but omits the failed patch and feedback from its second-attempt context.Source: §4.3 · p. 8
  • Without trajectory-local rewards, conditional recovery falls from 20.4% to 17.7% and Resolved@2 from 54.8% to 51.8% at selected one-seed checkpoints.The shared-reward variant assigns [1, 1] to a recovered chain with verifier outcomes [0, 1].Source: §4.3 · p. 8
  • Removing runtime-local penalties in the shared-reward configuration drops online success from 76.6% at step 28 to 2.3% at step 31.The run was not evaluated downstream, and the comparison does not isolate penalty effects under every reward scheme.Source: §4.3 · p. 8

4.4. Learning to Use Failure Evidence

  • At a matched two-attempt budget, feedback raises FC-SWE recovery to 19.0% versus 17.0% with resampling; adding verifier evidence to patch-only context raises recovery by 3.3 points.Feedback-conditioned retries are not uniformly better: Base and GRPO recover less with feedback than with resampling.Source: §4.4 · p. 9

4.5. Test-Time Scaling and Efficiency

  • After training with at most two attempts, FC-SWE reaches 70.7% Resolved@11, versus 67.3% for GRPO and 61.1% for Base.At the lowest evaluated budgets reaching 50% resolution, estimated token use is approximately 3.15M per task for FC-SWE, 4.39M for GRPO, and 8.22M for Base; these are discrete points, not interpolations.Source: §4.5 · pp. 9–10

5. Conclusion

The paper concludes that FC-SWE trains software engineering agents to recover from failed patches using trajectory-local rewards and active-set advantage estimation. Its AI-use statement describes assistance in research and manuscript work and author checks of AI-assisted claims.

5. Conclusion

  • FC-SWE trains SWE agents to recover from failed patches using trajectory-local rewards and active-set advantage estimation.On Qwen3.5-4B, it improves initial solving and recovery over GRPO; with at most two training attempts, it reaches 70.7% resolution at eleven test-time attempts.Source: §5 · p. 10

AI use statement

  • Generative AI tools assisted with hypothesis refinement, methodology feedback, literature review, result interpretation, and manuscript drafting and editing.The authors say they checked AI-assisted claims against implementation, logs, primary literature, and evaluation records, and take responsibility for the final content.Source: AI use statement · pp. 10–11

A. Verifier Choice and Evaluation Protocol

The study uses the official SWE-bench verification procedure as a common verifier rather than introducing a new one. Its multi-attempt evaluation measures verifier-assisted recovery under the paper’s shared comparison conditions.

A. Verifier Choice and Evaluation Protocol

  • The official SWE-bench verification procedure supplies binary patch outcomes and diagnostic output for task-specific tests in R2E-Gym training and SWE-bench Verified evaluation.The paper distinguishes this shared verification procedure from sharing training and evaluation tasks.Source: A. Verifier Choice and Evaluation… · p. 14
  • Base, GRPO, and FC-SWE use the same verification procedure, feedback construction, and evaluation attempt budgets in the controlled Qwen3.5-4B comparison, except where feedback content is ablated.This holds the verifier constant for the training comparison, without claiming it is optimal for deployment.Source: A. Verifier Choice and Evaluation… · p. 14
  • Multi-attempt results measure recovery with benchmark-test feedback and stop-on-success, not independent-sampling Pass@K; the three seeds are decoding seeds, not independent RL runs.The paper also cautions that this feedback access is not necessarily identical to that of other published systems.Source: A. Verifier Choice and Evaluation… · p. 14

B.1. Data, Initialization, and Checkpoint Selection

The appendix documents the training setup, recovery bookkeeping, and evaluation metrics, then uses selected cases to illustrate recovery and regression preservation. These examples are illustrative, not estimates of how often the behaviors occur.

B.1. Data, Initialization, and Checkpoint Selection

  • Training used 4,518 tasks from ten R2E-Gym repositories; the reported FC-SWE checkpoint was selected using multiple validation sets, including 100 SWE-bench Verified tasks.Training tasks had no repository overlap with SWE-bench Verified; the reported checkpoint was step 67.Source: §B.1 · p. 14

B.2. Hyperparameters

  • Sixteen tasks with eight initial chains each yield 128 initial attempts per update; up to 128 additional attempts activate after initial failures.Unused positions in the 256-row transport tensor are padding, not sampled trajectories.Source: §B.2 · p. 14, Table 3 · p. 15

B.3. Recovery Context and Trajectory Bookkeeping

  • Recovery context is constructed in a fixed order, while task identifiers define active-set comparison groups and inactive rows enter neither reward statistics nor the policy loss.Failure evidence is input context and receives no policy-loss weight.Source: §B.3 · p. 15

B.4. Serialization and Runtime Consistency

  • The implementation aligns serialization and message boundaries across tokenization, serving, the agent harness, and stored rollouts; prefix caching is disabled as an implementation setting, not a method requirement.Invalid tool calls are identified from runtime execution results rather than an additional text-schema check.Source: §B.4 · p. 15

B.5. Metrics and Seed Aggregation

  • Conditional recovery pools recovered tasks and initial failures across seeds, and success terminates the chain so already-correct patches generate no later attempts.The three seeds are decoding seeds, not independent RL training runs.Source: §B.5 · p. 16

C. Case Studies: Revising a Failed Patch and Preserving Existing Behavior

  • The case-study comparison uses selected examples with Round 0 and Round 1 denoting first and second attempts; these examples do not estimate behavior frequency.Target tests check the reported problem, while regression tests check whether previously working behavior still works.Source: C. Case Studies: Revising a Failed… · p. 16

C.1. Revising a Failed Attempt After Verifier Feedback

  • In the Astropy case, FC-SWE revised its failed patch after feedback and passed both target tests while preserving all 644 regression tests.GRPO repeated its patch and both target tests still failed; the policies received their own failed patches and feedback after clean resets.Source: §C.1 · pp. 16–17, Figure 6 · p. 17

C.2. Fixing the Reported Error Without Introducing Regressions

A Django case shows that apparently similar fixes can differ in regression behavior. The ablation design isolates some effects of failure conditioning and reward assignment, but not every implementation choice.

C.2. Fixing the Reported Error Without Introducing Regressions

  • Both Django patches passed the target test, but GRPO caused 30 regression-test failures while FC-SWE retained the per-pass reset and caused none.This first-attempt contrast concerns patch quality and regression testing, not successful second-attempt recovery.Source: §C.2 · pp. 17–18

D.1. What the Ablations Isolate

  • The training-time failure-context ablation removes failed patches and verifier feedback from recovery prompts while retaining retries, clean resets, trajectory-local rewards, and active-set estimation.The reward-sharing ablation broadcasts the final chain outcome to executed trajectories; its penalty-removal variant does not isolate penalty effects under trajectory-local rewards, and not every implementation choice is separately ablated.Source: §D.1 · p. 18

D.2. Shared-Reward Checkpoint Analysis

Checkpoint comparisons favor trajectory-local rewards over shared rewards on the reported metrics, but later training does not improve first-attempt and recovery performance uniformly. Additional feedback-content controls do not establish that the combined context is optimal.

D.2. Shared-Reward Checkpoint Analysis

  • 43.2% Resolved@1 and 54.8% Resolved@2 with trajectory-local rewards compare with 41.4% and 51.8% for shared rewards at selected checkpoints.The 3.0-point Resolved@2 gap comprises 1.8 points in first-attempt resolution and 1.2 in additional second-attempt resolutions; checkpoints are at different steps.Source: §D.2 · p. 18, Table 4 · p. 19
  • From steps 67 to 84, local-reward first-attempt resolution rises while recovery and Resolved@2 fall; removing runtime-local penalties in the shared-reward configuration drops online success from 76.6% at step 28 to 2.3% at step 31.The penalty-removal run has no downstream evaluation, and the missing table entry does not mean zero benchmark resolution.Source: §D.2 · p. 18

D.3. Additional Feedback-Content Controls

  • Verifier-only context reaches 20.4% conditional recovery, compared with 19.0% for verifier evidence plus the failed patch, 15.7% for patch-only context, and 17.0% without failure context.These point estimates do not establish that the patch is unnecessary, that combined context is optimal, or that verifier-only is statistically superior.Source: §D.3 · pp. 18–19
  • A same-task shuffled-evidence control is inconclusive about pairing feedback with a particular failed patch because different attempts can expose the same failing tests.The shuffle can retain task-level information and is not equivalent to removing verifier evidence.Source: §D.3 · p. 19

E.1. Per-Seed Resolution Counts

The appendix details test-time scaling, token accounting, and trajectory statistics. Its resolution–cost comparisons use discrete observed budgets and estimated costs, with explicit limits on interpreting those estimates as deployment savings.

E.1. Per-Seed Resolution Counts

  • The same policy is evaluated at test-time budgets through 11 attempts despite training chains using at most two; Figure 7(a) shows the fraction of first-attempt failures recovered by budget K.The three-seed union at K=11 resolves 78.8% for FC-SWE, 75.8% for GRPO, and 73.2% for Base, but permits up to three times as many attempts per task.Source: §E.1 · p. 19, Table 5 · p. 19, Figure 7 · p. 20

E.2. Token Accounting and Coverage

  • Token costs sum executed attempts through budget K, including unresolved tasks, and include repeated input-context processing; three missing Base trajectory logs undercount its cost.The comparison is approximate, not a direct FLOP measurement or measured end-to-end latency.Source: §E.2 · p. 20

E.3. Observed Resolution–Cost Operating Points

  • At observed discrete budgets reaching 50%, costs are 8.22M, 4.39M, and 3.15M tokens per task for Base, GRPO, and FC-SWE; at 60%, they are 13.57M, 6.29M, and 5.08M.The selected budgets are respectively 5, 3, and 2 at 50%, and 10, 5, and 4 at 60%; attained resolution can differ, so this is not an exact equal-accuracy speedup.Source: §E.3 · p. 20

E.4. Full Per-Budget Accounting

  • Table 6 reports token costs, executed attempts, and cumulative resolution; its hours column is a synthesized estimate for 500 tasks, not measured latency.Parallel scheduling, infrastructure overhead, and verifier execution are not fully characterized, so the estimate does not establish end-to-end speedup.Source: §E.4 · pp. 20–21, Table 6 · p. 21

F.1. Definitions and Averaging Population

  • Training statistics include active initial and recovery attempts for FC-SWE, while GRPO contributes initial attempts only; populations change as policies improve.Tool-call turns include invalid calls, generated length excludes inputs and tool observations, and logged truncation is a total-trajectory-token budget event rather than the 100-interaction limit.Source: §F.1 · p. 22

F.2. Observed Trends and Interpretation

FC-SWE and GRPO show similar tool-call turns but differ in generated tokens and logged truncation per active attempt over steps 61–70. These execution statistics are not total training cost, and cross-model reference rows use different timeouts.

F.2. Observed Trends and Interpretation

  • These per-attempt execution statistics do not measure total training cost because FC-SWE also generates recovery attempts; lower truncation alone does not establish successful termination or explain improved test-time efficiency.Test-time token efficiency is assessed separately in Section 4.5.Source: §F.2 · p. 22
  • Over steps 61–70, FC-SWE averages 62.0 tool-call turns and 16.2k generated tokens per active attempt, versus 60.4 turns and 17.8k tokens for GRPO; logged truncation is 5.2% versus 10.5%.Figure 8 reports tool-call turns, generated tokens, and logged truncation per active attempt.Source: Figure 8 · p. 22, §F.2 · p. 22
  • Qwen3.5-9B and Nemotron-3-Nano-4B reference rows use 60-minute timeouts, unlike the 30-minute controlled Qwen3.5-4B policies, so they do not isolate recovery training.Source: §F.2 · p. 22

G. Limitations and Future Work

The paper identifies verifier feedback, feedback quality, experimental coverage, and cost accounting as boundaries for its findings. It proposes deployment-realistic verifiers, broader training and evaluation, and compute-matched studies as future work.

G. Limitations and Future Work

  • The evaluation assumes benchmark-test feedback after each attempt, while deployment may require learned critics, agent-generated tests, or hybrid verification.Future evaluation should separate recovery guidance from held-out final scoring and assess verifier cost, false acceptance, false rejection, and diagnostic usefulness.Source: G. Limitations and Future Work · p. 23
  • Finite test suites do not guarantee semantic correctness or full satisfaction of an issue description, and recovery under incomplete, noisy, or misleading feedback remains to be evaluated.Cross-verifier evaluation and independent patch review could distinguish robust repair from adaptation to a particular feedback source.Source: G. Limitations and Future Work · p. 23
  • The controlled study uses Qwen3.5-4B and one training configuration; multiple decoding seeds measure inference variation, not variation across independently trained policies.The paper calls for multiple training seeds, model sizes, task distributions, and separately controlled optimization components to test generality.Source: G. Limitations and Future Work · p. 23
  • FC-SWE generates extra training trajectories, so the comparison is not matched in total training compute and does not establish end-to-end deployment savings.Future work should compare equal training-token or wall-clock budgets and report exact tokenization, complete logs, verifier cost, and measured latency.Source: G. Limitations and Future Work · p. 23
Loading 2610.07898v1…