PaperlayerView exact revision on arXiv

Source-linked AI summary

SWE-Ladder: Learning to Localize, Reproduce, and Patch via Progressive Privileged Self-Distillation

Xudong Cai, Zhengxi Lu, Yuchen Yan, Jian Shao, Jun Xiao, Yueting Zhuang, Yongliang Shen

arXiv:2610.33667v1cs.AIcs.SE

TL;DR

Repository-level repair agents receive a final test result as their reinforcement-learning signal, while full-patch OPSD lacks stage-specific guidance for localization and reproduction. SWE-LADDER routes an end-to-end agent’s steps to repair stages and distills matching evidence in repair order. It improves pass@1 on SWE-bench Verified under both R2E-Gym and OpenHands, with ablations indicating that stage order matters.

  • Problem

    Trajectory-level rewards and full-patch OPSD do not provide stage-specific guidance for localization or reproduction in multi-step repository repair.

    Source: §1 · pp. 1–2

  • Method

    SWE-LADDER uses a deterministic tool-call router to match teacher evidence to localization, reproduction, or patch generation while training an end-to-end agent.

    Source: §1 · p. 2

  • Results

    16.4% pass@1 with R2E-Gym and 17.6% with OpenHands, versus respective starting values of 9.6% and 13.0%.

    Source: §5 · p. 8

  • Takeaways & Limitations

    Ablations indicate that both stage-matched evidence and the order of repair stages matter for pass@1.

    Source: §5 · p. 8

    Evidence is limited to one Qwen3-4B policy, SWE-bench Verified, and two scaffolds, without independent random-seed repeats or confidence intervals.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Resolving a repository-level issue typically requires a software-engineering (SWE) agent to localize the faulty code, reproduce the failure, and patch the code. Reinforcement learning trains such agents from a single test result on the final patch, while on-policy self-distillation (OPSD) adds token-level supervision from a teacher with privileged evidence, exposing the full reference patch at every step. Neither provides stage-specific guidance for localization or reproduction. We propose SWE-Ladder, which turns the verified repair of each training instance into stage-specific evidence: the changed files and lines for localization, the fail-to-pass tests for reproduction, and the diff for patching. A rule-based router assigns each step of an end-to-end rollout to a stage based on its tool call, a frozen teacher sees only the matching evidence, and the stages are distilled in repair order while GRPO optimizes the full trajectory. With Qwen3-4B-Instruct-2507 on SWE-bench Verified under the R2E-Gym scaffold, SWE-Ladder improves pass@1 from 9.6% to 16.4%, compared with 10.4% for GRPO and 12.4% for OPSD with the full patch. Under the OpenHands scaffold, it improves pass@1 from 13.0% to 17.6%. Reversing the stage order under R2E-Gym reduces pass@1 to 11.4%.

1 Introduction

Trajectory-level rewards and full-patch distillation provide limited guidance about intermediate repair decisions. SWE-LADDER aligns teacher evidence with localization, reproduction, and patching, then distills those stages in repair order; it improves pass@1 across two scaffolds.

1 Introduction

  • The paper contrasts trajectory-level GRPO rewards, full-patch OPSD supervision, and SWE-LADDER’s stage-specific evidence introduced one stage at a time.Figure 1 summarizes the supervision signals.Source: Figure 1 · p. 1
  • Under GRPO, pass@1 moves from 9.6% to 10.4% while reproduction falls from 59.4% to 48.6%; full-patch OPSD raises pass@1 but lowers post-edit verification from 22.6% to 16.2%.The three diagnostics track editing reference-changed files, reproducing the failure with a script, and rerunning that script successfully after the last edit.Source: §1 · p. 1, Figure 2 · p. 2
  • Full-patch teacher evidence conflates localization, reproduction, and patching cues, potentially favoring editing over searching during localization.The proposed schedule introduces localization, reproduction, and patch generation in that order.Source: §1 · p. 2
  • SWE-LADDER keeps one end-to-end agent’s actions unconstrained while a deterministic tool-call router assigns localization, reproduction, or patch generation and gives the teacher the matching repair slice.Stages are distilled one at a time while GRPO optimizes the full trajectory.Source: §1 · p. 2
  • Introducing the stages in order, SWE-LADDER reaches 16.4% pass@1 and is the only method improving both localization and verification without lowering reproduction; on the OpenHands scaffold, pass@1 rises from 13.0% to 17.6%.Reversing the stage order lowers pass@1 to 11.4%.Source: §1 · p. 2
  • The contributions include stage-wise diagnostics showing outcome rewards do not consistently improve intermediate behavior and full-patch teaching lowers post-edit verification.The authors also report that removing a stage, reversing stage order, or training all stages at once lowers pass@1.Source: §1 · p. 2

2 Related Work

Prior SWE-agent training largely relies on trajectory-level feedback, while agentic distillation varies how teacher guidance is distributed. Repair subgoals have also been made explicit in pipelines or training data; SWE-LADDER instead aligns teacher evidence with subtasks while keeping the agent end-to-end.

2.1 Post-Training for SWE Agents

  • For RL-based SWE agents, feedback remains largely trajectory-level, with limited supervision for intermediate tool decisions.Related approaches include patch-similarity and verifier-based rewards, multi-turn RL, and lightweight SFT without additional multi-turn RL.Source: §2.1 · p. 2

2.2 On-Policy Distillation for Agents

  • Agentic distillation methods vary loss gating, step-level credit, trajectory depth, and evidence exposure; SWE-LADDER instead orders evidence across repair stages.Examples include SDAR, StepOPSD, SOD, TCOD, AR-OPD, ATESD, and EviSD.Source: §2.2 · pp. 2–3

2.3 Subtask-Oriented Software Repair

  • Prior repair methods fix subgoals into inference pipelines or training data, whereas SWE-LADDER keeps the agent end-to-end and aligns privileged evidence with subtasks.Unlike difficulty-ordered curricula, it keeps each task fixed and orders the evidence shown to the teacher.Source: §2.3 · p. 3

3 Method

SWE-LADDER trains one end-to-end repair policy by routing trajectory steps to localization, reproduction, or patch generation and distilling stage-matched privileged evidence. GRPO continues to optimize the full trajectory as the teacher evidence advances in repair order.

3 Method

  • SWE-LADDER targets localization, failure reproduction, and patch generation in sequence, using stage-specific privileged evidence while GRPO optimizes the full end-to-end trajectory.Only response tokens labeled for the active subtask receive OPSD supervision; GRPO covers all valid response tokens.Source: §3 · p. 3, §3.3 · p. 4

3.1 Rule-Based Stage Routing

  • A deterministic tool-level state machine labels each generated step LOC, REP, or PATCH, routes teacher context, and leaves tool choices, rollouts, and rewards unconstrained.State-dependent backtracking is allowed.Source: §3.1 · p. 3

3.2 Stage-Specific Privileged Evidence

  • Three verified-repair evidence views support distinct subgoals: changed-file and old-hunk clues for localization, failing-test evidence for reproduction, and the repair diff for patching.The evidence is factual content from the repository, tests, and verified repair, without knowledge generated by external large language models.Source: §3.2 · p. 3

3.3 Full-Trajectory GRPO Objective

  • GRPO applies across all valid model-generated response tokens throughout the end-to-end trajectory, regardless of the active training stage.Stage labels annotate distillation; they do not truncate a rollout at stage boundaries.Source: §3.3 · p. 4

3.4 Stage-Conditioned OPSD and Joint Objective

  • During each training iteration, the active stage selects the teacher’s privileged context, and OPSD supervises only response tokens assigned to that stage.The teacher is initialized from the student’s base checkpoint and remains frozen throughout training.Source: §3.4 · p. 4

3.5 Progressive Privilege Scheduling

  • SWE-LADDER trains in the fixed order LOC → REP → PATCH, carrying each phase forward from the preceding checkpoint.The order follows how repair uses evidence: patch evidence helps once the agent can find the code, and reproduction tests provide a basis for checking the patch.Source: §3.5 · p. 4

4 Experiments

Experiments evaluate SWE-LADDER with Qwen3-4B-Instruct-2507 on SWE-bench Verified, comparing repair performance, training dynamics, stage ablations, and checkpoint behavior. The results cover R2E-Gym and OpenHands scaffolds.

4.2 Main Results

  • 16.4% pass@1 for SWE-LADDER on R2E-Gym compares with 10.4% for GRPO and 12.4% for GRPO+OPSD; on OpenHands, it reaches 17.6% versus the base policy’s 13.0%.On R2E-Gym, SWE-LADDER uses 19.3 steps versus 26.5 for the base model and completes 98.6% of trajectories versus 36.0% for GRPO.Source: §4.2 · pp. 5–6, Table 2 · p. 6

4.5 Analysis

  • Only SWE-LADDER ends above the base model on localization, reproduction, verification, and pass@1; reproduction and verification dip during REP and recover during PATCH.The LOC phase improves all four metrics, while the REP phase improves localization but leaves reproduction and verification below the base model.Source: §4.2 · p. 6, Figure 4 · p. 6, §4.5 · p. 8

4.3 Training Dynamics

  • SWE-LADDER’s entropy and gradient norm remain relatively stable, while GRPO’s decline and its average response length nearly doubles.For GRPO, entropy falls from 0.0300 to 0.0061, gradient norm from 1.820 to 0.187, and response length rises from 8.29k to 16.73k tokens; SWE-LADDER’s corresponding measures are 0.0251 to 0.0221, 2.353 to 2.397, and 10.09k to 11.62k tokens. These observations do not establish a general stability guarantee.Source: §4.3 · pp. 6–7, Figure 5 · p. 6

4.4 Ablation Study

  • 16.4% pass@1 for LOC→REP→PATCH exceeds 11.4% for PATCH→REP→LOC and 12.6% for REP→LOC→PATCH.Removing LOC, REP, or PATCH lowers pass@1 from 16.4% to 11.2%, 13.2%, or 14.0%, respectively; the all-stages-at-once variant reaches 12.4%.Source: §4.4 · p. 7, Table 4 · p. 7, Table 3 · p. 7
  • The neural router achieves higher stage-label agreement than rule-based routing, 98.63% versus 92.0%, but yields lower downstream pass@1.Matched evidence also outperforms swapped evidence: for PATCH targets, matched PATCH evidence reaches 13.2% pass@1 versus 11.8% with LOC evidence.Source: §4.4 · pp. 7–8, Table 5 · p. 8

5 Conclusion

SWE-LADDER studies what privileged evidence a teacher should receive when training an end-to-end software-engineering agent. It gives the teacher only evidence for the current repair stage and introduces stages in repair order.

5 Conclusion

  • SWE-LADDER gives the teacher only current-stage evidence and introduces localization, reproduction, and patch generation in repair order.The conclusion reports pass@1 gains from 9.6% to 16.4% with R2E-Gym and from 13.0% to 17.6% with OpenHands; ablations indicate stage-matched evidence and stage order matter.Source: §5 · p. 8

Limitations

The evaluation covers one Qwen3-4B policy, SWE-bench Verified, and two agent scaffolds, with main ablations and stage-wise diagnostics confined to R2E-Gym. Single checkpoint lineages and reliance on verified-repair metadata and a tool-call parser limit the strength and scope of the evidence.

Limitations

  • Evidence is limited to one Qwen3-4B policy, SWE-bench Verified, and two scaffolds; transfer to larger backbones, other benchmarks, or substantially different tool interfaces is unestablished.Main ablations and stage-wise diagnostics use R2E-Gym, despite the second-scaffold OpenHands check.Source: Limitations · p. 9
  • Training curves and router comparisons use one checkpoint lineage without independent random-seed repeats or confidence intervals, so observed differences are configuration-specific evidence.The method also depends on verified-repair metadata and a parser that maps tool calls to repair stages.Source: Limitations · p. 9

Ethics Statement

The work uses public benchmark issues, open-weight Qwen3 models, a hosted Qwen3.8-Max model for rollout annotation, and public agent scaffolds. Evaluation stays in isolated benchmark sandboxes, and privileged repair evidence is used only during training.

Ethics Statement

  • The study uses publicly available SWE-bench Verified issues, open-weight Qwen3 models, hosted Qwen3.8-Max for rollout annotation, and public R2E-Gym and OpenHands scaffolds.It recruits no human participants or collects new personal data.Source: Ethics Statement · p. 9
  • Agents are evaluated in isolated benchmark sandboxes and do not act on external systems; deployment beyond these sandboxes requires additional safety validation, permission controls, and human oversight.Privileged repair evidence is absent at inference.Source: Ethics Statement · p. 9

A.1 Stage-wise Capability Evaluation

The appendix defines stage-wise behavioral diagnostics, details how tool calls drive stage labels and token alignment, and compares neural and rule-based routers. The rule-based router yields the stronger downstream training result despite lower label agreement.

A.1 Stage-wise Capability Evaluation

  • Figure 2’s diagnostics measure localization, reproduction, post-edit verification, and solve behavior; localization is macro-averaged task-level recall, while the other metrics are trajectory-level percentages.Localization tracks edits to reference-changed non-test files; reproduction requires running a script and observing failure, while verification checks for a successful rerun after the last edit.Source: §A.1 · p. 12

A.2 Stage Annotation and Token Alignment

  • The rule-based router proposes LOC, REP, or PATCH from parsed tool calls, advances through LOC→REP→PATCH, and uses state-dependent backtracking; resulting labels mark the same step’s reasoning and action tokens.The annotation does not constrain tool choices, rollouts, or rewards; OPSD uses the active-stage indicator to select teacher-supervised response tokens.Source: §A.2 · p. 12

A.3 Neural Router and Rule-Based Router Comparison

  • The rule-based router’s 92.0% agreement is below the neural router’s 98.63%, yet it gives the stronger downstream training result in the comparison.Agreement is measured against Qwen3.8-Max annotations, not independently adjudicated semantic labels; the two routers’ agreement and solve rate assess different properties.Source: §A.3 · p. 13

A.4 Evaluation Protocols

The appendix specifies separate evaluation protocols for the two scaffolds, then details termination-aware reward shaping and stage-masked top-K distillation. The implementation keeps the full-response GRPO objective while restricting OPSD to active-stage response tokens.

A.4 Evaluation Protocols

  • The OpenHands and R2E-Gym configurations use separate scaffolds and evaluation harnesses, each with at most 150 agent steps and temperature 0.7.OpenHands uses a 128K context window and R2E-Gym uses 64K; their results are reported separately.Source: §A.4 · p. 13

A.5 Termination-Aware Reward Shaping

  • Termination-aware shaping discounts successful verifier results for soft-overlong termination and penalizes voluntary unsolved finishes only when no successful edit was made.Exploration-tool reward and the no-tool-call penalty are both set to zero in the main training chain.Source: §A.5 · pp. 13–14

A.6 Top-K Distillation and Systems Configuration

  • OPSD targets are aligned to student-sampled prefixes, but only valid response positions assigned to the active stage contribute to its loss.GRPO averages over valid response tokens, whereas OPSD averages over active-stage tokens; a rollout with none contributes zero OPSD loss.Source: §A.6 · p. 14, §A.5 · p. 14

A.7 Progressive Privilege Scheduling and Training Configuration

SWE-LADDER schedules privileged distillation through localization, reproduction, and patching phases, while preserving end-to-end rollouts and full-response GRPO. Its appendix describes stage evidence views and training configuration.

A.7 Progressive Privilege Scheduling and Training Configuration

  • Training proceeds through LOC, REP, then PATCH phases, with each phase continuing from its predecessor and the active stage selecting teacher context and distillation tokens.The example configuration allocates 30, 20, and 20 iterations respectively; inference deploys one end-to-end policy without a teacher or stage-specific policies.Source: §A.7 · pp. 14–15

B Stage-Routed Privileged Information

B.1 Representative Privileged-Information Views

  • The privileged evidence examples are supplied to the frozen teacher, not included in the student prompt.They are rendered from the structured fields described in the main text.Source: §B.1 · p. 15

C.1 Training Dynamics

Training-dynamics plots compare individual runs over a common 70-iteration horizon, separating shaped training reward from held-out pass@1. GRPO’s diagnostics deteriorate, while SWE-LADDER does not show the same decline, though its reward is noisy and non-monotonic.

C.1 Training Dynamics

  • GRPO’s mean entropy fell from 0.0300 to 0.0061 and gradient norm from 1.820 to 0.187, while its mean generated tokens rose from 8.29k to 16.73k.The comparison is between iterations 1–10 and 61–70.Source: §C.1 · p. 15
  • SWE-LADDER’s entropy means were 0.0251 and 0.0221, and its gradient-norm means were 2.353 and 2.397 across the same iteration windows.Its mean generated length rose from 10.09k to 11.62k; reward remained noisy and did not increase monotonically.Source: §C.1 · p. 15
  • The plots provide diagnostic context for stage-aligned comparisons, but do not establish a general failure of GRPO+OPSD or SDAR.Those baselines retain additional late-training optimization signal, and preserving entropy or gradient magnitude alone does not explain SWE-LADDER’s held-out performance.Source: §C.1 · p. 15

C.2 Metric Definitions

The metric notes distinguish logged optimization diagnostics and generated-token counts from broader response-length fields. They also caution against treating shaped reward as benchmark pass@1 or the teacher-top-K loss as calibrated KL.

C.2 Metric Definitions

  • Entropy and gradient norm retain the implementation’s logged scale rather than being independently recalculated policy statistics.Generated-token counts exclude environment output included in the broader response_length/mean field.Source: §C.2 · p. 15
  • The reported reward includes termination-aware shaping (Appendix A.5) and should not be interpreted as SWE-bench pass@1; teacher-top-K loss is not plotted as a calibrated KL distance for reasons discussed in Appendix A.6.Source: §C.2 · p. 15
  • The accompanying visuals identify localization evidence, reproduction evidence, patch diffs, and a representative router trace with backtracking.The router-trace example notes that rerunning reproduction in PATCH demotes the state to REP, while repeated inspection returns it to LOC.Source: Table 7 · p. 16, Figure 9 · p. 16, Figure 10 · p. 17
Loading 2610.33667v1…