PaperlayerView exact revision on arXiv

Source-linked AI summary

Teaching Agents to Code Reliably

Muhammad Ahmed Mohsin, Myeongsoo Kim, Kangrui Ruan, Shweta Garg, Varun Kumar, Murali Krishna Ramanathan

arXiv:2610.03984v1cs.AIcs.LG

TL;DR

Autonomous coding agents must produce useful repairs and reliable evidence for selecting among them, but extra attempts can repeat flawed edits or tests. This paper trains agents to explore locations and repair approaches and to discriminate with tests scored against incorrect variants. Training improves held-out repair results and transfers to two external suites, while DeepSWE shows partial credit without full resolution.

  • Problem

    Extra inference-time attempts do not guarantee a correct selected patch, because agents may repeat incorrect repairs and tests may accept behavior that matches a patch’s mistaken interpretation.

    Source: §1 · p. 1

  • Method

    The approach combines exploration of edit locations and repair approaches with weighted trajectory supervision and reinforcement learning that scores tests against incorrect programs.

    Source: §1 · p. 1, §2.3 · p. 5

  • Results

    Gains span SWE-bench Pro and SWE-PolyBench, while DeepSWE shows partial credit without full resolution; on held-out SWE-bench Verified, greedy pass@1 rises from 31.9% to 43.0%.

    Source: §1 · p. 2, §5 · p. 9

  • Takeaways & Limitations

    Reliable repair rests on learning fault localization, edit variation, and candidate verification; training a verifier against its own work requires manufactured negatives.

    Source: §5 · p. 9

    Detecting all scored incorrect variants does not establish correctness beyond that set.

    Source: §3.6 · p. 8

Abstract

from arXiv · show

Autonomous coding agents solve repository issues by reading code, running commands, editing files, and submitting patches. Extra inference-time compute yields gains only when it produces a useful repair and supplies reliable evidence for choosing one. Three behaviors decide both, and we argue they are teachable rather than byproducts of scale, so a policy can carry them instead of a scaffold. Location diversity remains narrow, since attempts return to the same site and extra samples add no coverage. Edit diversity is left unexploited, since methodologies that differ resolve complementary issues no single run reaches. Verification misleads, since a test the agent writes for its own patch accepts many incorrect ones. Directing search by execution feedback and scoring each patch against its own reverted tree resolves 52.8% of SWE-bench Verified using 48.1% of the agent-steps an eight-sample baseline spends. Training moves these behaviors into the policy. On the 270 issues held out from SFT and RL training, weighted supervised fine-tuning raises pass@1 from 31.9% to 35.2% and pass@8 from 46.7% to 51.1%. A reinforcement objective then trains the verifier against gold-labeled repairs and incorrect variants, crediting the assertions that detect them. It raises pass@1 to 43.0% and pass@8 to 60.7%, lifts verifier precision from 26.8% to 41.7%, and more than halves false acceptance. Resolution improves on two of three out-of-distribution suites and verifier precision on all three, and the gains hold at 7B, 14B, and 30B against published coder baselines.

1 Introduction

Extra attempts help only when they produce correct repairs and reliable evidence for selecting among them. The paper proposes training agents to explore edit locations and approaches, use execution feedback, and discriminate between correct and incorrect repairs.

1 Introduction

  • Repeated attempts can revisit the same edit location, and an agent-written test can accept incorrect behavior when it shares the patch’s mistaken interpretation.More candidates do not guarantee that the selected patch is correct.Source: §1 · p. 1
  • Candidate coverage measures whether any correct repair is present; selection quality measures whether the agent returns one of those correct candidates.Verification affects both the candidates generated and the patch returned.Source: §1 · p. 1
  • Different problem-solving preambles produce complementary repairs, while repeated self-verification changes few submitted patches and proactive localization visits more sites but resolves fewer issues.These diagnostic observations motivate using execution feedback to guide edits and tests’ ability to detect incorrect repairs to evaluate verification.Source: §1 · p. 1
  • The approach combines test-time exploration of locations and repair approaches with differential verification, then trains the policy in two stages.Differential verification runs a reproduction on both the patched and reverted repository.Source: §1 · p. 1
  • On 270 held-out issues, the two training stages raise greedy pass@1 from 31.9% to 43.0%, while eight-attempt oracle coverage rises from 46.7% to 60.7%.Selected-patch accuracy rises from 33.5% to 50.5%, with selection efficiency increasing from 0.72 to 0.83.Source: §1 · p. 2

2 Methodology

The methodology combines execution-guided repair search with weighted trajectory supervision and reinforcement learning for test discrimination. It trains on curated trajectories and scores tests against incorrect variants and sibling repairs.

2.1 Test-Time Scaling Methods on Coding Agents

  • The diagnostic analyzes all 500 SWE-bench Verified issues, selects 230 for training curation, and holds the remaining 270 out of SFT and RL training.Search procedures share a backbone and scaffold, but have different attempt limits and stopping rules, so comparisons reflect accuracy-cost trade-offs.Source: §2.1 · p. 2
  • Differential-verification search resolves 52.8% of issues versus 46.8% for the eight-sample control, using 48.1% of its agent-steps.It pairs up to eight locations and repair approaches, runs reproductions on patched and reverted trees, and uses repository tests and changed-line execution to stop or prune search.Source: §2.1 · p. 2

2.2 Supervised Fine-Tuning on Weighted Agent Trajectories

  • The SFT corpus retains 5,061 examples from 230 issues, including contrastive-edit and reproduction examples; within each issue, examples are ranked by tiered verifier/gold-label weights, with ties broken by fewer steps.Weights are 1.0, 0.6, 0.25, or 0; only T1 requires a gold label.Source: §2.2 · p. 2

2.3 Reinforcement Learning for Discriminative Verification

  • The reinforcement objective evaluates generated tests against incorrect programs, measuring which incorrect behaviors each test detects.The scoring set includes oracle-rejected variants and sibling repairs; candidates passing gold tests are excluded from the negative set. The reward combines correctness, verification, edit, and process terms, with correctness weighted 3.0 and verification 1.0.Source: §2.3 · pp. 3–4
  • Detecting every scored variant does not establish correctness outside that finite set.The weighted auxiliary reward terms have a combined maximum of 2.6, below the correctness weight of 3.0, so correct rollouts receive higher scalar reward than incorrect ones.Source: §2.3 · p. 4
  • Assertion credit allocates each detected mutant’s class-weighted unit among assertions that detect it, then assigns that credit to tokens in those assertions.The gate opens when the patch resolves the issue or the test detects every scored mutant; complete detection can therefore earn assertion credit even for an incorrect patch, while unsigned assertion credit does not penalize rejecting correct alternatives.Source: §2.3 · p. 5

3 Evaluation

Evaluation tests repair accuracy, candidate coverage and selection, verifier discrimination, reward components, and transfer across held-out and external suites. Training improves held-out resolution and selection, while results also expose limits in transfer scope and verifier generalization.

3.2 Repair Accuracy

  • On the Verified split held out from training, RL from SFT raises pass@1 from 35.2% to 43.0%.SFT first raises pass@1 from 31.9% to 35.2%.Source: §3.2 · p. 5

3.3 Candidate Coverage and Selection

  • Oracle pass@8 rises from 46.7% to 60.7%, Selected@8 from 33.5% to 50.5%, and selection efficiency from 0.72 to 0.83.The same policy reaches 1.98 distinct localizations per issue versus 1.41 for the base model.Source: §3.3 · p. 6

3.4 Verifier Discrimination on Fixed Candidates

  • After training, the median generated test has 11 assertions, compared with none before training, and covers 3 of 5 fault classes, compared with none before training.Its length falls from 42 to 33 lines.Source: §3.4 · p. 6

3.5 Reward Controls

  • Assertion credit raises pass@1 to 43.0% and verifier precision to 41.7%, compared with 40.7% and 38.4% under scalar verification reward.Removing mutation evaluation reduces pass@1 from 43.0% to 38.1%, the largest reduction among the component removals reported.Source: §3.5 · p. 7

3.6 Transfer and Limitations

  • RL improves pass@1 over SFT by 8.3 points on train-exposed issues, 7.7 on test issues from training repositories, and 7.8 on repositories excluded from training.The resulting pass@1 values are 45.7%, 44.4%, and 40.0%, respectively; transfer evaluations also include SWE-PolyBench and SWE-bench Pro.Source: §3.6 · p. 8
  • DeepSWE full resolution remains zero at this model scale despite improved internal measures, including tasks with at least one satisfied fail-to-pass test rising from 17 to 21 of 113.This is partial test satisfaction, not completed repair.Source: §3.6 · p. 8
  • Verifier rejection declines across harder incorrect-repair sources, from 91.0% for familiar-operator mutants to 74.0% for natural incorrect repairs from independent generators.Complete detection of a finite mutant set does not establish correctness beyond that set.Source: §3.6 · p. 8
  • The 7B and 14B evaluations improve, but published coder results are not matched comparisons and agent-step savings exclude token-length and test-execution costs.The held-out split is within a benchmark used for method development; it is not an out-of-distribution evaluation.Source: §3.1 · p. 5, §3.6 · p. 8

4 Related Work

The paper situates its work among repository-agent interfaces, training and test-generation methods, mutation rewards, and revision loops. It distinguishes its approach through a policy that generates both patches and tests and learns from mutation-based signals.

Repository search and feedback

  • Repository-agent work studies inspection, editing, execution, localization, and additional attempts; this diagnostic holds backbone and scaffold fixed to compare location search and verification.Generated tests are also used to measure candidate coverage separately from selected-patch accuracy.Source: §4 · p. 9

Training repair and test generation

  • CodeRL, SWE-Gym, and CURE study execution-feedback training, repository agents and verifiers, or coder–tester co-training; this policy generates both patches and tests.Its reward combines patch correctness with detection of oracle-rejected variants.Source: §4 · p. 9

Mutation rewards and credit assignment

  • MIST-RL and Ockhamareto use mutation rewards; this work instead divides detected-mutant weight among detecting assertions and gates a positive-advantage bonus.The trajectory reward prioritizes repair correctness; the distinction concerns reward and credit assignment, not introducing mutation testing or token-level attribution.Source: §4 · p. 9

Test and patch revision

  • CoHarden is the closest comparison for the revision loop, while this work also trains the policy from those signals and evaluates its verifier on frozen candidates and independent-generator repairs.Evaluation includes correct alternatives and incorrect repairs from independent generators.Source: §4 · p. 9

5 Conclusion

The conclusion frames reliable repair as depending on three learnable behaviors and reports gains from weighted supervision and mutation-based reinforcement learning. It also identifies manufactured negatives as a general constraint when a proposer scores its own work.

5 Conclusion

  • Reliable repair rests on locating faults, varying edits, and verifying which candidate to return; weighted supervision and mutation-based reinforcement learning improve these behaviors.The first two affect candidate coverage, while verification affects selection quality, and the paper estimates their contributions as additive.Source: §5 · p. 9
  • Verifier performance is weakest on repairs from independent generators, where rejection is 74.0%, versus 91.0% for familiar-operator mutants.The paper proposes wider negative distributions and signed credit as improvement directions; the latter raises correct-alternative acceptance to 84.0% in an ablation, not the full evaluation.Source: §5 · p. 9

AI Use Statement

The authors report limited uses of generative AI and coding assistants, while retaining responsibility for reviewed content.

AI Use Statement

  • Generative AI assisted with grammar, clarity, and concision, while coding assistants handled routine tasks such as plotting, table generation, and refactoring.The authors state AI was not used to develop ideas, design methods or experiments, or generate or interpret results; they reviewed the assisted content and take responsibility.Source: AI Use Statement · p. 10

Ethics Statement

The work uses public models and benchmarks and reports no human subjects, personally identifiable information, or new data collection.

Ethics Statement

  • The study uses publicly available models and benchmarks, with no human subjects, personally identifiable information, or new data collection.Source: Ethics Statement · p. 10

Reproducibility Statement

The statement maps the paper’s methods, evaluation protocol, implementation details, training settings, search configurations, and ablations to their locations in the paper.

Reproducibility Statement

A Appendix

  • Appendices A.1, B.13, and B.15 contain search rules, prompts, and experiment configurations; Appendices A.3 and B report ablations.The appendix also includes reward ablations, comparator studies, a worked reward example, transfer results, and model-scale results.Source: Reproducibility Statement · p. 10, A Appendix · p. 11

A.1 Test-time search procedures

The search procedures test how location and methodology diversity affect repair coverage, then combine reactive exploration with differential verification. The combined procedure improves resolution over an eight-sample control while using fewer agent-steps.

Self-verification

  • Self-verification resolves 184/500 (36.8%) at 189,371 agent-steps, with 78–85% of audit passes changing no code.Across four attempts it reaches only 1.10 distinct locations and 1.31 distinct edits.Source: §A.1 · p. 12

Single-axis diagnosis

  • The single-axis diagnosis resolves 189/500 (37.8%) at 41,332 agent-steps, the study’s cheapest run at 219 steps per solve.Source: §A.1 · p. 12

Methodology diversity

  • Eight problem-solving preambles reach 305/500 oracle coverage (61.0%), exceeding any single preamble and revealing complementary repairs.The label-using union is not a deployable selection rule.Source: §A.1 · p. 12

Directed exploration

  • Banning confirmed-wrong edits and stopping on strong positives resolves 217/500 (43.4%), reaching 1.84 locations and 2.79 edits.Early stops occur on 65 issues; proactive localization alone visits 2.46 ranked sites yet resolves 210/500 (42.0%), below blind retries.Source: §A.1 · p. 12

Reactive location diversity

  • Reactive location-banning resolves 225/500 (45.0%), exceeding reactive edit-banning (217), blind retries (213), and proactive site-planning (210).It uses the ban and early-stop rules described in Appendix B.14.Source: §A.1 · p. 12

Differential-verification search

  • 52.8% resolution versus 46.8% for the eight-sample control uses 48.1% of its agent-steps.Differential-verification search averages 503 steps per solve versus 1,179; removing the two-tree differential reduces resolution to 47.4%, while early stopping reduces steps per solve to 728 at unchanged resolution.Source: §A.1 · p. 12

A.2 Reinforcement learning results by starting point

The section reports how the reinforcement objective performs from different starting policies, alongside ablations, implementation boundaries, and transfer evidence. Its examples illustrate mutation-based test discrimination and diverse edit locations.

A.2 Reinforcement learning results by starting point

  • From the supervised policy, reinforcement learning adds 7.8 pass@1 points and 14.9 verifier-precision points, versus gains of 4.0 and 9.6 from the base policy.Both starting points improve on every metric; the verifier comparison uses a shared bank of labeled repairs.Source: §A.2 · p. 12

A.3 Reinforcement Learning Ablations

  • The ablations vary semantic-mutant generation, correctness/verifier reward weights, and GRPO group size; every semantic-mutant configuration beats deterministic-only mutants on all three metrics.The trained setting uses wc:wv = 3.0:1.0 at k=6; increasing group size from k=6 to k=16 adds 0.3 pass@1 and 0.9 precision for more than twice the rollouts per instance.Source: §A.3 · p. 13, Table 9 · p. 14, Table 10 · p. 15

A.4 RL objective implementation details and boundary cases

  • No valid negatives means no kill-fraction or fixpoint credit; infrastructure errors earn no behavioral-kill credit, and failing a correct repair incurs an honesty penalty.The implementation also normalizes assertion credit by assertion span length and uses independently executable test nodes to reconstruct the full matrix; the Shapley reading makes no convergence claim.Source: §A.4 · p. 13

A.5 Comparator Studies

  • Published comparator values are context, not a matched ranking, because evaluated subsets and inference protocols differ.Oracle pass@8 measures candidate existence, while best-of-16 reports a selected candidate under a different budget.Source: §A.5 · p. 13

A.6 A worked example of the mutation-verifier reward

  • In the worked example, the adversary creates eight behavior-changing variants across five fault classes, and the test reaches the fixpoint across those classes.Assertions must fail to eliminate a variant; the four parallel attempts also span distinct edit locations, which the edit term rewards.Source: §A.6 · p. 14

A.7 DeepSWE partial-credit breakdown

  • DeepSWE’s binary grader resolves zero tasks for either policy, so the comparison uses partial credit against fail-to-pass tests.Source: §A.7 · p. 14

A.8 Model-scale trend

  • The section reports repeating the full pipeline on 7B and 14B backbones, and points to Table 14 for the results.Source: §A.8 · p. 15

B Rigor and reproducibility

  • The rigor appendix covers training settings, metric definitions, additional controls, and the populations used for evaluation and diagnostic analysis.Unless another population is stated, Verified evaluations use 270 issues held out from SFT and RL; the diagnostic and differential-verification ablation use all 500 issues.Source: B Rigor and reproducibility · p. 16

B.1 Protocol for the main-text rigor results

The protocol separates issue exposure and controls reward-training comparisons. A generator/verifier swap experiment finds that replacing either role contributes independently to resolution.

B.1 Protocol for the main-text rigor results

  • Table 5a separates training issues, test issues from training repositories, and test issues from repositories excluded from training.Source: §B.1 · p. 17
  • Reward controls use one SFT checkpoint and matched training issues, update budgets, and rollout budgets; the shuffled-credit control permutes assignment across assertion tokens while preserving total credit mass.Source: §B.1 · p. 17

B.2 Generator and verifier swap

  • With the generator fixed at base, replacing only the verifier raises resolution from 31.9% to 36.5% and selection accuracy from 52.0% to 66.0%.Replacing only the generator reaches 38.5%; the interaction estimate is −0.1 points with a 95% interval of [−2.6, 2.4], indicating additive rather than multiplicative contributions in this comparison.Source: §B.2 · p. 17

B.3 Cross-error-source transfer

Cross-error-source evaluation shows that verifier rejection declines as incorrect repairs move farther from training, while remaining above SFT and accepting correct alternatives. The remaining ablations distinguish the effects and scope of trajectory statistics, assertion credit, weighting, and operational metrics.

B.3 Cross-error-source transfer

  • Rejection falls from 91.0% on familiar operators to 74.0% on natural incorrect repairs, while the RL verifier stays 11–12 points above SFT and accepts 78.0% of correct alternatives on natural repairs.Across all four error sources, rejection declines gradually rather than collapsing.Source: §B.3 · p. 17, Table 16 · p. 18

B.4 Trajectory Statistics

  • Reproduce-first behavior decreases from 94.4% to 92.1% on Verified and increases on four transfer suites; other trajectory changes do not establish separate causal effects on resolution.Source: §B.4 · p. 17

B.5 Reproducibility

  • With the generator fixed, swapping in the trained verifier raises held-out Verified pass@1 from 31.9% to 36.5% and selection accuracy from 52.0% to 66.0%.The reported backbone is Qwen3-Coder-30B-A3B-Instruct; training uses a 40k prompt and 65k response context.Source: §B.5 · p. 17, Table 15 · p. 18

B.8 Weighted supervised fine-tuning ablation

  • Full trajectory weighting raises resolution from 33.7% under uniform weights to 35.2%, while distinct localizations rise from 1.55 to 1.98.The exploration term concentrates the diversity gain; interface-compliance tuning reaches 32.6% resolution and 1.45 distinct localizations.Source: §B.8 · p. 18

B.6 Zero-survivor audit

  • Strong-adversary training lowers train-time completion but raises held-out detection from 66.0% to 85.0% versus deterministic-only training.Train-time completion counts training instances; held-out detection is measured on a fresh mutant set after the test is frozen.Source: §B.6 · p. 18, Table 17 · p. 19

B.7 Assertion-credit ablation

  • The signed two-sided assertion-credit rule raises correct-alternative acceptance from 78.0% to 84.0% and cross-generator discrimination from 74.0% to 78.0%, with pass@1 increasing by 0.3.The signed variant is evaluated only in this ablation; other results use unsigned assertion credit, and its transfer-benchmark performance is not established.Source: §B.7 · p. 18, Table 18 · p. 19

B.9 Operational metric definitions

  • Coverage measures whether a candidate set contains a correct repair, selection accuracy whether the chosen candidate is correct, and selection efficiency their ratio.The definitions also distinguish acceptance coverage, correct-alternative acceptance, cross-generator rejection, zero-survivor rate, and distinct localizations.Source: §B.9 · pp. 18–19

B.10 Supervised-fine-tuning weighting ablation

  • Adding the exploration factor raises distinct localizations from 1.74 to 1.98 on Verified and from 3.70 to 4.13 on SWE-PolyBench, with matching resolution gains on both suites.The comparison holds the corpus and other weighting factors fixed; the λ=0 row removes only the exploration factor.Source: §B.10 · p. 19

B.11 Differential-verification component ablation

This section reports a test-time ablation of differential-verification search, covering adversarial self-auditing and chains conditioned on different coding methodologies. It also specifies how reactive attempts use execution evidence to explore repair locations, verify broadly, and interpret k=8 across three experiment axes.

B.12 Self-Verification Hook

  • The self-verification hook prompts an agent to treat its patch as wrong, reconstruct expected behavior from the issue, inspect root cause, test adversarially, and check edge cases and regressions.The audit can be applied up to three times per chain; it warns against inferring correctness from the current code’s output.Source: §B.12 · p. 20

B.11 Differential-verification component ablation

  • 1.6 points from location pruning and a 3.6-point loss from halving cell count are reported in the test-time-only DVS ablation.The experiment removes one component at a time and varies cell count on 500 issues.Source: §B.11 · p. 19, Table 21 · p. 21

B.13 The Eight Methodology Preambles

  • Eight parallel chains receive methodology preambles rather than different temperatures, covering a pragmatic baseline, TDD, BDD, incremental delivery, and a TDD–Agile hybrid.The approaches emphasize, respectively, neutral engineering, test-first cycles, observable scenarios, small verified increments, and combining those increments with TDD.Source: §B.13 · pp. 21–22
  • Other preambles emphasize finding and fixing root causes, defending against unprovided inputs, and exploring the repository broadly before editing.The root-cause approach traces the failure to its origin; the defensive approach checks adversarial conditions; the exploration-first approach builds a repository model before choosing a location.Source: §B.13 · pp. 23–24
  • The verification preamble calls for running the tests and checks for surfaced edge cases, avoiding edits to existing tests, cleaning scratch artifacts, and submitting only after verifying the fix.It emphasizes understanding the issue before making a surgical change.Source: §B.13 · p. 24

B.14 Reactive Location-Ban Logic

  • The reactive walk bans a location only after a confirmed-vacuous differential, freezes a plausible site after a weak pass to vary the edit, and stops at the first strong pass.A second guard prevents banning when an edit is not real and on-plan; missing or inconclusive differentials are not treated as proof that a location is wrong.Source: §B.14 · p. 24
  • Prior executed patches that fail to resolve the issue make their source locations hard constraints: a fresh attempt must fix the bug in a different file or function.The prompt frames those locations as ruled out by the reproduction and existing tests, while leaving the repair approach open.Source: §B.14 · pp. 24–25
  • The new-location procedure derives intended behavior from the requirement, traces the fault to a different site, then applies a minimal fix and reruns the reproduction, tests, and relevant edge cases.It prohibits test-file edits and permits returning to a banned site only sparingly, with an explanation of why the earlier attempt failed and how the new change differs.Source: §B.14 · p. 25

B.15 Experiment Configurations

  • The experiment’s three k=8 axes differ: pass@8 uses one greedy and seven temperature samples, methodology diversity uses eight distinct preambles, and coverage-space arms use eight candidate cells.Pass@8 samples are independent but not identically distributed; diversity comes from temperature, distinct methodology conditioning, or variation in cell diversity enforcement and verification, respectively.Source: §B.15 · p. 25
Loading 2610.03984v1…