PaperlayerView exact revision on arXiv

Source-linked AI summary

Cross-Benchmark Transfer from RL on Agentic Coding Tasks

Sushant Mehta, Logan Ritchie, Edwin Chen

arXiv:2610.00890v1cs.LGcs.AIcs.SE

TL;DR

Coding agents can leave requirements incomplete, test too narrowly, regress existing behavior, or trust unchecked assumptions; it is unclear whether RL on expert-built tasks teaches completion habits that transfer across benchmarks and harnesses. The paper trains Kimi K2.7 Code with RL on 1,700 expert-built coding tasks and evaluates it on six external benchmarks. The trained model improves on all six, with significant gains across five independent task sets.

  • Problem

    It is unclear whether RL on expert-built coding tasks teaches general task-completion ability that transfers beyond benchmarks sharing the training format.

    Source: §1 · p. 1

  • Method

    The authors post-train Kimi K2.7 Code with RL on 1,700 expert-built repository and terminal tasks, then evaluate it on six external benchmarks.

    Source: §1 · pp. 1–2

  • Results

    4.7 to 20.0 pp: the trained model improves on all six external benchmarks, with pooled gains across five independent task sets significant at p < 0.001.

    Source: §6.1 · p. 7, §8 · p. 10

  • Takeaways & Limitations

    On newly solved tasks, the trained model completes specifications, tests requirements, protects existing behavior, and builds its own ground truth.

    Source: §1 · p. 2, §8 · p. 10

    The training task families mirror SWE-bench and Terminal-Bench formats, although no training task overlaps with or derives from an evaluation benchmark task.

    Source: §3.3 · p. 4

Abstract

from arXiv · show

Coding agents often fail in the last mile: they build most of a feature but drop a requirement, test only the cases their implementation already handles, break behavior that was supposed to stay intact, or validate against an unchecked assumption. We ask whether reinforcement learning (RL) on expert-built agentic coding tasks closes this gap, and whether what the agent learns transfers beyond the training distribution. We post-train Kimi K2.7 Code, a 1T-parameter (32B active) open-weight mixture-of-experts model, with RL alone on 1,700 tasks: 1,000 repository tasks graded by hidden fail-to-pass tests and by pass-to-pass tests of existing behavior, and 700 terminal tasks graded by expert-written hidden verifiers. The reward is the fraction of target checks passed and drops to zero if any pass-to-pass test fails. One epoch of GSPO on a rank-32 LoRA adapter improves pass@1 on each of the six external benchmarks we evaluated, across three agent harnesses: SWE-Bench Pro (60.1 to 64.8), DeepSWE (31.0 to 43.4), Terminal-Bench 2.1 (67.4 to 82.0), Terminal-Bench 3 (1.4 to 12.1), Terminal-Bench 4 (0.0 to 7.6), and SWE-Marathon (5.0 to 25.0). Pooled over the five independent task sets (Terminal-Bench 4 revises Terminal-Bench 3), the improvement is significant (p < 0.001), and it remains significant on the three sets released after the training data was collected (p = 0.004); the model also improves under both harnesses never used in training. Median trajectories on DeepSWE and Terminal-Bench 3 are 24-35% shorter in agent steps. The base model's failed DeepSWE runs are mostly near-misses, and on the tasks the trained model newly solves, paired trajectories show it avoiding each of the four failure modes above.

1 Introduction

The paper tests whether RL on expert-built coding tasks teaches agents to finish realistic work and transfer across benchmarks, harnesses, and horizons. One RL run improves all six external benchmarks; paired trajectories suggest the transfer is a way of finishing work, not domain knowledge.

1 Introduction

  • The study asks whether RL improves task completion beyond familiar benchmark conventions and transfers across benchmarks, agent harnesses, and horizons.The motivation is that an agent that learns one benchmark’s conventions is less valuable than one that finishes realistic work.Source: §1 · p. 1
  • One RL run trains Kimi K2.7 Code on 1,700 expert-built tasks, using hidden checks for requested behavior and pass-to-pass tests to guard existing behavior.The reward gives partial credit for satisfied checks but drops to zero if any pass-to-pass test fails.Source: §1 · pp. 1–2
  • Pass@1 improved by 4.7 to 20.0 percentage points across all six external benchmarks, with significant pooled gains and improvements under both harnesses absent from training.The pooled improvement across five independent task sets is significant (p < 0.001) and remains significant on three post-collection sets (p = 0.004).Source: §1 · p. 2, Figure 1 · p. 2
  • The trained model avoids four recurring near-miss failures on tasks it newly solves: lost requirements, narrow testing, silent regressions, and weak ground truth.It implements the specification, derives tests from requirements, tracks existing behavior, and builds ground truth when none is provided.Source: §1 · p. 2
  • The paper’s paired trajectory analysis and regression-gated partial-credit reward link the observed behavior changes to properties favored by the training tasks and reward.The authors conclude that what transferred was a way of finishing work, not domain knowledge the base model largely had.Source: §1 · pp. 2–3

2 Related Work

Prior work applies RL to coding agents, builds executable training environments, and studies transfer, but commonly evaluates on benchmarks resembling training. This paper contrasts that pattern with expert-built tasks and transfer tests across benchmarks, harnesses, and horizons.

RL for software-engineering agents

  • Prior coding-agent RL mostly mines or synthesizes tasks at scale and evaluates on SWE-bench-style benchmarks matching its training format; this work instead uses expert-built tasks and tests broader transfer.Its tasks enumerate individually gradable requirements and gate partial credit on regressions.Source: §2 · p. 3

Benchmarks for agentic coding

  • Coding benchmarks span repository changes, containerized command-line tasks, original repository features, and multi-hour system builds, with scores depending on the agent harness.Harnesses determine the tools, prompts, and control loop available to the model.Source: §2 · p. 3

Partial-credit and rubric-based rewards

  • The reward preserves partial credit for requested behavior while assigning zero whenever a pass-to-pass test fails, avoiding an exchange between new requirements and existing behavior.This addresses the specific tradeoff that partial credit can introduce when satisfying new checks at the expense of existing behavior.Source: §2 · p. 3

Transfer from post-training

  • Earlier work reports RL transfer across task distributions; here both training and evaluation concern coding, but benchmarks, harnesses, and horizons differ.The cited examples include transfer from office-work and constraint-following post-training to external evaluations.Source: §2 · p. 3

3 Training Data

The training set combines expert-built repository and terminal tasks whose hidden checks grade requested behavior. Its specifications enumerate gradable requirements, protect existing behavior, and remain separate from the evaluated benchmark tasks.

3 Training Data

  • The training set contains 1,700 expert-built tasks from Surge AI’s agentic coding collections, in two common evaluation formats; each pairs a visible specification with hidden checks of requested behavior.Source: §3 · p. 4

3.1 Two task families

  • Repository tasks use hidden fail-to-pass tests for requested changes and pass-to-pass tests for existing behavior; terminal tasks use expert-written hidden verifiers to inspect final environment state.Repository specifications state concrete requirements, including edge cases, interfaces, restrictions, and what must remain unchanged.Source: §3.1 · p. 4

3.2 Properties that shape the training signal

  • Task properties support partial credit while discouraging regressions: requirements are individually gradable, graders are hidden, existing behavior is protected, and no reference solution is provided.The reward gives partial credit per check but is gated on regressions.Source: §3.2 · p. 4

3.3 Separation from the evaluation benchmarks

  • Training tasks predate DeepSWE, Terminal-Bench 3 and 4, and SWE-Marathon, and none overlaps with or derives from any of the six evaluated benchmarks.The repository and terminal formats mirror SWE-bench and Terminal-Bench, but the tasks were not selected to improve benchmark scores.Source: §3.3 · p. 4

4 Method

The method uses one RL epoch over repository and terminal tasks, with a reward that balances requested behavior against regressions. GSPO updates a rank-32 LoRA adapter in a minimal, isolated agent loop.

4 Method

  • Training runs one RL epoch over 1,700 tasks in two stages: repository tasks first, then terminal tasks ordered by difficulty.The run uses no supervised stage.Source: §4 · p. 4

4.1 Reward

  • The reward is zero if any pass-to-pass test fails; otherwise, it gives partial credit equal to the fraction of requested checks satisfied.Terminal tasks have no pass-to-pass tests, so their reward is the fraction of verifier criteria satisfied.Source: §4.1 · pp. 4–5

4.2 Policy optimization

  • GSPO uses group-standardized advantages and a length-normalized sequence ratio; equal-reward groups contribute no gradient.The method corrects rollout-engine and trainer mismatch with truncated importance sampling and applies no KL penalty or entropy bonus.Source: §4.2 · p. 5

4.3 Rollouts and infrastructure

  • Rollouts use a minimal bash-only agent loop in an isolated sandbox, while training updates a rank-32 LoRA adapter on attention and MLP projections.Generation and policy updates run co-located on 64 NVIDIA H200 GPUs; all results use a checkpoint fixed in advance, with no benchmark-tuned hyperparameters.Source: §4.3 · pp. 5–6, §4.2 · p. 5

5 Evaluation Protocol

The evaluation compares the base and trained models on six benchmarks using a fixed harness per benchmark and pass@1 scores from final-checkpoint runs.

5 Evaluation Protocol

  • The evaluation covers six benchmarks; Terminal-Bench 4 revises Terminal-Bench 3, so the two are treated as one family.These six are the only benchmarks evaluated.Source: §5 · p. 6
  • Each benchmark uses one unchanged harness for both models, with three harnesses overall and two whose tool interface, prompts, and control flow differ from training.Terminus 2 is used for Terminal-Bench 2.1 and Claude Code for SWE-Marathon.Source: §5 · p. 6
  • Pass@1 uses a single final-checkpoint run; all baselines are in-house except Terminal-Bench 2.1, whose public baseline averages three Terminus 2 runs.Table 2 reports Pass@1 (%) before and after RL, with gains calculated from rounded scores.Source: §5 · p. 6, Table 2 · p. 7
  • Of the trained model’s 74 Terminal-Bench 4 runs, 8 harness failures were excluded; 8 of the remaining 66 passed..Source: §5 · p. 7

6 Results

RL improves benchmark results across five independent task sets, including newer benchmarks and untrained harnesses, while reducing median agent steps on two benchmarks.

6.1 The trained model improves on all six benchmarks

  • Pooled over five independent task sets, gains reject equal pass rates at p < 0.001; gains on three post-collection task sets are also significant (p = 0.004).Terminal-Bench 3 and 4 are counted once through Terminal-Bench 4.Source: §6.1 · p. 7, §6.2 · p. 7

6.2 Where the gains appear

  • The four benchmarks released after training-data collection improve by 7.6 to 20.0 pp; gains exceed 14 pp on both untrained harnesses, including multi-hour SWE-Marathon builds.The gains span the difficulty range: RL closes 45% of the remaining gap on Terminal-Bench 2.1 and raises passes from 1 of 74 and 0 of 66 to 8 of 66 and 5 of 66 on Terminal-Bench 3 and 4.Source: §6.2 · p. 7

6.3 Fewer agent steps

  • On DeepSWE, median trajectory length fell from 150 to 98 agent steps (−35%); on Terminal-Bench 3, it fell from 102 to 78 (−24%).Counts cover all tasks on each benchmark, passed or failed.Source: §6.3 · p. 7

7 What Changed: A Paired Trajectory Analysis

Paired trajectories on tasks newly solved by the trained model expose recurring base-model failures and contrasting behaviors; DeepSWE failure statistics cover every failed in-house base run.

7 What Changed: A Paired Trajectory Analysis

  • The analysis pairs base and trained trajectories on the same tasks, focusing on DeepSWE, Terminal-Bench 3, and SWE-Marathon tasks the base failed and trained model passed.The paired cases are newly solved tasks; DeepSWE near-miss statistics cover every failed in-house base run.Source: §7 · p. 7

7.1 The base model’s DeepSWE failures are mostly near-misses

  • Among 83 failed in-house DeepSWE runs, 59% passed at least 80% of target tests, the median passed 86%, and 84% preserved every pass-to-pass test.A test fraction is not a requirement fraction: one dropped requirement in an example caused 16 of 137 tests to fail.Source: §7.1 · p. 8, Figure 2 · p. 8

7.2 Four failure modes and the behaviors after RL

  • The four recurring failure modes are lost requirements, narrow testing, silent regressions, and weak ground truth, aligned with four corresponding task properties.The paired analysis contrasts each base-model failure with trained-model behavior on the same task.Source: §7.2 · p. 8, Table 3 · p. 8

7.2.1 Lost requirements →implementing the entire specification

  • In a FastAPI task, the base model put two required methods at module level and passed 121 of 137 tests; the trained model made them instance methods and passed all 137.The missing interface requirement accounted for the 16 failed tests; requirement-level checks connect omissions to reward.Source: §7.2.1 · pp. 8–9

7.2.2 Narrow testing →testing the requirement, not the implementation

  • For a function-default task, the base model tested only anonymous functions, while the trained model tested the named form specified and additional variants.The base model’s implementation did not handle the named form, and both hidden test suites failed.Source: §7.2.2 · p. 9

7.2.3 Silent regressions →protecting existing behavior

  • When eliminating layout shift, the base model deleted two components; the trained model preserved them while changing how their visual effects were applied.The task also required preserving visible elements, styling, and analytics side effects.Source: §7.2.3 · p. 9

7.2.4 Weak ground truth →reconstructing ground truth

  • Without reference functions for a two-dimensional solver, the base model assumed a one-dimensional problem; the trained model used manufactured solutions as an independent check.The training tasks share a precise specification, hidden grader, and lack of oracle, though they do not include PDE solvers, hardware emulators, or compressors.Source: §7.2.4 · pp. 9–10

7.3 A common thread

  • Across all four modes, the trained model treats the specification as a full contract and its work as a hypothesis to test, while taking fewer steps on DeepSWE and Terminal-Bench 3.These engineering habits are consistent with gains across benchmarks and harnesses.Source: §7.3 · p. 10

8 Conclusion

The conclusion reports cross-benchmark gains from one RL epoch on expert-built coding tasks. In newly solved tasks, paired trajectories show behaviors the authors characterize as a way of finishing work.

8 Conclusion

  • One RL epoch on 1,700 expert-built coding tasks improved Kimi K2.7 Code by 4.7 to 20.0 pp on all six external benchmarks evaluated.The gains include four benchmarks released after data collection, two harnesses absent from training, and multi-hour builds with no training counterpart.Source: §8 · p. 10
  • On newly solved tasks examined, the trained model implemented the full specification, tested requirements, protected existing behavior, and built its own ground truth.The conclusion links each behavior to properties of the training tasks and reward.Source: §8 · p. 10
  • The authors conclude that the transferred capability was a way of finishing work, rather than domain knowledge the base model largely had..Source: §8 · p. 10

A.1 Scoring

The in-house benchmark results are single runs, with task-specific handling of unscored or unavailable runs.

A.1 Scoring

  • On Terminal-Bench 3, excluding 8 harness-failure runs leaves 8 of 66 trained-model passes; counting them as failures gives 8 of 74 (10.8%) and a 9.4 pp gain (p = 0.016).The base model passes 1 task; denominators from 69 to 74 yield 1.4%.Source: §A.1 · p. 15
  • All in-house results are single runs.Source: §A.1 · p. 15
  • On SWE-Marathon, the A100-dependent task was unavailable and counted as failed for both models.Source: §A.1 · p. 15

A.2 Statistics

Pooled analysis finds a significant improvement across five independent task sets, including on three sets released after data collection; Table 4 reports pass counts and uncertainty.

A.2 Statistics

  • Across five independent task sets, the pooled gain is 7.0 pp (95% CI [3.0, 11.0]; p = 7 × 10^-4), and across three post-collection sets, p = 0.004.Terminal-Bench 3/4 is counted once through Terminal-Bench 4, the more conservative choice.Source: §A.2 · p. 15
  • The intervals and tests treat runs as independent samples, which is conservative when the checkpoints’ same-task outcomes are positively correlated.Source: §A.2 · p. 15
  • Table 4 reports pass counts, 95% confidence intervals for gains, and two-sided Boschloo exact-test p-values.Source: Table 4 · p. 15, §A.2 · p. 15
Loading 2610.00890v1…