PaperlayerView exact revision on arXiv

Source-linked AI summary

CodeMidas: Scaling Agentic Coding RL Environments from Code Itself

Bowen Ye, Lei Li, Shicheng Li, Zihao Yue, Linghao Zhang, Hanglong Lv, Yuanxin Liu, Wenhan Ma, Hao Tian, Rang Li, Jinhao Dong, Yikai Zhao, Xiangwei Deng, Hailin Zhang, Liang Zhao, Qi Liu, Lingpeng Kong, Tong Yang, Fuli Luo

arXiv:2609.22068v1cs.AI

TL;DR

Coding RL needs diverse tasks and reliable verifiers, but existing environment pipelines often rely on development artifacts. CodeMidas constructs executable environments from implemented code using source code as its only task-specific input. Training on these tasks improves performance across five external benchmarks, with gains spanning diverse forms of software work.

  • Problem

    Coding RL needs diverse tasks and reliable verifiers, while existing environment pipelines often rely on development artifacts such as issues and commits.

    Source: §1 · p. 1

  • Method

    CodeMidas turns implemented functionality into executable coding environments using source code as its only task-specific input, with tests informed by original-code execution and post-rollout filtering.

    Source: §1 · p. 2

  • Results

    Training on 5,545 CodeMidas tasks improves MiMo-V2.5 across all five external benchmarks; DeepSWE pass rate rises from 10.0% to 21.7%.

    Source: §1 · p. 2, §6 · p. 11

  • Takeaways & Limitations

    The findings point to implemented source-code functionality as a basis for scaling coding RL environments that support generalization across diverse software work.

    Source: §1 · p. 2, §6 · p. 11

Abstract

from arXiv · show

Training capable coding agents via reinforcement learning (RL) requires diverse tasks with reliable verifiers. Open-source codebases offer a rich source of such tasks, while existing methods typically rely on development artifacts such as issues and commits, limiting the range of tasks that can be extracted. To better scale RL environments, we present CodeMidas, an agentic pipeline that turns implemented functionality in existing codebases into executable RL environments using source code as its only task-specific input. CodeMidas allocates agentic compute to every stage of environment construction: agents explore implemented functionality to formulate behavioral specifications, construct tests grounded in execution of the original code, and validate and filter candidate tasks through execution checks and repeated solution rollouts. The resulting dataset has 5,545 training tasks from 3,185 open-source codebases spanning 23 programming languages and 15 technical domains. Training MiMo-V2.5 on these tasks with GRPO improves performance on all five diverse benchmarks, covering issue repair (DeepSWE + 11.7%), whole-program construction (ProgramBench +17%), and terminal work (Terminal-Bench v2.1 +8.5%). Ablations show that increasing the number of high-quality training tasks improves performance. Trajectory analysis shows the RL-trained agent demonstrates better behaviors like increasing codebase exploration and more diverse self-verification. These results establish source code as a scalable foundation for constructing RL environments that improve coding agents across diverse software tasks.

1 Introduction

Existing coding-RL environment pipelines often depend on development records, tests, or documentation, motivating tasks derived directly from implemented functionality. CodeMidas builds executable environments from source code and improves performance across diverse software benchmarks.

1 Introduction

  • Existing pipelines draw tasks from issues, pull requests, commits, tests, or documentation, whose coverage bounds the tasks they can supply.Codebases offer public interfaces and observable behavior for defining tasks, original-code execution for test expectations, and project context for development environments.Source: §1 · pp. 1–2
  • The pipeline identifies existing functionality, formulates behavioral task statements, adapts codebases, builds tests informed by original-code execution, and filters tasks after solution rollouts.Filtering probes for leakage, reviews verifier decisions against requirements, and uses rollout success rates to select tasks.Source: §1 · p. 2
  • Training MiMo-V2.5 on CodeMidas tasks improves performance on all five external benchmarks, spanning issue repair, whole-program construction, code translation, and terminal work.DeepSWE pass rate rises from 10.0% to 21.7%, ProgramBench Almost Solved from 4.5 to 21.5, and Terminal-Bench v2.1 pass rate from 63.7% to 72.2%.Source: §1 · p. 2
  • CodeMidas constructs 5,545 verifiable training tasks from 3,185 open-source codebases spanning 23 programming languages and 15 technical domains.The tasks train MiMo-V2.5 using GRPO.Source: §1 · p. 2
  • Increasing high-quality tasks from 1k to 3k to 5,545 progressively raises scores on SWE-bench Pro, DeepSWE, and CodeMidas Val; the 3k subset also beats an unfiltered 8k baseline on all three.During RL, agents explore codebases more and perform more varied self-verification; agent-written checks are associated with higher success, and behavioral changes also appear on external tasks.Source: §1 · p. 2

2 Related Work

Prior coding-environment pipelines draw on development records, existing tests, or documentation, while reward and verification work explores execution and learned assessment. CodeMidas uses source code as its only task-specific input and execution-based rewards with GRPO.

2.1 Building coding RL environments

  • Development-record pipelines seed tasks from issues, pull requests, or commits, while other approaches build tasks around tests or documentation.CodeMidas instead derives behavioral statements and execution-grounded tests from source code as its only task-specific input.Source: §2.1 · pp. 2–3, Table 1 · p. 3

2.2 Rewards and verification for coding agents

  • Coding-agent reward approaches include learned critics, reference-patch similarity, process rewards, rubrics, execution feedback, and synthesized-test rewards.CodeMidas uses GRPO with synthesized-test execution rewards, without a reward model or learned verifier.Source: §2.2 · p. 3
  • Test oracles can miss incorrect programs or accept incorrect patches, while specifications and restrictive tests can leave gaps.CodeMidas combines execution-consistency checks with adversarial leakage probes, solution reviews, and rollout-outcome filtering.Source: §2.2 · p. 3

3 Method

CodeMidas constructs and filters coding environments from source code through task design, execution-grounded test construction, environment preparation, and agent rollouts. It retains a diverse dataset of 5,545 tasks from 3,185 codebases.

3 Method

  • Each task pairs a statement and containerized development environment with a hidden executable verifier that returns a binary execution reward at grading.The verifier stays outside the solver’s environment and is injected only for grading.Source: §3 · p. 4

3.1 Task Design and Codebase Adaptation

  • Agents identify public entry points and observable behavior, remove the target implementation, and revise task boundaries and statements while preserving shared components and project context.Statements specify inputs, observable behavior, and required public interfaces while leaving internal helpers and algorithms open.Source: §3.1 · pp. 4–5

3.2 Execution-grounded Test Construction

  • Tests map behavioral requirements to inputs and boundary cases, execute public entry points in the reference codebase, and record requirement-linked outcomes.Review removes assertions unsupported by the statement, including incidental wording, ordering, or internal structure; revised tests are rerun against the reference solution.Source: §3.2 · p. 5

3.3 Environment Preparation

  • Environment preparation installs declared dependencies and removes artifacts that could reveal the deleted implementation, while retaining required packages, fixtures, and build support.Execution consistency requires both starting-codebase runs to fail and all four reference-solution runs to pass across six fresh containers.Source: §3.3 · p. 5

3.4 Post-rollout Environment Filtering

  • Before training, rollout filtering probes for exploitable leakage, reviews agent implementations for verifier false positives or false negatives, and retains tasks with both passing and failing attempts.All-pass or all-fail outcomes may reflect difficulty or defects, but do not reveal the cause.Source: §3.4 · pp. 5–6

3.5 Dataset Overview

  • The retained dataset contains 5,545 tasks from 3,185 codebases across 23 programming languages and 15 technical domains.The ten most frequent languages account for 5,445 tasks (98.2%); systems software (17.4%), web technologies (14.6%), and developer tools (13.6%) together account for 45.6% of tasks.Source: §3.5 · pp. 6–7, Figure 2 · p. 6, Figure 3 · p. 6
  • Reference patches have a median size of 142 lines and an interquartile range of 66–305 lines; 65.9% touch at least two source files.The count includes added or deleted source lines, comments, and blank lines.Source: Figure 4 · p. 6, §3.5 · p. 7

4 Experiments

The experiments train MiMo-V2.5 on CodeMidas tasks with GRPO and evaluate it against the initial policy on five external benchmarks and CodeMidas Val. RL improves performance across external evaluations, while pass rate on CodeMidas Val rises during training.

4.2 Results

  • Figure 5 compares initial and RL scores across five external benchmarks, with right-side labels reporting absolute improvements in percentage points.ProgramBench reports Almost Solved; the other evaluations report pass rate.Source: Figure 5 · p. 7

4.1 Experimental Setup

  • MiMo-V2.5 is trained on 5,545 CodeMidas tasks with GRPO, binary execution rewards, batch size 32, and 32 rollouts per task.The initial policy and RL checkpoints use identical evaluation settings.Source: §4.1 · p. 7
  • Evaluation covers five official external benchmark task sets and a separate 200-task CodeMidas Val sample, with three attempts per validation task.Training tasks are disjoint from CodeMidas Val and all five external benchmark task sets.Source: §4.1 · p. 7

4.2 Results

  • CodeMidas Val pass rate rises from 35.0% to 44.7%, remaining roughly 8–10 percentage points above the initial rate from step 40 onward.The gains accompany longer trajectories.Source: §4.2 · p. 8, Figure 6 · p. 8

5 Analysis

The analysis compares training pools of different sizes and quality, then examines behavioral changes during RL and on held-out tasks. Larger high-quality pools perform better, while exploration and self-verification become more frequent or varied.

5.1 Task Scale and Quality

  • The full high-quality dataset leads at every evaluated CodeMidas Val checkpoint from step 40 through step 70, reaching 44.73 at step 70.The 1k setting reaches 41.30 at step 30, and the 3k setting reaches 43.22 at step 65.Source: §5.1 · pp. 8–9, Figure 7 · p. 8
  • DeepSWE scores rise from 17.57 to 19.05 to 21.70, while CodeMidas Val scores increase from 41.30 to 43.22 to 44.73 across the 1k, 3k, and 5k pools.These results support scaling high-quality training data.Source: §5.1 · p. 9
  • The full high-quality dataset exceeds the vanilla 8k sample by 0.59, 4.59, and 4.49 percentage points on SWE-bench Pro, DeepSWE, and CodeMidas Val, respectively.The high-quality 3k subset also outperforms the vanilla 8k sample on all three evaluations; the comparison supports the combined value of cleaning, execution checks, and post-rollout filtering.Source: §5.1 · p. 9

5.2 Behavioral Changes and Generalization

  • From early to late training, exploration calls increase from 27.2 to 40.1, drafting ratio from 0.36 to 0.63, and verification commands from 2.03 to 2.53.Exploration counts read/search calls before the first edit; drafting measures overlap with preceding reasoning; self-verification counts distinct commands after the final edit.Source: §5.2 · p. 9, Table 2 · p. 10
  • On CodeMidas Val, rollouts with agent-written and executed checks have a mean pass rate 4.2 percentage points higher than those without (95% CI: 1.8–6.6).Exploration and drafting differences have confidence intervals that include zero.Source: §5.2 · p. 9
  • Behavioral changes extend to held-out benchmarks: exploration increases on all three, drafting ratios rise on SWE-bench Pro and ProgramBench, and interaction length changes by task type.Mean interaction length rises from 37.3 to 50.1 turns for SWE-bench Pro issue repair and falls from 155.1 to 122.8 turns for ProgramBench construction; on ProgramBench, greater exploration accompanies shorter interactions.Source: §5.2 · p. 10, Table 3 · p. 10

6 Conclusion

The conclusion presents existing codebases as a source of coding RL data and reports that CodeMidas training improves performance across diverse software tasks. Its analyses also connect larger high-quality pools and changing agent behaviors with the reported gains.

6 Conclusion

  • Training on 5,545 tasks improves MiMo-V2.5 across five external benchmarks, with gains spanning diverse forms of software work.The analysis also finds that expanding high-quality task pools improves performance and that a smaller filtered dataset can outperform a larger unfiltered one.Source: §6 · p. 11
  • During training, agents explore codebases more and perform more varied self-verification, while agent-written checks are associated with higher success rates.These behavioral changes also appear on external tasks.Source: §6 · p. 11
  • CodeMidas turns implemented functionality in existing codebases into training environments as a practical route to scaling coding RL from code itself.The conclusion frames source code as a basis for improving coding agents across diverse software tasks.Source: §6 · p. 11

A Training and Evaluation Configuration

Appendix Table A1 lists the evaluation setup and shared training settings for the four task pools used in Section 5.1.

A Training and Evaluation Configuration

B Behavioral Metric Definitions

The section defines three behavioral metrics: codebase exploration before editing, code drafting based on prior reasoning, and self-verification after editing.

B Behavioral Metric Definitions

  • Codebase exploration counts distinct read/search requests made before the first codebase edit.Requests with the same target file and line range, or the same search query, scope, and options, count once; equivalent tool and shell requests are duplicates.Source: B Behavioral Metric Definitions · p. 15
  • Code drafting measures the fraction of sampled Write/Edit code fragments that appeared in reasoning before the corresponding write.It samples distinct 16-character fragments at a stride of four characters and divides matching fragments by all sampled fragments across write actions.Source: B Behavioral Metric Definitions · p. 15
  • Self-verification counts distinct verification commands run after the final codebase edit.The count includes project tests, inline checks, temporary test programs, and local program runs; repeated commands count once.Source: B Behavioral Metric Definitions · p. 15

Related papers

Loading 2609.22068v1…