Training Software Engineering Agents and Verifiers with SWE-Gym

Introduction

• Prior SWE-agent gains relied on interfaces and prompting, while progress was limited by proprietary models and the lack of suitable real-world training environments.Existing resources lacked developer action trajectories, executable environments, or reproducible reward signals; other datasets used synthetic or isolated tasks rather than realistic repository-level coding problems.Source: §1 · pp. 1–2
• 491 training trajectories showed continued performance gains without saturation, while greater inference-time compute steadily improved results.The figure describes roughly log-linear gains as more candidate trajectories are sampled and selected by the verifier.Source: Figure 1 · p. 1, §1 · p. 2
• SWE-Gym provides 2,438 Python tasks from 11 open-source repositories, with pre-installed dependencies and executable test verification for real-world GitHub issues.The environment is designed to support training both language-model agents and verifiers.Source: §1 · p. 2
• Fine-tuning a 32B Qwen-2.5 coder on 491 SWE-Gym trajectories improved resolution by +12.3% to 15.3% on SWE-Bench Lite and +13.6% to 20.6% on Verified.The agent uses the OpenHands scaffold.Source: §1 · p. 2
• Self-improvement with the MoatlessTools scaffold boosted SWE-Bench Lite performance to 19.7%.The model interacts with SWE-Gym, receives reward, and improves through rejection-sampling fine-tuning.Source: §1 · p. 2
• 32.0% on SWE-Bench Verified (+11.4% absolute) and 26.0% on SWE-Bench Lite established a new state-of-the-art among systems with publicly accessible weights.A verifier trained on test-suite outcomes estimates each trajectory’s probability of success and selects the highest-scoring sampled trajectory.Source: §1 · p. 2

Agents that solve GitHub issues

• Existing GitHub-issue agents use specialized staged workflows or general-purpose long-horizon prompting, while the most successful systems rely on proprietary models and specialized workflows.Specialized workflows reduce planning horizons but require substantial engineering and may fail at intermediate steps; general prompting is more flexible but demands stronger models and can be computationally expensive. Agents take an issue and repository as input and produce a patch verified by human-written tests.Source: §2 · p. 2

Environments for training software agents

• Prior datasets lack key ingredients for training on realistic repository-level tasks, whereas SWE-Gym pairs real GitHub issues with executable unit tests.SWE-Bench's training split lacks executable environments and success signals; HumanEval targets standalone code generation, and R2E has 246 synthetic instances. SWE-Gym provides real-world tasks, pre-installed dependencies, and executable test verification.Source: §2 · pp. 2–3, Table 1 · p. 3

Post-training: From chatbots and reasoners to agents

• Post-training methods rely on demonstrations or reliable reward signals, both largely absent in software engineering, while SWE-Gym provides expert-written tests as rewards for agent training.Prior work explores supervised or reinforcement learning for chatbots and reasoners, trajectory distillation, and self-improvement guided by rewards; the paper reports that SWE-Gym can build strong SWE agents without prompt engineering.Source: §2 · p. 3

3. SWE-Gym Environment

• SWE-Gym contains 2,438 real-world tasks from 11 Python repositories, with executable environments and expert-validated tests for producing test-passing code patches.Its repositories are separate from SWE-Bench’s to avoid contamination.Source: §3 · p. 3

3.1. Dataset Construction

• After extracting 64,689 raw instances across 358 repositories, the authors configured environments and filtered them to 2,438 validated tasks from 11 repositories.The 358 repositories met age, star, code-size, pull-request, and contributor thresholds. Environment setup and validation used about 200 human annotation hours and 10,000 CPU core hours; pre-built Docker images total 6 TB.Source: §3.1 · pp. 3–4

3.2. SWE-Gym Lite

• SWE-Gym Lite contains 230 generally easier, self-contained tasks, excluding multi-file edits, poorly described issues, complex gold diffs, and tests focused on error-message validation.The subset follows the SWE-Bench Lite filtering pipeline.Source: §3 · p. 3, §3.2 · p. 4

3.3. Dataset Statistics

• Repository task counts follow a long-tail distribution: pandas accounts for nearly one-third of instances, while bokeh represents about one percent.Source: §3.3 · p. 4, Figure 2 · p. 4
• SWE-Gym tasks are on average harder than SWE-Bench tasks: codebases have fewer files but similar lines of code, gold patches edit more lines and files, and models perform worse.The authors suggest the lower performance may reflect both SWE-Bench overfitting and inclusion of sophisticated repositories such as pandas and MONAI.Source: §3.3 · pp. 4–5

4.1. Setting

• The experiments compare OpenHands CodeAct (Wang et al. 2024c) and MoatlessTools (¨Orwall 2024), using SWE-Bench Lite and Verified resolution rate and empty-patch rate for evaluation.The baseline policy-improvement method fine-tunes the base language model on successful SWE-Gym trajectories; the benchmarks are cited as Jimenez et al. (2024).Source: §4.1 · p. 5

4.2. Training General-Purpose Prompting Agents

• Fine-tuning on 491 SWE-Gym trajectories raised OpenHands resolution from 3.0% to 15.3% on SWE-Bench Lite and from 7.0% to 20.6% on Verified.The reported improvements are up to 12.3% and 13.6%, respectively.Source: §4.2 · p. 5, Table 3 · p. 6
• SWE-Gym fine-tuning reduced stuck-in-loop rates by 4.6–18.6% across Lite and Verified, except for a 1.5% increase for the 32B model on Lite.Even the zero-shot 32B model repeated actions in 29.4% of Verified tasks; fine-tuning also coincided with lower empty-patch rates.Source: §4.2 · p. 5, Table 3 · p. 6
• For OpenHands, the 32B model’s SWE-Bench Lite resolution rate fell from 15.3% to 8.7% when fine-tuned on a mixture of 868 on-policy and 491 off-policy trajectories.The authors conclude that this self-improvement setup was not yet working; the on-policy trajectories came from six samples per SWE-Gym instance at temperature 0.5.Source: §4.2 · pp. 5–6

4.3. Self-Improvement with Specialized Workflow

• MoatlessTools’ constrained specialized workflows outperformed OpenHands on zero-shot SWE-Bench Lite resolution: 7% versus 1.0% for 7B and 19% versus 3.0% for 32B.The scaffold constrains actions to predefined workflows, reducing task horizons compared with OpenHands’ long-horizon planning.Source: §4.3 · p. 6
• Repeated successful-task sampling favors easier tasks, so per-instance capping balances dataset size against bias; a cap of 2 slightly outperformed the full dataset and sped training.The method ranks trajectories by model-response rounds, preferring those requiring fewer rounds; low caps reduce dataset size and performance, while high caps skew toward easier tasks.Source: §4.3 · p. 6
• With MoatlessTools, the 7B model improved from 7.0% to 10.0% resolution after two iterations, while the 32B model rose from 19.0% to 19.7% after one and gained no further.The authors attribute the 32B model’s limited gains to the scaffold’s restricted action space and rejection-sampling fine-tuning.Source: §4.3 · p. 6, Table 4 · p. 6

5.1. Inference-Time Scaling with Verifiers

• An outcome-supervised reward model scores the task statement, agent trajectory, and current git diff to estimate success and rerank sampled SWE-agent trajectories.The section presents learned verifiers as a way to improve performance through inference-time scaling.Source: §5.1 · pp. 6–7

5.1.1. VERIFIER FOR GENERAL-PURPOSE PROMPTING

• Pass@k rises from 20.6 to 37.8% as k grows from 1 to 8 and reaches 42.8@k=16; Best@k improves from 20.6@1 to 29.8@8 and 32.0@16.The gap between Pass@k and Best@k reflects imperfect verifier performance; LoRA fine-tuning reaches 29.8@8 versus 27.2@8 with full-parameter fine-tuning.Source: §5.1.1 · p. 7, Figure 3 · p. 7

5.1.2. VERIFIER FOR SPECIALIZED WORKFLOW

• On SWE-Bench Lite, a 7B verifier raises its 7B policy from 10 to 13.3%, while a 32B verifier raises its 32B policy from 19.7 to 26.3%.The 7B verifier plateaus after k=4, whereas the 32B verifier continues improving through k=8.Source: §5.1.2 · p. 8, Figure 4 · p. 8

5.2. Training-Time Scaling with Data

• Randomly adding training trajectories consistently improves resolution, especially for the 32B model, suggesting current repository and instance diversity is not yet a bottleneck.Among deduplicated approaches, repository scaling performs better initially at 25% data; random scaling's higher final performance likely reflects its larger dataset (491 versus 294 trajectories).Source: §5.2 · p. 8, Figure 5 · p. 8

Conclusions

• SWE-Gym supports improvements in both agents and verifiers for complex software tasks, with potential for continued performance gains as compute increases.The environment combines real-world Python tasks, repository-level context, pre-configured execution environments, and test verifications.Source: §6 · p. 9

Automatic Environment Synthesis

• SWE-Gym’s limited repository, task-type, and language diversity motivates automated environment creation, test-case generation, and task generation.The authors identify environment synthesis as a critical next step.Source: §6 · p. 9

Self-Improvement with Reinforcement Learning

• The authors describe self-improvement results as modest and propose large-scale online reinforcement learning to improve language model agents.Source: §6 · p. 9

Human-Agent Interaction

• Current SWE settings neglect human-in-the-loop collaboration, motivating user simulation and learning from offline human-agent interaction data.The goal is to develop collaborative agents aligned with humans.Source: §6 · p. 9

• Current models are research artifacts rather than production-ready tools, but could support critical open-source infrastructure and make software development more accessible.The potential benefit spans industries and includes increased developer productivity.Source: Impact Statement · p. 9
• As agents become more capable, they may affect software engineering jobs and raise questions about code ownership, licensing, and attribution.Source: Impact Statement · p. 9
• Techniques focused here on legitimate engineering tasks could also be misused to create malicious code, motivating responsible-deployment work on security, safety, and economic impacts.Source: Impact Statement · p. 9

A. Comparison with Concurrent Works

• Ma et al. (2024) train Lingma SWE-GPT with rejection-sampling fine-tuning on a comparable dataset, filtering trajectories with manual heuristics rather than executable unit-test feedback.The model weights are public, but the training pipeline and dataset are not.Source: A. Comparison with Concurrent Works · p. 13
• Golubev et al. (2024) and Badertdinov et al. (2024) collect 6,415 executable GitHub task instances, about 1.5 times SWE-Gym’s dataset size, by discarding tasks that fail a general setup script.The setup approach biases their environment away from tasks with complex dependencies; their dataset and agent trajectories are public, but their model is not.Source: A. Comparison with Concurrent Works · p. 13
• Golubev et al. (2024) study agent and verifier training and lookahead action selection; their best system achieves 40% accuracy on SWE-Bench Verified using 80,036 trajectories and a 72B model.The passage contrasts these resources with thousands of trajectories in the authors’ experiments and a 32B model.Source: A. Comparison with Concurrent Works · p. 13
• SWE-Gym achieves competitive results with significantly lower compute and a smaller model size (32B vs 72B), while releasing its dataset, model weights, trajectories, and training pipeline.The authors also describe their work as providing comprehensive analysis of agent and verifier training, scaling behaviors, and positive results on agent self-improvement.Source: A. Comparison with Concurrent Works · p. 13

B.4. Details of OpenHands Trajectory Sampling

• Sampling used 30-turn limits for D0 and D1, then 50-turn runs with GPT-4o and Claude-3.5-Sonnet on SWE-Gym Lite and GPT-4o on the full set.D0 comprised 19 resolved trajectories from GPT-4o at temperature 0; varying temperatures produced D1 with 106 resolved trajectories.Source: §B.4 · p. 15
• The final D2 training set contains 491 successful trajectories, combining 106 + 19 + 67 + 299 trajectories.Infrastructure errors may mean some reported success counts underestimate the actual number of successful trajectories.Source: §B.4 · p. 16

B.5. MoatlessTools ORM Prompt

• Unlike Zhang et al. (2024a), the MoatlessTools ORM obtains context directly from the evaluated agent trajectory instead of using a proprietary model for context extraction.The prompt evaluates candidate patches as a Python software-engineering code reviewer.Source: §B.5 · p. 16
• The MoatlessTools prompt supplies the issue, relevant code spans with file paths and line numbers, and the candidate patch; it instructs the verifier to adjust its score if spans are insufficient.The prompt also provides original and patched code hunks.Source: §B.5 · pp. 16–17

{code_spans}

• The code-spans prompt asks whether the patch resolved the issue and requires a True-or-False response.Source: {code_spans} · p. 17

B.6. OpenHands ORM Prompt

• The OpenHands ORM judges task completion, edge-case and requirement handling, errors in the final solution, and verification, using the interaction log and a YES-or-NO judgment.Source: §B.6 · p. 17
• During training, the OpenHands verifier prompt includes the resolved-status judgment; at inference, the trained verifier predicts probabilities for Yes and No.Source: §B.6 · p. 18