Open-SWE-Traces: Advancing Dual-Mode Multilingual Distillation for Software Engineering Agents

• 207,489 multilingual trajectories address the shortage of large-scale interaction traces and executable environments for agent development.Source: §1 · pp. 1–2
• Dual-mode distillation combines reasoning-heavy MiniMax-M2.5 traces with direct tool-use traces from Qwen3.5-122B.Source: §1 · pp. 1–2
• Thinking-enabled traces reduce average OpenHands assistant turns from 94.08 to 58.22, a 38% increase in execution efficiency.Source: §1 · pp. 1–2, Table 1 · p. 2
• 61.7% on SWE-bench Verified, 57.1% on SWE-bench Multilingual, and 36.8% on SWE-bench Pro were achieved by the best fine-tuned model.Source: §1 · p. 2
• The corpus supports analysis of base models, filtering strategies, multilingual scaling, thinking versus non-thinking modalities, and generalization across execution harnesses.Source: §1 · p. 2

2.1 Repository Selection and Criteria

• The dataset selects SWE-rebench v2 instances in Python, Java, C, C++, JavaScript, TypeScript, Rust, Go, and PHP under MIT, Apache-2.0, or BSD licenses.Source: §2.1 · p. 2

2.2 Agentic Trajectory Synthesis

• MiniMax-M2.5 and Qwen3.5-122B generate heterogeneous traces within OpenHands and SWE-agent while agents explore repositories, run commands, and iteratively refine patches.Source: §2 · p. 2, §2.2 · p. 3

2.3 Multi-Stage Quality Filtering

• The filtering pipeline aggregates logs, discards runtime failures, removes incomplete or invalid patches and malformed tool use, then standardizes role, content, tool-call, and reasoning fields.Source: §2.3 · pp. 3–4
• AST-based TrajectoryScanner removes git-hacking attempts that inspect repository metadata or internal histories for solution leaks.Source: §2.4 · p. 4

2.4 Dataset Statistics

• 207,489 trajectories span nine languages, with 51.7% containing MiniMax-M2.5 reasoning traces and 50.8% synthesized through OpenHands.Source: §2.5 · p. 4, Table 2 · p. 4
• 65,244 trajectories resolved their issues in unit testing, yielding an overall pass rate of approximately 40.6%.Source: §2.5 · p. 4

3.1 Experiment Setup

• The evaluation uses MOpenHands and MSWE-agent scaffolds, Qwen3-30B-A3B Thinking, Instruct, and Code variants, and three SWE-bench benchmarks.The dual-mode model uses the full corpus, while Thinking and Instruct variants use their respective trace subsets.Source: §3.1 · pp. 4–6

3.2 Experiment Results

• OPEN-SWE-AGENT-INSTRUCT and OPEN-SWE-AGENT-THINKING improve by over 30.0% versus their base models on SWE-bench Verified.The comparison is reported for the monolingual evaluation in table 3.Source: §3.2 · p. 6
• 61.7% resolve rate is achieved on SWE-bench Verified by OPEN-SWE-AGENT in the no-think setting, compared with 59.3% in think mode.These results represent performance under the primary evaluation settings described for the model.Source: §3.2 · p. 6
• OPEN-SWE-AGENT improves over its base model by 7.7–10.1 absolute points, reaching 59.3% in think mode and 61.7% in no-think mode.The base model performance reported for comparison is 51.6%.Source: §3.2 · p. 6
• 57.1% resolved rate is achieved on SWE-bench Multilingual, while SWE-bench Pro reaches 36.8% under the MSWE-agent framework.The multilingual score rises from 33.5% to 57.1%, and Pro performance rises from 28.4% to 36.8%.Source: §3.2 · pp. 6–7
• On SWE-bench Multilingual, distilled models underperform in think mode relative to no-think, while MOpenHands produces framework-specific performance drops and loops in 5–10% of cases.The paper hypothesizes that internal reasoning requires more extensive training and reports exhaustion of the maximum turn limit in these loops.Source: §3.2 · p. 7

4 Ablation and Analyses

• The ablation study evaluates how training configurations affect agentic capabilities across environments and reasoning modalities.The analyzed factors include harness transfer, multilingual versus Python-only data, and resolved-only versus full trajectory inclusion.Source: §4 · p. 7

4.1 Cross-Harness Generalization

• Cross-harness transfer is achievable but incurs a consistent performance penalty across Python-centric and multilingual benchmarks.The degradation is attributed to overfitting to each framework’s interaction patterns, action spaces, or observation formats.Source: §4.1 · p. 7
• Models trained on MSWE-agent transfer more stably to MOpenHands, whereas models trained on MOpenHands suffer steeper drops when evaluated with MSWE-agent.The analysis identifies MSWE-agent as the more robust baseline for cross-harness generalization.Source: §4.1 · p. 7

4.2 Cross-Lingual Transfer

• Multilingual data raises SWE-bench Multilingual no-think performance from 43.1% to 57.1%, a +14% absolute gain over Python-only training.On SWE-bench Verified, multilingual data also slightly improves think-mode performance from 54.9% to 58.1%.Source: §4.2 · p. 7

4.3 Data Filtering Impact

• Including unresolved trajectories improves SWE-bench Multilingual from 40.5% to 47.6% in think mode and from 49.6% to 57.1% in no-think mode.The full corpus also improves SWE-bench Verified think performance from 55.3% to 58.1%.Source: §4.3 · p. 7

4 Ablation and Analyses

• The paper’s overall hierarchy of performance drivers places multilingual data and full trajectory inclusion above harness consistency, which remains a technical challenge.Peak performance requires careful alignment between training and evaluation environments.Source: §4.3 · p. 7

SWE Benchmarks

• SWE benchmarks evolved from SWE-bench and SWE-bench-Verified toward evaluations of multimodal integration, crosslinguistic proficiency, and long-horizon reasoning.The cited benchmark ecosystem probes multiple specialized dimensions of autonomous software engineering.Source: §5 · p. 8

SWE Datasets

• SWE programming datasets use synthetic generation, real-world mining, learned surrogate feedback, and hybrid combinations of authentic pull requests with synthetic instances.Examples include R2E-Gym, SWE-smith, SWE-Mirror, SWE-Gym, SWE-rebench, SWE-World, and SWE-Lego.Source: §5 · p. 8

SWE Models and Agents

• Autonomous coding research is shifting from environment-based prompting toward model-centric optimization through mid-training, expert-trajectory supervised fine-tuning, and reinforcement learning.Early systems such as SWE-agent and OpenHands relied on environment-based prompting, while newer work embeds reasoning into model weights.Source: §5 · p. 8

• 61.7% on SWE-bench Verified, 57.1% on Multilingual, and 36.8% on SWE-bench Pro demonstrate strong evaluation results.Source: §6 · p. 8
• Over 200,000 trajectories across nine languages address the scarcity of large-scale agentic software-engineering data.Source: §6 · p. 8
• Combining explicit thinking modalities with high-quality behavioral traces is presented as an effective path for developing specialized autonomous agents.Source: §6 · p. 8

• Teacher-model biases and systemic errors may be inherited by students trained from Minimax-M2.5 and Qwen3.5-122B.Source: Limitations · p. 9
• Infrastructure dependencies and environmental factors can cause minor score fluctuations that affect precise reproducibility.Triplerun aggregation mitigates, but does not eliminate, variance from LLM stochasticity and volatile execution environments.Source: Limitations · p. 9

• Agent trajectories come from open-source repositories using permissive licenses including MIT, Apache 2.0, and BSD.Source: Ethics Statement · p. 9
• Automated filtering pipelines redact Personally Identifiable Information and sensitive credentials from the released data.Source: Ethics Statement · p. 9
• Gemini assisted with manuscript language refinement and generation of the Figure 1 visual data-flow illustration.Source: Ethics Statement · p. 9

• The SFT and inference hyperparameters are detailed in Tables 6a and 6b.Source: A Implementation Details · p. 12
• Table 6 provides a comprehensive breakdown of OPEN-SWE-AGENT hyperparameters.Source: Table 6 · p. 12