Source-linked AI summary
Qwen3-Coder-Next Technical Report
Ruisheng Cao, Mouxiang Chen, Jiawei Chen, Zeyu Cui, Yunlong Feng, Binyuan Hui, Yuheng Jing, Kaixin Li, Mingze Li, Junyang Lin, Zeyao Ma, Kashun Shum, Xuwu Wang, Jinxi Wei, Jiaxi Yang, Jiajun Zhang, Lei Zhang, Zongmeng Zhang, Wenting Zhao, Fan Zhou
TL;DR
Coding agents need strong long-horizon reasoning, tool use, and recovery while remaining efficient to deploy. Qwen3-Coder-Next addresses this through large-scale executable-task synthesis and execution-feedback training, achieving competitive multi-benchmark performance with 80 billion total and 3 billion active parameters. The model is therefore positioned as an open-weight option for practical coding-agent development, although complex large-scale software tasks remain a limitation.
Problem
Coding agents require long-horizon interaction with execution environments and cascading-failure recovery, creating a need for scalable, verifiable, interaction-rich training signals.
Method
The paper combines staged model training with large-scale synthesis of executable coding tasks, reproducible environments, and learning from execution feedback.
Results
Qwen3-Coder-Next achieves strong performance across agentic coding benchmarks, including 70.6% with SWE-Agent, 71.1% with MiniSWE-Agent, and 71.3% with OpenHands on SWE-Bench Verified.
Takeaways & Limitations
With 80 billion total and 3 billion active parameters, the model balances coding-agent capability with practical inference efficiency and deployment cost.
Takeaways & Limitations
The model remains behind frontier proprietary models on highly complex, large-scale software engineering tasks and may require more interaction turns for some complex tasks.
Abstract
from arXiv · showhide
We present Qwen3-Coder-Next, an open-weight language model specialized for coding agents. Qwen3-Coder-Next is an 80-billion-parameter model that activates only 3 billion parameters during inference, enabling strong coding capability with efficient inference. In this work, we explore how far strong training recipes can push the capability limits of models with small parameter footprints. To achieve this, we perform agentic training through large-scale synthesis of verifiable coding tasks paired with executable environments, allowing learning directly from environment feedback via mid-training and reinforcement learning. Across agent-centric benchmarks including SWE-Bench and Terminal-Bench, Qwen3-Coder-Next achieves competitive performance relative to its active parameter count. We release both base and instruction-tuned open-weight versions to support research and real-world coding agent development.
1 Introduction
Qwen3-Coder-Next is an open-weight coding-agent model designed to combine broad agentic capability with a small active inference footprint. Its training stack scales executable task synthesis and environment-feedback learning, yielding competitive coding performance and deployment efficiency.
- Qwen3-Coder-Next uses hybrid attention and Mixture-of-Experts architecture for coding agents and local development, with 80 billion total parameters and 3 billion active per forward pass.
- Its agentic training stack synthesizes executable tasks, constructs reproducible environments, and learns from execution feedback for code editing, tool use, and fault recovery.
- The staged training pipeline continues pretraining, applies supervised fine-tuning, specializes expert models across coding domains, and distills their capabilities into one unified model.
- Qwen3-Coder-Next achieves competitive SWE-Bench Pro performance and strong results across coding-agent, general-coding, knowledge, and reasoning benchmarks relative to its active compute.
2 Scaling up Agentic Training
The paper scales agentic training by pairing diverse verifiable coding tasks with executable environments and high-throughput execution infrastructure. Its synthesis pipeline combines real-world GitHub issues with controlled task generation to support execution-based validation at scale.
- Scaling agentic training requires both a reliable pipeline for synthesizing verifiable tasks with executable environments and infrastructure that efficiently returns execution feedback.
- Two complementary task-generation approaches mine GitHub pull requests for runnable bug-fixing environments and synthesize new instances within existing executable datasets.
- GitHub-derived tasks decompose pull requests into buggy states, fixes, and test patches, while environment-building agents create Docker environments and verification scripts.
- Controlled bug synthesis uses model-driven rewriting, semantic perturbations, and rule-based transformations, retaining bugs that fail existing tests and are resolved by patch reversion.
- MegaFlow uses a cloud-native execution framework in which each agentic task runs as an Argo workflow with rollout, evaluation, and post-processing stages.
3 Mid-training
Mid-training adapts Qwen3-Next toward code reasoning, repository understanding, and agent interaction while balancing natural and synthetic data. Experiments examine long-context coding, data quality, scaffold scaling, and tool-oriented trajectories.
- 3.1 Mid-training: Mid-training targets code reasoning, repository-level understanding, and agent-style interaction patterns starting from the pretrained Qwen3-Next base model.
- 3.1 Mid-training: Natural data supports general intelligence and robustness, whereas excessive synthetic data can cause over-specialization, reduced response diversity, and weaker adaptation.
- 3.1 Mid-training: The corpus is primarily natural data with a smaller synthetic component intended to preserve response diversity and general-purpose capabilities.
- 3.1.1 Natural Data: Repository-level training expands context length from 32,768 to 262,144 tokens to capture cross-file dependencies and broader context.
- 3.1.1 Natural Data: Web documents are rewritten into normalized structured text to remove advertisements, irrelevant HTML, and formatting artifacts.
- 3.1.2 Synthetic Data: Synthetic multi-turn trajectories are generated across multiple agent frameworks and filtered for termination, task success, and valid tool calls.
- 3.1.2 Synthetic Data: Within-scaffold performance consistently improves with more mid-training tokens, while cross-scaffold transfer remains limited.
4 Post-training
Post-training combines verified agentic data, specialized expert training, and distillation into a unified coding-agent model. It also improves robustness to tool-format variation and coding capabilities through diverse templates and reinforcement learning.
- Supervised fine-tuning: Supervised fine-tuning uses proprietary corpora, verified agentic trajectories, and documentation-grounded open-domain QA to broaden alignment data.Verification executes proposed code or commands and evaluates compiler outputs, runtime errors, and environment changes.
- Expert specialization: Specialized experts target software engineering, QA, web development, and UX coding workflows before their capabilities are distilled into one unified model.The Web Development expert is trained on execution-valid trajectories emphasizing visual and runtime consistency.
- Tool-format generalization: Diverse tool chat templates expose the model to different tool definitions, invocation formats, response wrappers, and representations including JSON, Python, XML, and TypeScript.The qwen3_coder XML-style format is designed for string-heavy arguments and long code snippets without nested quoting.
- Tool-format generalization: SWE-bench Verified performance improves as tool-call template diversity increases, even with identical data volume and training configuration.The reported trend indicates that format diversity improves generalization to new tool-calling formats at deployment time.
- Tool-format generalization: The in-house template-following benchmark evaluates precise tool-call compliance across prompt templates and schemas from community IDE/CLI environments.The model follows consistently across all five environments, whereas some models vary substantially across formatting conventions.
- Unified deployment: Distillation preserves instruction following while transferring expert strengths, enabling one model to handle diverse coding domains without expert routing or multi-model orchestration.This supports practical deployment across real-world agentic coding scenarios.
5 Experiments
Qwen3-Coder-Next is evaluated across agentic, coding, general knowledge, reasoning, and mathematics benchmarks under standardized comparisons. Results show competitive performance with substantially larger models while retaining broad capabilities.
- SWE coding tasks: 70.6%, 71.1%, and 71.3% are achieved on SWE-Bench Verified with SWE-Agent, MiniSWE-Agent, and OpenHands, respectively.The model remains competitive with substantially larger frontier and open-weight models across all three agent scaffolds.
- SWE coding tasks: Qwen3-Coder-Next remains competitive on SWE-Bench Pro, which emphasizes longer-horizon software engineering tasks, despite significantly larger open-weight competitors.The result indicates a favorable efficiency–performance trade-off under multilingual and high-difficulty evaluation conditions.
- Command-line interface tasks: Qwen3-Coder-Next demonstrates consistent Terminal-Bench 2.0 performance across XML- and JSON-based tool schemas and multiple agent scaffolds.The evaluation also notes clear room for improvement in complex tool-use tasks.
- Broader coding benchmarks: The model is evaluated on function-level coding, competitive programming, full-stack development, text-to-SQL, and multilingual code editing benchmarks.Results are reported in Tables 6 and 7, whose captions identify these benchmark groups.
- General knowledge and reasoning: Qwen3-Coder-Next remains strong on general knowledge and reasoning, slightly improving on MMLU-Redux and GPQA while staying close on MMLU, MMLU-Pro, and SuperGPQA.These comparisons use Qwen3-Next as the baseline.
- Competitive mathematics: Qwen3-Coder-Next substantially outperforms Qwen3-Next across all reported competitive mathematics benchmarks, with large gains on HMMT25 Feb and AIME25.The paper attributes these results to transfer of strong code reasoning capabilities to mathematics reasoning capabilities.
6 Conclusion, Limitation, and Future Work
The paper concludes that Qwen3-Coder-Next combines agentic coding capability with practical inference efficiency through a hybrid mixture-of-experts architecture and scaled executable-task training. It also identifies capability gaps and future directions relative to frontier proprietary models.
- Conclusion: Qwen3-Coder-Next uses 80 billion total parameters but only 3 billion active parameters per forward pass to balance coding capability with inference efficiency.The model is designed for real-world agentic software development tasks.
- Conclusion: Large-scale synthesis of executable coding tasks and execution-feedback learning improves tool-use robustness, long-context coding ability, and multi-domain coding performance.These improvements are presented as outcomes of scaling agentic training through mid-training and reinforcement learning.
- Limitations and future work: The smaller active compute footprint and lower total training compute introduce capability trade-offs, including weaker performance on highly complex large-scale software engineering tasks.The authors also identify greater interaction-turn requirements for some complex tasks and frontend and UI capability as areas for improvement.
- Limitations and future work: Future work targets harder and more realistic software projects, improved reasoning efficiency and long-horizon planning, frontend and UI capability, and agentic cybersecurity tasks.The authors additionally mention integrating visual capability into future agent models.
7 Authors
The paper lists core contributors and contributors separately, with authors arranged alphabetically by last name.
- Core contributors: The core contributors are Ruisheng Cao, Mouxiang Chen, Jiawei Chen, Zeyu Cui, Yunlong Feng, Binyuan Hui, Yuheng Jing, Kaixin Li, Mingze Li, Junyang Lin, and others.The passage provides the complete core-contributor list.
- Contributors: The contributor list includes Tianyi Bai, Keqin Bao, Chen Cheng, Yizhong Cao, Xiaodong Deng, and many others.The passage provides the complete contributor list.
- Author ordering: Authors are listed alphabetically by their last names.
A.1 Data Statistics for Synthesized Tasks
This appendix reports data statistics for synthesized tasks, covering real-world repository instances and generated task instances produced through workflow-based bug synthesis. The synthesis pipeline averages 169.7 sampled bugs per code repository.
- Real-world repositories: Table 10 reports data statistics for real-world repository instances built from GitHub pull requests.
- Synthesized tasks: Table 11 reports data statistics for generated task instances created through workflow-based pipelined bug synthesis.
- Synthesized tasks: 169.7 bugs or tasks are sampled on average for each code repository.The tasks are synthesized from open-source projects using different bug-sampling strategies.
A.2 The Checklist of Tool Chat Templates for Scaling
The scaling setup uses 21 tool chat templates that vary mainly in tool-definition and tool-call formats.
- 21 tool chat templates were used to scale training across varied tool-definition and tool-call formats.The templates came from open-source models and agent scaffolds, including variants adapted from harmony, Cline, and Aone Copilot.
A.3 Detailed Implementation on Best-Fit-Packing
Best-fit packing avoids fragmentation by placing documents into samples with sufficient remaining capacity, while fragmentation and padding rates quantify packing costs.
- Best-fit packing treats sample construction as bin packing, placing each document into a sample with enough remaining capacity.The approach avoids context hallucination and head-side truncation associated with traditional concatenation and splitting.
- Fragmentation rate measures the proportion of fragmented documents among all training documents.
- Padding rate measures padding tokens relative to all training tokens.
A.3.1 Two Variants of Sample Packing Strategy
The packing variants trade off fragmentation, padding, and efficiency, while long-document handling and cybersecurity evaluations define practical operating boundaries and reported outcomes.
- Two Variants of Sample Packing Strategy: Restart Last Document forces each sample to begin at a document boundary, eliminating head fragmentation while preserving zero padding and tail fragmentation.The strategy also reweights tokens toward the beginnings of long documents.
- Two Variants of Sample Packing Strategy: Pad Last Document fills the previous sample’s final fragmented document with masked padding, eliminating fragmentation at the cost of training efficiency.Fair comparison requires scaling total training tokens by 1 / (1 − padding_rate).
- Long-Document Handling: For documents exceeding the context length, the method offers split, slide, and drop preprocessing strategies.Split creates context-length chunks, slide uses overlapping windows, and drop removes extremely long documents before packing.
- Ablation Study on Sample Packing Strategy: 20.84% similarity and 24.34% empty rate were achieved by augmenting best-fit packing with drop for extremely long documents.Best-fit packing achieved 17.82% versus padding’s 16.86% with 22% fewer tokens.
- Ablation Study on Sample Packing Strategy: The main experiments adopt split, because extending model context length may be preferable to more elaborate long-context algorithms.
- Cybersecurity Experiments: Qwen3-Coder-Next achieves comparable CTI-analysis performance to other open frontier models but remains weaker on root cause mapping and threat actor attribution.The evaluation uses six AthenaBench-Mini tasks and reports results with greedy decoding.
- Cybersecurity Experiments: Qwen3-Coder-Next reaches competitive secure-coding performance, including 56.32% func-sec@1 on CWEval and 61.2 versus 52.5 on SecCodeBench generation against Claude-Opus-4-5 without security hints.