Source-linked AI summary
IQuest-Coder-V1 Technical Report
Jian Yang, Wei Zhang, Shawn Guo, Zhengmao Ye, Lin Jing, Shark Liu, Yizhi Li, Jiajun Wu, Cening Liu, X. Ma, Yuyang Song, Siwei Wu, Yuwen Li, L. Liao, T. Zheng, Ziling Huang, Zelong Huang, Che Liu, Yan Xing, Renyuan Li, Qingsong Cai, Hanxu Yan, Siyue Wang, Shikai Li, Jason Klein Liu, An Huang, Yongsheng Kang, Jinxing Zhang, Chuan Hao, Haowen Wang, Weicheng Gu, Ran Tao, Mingjie Tang, Peihao Wu, Jianzhou Wang, Xianglong Liu, Weifeng Lv, Bryan Dai
TL;DR
Open-weights code models remain behind proprietary leaders on long-horizon reasoning and complex multi-file codebases. IQuest-Coder-V1 addresses this gap with a code-flow, multi-phase training pipeline and specialized model paths, achieving strong results across code-intelligence benchmarks while introducing a recurrent Loop variant for deployment trade-offs.
Problem
Open-weights code models lag proprietary leaders in long-horizon reasoning and navigating complex, multi-file codebases.
Method
IQuest-Coder-V1 uses code-flow training across pre-training, reasoning- and agentic-data mid-training, and bifurcated thinking and instruction post-training paths.
Results
The models achieve strong performance across agentic software engineering, competitive programming, and tool-use evaluations, including a 76.2 score on SWE-bench Verified.
Takeaways & Limitations
The series provides specialized reasoning and assistance models, plus a recurrent Loop variant and open training checkpoints for research and production-oriented agentic coding systems.
Abstract
from arXiv · showhide
In this report, we introduce the IQuest-Coder-V1 series-(7B/14B/40B/40B-Loop), a new family of code large language models (LLMs). Moving beyond static code representations, we propose the code-flow multi-stage training paradigm, which captures the dynamic evolution of software logic through different phases of the pipeline. Our models are developed through the evolutionary pipeline, starting with the initial pre-training consisting of code facts, repository, and completion data. Following that, we implement a specialized mid-training stage that integrates reasoning and agentic trajectories in 32k-context and repository-scale in 128k-context to forge deep logical foundations. The models are then finalized with post-training of specialized coding capabilities, which is bifurcated into two specialized paths: the thinking path (utilizing reasoning-driven RL) and the instruct path (optimized for general assistance). IQuest-Coder-V1 achieves state-of-the-art performance among competitive models across critical dimensions of code intelligence: agentic software engineering, competitive programming, and complex tool use. To address deployment constraints, the IQuest-Coder-V1-Loop variant introduces a recurrent mechanism designed to optimize the trade-off between model capacity and deployment footprint, offering an architecturally enhanced path for efficacy-efficiency trade-off. We believe the release of the IQuest-Coder-V1 series, including the complete white-box chain of checkpoints from pre-training bases to the final thinking and instruction models, will advance research in autonomous code intelligence and real-world agentic systems.
1. Introduction
IQuest-Coder-V1 targets the gap between open-weight and proprietary code models with a four-pillar, multi-phase Code-Flow pipeline. Its findings emphasize repository transitions, staged reasoning data, and distinct post-training paths for agentic coding.
- IQuest-Coder-V1 is a dense 7B–40B model family designed to narrow open-weight models’ gap with proprietary leaders in long-horizon and multi-file coding.
- The Code-Flow pipeline combines high-quality pre-training and annealing, dual-phase mid-training, bifurcated post-training, and an efficient recurrent architecture.
- Repository transition data provides a superior task-planning signal compared with usual static snapshot files alone.
- Injecting 32k reasoning and agentic trajectories after code annealing but before post-training stabilizes performance under distribution shifts.
- The RL-based thinking path produces autonomous error recovery in long-horizon SWE and coding-contest tasks, unlike standard Instruct SFT paths.
- Evaluations cover coding assistants and multiple code-intelligence benchmarks, with reported improvements over existing open-source alternatives.
2. Model Architecture
LoopCoder uses shared transformer blocks executed recurrently, while the broader training pipeline progresses from foundational data through longer-context mid-training to specialized thinking or instruction models.
- LoopCoder Architecture: LoopCoder executes shared transformer blocks in two fixed iterations, using position-shifted hidden states in the first iteration.
- LoopCoder Architecture: In the second iteration, global attention accesses first-iteration key-value pairs while local attention attends to preceding second-iteration tokens causally.
- Stage 1: Pre-Training & Annealing: Pre-training and annealing establish foundational language understanding and code-generation capabilities before later stages.
- Stage 2: Mid-Training: Mid-training advances from 32k reasoning, agentic, and code data to 128k contexts, producing base models for post-training.
- Stage 3: Post-Training: Post-training branches into thinking models using reasoning SFT and RL, or instruction models using general and code instruction tuning followed by RL.
3. Pre-training
The pre-training strategy combines multilingual and repository-aware data construction with completion objectives and a two-stage mid-training process that scales context length.
- Syntactically related programming languages can outperform naive single-language upsampling when training tokens are limited, supporting multilingual code-data allocation.
- General-corpus processing uses cleaning, hierarchical exact and fuzzy deduplication, and decontamination to construct foundational data.
- Repository evolution data forms (R_old, P, R_new) triplets from mature 40%–80% lifecycle commits to represent meaningful development changes.
- Fill-In-the-Middle predicts a document’s middle segment from its prefix and suffix, with repository-level FIM adding semantically similar same-repository snippets.
- Heuristic completion uses random boundary splitting for large-block generation and random line splitting for typical line-level interaction.
- Mid-training uses 32K and 128K contexts with reasoning QA, agent trajectories, commits, and file- or repository-level FIM data.
4. Post-Training
Post-training converts pretrained models into specialized code-intelligence systems through supervised fine-tuning and reinforcement learning over verified, diverse programming tasks.
- Post-training combines supervised fine-tuning and reinforcement learning across code engineering, mathematics, agentic capabilities, and conversation.
- Training data is generated by frontier models and filtered through deterministic execution checks, reward models, rule-based validation, and multi-agent debate.
- The methodology covers API orchestration, full-stack engineering, competitive programming, code editing, repository-scale engineering, tool use, and GUI agents.
- Infrastructure uses sequence packing, cosine learning-rate schedules, and a three-phase curriculum from basic instruction following to adversarial examples.
- GRPO with clip-Higher trains on competition-code test-pass rates, while SWE-RL uses scalable sandbox infrastructure for real-world software tasks.
5.1. Baselines
The evaluation compares IQuest-Coder-V1 with a broad set of state-of-the-art code models across diverse parameter sizes, tuning strategies, and code-intelligence tasks.
- The baselines include instruction-tuned, base, and reasoning-enhanced code models from leading closed-source and open-source systems.The comparison spans models from Anthropic, OpenAI, Google, Alibaba, DeepSeek, Mistral, Moonshot, ZhiPu, Kuaishou, and BigCode.
- The compared systems cover a wide parameter range and different tuning strategies.
- CrossCodeEval, code-generation, and code-reasoning evaluations define the main comparison areas in this section.
5.2. Experiments on Base Models
The base-model experiments evaluate repository-level code completion, testing whether models can use cross-file context across multiple programming languages.
- CrossCodeEval measures cross-file code completion in Python, Java, TypeScript, and C#.The benchmark is multilingual and explicitly targets repository-level completion scenarios.
- The benchmark assesses how effectively code LLMs leverage cross-file context.
5.3. Evaluation on Instruct Models and Reasoning model
The instruct and reasoning models are evaluated across code generation, code reasoning, efficiency, Text-to-SQL, agentic software workflows, general tool use, and safety. Results show consistently strong performance, including a SWE-bench Verified score of 76.2 and gains on inverse code reasoning.
- Code Generation: The models achieve consistently strong performance across functional correctness, compositional coding, full-stack development, and contamination-aware code-generation evaluations.The evaluations use EvalPlus, BigCodeBench, FullStackBench, and LiveCodeBench.
- Code Reasoning: The models perform strongly on forward code execution and show clear gains on the more challenging Output-to-Input reasoning setting.CRUXEval evaluates I2O and O2I over 800 concise Python functions.
- Code Efficiency: The models produce solutions that are correct and competitive in runtime under Mercury’s distribution-based efficiency evaluation.Mercury evaluates 256 Python problems using runtime distributions and the percentile-based Beyond metric.
- Text-to-SQL: The models perform strongly on Spider and BIRD, which test schema linking, structurally correct SQL generation, value grounding, and realistic database settings.
- Agentic and Tool Use: The evaluation also covers realistic terminal workflows, generalist web agents, heterogeneous API tool use, and safety behavior across harmful and benign prompts.The cited benchmarks include Terminal-Bench, Mind2Web, BFCL, and the Tulu 3 safety suite.
- Agentic Coding Tasks: 76.2 is the reported score on SWE-bench Verified for issue-driven repository patching under standardized Docker-based unit-test verification.SWE-bench Verified contains 500 curated instances.
Conclusion
IQuest-Coder-V1 combines code-flow pre-training with multi-phase evolutionary training to support complex code-intelligence tasks. The Loop variant addresses deployment trade-offs, while specialized models and open checkpoints support further research and production-oriented agentic systems.
- IQuest-Coder-V1 uses code-flow pre-training and multi-phase evolutionary training for autonomous software engineering.
- Dynamic repository transitions and repository-scale reasoning trajectories establish logical foundations for complex code-intelligence tasks.
- The models demonstrate strong performance across agentic software engineering, competitive programming, and tool-use benchmarks.
- The Loop variant uses recurrent architectural innovations to optimize the capacity-efficiency trade-off for deployment.
- Specialized thinking and instruction models, together with open training checkpoints, are released to support code-intelligence research and agentic systems.
6. Contributions and Acknowledgements
The paper acknowledges its authors, core contributors, contributors, leadership and senior advisory committee, and corresponding author.
- The authors are listed in order from Jian Yang through Bryan Dai.
- Jian Yang, Wei Zhang, Shawn Guo, Zhengmao Ye, Lin Jing, Shark Liu, Yizhi Li, and Jiajun Wu are identified as core contributors.
- The contributor list includes Cening Liu, Xi Lin, Yuyang Song, Siwei Wu, and additional named collaborators associated with IQuest Coder.
- Ran Tao, Mingjie Tang, Peihao Wu, Jianzhou Wang, Xianglong Liu, and Weifeng Lv form the leadership and senior advisory committee.
- Bryan Dai is identified as the corresponding author.