Source-linked AI summary
Scaling the Horizon, Not the Parameters: Reaching Trillion-Parameter Performance with a 35B Agent
Lei Bai, Zongsheng Cao, Yang Chen, Zhiyao Cui, Shangheng Du, Yue Fan, Shiyang Feng, Zijie Guo, Haonan He, Liang He, Xiaohan He, Shuyue Hu, Yusong Hu, Songtao Huang, Yichen Jiang, Hao Li, Xin Li, Dahua Lin, Weihao Lin, Fenghua Ling, Dongrui Liu, Zhuo Liu, Wenjie Lou, Runmin Ma, Chunjiang Mu, Haoyang Peng, Tianshuo Peng, Jinxin Shi, Luohe Shi, Boyuan Sun, Zelin Tan, Shengji Tang, Yan Teng, Qianyi Wang, Xiaosong Wang, Yiming Wu, Yi Xie, Xiangchao Yan, Jingqi Ye, Peng Ye, Fangchen Yu, Jiakang Yuan, Bihao Zhan, Bo Zhang, Chen Zhang, Shufei Zhang, Shuaiyu Zhang, Wenlong Zhang, Yiqun Zhang, Junpeng Zhao, Zhijie Zhong, Bowen Zhou, Yuhao Zhou
TL;DR
Long-horizon agents need infrastructure that links knowledge, actions, observations, and verification without relying solely on trillion-parameter scale. Agents-A1 addresses this with a 35B MoE model trained through domain-routed on-policy distillation, outperforming 1T-parameter models on several long-horizon benchmarks while remaining strong on others.
Problem
Long-horizon agent training lacks unified knowledge-action infrastructure connecting external knowledge, actions, observations, and verification signals for grounded supervision.
Method
Agents-A1 combines a long-horizon knowledge-action infrastructure with full-domain supervision, domain teachers, and domain-routed on-policy distillation to unify heterogeneous agent abilities.
Results
Agents-A1 outperforms 1T-parameter models on SEAL-0, IFBench, HiPhO, FrontierScience-Olympiad, and MolBench-Bind, while also achieving strong results on SciCode, HLE, and BrowseComp.
Takeaways & Limitations
The work demonstrates a practical path toward scaling agent horizons with a 35B model across heterogeneous long-horizon science and research capabilities.
Takeaways & Limitations
The model’s long-horizon abilities remain limited by basic atomic abilities such as planning, reflection, long-context summarization, and identifying important past information.
Abstract
from arXiv · showhide
We introduce Agents-A1, a 35B Mixture-of-Experts Agentic Model that reaches trillion-parameter-level performance by scaling the agent horizon. We investigate agent-horizon scaling from two perspectives: scaling long-horizon trajectories and scaling heterogeneous agent abilities. To support this goal, we build a long-horizon knowledge-action infrastructure that connects external knowledge, actions, observations, and verifier outcomes, producing agentic trajectories with an average length of 45K tokens. Based on this, we train Agents-A1 with a three-stage recipe. First, we perform full-domain supervised fine-tuning to align the base model with broad agentic behaviors. Second, we train domain-level teacher models to capture specialized expertise in each domain. Third, we propose a multi-teacher domain-routed on-policy distillation with salient vocabulary alignment to improve knowledge transfer efficiency across different domains, unifying six heterogeneous domains into one deployable student model. Agents-A1 achieves strong and broad performance for long-horizon agent benchmarks. Compared with 1T-parameter model such as Kimi-K2.6 and DeepSeek-V4-pro, Agents-A1 achieves leading results on SEAL-0 (56.4), IFBench (80.6), HiPhO (46.4), FrontierScience-Olympiad (79.0), and MolBench-Bind (56.8), and remains highly competitive on SciCode (44.3), HLE (47.6) and BrowseComp (75.5). We hope this work provides the community with a practical path for scaling the horizon using a 35B agent that can reach or match the performance of 1T models on long-horizon tasks.
1. Introduction
Agents-A1 addresses long-horizon agent challenges by scaling trajectories and heterogeneous abilities rather than relying only on parameter scaling. It combines a knowledge-action infrastructure with three-stage multi-domain training to achieve strong long-horizon performance using a 35B MoE model.
- Introduction: Long-horizon agents must acquire information, decompose tasks, use tools, verify intermediate results, and continuously adapt strategies.These requirements arise in software engineering, scientific research, and complicated decision making.
- Introduction: Existing approaches primarily scale parameters, making comparable agentic competence difficult without similar model scale, data, and training resources.Parameter scaling internalizes reasoning patterns, tool-use behaviors, and domain knowledge.
- Introduction: The proposed infrastructure supports useful information gathering, issue summarization, tool selection, task execution, and verification across heterogeneous agentic abilities.These abilities include multi-step retrieval, tool use, executable iteration, constraint tracking, and result reflection.
- Introduction: Agents-A1 is a 35B Mixture-of-Experts agentic model built with infrastructure connecting external knowledge, actions, observations, execution results, and verification signals.The infrastructure produces long-horizon trajectories averaging 45K tokens.
- Introduction: The model uses three stages: full-domain supervised fine-tuning, domain-specific teacher training, and multi-teacher domain-routed on-policy distillation with salient vocabulary alignment.The recipe transfers specialized expertise across domains while reducing conflicts between different reasoning patterns.
- Introduction: Agents-A1 outperforms 1T-parameter models on SEAL-0, IFBench, HiPhO, FrontierScience-Olympiad, and MolBench-Bind, while achieving strong results on SciCode, HLE, and BrowseComp.The comparison is against Kimi-K2.6 and DeepSeek-V4.
2. Knowledge-Guided General Agent Training with Specialized Teachers
Agents-A1 uses a three-stage training pipeline that combines broad supervised alignment, specialized domain teachers, and multi-teacher on-policy distillation. Its knowledge-action infrastructure preserves verifiable agent processes, while domain-routed guidance consolidates heterogeneous capabilities into one deployable student.
- Three-stage training: The three-stage pipeline first performs full-domain supervised fine-tuning, then trains specialized domain teachers, and finally consolidates them through multi-teacher on-policy distillation.The resulting objective is a single deployable student model spanning heterogeneous agent capabilities.
- Knowledge-action infrastructure: KAG represents information acquisition, tool calling, executable iteration, evidence verification, and constraint tracking as linked evidence, actions, observations, and verifier outcomes.This infrastructure provides process-level supervision beyond final answers.
- Knowledge-action infrastructure: Unlike conventional knowledge graphs, KAG preserves how answers are acquired, tested, revised, and verified, retaining successful and failed evidence-backed trajectories for reproducible long-horizon supervision.Its process-level records support cross-step credit assignment.
- Knowledge-action infrastructure: A proposer–solver–verifier game iteratively expands each domain graph by proposing constrained tasks, solving them with retrieval and tools, and verifying answers, evidence, execution, and trajectories.Candidates must be verifiable, valid, process-informative, evidence-covering, and unambiguously specified.
- Multi-teacher distillation: Domain-routed multi-teacher OPD with salient vocabulary alignment transfers specialized policies while limiting losses to trainable generated tokens and normalizing updates across active domains.Hard routing preserves domain-specific teacher preferences and avoids incompatible token-level signals.
3. Multi-domain Data Pipeline
The multi-domain pipeline instantiates knowledge-action graphs across search, machine learning engineering, scientific reasoning, and instruction following. It combines structured problem construction, executable or tool-augmented trajectories, verification, and quality control to produce long-horizon training data.
- Search: Search-domain KAGs connect wiki evidence, hyperlink-induced entity transitions, retrieved observations, and verifiers that check answer recovery and supporting paths.Controlled random walks filter candidate nodes by entity repetition, title duplication, page validity, connectivity, degree, and text length before LLM selection.
- Search: Deep-research trajectories use search, read_page, and code tools in the real Internet environment, with a 256K-token context window and post-processing that removes incorrect or degenerate runs.The code tool supports sandboxed Python computation, while quality control removes wrong answers, overly short histories, and obvious guesses.
- Machine Learning Engineering: MLE-domain KAGs represent executable solution search through edits, experiments, tree navigation, observations, and grader-based verification, retaining valid and competitive submissions across multiple teacher runs.The harness records logs, exceptions, metrics, artifacts, and submission validity; trajectories are replayed, trimmed, deduplicated, and serialized with loss masks.
- Scientific Reasoning: Scientific KAG expansion yields approximately 15K higher-difficulty, interaction-rich problem instances for long-horizon reasoning, with training trajectories spanning verified no-tool and tool-augmented solutions.The graph encodes problem statements, keywords, and solution components, while reasoning actions include decomposition, transformation, retrieval, and computation.
- Instruction Following: A self-built long-context pipeline produces 10K verified QA instances by graphing document facts and synthesizing multi-hop tasks with injected local rules or distractors.The pipeline extracts entities, attributes, relations, numerical values, and other salient facts before constructing evidence graphs compatible with the KAG representation.
4. Three-stage Training Recipe
The three-stage recipe begins with broad-domain SFT on long-horizon trajectories, then trains specialized domain teachers, and finally consolidates them into one deployable student through multi-teacher on-policy distillation.
- Stage 1: Full-domain SFT: SFT aligns Qwen3.5-35B-A3B with helpful, clear, context-aware instruction following using response-only cross-entropy loss.Instruction tokens are masked, while sample packing concatenates short examples to improve throughput and GPU utilization without cross-contamination.
- Stage 1: Full-domain SFT: ~100K trajectories averaging 45K tokens form the diverse, multi-domain SFT dataset, with rigorous filtering, deduplication, and human review.The data covers deep research, coding and engineering, and scientific problem-solving, emphasizing long-horizon reasoning across extended contexts.
- Stage 2: Domain-level teachers: Domain-level teachers specialize agentic abilities through domain-specific training, including search-tool use and scientific problem-solving with interactive behavior.The search teacher learns decomposition, web search, page reading, and answer synthesis; scientific training targets derivations, specialized knowledge, computation, and symbolic manipulation.
- Stage 2: Domain-level teachers: Dense outcome and process rewards, plus hard-task data with near-miss trajectories, address sparse and noisy rewards in long-horizon scientific-agent training.Hard examples create gradient-bearing contrast within groups, while data reuse repeatedly leverages high-quality batches during RL post-training.
- Stage 3: Multi-teacher OPD: After full-domain SFT and teacher training, multi-teacher OPD optimizes the student on its own rollouts under domain-specific teacher guidance to produce one deployable model.The method includes a detailed OPD objective with salient vocabulary alignment for transferring specialized knowledge across domains.
5. Experimental Results · 5.1. Evaluation Setting
Section 5.1 defines evaluation settings across long-horizon search, engineering, scientific research, long-context instruction following, general agentic tasks, molecular benchmarking, and autonomous tool-code exploration. It specifies benchmark protocols, tool configurations, scoring procedures, simulators, repeated runs, and execution budgets where provided.
- 5.1. Evaluation Setting: Long-horizon search covers GAIA, BrowseComp, XBench-DeepResearch, and SEAL-0, with search, webpage-visit, and Python code tools.The search tool returns the top-50 results per query, the visit tool summarizes fetched webpages, and the code tool runs Python in a remote sandbox.
- 5.1. Evaluation Setting: Engineering evaluation uses official SciCode and MLE-Bench-Lite protocols for sequential scientific coding and end-to-end machine-learning engineering.MLE-Bench-Lite covers 22 Kaggle competitions and requires autonomous data exploration, model training, and submission generation.
- 5.1. Evaluation Setting: MLE-Bench-Lite reports medal rate—the fraction of competitions earning at least bronze—averaged over three seeds, with each task isolated on an H200 for 12 hours.Submissions are graded against the original competition leaderboard and mapped to bronze, silver, or gold medals.
- 5.1. Evaluation Setting: Scientific research evaluation includes HLE with tools, HiPhO, FS-O, and FS-R, reporting official baseline scores when available.HLE with tools evaluates expert-level reasoning with external tool use; Qwen3.6-35B-A3B lacks an official result and is evaluated through the same tool-augmented pipeline.
- 5.1. Evaluation Setting: Long-context and instruction-following evaluation uses LongBench V2, IFBench, and IFEval under official or benchmark-provided scripts.LongBench V2 contains 503 multiple-choice questions spanning diverse long-context understanding tasks.
- 5.1. Evaluation Setting: General agentic evaluation uses pass@1 averaged across domains on τ2-Bench and VitaBench, with DeepSeek-V3.2 replacing GPT-4.1 as the user simulator.τ2-Bench covers retail, telecom, and airline, while VitaBench covers cross-domain, delivery, in-store, and OTA settings.
- 5.1. Evaluation Setting: MolBench reports Binding Affinity Comparison scores over three repeated runs, while MatTools reports completion over 138 subtasks after autonomous exploration of a nearly 100K-line codebase.Both evaluations follow their stated official or autonomous code-exploration settings and average results over three runs.
- 5.1. Evaluation Setting: For τ2-Bench, the reported Qwen3.5-35B-A3B results are 81.2 officially and 33.0 reproduced, with further discussion deferred to Section 5.2.1.The passage explicitly distinguishes the official result from the reproduced result.
5.2. Results and Observations
The experiments show that specialized search, science, long-context, and tool-calling post-training substantially improve targeted capabilities, while Agents-A1’s routed distillation transfers these strengths into a competitive unified 35B model. Agents-A1 leads same-scale baselines on several difficult tasks but remains weaker than larger models on some engineering-oriented evaluations.
- Domain teacher results: Search-enhanced training outperforms Qwen3.5-35B-A3B across all four search benchmarks, with GAIA rising from 59.8 to 85.4 (+25.6).On HLE, the score increases from 47.4 to 50.3 (+2.9).
- Domain teacher results: Science-enhanced training shows comprehensive superiority over Qwen3.5-35B-A3B, especially on FS-R, improving intrinsic reasoning and extrinsic tool-use interaction.The passage attributes these gains to the proposed two-stage SFT in scientific scenarios.
- Domain teacher results: Long-context RL raises LongBench v2 from 59.0 to 62.4 while consistently improving instruction-following and long-context evaluations over Qwen3.5-35B-A3B.The improvement is attributed to better retrieval and understanding of relevant evidence from longer, more difficult contexts.
- Domain teacher results: Tool-enhanced post-training raises the τ2-Bench average from 32.53 to 82.50, including Airline from 16.00 to 72.00 and Retail from 30.70 to 82.50.The passage reports consistent gains across Airline, Retail, and Telecom, as well as substantial improvements on VitaBench.
- Agents-A1 results and observations: Agents-A1 is a strong 35B-level model that competes with 1T-level models, surpassing same-scale baselines and several larger models in multi-step search, scientific research, and long-instruction following.It achieves 44.3 on SciCode and 43.9 on MLE-Bench-Lite, leading all same-scale 35B baselines on both, but GPT-5.5 reaches 72.7 on MLE-Bench-Lite.
- Agents-A1 results and observations: Multi-teacher on-policy distillation reduces the full-domain SFT performance drop caused by differing long-instruction and long-horizon search thinking patterns, balancing heterogeneous capabilities in one model.OPD transfers teacher strengths into a unified policy but does not always outperform each specialized domain teacher.
5.3. Long-Horizon Task Applications
Agents-A1 demonstrates long-horizon task capability by autonomously optimizing a machine-learning pipeline through iterative diagnosis and intervention, while also reconstructing and analyzing a real tropical cyclone from best-track data. These applications show integrated data organization, algorithmic improvement, diagnostic computation, and result synthesis.
- Machine Learning Engineering: Agents-A1 starts from a naive CNN baseline and autonomously applies temporal analysis, audio augmentation, and localized interventions for right-whale call detection.The evaluation requires end-to-end optimization over a long-horizon machine-learning engineering run.
- Machine Learning Engineering: Agents-A1 identifies temporal train-test domain shift and combines dataset diagnosis, representation design, augmentation, and iterative evaluation to improve generalization.The optimization trajectory shows consistent improvement across multiple iterations and produces an interpretable multi-step solution beyond isolated hyperparameter tuning.
- Earth Science: Agents-A1 automatically identifies IBTrACS and completes data extraction, cleaning, derived analysis, visualization, and interpretation for Severe Cyclonic Storm Nargis.The task evaluates end-to-end Earth science analysis using real best-track data.
- Earth Science: Agents-A1 reconstructs Nargis’s major evolution with reasonable fidelity and derives track length, translation speed, heading variation, and intensity evolution from multiple operational estimates.It preserves WMO/IMD and JTWC/USA intensity estimates rather than conflating them.
6. Limitation and Future Work
Agents-A1 is a 35B MoE model built to explore agent-horizon scaling, drawing its abilities from a baseline model, unified knowledge-action infrastructure, and domain-routed on-policy distillation. Future work will improve atomic abilities that support goal-consistent, efficient long-horizon interaction and long-process task solving.
- Limitations: Agents-A1 is a 35B MoE model exploring agent-horizon scaling through baseline abilities, unified knowledge-action infrastructure, and domain-routed on-policy distillation.These are identified as the model’s three main sources of learned agentic abilities.
- Future Work: Long-horizon agents require planning before reasoning, reflection before acting, long-context summarization, and identification of important past information.The paper presents these atomic abilities as important for maintaining goal consistency and efficiency during long-horizon task solving.
- Future Work: Future work will improve these atomic abilities to strengthen Agents-A1’s performance on long-process tasks.The authors plan to use the improvements as a starting point for further advancing long-interaction agents.
A. Appendix · A.1. Contributions and Acknowledgments
The appendix acknowledges contributors across the knowledge-action infrastructure, training methods, long-horizon search, engineering, scientific research, instruction following, tool-calling, evaluation, and deployment. It assigns named contributors to each area.
- A.1. Contributions and Acknowledgments: Knowledge-action infrastructure was contributed by Zongsheng Cao, Bihao Zhan, and Zhijie Zhong.
- A.1. Contributions and Acknowledgments: Full-domain SFT was contributed by Yue Fan and Tianshuo Peng, while multi-teacher OPD was contributed by Shiyang Feng, Yi Xie, and Songtao Huang.
- A.1. Contributions and Acknowledgments: Long-horizon Search was contributed by Tianshuo Peng, Zhijie Zhong, Jinxin Shi, Runmin Ma, Jiakang Yuan, Yusong Hu, and Yue Fan.
- A.1. Contributions and Acknowledgments: Engineering Tasks were contributed by Xiangchao Yan, Shangheng Du, Shuaiyu Zhang, Junpeng Zhao, Jinxin Shi, Yiming Wu, and Boyuan Sun.
- A.1. Contributions and Acknowledgments: Scientific Research was contributed by Fangchen Yu, Shengji Tang, Zhuo Liu, Jingqi Ye, Yichen Jiang, Haonan He, and Weihao Lin.
- A.1. Contributions and Acknowledgments: Instruction Following and Context Learning was contributed by Xiaohan He, Songtao Huang, Zhijie Zhong, and Shiyang Feng.
- A.1. Contributions and Acknowledgments: General and Scientific Tool-calling, Evaluation and Deployment were contributed by their respective named teams, including Yiqun Zhang, Chen Zhang, Yuhao Zhou, and Luohe Shi.General and Scientific Tool-calling lists eight contributors; Evaluation and Deployment lists six.
Scientific Directors and Advisors
The scientific directors and advisors are Wenlong Zhang, Fenghua Ling, Xin Li, Yan Teng, Dongrui Liu, Shufei Zhang, Liang He, Xiaosong Wang, Peng Ye, Shuyue Hu, Dahua Lin, and Bowen Zhou.
- Scientific Directors and Advisors: The listed scientific directors and advisors include Wenlong Zhang, Fenghua Ling, Xin Li, and Yan Teng.
- Scientific Directors and Advisors: The list also includes Dongrui Liu, Shufei Zhang, Liang He, and Xiaosong Wang.
- Scientific Directors and Advisors: Additional scientific directors and advisors are Peng Ye, Shuyue Hu, Dahua Lin, and Bowen Zhou.
Project Co-lead
The section identifies a project co-lead who made a key contribution to the project.
- The project co-lead made a key contribution to the project.