PaperlayerView exact revision on arXiv

Source-linked AI summary

LongCat-Flash-Thinking-2601 Technical Report

Meituan LongCat Team, Anchun Gui, Bei Li, Bingyang Tao, Bole Zhou, Borun Chen, Chao Zhang, Chao Zhang, Chen Gao, Chen Zhang, Chengcheng Han, Chenhui Yang, Chuyu Zhang, Cong Chen, Cunguang Wang, Daoru Pan, Defei Bu, Dengchang Zhao, Di Xiu, Dishan Liu, Dongyu Ru, Dunwei Tu, Fan Wu, Fengcheng Yuan, Fengcun Li, Gang Xu, Guanyu Wu, Guoyuan Lin, Haibin Wang, Hansi Yang, Hao Yang, Haonan Yan, Haoxiang Ma, Haoxing Wen, Hongyan Hao, Hongyin Tang, Hongyu Zang, Hongzhi Ni, Hui Su, Jiacheng Zhang, Jiahong Zhou, Jiahuan Li, Jiaming Wang, Jian Yang, Jianfei Zhang, Jianhao Xu, Jianing Wang, Jiapeng Zhu, Jiaqi Sun, Jiarong Shi, Jiarui Zhao, Jingang Wang, Jinluan Yang, Jinrui Ding, Jinwei Xiao, Jiyuan He, Juncan Xu, Kefeng Zhang, Keheng Wang, Li Wei, Lianhui Ma, Lin Qiu, Lingbing Kong, Lingchuan Liu, Linsen Guo, Mengshen Zhu, Mengxia Shen, Mingyang Zhu, Peiguang Li, Peng Pei, Peng Zhao, Pengcheng Jia, Pengtao Zhang, Ping Liu, Qi Gu, Qiong Huang, Qiyuan Duan, Quanchi Weng, Rongxiang Weng, Rongzhi Zhang, Rumei Li, Shanglin Lei, Shengnan An, Shijun Dai, Shizhe Wu, Shuaikang Liu, Shuang Zhou, Shuo Wang, Songyuan Zhao, Tao Liang, Tianhao Hu, Tianze Chen, Wei Liu, Wei Shi, Wei Wang, Weifeng Tang, Wenjie Shi, Wenlong Zhu, Wentao Chen, Wentao Shi, Xi Su, Xiandi Ma, Xiangcheng Liu, Xiangyu Xi, Xiangyuan Liu, Xiangzhou Huang, Xiao Liu, Xiaodong Cai, Xiaolong Chen, Xiaowei Shi, Xiaoyu Li, Xin Chen, Xingchen Liu, Xuan Huang, Xuezhi Cao, Xunliang Cai, Yan Chen, Yang Bai, Yang Liu, Yang Yang, Yang Zheng, Yanyu Chen, Yaoming Wang, Yaoming Zhu, Yaorui Shi, Yaqi Huo, Yerui Sun, Yi Zhang, Yi-Kai Zhang, Yifan Lu, Yifan Zhao, Yihao Chen, Yitao Zhai, Yongjing Yin, Yongwei Zhou, Youshao Xiao, Yu Wang, Yu Yang, Yuchen Xie, Yuchen Yu, Yuchuan Dai, Yue Xu, Yueqing Sun, Yufei Zhang, Yuhuai Wei, Yulei Qian, Yunfan Liang, Yunke Zhao, Yuwei Jiang, Yuxin Bian, Yuxin Chen, Yuxin Liu, Zeyang Yu, Zhao Yang, Zhengsheng Huang, Zhengyu Chen, Zhijian Liu, Zhikang Xia, Zhimin Lin, Zhiyuan Yao, Zhuofan Chen, Zhuowen Han, Zijian Zhang, Ziran Li, Ziwen Wang, Ziyuan Zhuang

arXiv:2601.16725v2cs.AI

TL;DR

Complex real-world tasks require reasoning models to interact robustly with external tools across long-horizon, heterogeneous, and noisy environments. LongCat-Flash-Thinking-2601 addresses this through unified training, scalable environment construction, noise-aware reinforcement learning, and Heavy Thinking, achieving leading open-source agentic benchmark performance and broad real-world generalization.

  • Problem

    Agentic tasks demand long-horizon interaction, diverse environments, and robustness to real-world noise, while suitable trajectories and scalable training setups remain limited.

  • Method

    The model combines unified multi-domain training, executable environment scaling, synthesized and verified agentic data, noise-aware reinforcement learning, and Heavy Thinking test-time scaling.

  • Results

    The model achieves state-of-the-art performance among open-source models across agentic benchmarks and generalizes to out-of-distribution real-world agentic scenarios.

  • Takeaways & Limitations

    Diverse executable environments, explicit noise incorporation, and joint expansion of reasoning width and depth support broad agentic reasoning capability.

  • Takeaways & Limitations

    Environment expansion is constrained by the need to maintain cross-tool database consistency and satisfy dependency constraints.

Abstract

from arXiv · show

We introduce LongCat-Flash-Thinking-2601, a 560-billion-parameter open-source Mixture-of-Experts (MoE) reasoning model with superior agentic reasoning capability. LongCat-Flash-Thinking-2601 achieves state-of-the-art performance among open-source models on a wide range of agentic benchmarks, including agentic search, agentic tool use, and tool-integrated reasoning. Beyond benchmark performance, the model demonstrates strong generalization to complex tool interactions and robust behavior under noisy real-world environments. Its advanced capability stems from a unified training framework that combines domain-parallel expert training with subsequent fusion, together with an end-to-end co-design of data construction, environments, algorithms, and infrastructure spanning from pre-training to post-training. In particular, the model's strong generalization capability in complex tool-use are driven by our in-depth exploration of environment scaling and principled task construction. To optimize long-tailed, skewed generation and multi-turn agentic interactions, and to enable stable training across over 10,000 environments spanning more than 20 domains, we systematically extend our asynchronous reinforcement learning framework, DORA, for stable and efficient large-scale multi-environment training. Furthermore, recognizing that real-world tasks are inherently noisy, we conduct a systematic analysis and decomposition of real-world noise patterns, and design targeted training procedures to explicitly incorporate such imperfections into the training process, resulting in improved robustness for real-world applications. To further enhance performance on complex reasoning tasks, we introduce a Heavy Thinking mode that enables effective test-time scaling by jointly expanding reasoning depth and width through intensive parallel thinking.

1 Introduction

LongCat-Flash-Thinking-2601 extends general reasoning toward long-horizon, tool-mediated agentic tasks through specialized training, environment scaling, and Heavy Thinking. It reports leading open-source performance across agentic benchmarks and generalization to out-of-distribution real-world scenarios.

  • Agentic reasoning applies complex problem-solving through adaptive interaction with external environments, extending reasoning beyond intrinsic capabilities.
  • The 560B-parameter MoE model uses agentic mid-training with synthesized structured trajectories to initialize scarce long-horizon interaction behavior.The model activates 27B parameters on average per token.
  • The post-training pipeline scales reinforcement learning across diverse environments and more than 20 domains to acquire generalizable agentic skills.
  • Heavy Thinking jointly expands reasoning width and depth through parallel exploration and iterative refinement for challenging reasoning and agentic tasks.
  • 73.1% on BrowseComp, 77.7% on RWSearch, 88.2% on τ 2-Bench, and 29.3% on VitaBench establish leading open-source results across agentic search and tool use.

2 Pre-Training

The pre-training strategy augments the original recipe with synthesized agentic trajectories derived from text and executable environments. These trajectories increase structural complexity, planning demands, and execution consistency through targeted decomposition and verification.

  • The model retains the original pre-training data distribution while adding structured agentic trajectories to address scarce long-horizon reasoning, planning, and interaction data.
  • Text-driven synthesis: Text-driven synthesis mines multi-step workflows, extracts tool schemas, and converts them into diverse multi-turn user-agent interactions with quality filtering.
  • Tool decomposition hides parameters in the environment, while reasoning decomposition supplies alternative actions requiring the model to select appropriate candidates.
  • Environment-grounded synthesis: Executable environments ground trajectories in tool dependencies, controlled tool-chain sampling, and verified execution to preserve logical correctness.
  • Planning-oriented augmentation targets decomposition, alternative exploration, and intermediate decision-making because planning behavior is poorly represented in existing data.

3 Scaling Reinforcement Learning

The reinforcement-learning approach trains domain-specialized experts under a unified multi-domain framework before consolidating them into one general model. It addresses scalable environment construction and the infrastructure demands of heterogeneous, long-tailed, multi-turn agentic rollouts.

  • Domain-specialized expert models are trained under a shared post-training framework and consolidated into one general model through model-level and data-level merging.
  • Agentic reinforcement learning requires scalable environments, high-quality cold-start data, calibrated tasks, and asynchronous infrastructure for long-tailed multi-turn rollouts.
  • Figure 3 depicts automated environment construction from high-level domain specifications into tools, database and tool schemas, and a verified dependency graph.

3.1 RL Preparation

The paper prepares agentic reinforcement learning with executable, diverse environments, structured task and data construction, and controlled complexity growth that preserves reliable supervision.

  • Preparation requirements: Agentic reinforcement learning requires a capable initial policy, principled tasks, and scalable environments for long-horizon interaction.The preparation pipeline also targets high-quality cold-start data and stable learning across heterogeneous domains.
  • Environment construction: The environment pipeline converts domain specifications into executable tool graphs with unified schemas, generated databases, and corresponding tool code.The resulting graphs cover over 20 domains and contain more than 60 tools organized in dense dependency structures.
  • Task-set design: Environment difficulty combines interaction complexity from clarification, planning, and multi-step prompts with structural complexity in the underlying tool graph.Structural complexity includes the number of involved tool nodes and their connectivity density.
  • Verifiability-preserving expansion: Uncontrolled graph expansion can break cross-tool database consistency, causing valid trajectories to fail and producing biased negative rewards.The paper therefore expands executable tool chains rather than injecting random tools.
  • Verifiability-preserving expansion: Controlled graph growth progressively increases environment complexity while preserving executability and reliable supervision.The procedure can add further seed chains from unused tool nodes and guarantees at least 20 tools in each constructed environment.
  • Cold-start training: Cold-start training prioritizes diverse reasoning patterns and stable interaction formats to enable large-scale exploration while retaining general thinking capability.The data pipeline combines curated existing trajectories with quality control, executability verification, and filtering for general thinking data.

3.2 Scalable Asynchronous Agentic RL Framework

The scalable asynchronous framework addresses long-tailed, latency-variable multi-turn rollouts through coordinated streaming components, distributed execution, and memory-aware prefill-decode disaggregation.

  • Rollout challenges: Multi-turn agentic rollouts repeatedly interleave language-model generation, environment execution, and reward assessment, producing long-tailed trajectories and unpredictable latency.These characteristics create infrastructure challenges beyond single-turn reasoning workloads.
  • System architecture: A producer-consumer architecture separates rollout management, sample-queue staleness control, and training coordination across nodes and workers.Remote procedure calls coordinate the components while CPUs and accelerators execute the tasks.
  • Asynchronous execution: A fully streaming asynchronous pipeline removes batch barriers so generation, environment execution, and reward computation proceed at individual-sample granularity.The design targets reduced device idleness within rollouts and between rollout and training.
  • Large-scale training: Up to 32,000 environments can run across roughly 400 machines with thousands of accelerators, requiring decomposition to avoid rollout-manager bottlenecks.The framework addresses scaling pressure caused by frequent CPU-level interactions across many environments.
  • Memory-aware generation: Prefill-decode disaggregation with CPU-resident KV-cache swapping addresses workload imbalance from long contexts and insufficient on-device cache capacity.Asynchronous chunked transfers and concurrent swapping reduce recomputation overhead and sustain accelerator throughput.
  • Runtime performance: 63% request load ratio was achieved throughout production rollouts, while staleness controls deliberately limited full utilization to manage request freshness.A two-phase request-per-device policy also limits recomputation during later load-balancing stages.

3.3 RL Training Strategy

Agentic reinforcement learning must handle multi-turn interactions, heterogeneous task difficulty, diverse environments, and imperfect feedback. The training strategy combines context management, adaptive task allocation, curriculum learning, self-verification, and robust multi-domain training to improve stability, efficiency, generalization, and resilience.

  • General Training Strategy: Agentic RL introduces multi-turn context growth and unpredictable environment feedback, making effective context management necessary under limited context windows.The strategy preserves the most informative context for long-horizon trajectories.
  • Training Strategy with Scaled Environment: Multi-domain training and explicit environmental noise injection target generalization and robustness across heterogeneous, imperfect real-world settings.Training with noise achieves comparable or slightly better standard-benchmark performance while substantially improving results under noisy conditions.
  • General Training Strategy: Curriculum learning organizes training by task difficulty and capability requirements, progressively structuring optimization around the model’s evolving abilities.Task difficulty is quantified using pre-optimization pass rate, with lower pass rates indicating harder tasks.
  • General Training Strategy: Dynamic rollout allocation prioritizes tasks whose difficulty matches current capability, using real-time training metrics and a dynamic value function to estimate learning value.A heap-based greedy algorithm computes rollout allocation that maximizes aggregate learning value for the current batch.
  • General Training Strategy: On-policy self-verification is dynamically activated when generation stagnates, using trajectory evaluation as an auxiliary signal that accelerates convergence and improves generation.Verification emphasizes challenging cases and couples its influence to generated-trajectory quality to discourage shortcut behavior.
  • Training Strategy with Scaled Environment: Multi-domain environment training improves average task completion, especially on challenging environments, while supporting strong generalization to randomly generated environments.The training reward shows a stable increasing trend, indicating stability at scale under the algorithm–infrastructure co-design.

4 Test-Time Scaling Through Heavy Thinking

Heavy Thinking scales test-time computation through parallel reasoning followed by summary-based refinement. It expands reasoning width and depth, and empirically outperforms self-consistency across long-form, tool-integrated, and agentic reasoning settings.

  • Test-Time Scaling Through Heavy Thinking: Test-time scaling improves complex reasoning by expanding inference computation through greater reasoning depth and breadth.The framework builds on long reasoning with self-reflection and breadth-expanding methods such as self-consistency and tree search.
  • Test-Time Scaling Through Heavy Thinking: Heavy Thinking generates multiple parallel reasoning trajectories, then uses a summary model to combine and refine the candidate solutions.The thinking and summary modules may share parameters or use distinct models.
  • Test-Time Scaling Through Heavy Thinking: Heavy Thinking consistently outperforms self-consistency across long chain-of-thought, tool-integrated, and fully agentic tool-use settings, with larger advantages at higher test-time budgets.An additional reinforcement-learning stage is tailored specifically to the summary phase.

5 Evaluation

The evaluation spans mathematical reasoning, agentic search, agentic tool use, general QA, and coding, using benchmark protocols that include noisy and randomly generated task settings. LongCat-Flash-Thinking-2601 achieves strong results across these evaluations, including leading open-source performance on several challenging benchmarks.

  • The evaluation covers mathematical reasoning, agentic search, agentic tool use, general reasoning, and coding.
  • Evaluation protocols include noise-augmented benchmarks and randomly generated complex tasks across diverse scenarios and domains.The noise pipeline repeatedly injects realistic noise into τ 2 and Vita benchmarks, while Random Complex Tasks samples 100 tasks spanning more than four domains.
  • Mathematical Reasoning: LongCat-Flash-Thinking-2601 achieves a perfect score on AIME-2025, 86.8 on IMO-AnswerBench, and open-source state-of-the-art performance on AMO-Bench.With heavy mode, performance on the English and Chinese AMO-Bench versions is comparable.
  • Agentic Search: 73.1 on BrowseComp and 77.7 on BrowseComp-ZH are state-of-the-art results among evaluated models, while RWSearch reaches 79.5, second only to GPT-5.2-Thinking.BrowseComp results are reported with context management enabled.
  • Agentic Tool Use: LongCat-Flash-Thinking-2601 demonstrates state-of-the-art open-source agentic tool use, including competitive results on noisy variants and randomly generated complex tasks.The reported results indicate robustness to real-world environmental noise and generalization to previously unseen task distributions.
  • General QA and Coding: The model maintains strong general QA and coding performance, including 25.2 on text-only HLE, 85.2 on GPQA-Diamond under heavy mode, and competitive SWE-bench Verified results.It also ranks among top open-source models on LiveCodeBench, OJBench, OIBench, and agentic coding evaluations.

6 One More Thing: Zig-Zag Attention Design

Zigzag Attention reduces long-context inference costs by combining sparse local attention with preserved prefix tokens and alternating sparse and full-attention layers. The design targets the additional latency of heavy-thinking inference and is integrated through structured mid-training.

  • Long reasoning traces make full attention increasingly expensive, while parallel reasoning in heavy-thinking mode further amplifies inference latency.
  • Zigzag Attention: Zigzag Attention restricts each query to recent local tokens and a small set of initial tokens, reducing computation and memory complexity while retaining short-term context and global anchors.W is the local context window size, and B is the number of preserved prefix tokens.
  • Zigzag Connectivity: Approximately 50% of full-attention layers are replaced with SSA layers, while the remaining layers retain MLA-based full attention.Alternating sparse and full-attention layers preserves global information through cross-layer composition and improves hardware utilization.
  • Zigzag Integration: Zigzag Attention is introduced during mid-training by replacing low-importance layers with SSA layers, followed by long-context training and YaRN positional-encoding extension.The resulting model supports context lengths of up to 1M tokens.
  • Figure 13 compares performance against relative cost, with arrow percentages denoting the reduced cost for the corresponding dataset benchmarks.

7 Conclusion

The paper concludes that LongCat-Flash-Thinking-2601 is an open-weight 560B MoE model built through an end-to-end pipeline for long-horizon, interaction-driven agentic reasoning. Its environment scaling, noise-aware training, and Heavy Thinking Mode support strong open-source benchmark performance across agentic tasks.

  • LongCat-Flash-Thinking-2601 is an open-weight 560B MoE reasoning model developed through a unified pipeline co-designing architecture, training, data, and infrastructure.
  • The core innovations are scalable multi-domain reinforcement learning, curriculum-based training with real-world environmental noise, and Heavy Thinking Mode for expanding reasoning width and depth.
  • The holistic design yields state-of-the-art performance among open-source models across a wide range of agentic benchmarks and narrows the gap with leading closed-source models.

Contributions

This supplied contribution section contains author-list information rather than substantive technical contributions.

  • The author listing is alphabetized, with identical English names ordered by Chinese stroke count and pronunciation.
  • An asterisk marks authors who have departed from the team.
  • The supplied passage also lists the paper’s authors without describing a technical contribution.

A Optimal Hyperparameter Prediction

The paper addresses the computational cost of identifying optimal hyperparameters for large-scale mid-training by predicting efficient configurations. Its approach maps hyperparameters to compute cost and uses validation loss to predict suitable settings for continual training.

  • Large-scale mid-training makes optimal hyperparameter identification difficult because the search space is vast and computational costs are high.
  • Hyperparameter Mapping: The method first trains small models with varied hyperparameters and maps optimal configurations to computational cost using validation loss and FLOPS.
  • Hyperparameter Prediction: For each continual training checkpoint, the method estimates equivalent compute cost from validation loss to predict suitable hyperparameters.
  • The approach improves model performance with minimal computational overhead by enabling efficient continual-training hyperparameter selection.

B Token Threshold Context Management Performance Evaluation

BrowseComp Pass@1 accuracy increases with summary context length up to 80K tokens, then declines at 100K. The study therefore fixes 80K as the summarization trigger threshold for subsequent experiments.

  • 63.86% at 20K tokens rises to a peak of 66.58% at 80K, then drops to 65.9% at 100K.
  • The evaluation uses BrowseComp Pass@1 accuracy across summary context lengths with a maximum context turn limit of 500.
  • The authors fix 80K context length as the summarization trigger threshold in subsequent experiments.
Loading 2601.16725v2…