PaperlayerView exact revision on arXiv

Source-linked AI summary

GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Models

GLM-4. 5 Team, :, Aohan Zeng, Xin Lv, Qinkai Zheng, Zhenyu Hou, Bin Chen, Chengxing Xie, Cunxiang Wang, Da Yin, Hao Zeng, Jiajie Zhang, Kedong Wang, Lucen Zhong, Mingdao Liu, Rui Lu, Shulin Cao, Xiaohan Zhang, Xuancheng Huang, Yao Wei, Yean Cheng, Yifan An, Yilin Niu, Yuanhao Wen, Yushi Bai, Zhengxiao Du, Zihan Wang, Zilin Zhu, Bohan Zhang, Bosi Wen, Bowen Wu, Bowen Xu, Can Huang, Casey Zhao, Changpeng Cai, Chao Yu, Chen Li, Chendi Ge, Chenghua Huang, Chenhui Zhang, Chenxi Xu, Chenzheng Zhu, Chuang Li, Congfeng Yin, Daoyan Lin, Dayong Yang, Dazhi Jiang, Ding Ai, Erle Zhu, Fei Wang, Gengzheng Pan, Guo Wang, Hailong Sun, Haitao Li, Haiyang Li, Haiyi Hu, Hanyu Zhang, Hao Peng, Hao Tai, Haoke Zhang, Haoran Wang, Haoyu Yang, He Liu, He Zhao, Hongwei Liu, Hongxi Yan, Huan Liu, Huilong Chen, Ji Li, Jiajing Zhao, Jiamin Ren, Jian Jiao, Jiani Zhao, Jianyang Yan, Jiaqi Wang, Jiayi Gui, Jiayue Zhao, Jie Liu, Jijie Li, Jing Li, Jing Lu, Jingsen Wang, Jingwei Yuan, Jingxuan Li, Jingzhao Du, Jinhua Du, Jinxin Liu, Junkai Zhi, Junli Gao, Ke Wang, Lekang Yang, Liang Xu, Lin Fan, Lindong Wu, Lintao Ding, Lu Wang, Man Zhang, Minghao Li, Minghuan Xu, Mingming Zhao, Mingshu Zhai, Pengfan Du, Qian Dong, Shangde Lei, Shangqing Tu, Shangtong Yang, Shaoyou Lu, Shijie Li, Shuang Li, Shuang-Li, Shuxun Yang, Sibo Yi, Tianshu Yu, Wei Tian, Weihan Wang, Wenbo Yu, Weng Lam Tam, Wenjie Liang, Wentao Liu, Xiao Wang, Xiaohan Jia, Xiaotao Gu, Xiaoying Ling, Xin Wang, Xing Fan, Xingru Pan, Xinyuan Zhang, Xinze Zhang, Xiuqing Fu, Xunkai Zhang, Yabo Xu, Yandong Wu, Yida Lu, Yidong Wang, Yilin Zhou, Yiming Pan, Ying Zhang, Yingli Wang, Yingru Li, Yinpei Su, Yipeng Geng, Yitong Zhu, Yongkun Yang, Yuhang Li, Yuhao Wu, Yujiang Li, Yunan Liu, Yunqing Wang, Yuntao Li, Yuxuan Zhang, Zezhen Liu, Zhen Yang, Zhengda Zhou, Zhongpei Qiao, Zhuoer Feng, Zhuorui Liu, Zichen Zhang, Zihan Wang, Zijun Yao, Zikang Wang, Ziqiang Liu, Ziwei Chai, Zixuan Li, Zuodong Zhao, Wenguang Chen, Jidong Zhai, Bin Xu, Minlie Huang, Hongning Wang, Juanzi Li, Yuxiao Dong, Jie Tang

arXiv:2508.06471v1cs.CL

TL;DR

Existing models show strong but specialized performance, motivating an open-source system that unifies agentic, reasoning, and coding capabilities. The paper introduces MoE models with hybrid reasoning modes and multi-stage training, achieving strong ARC results and a high overall ranking. It releases GLM-4.5, GLM-4.5-Air, and evaluation tools for research and reproducibility.

  • Problem

    A single open-source model that excels across agentic, reasoning, and coding tasks remains limited despite strong proprietary-model results in specific ARC domains.

  • Method

    The paper develops GLM-4.5 and GLM-4.5-Air as MoE models with hybrid thinking and non-thinking modes, trained through supervised fine-tuning and reinforcement-learning methods for reasoning and agentic tasks.

  • Results

    GLM-4.5 ranks 3rd overall across evaluated open-source and proprietary models, with strong performance across agentic, reasoning, and coding benchmarks.

  • Takeaways & Limitations

    The released model series and evaluation toolkit provide open-source systems and resources for advancing applications and research in reasoning and agentic AI.

Abstract

from arXiv · show

We present GLM-4.5, an open-source Mixture-of-Experts (MoE) large language model with 355B total parameters and 32B activated parameters, featuring a hybrid reasoning method that supports both thinking and direct response modes. Through multi-stage training on 23T tokens and comprehensive post-training with expert model iteration and reinforcement learning, GLM-4.5 achieves strong performance across agentic, reasoning, and coding (ARC) tasks, scoring 70.1% on TAU-Bench, 91.0% on AIME 24, and 64.2% on SWE-bench Verified. With much fewer parameters than several competitors, GLM-4.5 ranks 3rd overall among all evaluated models and 2nd on agentic benchmarks. We release both GLM-4.5 (355B parameters) and a compact version, GLM-4.5-Air (106B parameters), to advance research in reasoning and agentic AI systems. Code, models, and more information are available at https://github.com/zai-org/GLM-4.5.

1 Introduction

The paper targets a unified open-source model with agentic, reasoning, and coding abilities, introducing GLM-4.5 and GLM-4.5-Air with hybrid reasoning modes. GLM-4.5 achieves strong ARC benchmark performance and ranks highly across evaluated models.

  • The paper identifies agentic abilities, complex reasoning, and advanced coding as three interconnected capabilities for generalist models.
  • GLM-4.5 and GLM-4.5-Air unify these capabilities through thinking mode for complex tasks and non-thinking mode for instant responses.GLM-4.5 uses a 355B-parameter MoE architecture with 32B activated parameters.
  • 70.1% on TAU-Bench, 91.0% on AIME 24, and 64.2% on SWE-bench Verified demonstrate strong agentic, reasoning, and coding performance.The broader results include 77.8% on BFCL v3, 79.1% on GPQA, and 37.5% on Terminal-Bench.
  • GLM-4.5-Air is a 106B-parameter MoE model that matches or exceeds Qwen3-235B-A22B and MiniMax-M1 at the 100B scale.
  • GLM-4.5 ranks 3rd overall across 12 ARC benchmarks and 2nd on agentic tasks, while using half the parameters of DeepSeek-R1 and one-third those of Kimi K2.GLM-4.5-Air ranks 6th overall and GLM-4.5 ranks 3rd on coding tasks.
  • Both models and an evaluation toolkit are open-sourced to support model use, research, and reproducibility of benchmark results.

2 Pre-Training

GLM-4.5 combines a parameter-efficient MoE architecture with curated multi-stage pre-training and mid-training. The recipe emphasizes code, mathematics, science, long-context, repository-level dependencies, and agent trajectories.

  • Architecture: GLM-4.5 and GLM-4.5-Air use MoE architectures designed to improve computational efficiency in training and inference.The architecture uses loss-free balance routing, sigmoid gates, deeper models, Grouped-Query Attention, partial RoPE, QK-Norm, and MTP layers.
  • Pre-Training Data: Pre-training uses documents from webpages, social media, books, papers, and code repositories, with source-specific processing pipelines.
  • Pre-Training Data: Higher-quality web, multilingual, code, mathematics, and science documents are up-sampled, while low-quality or redundant data is filtered or removed.Code data also uses language-specific quality tiers and Fill-In-the-Middle training; SemDedup removes similar templated webpages.
  • Pre-Training: The two-stage pre-training process begins with general web documents and then up-samples coding, mathematics, and science data.
  • Mid-Training: Mid-training adds domain-specific instruction data after pre-training to boost performance in important application areas.
  • Mid-Training: Repo-level code training concatenates files and related GitHub issues, pull requests, and commits, extending sequence length from 4K to 32K.This setup targets cross-file dependencies and larger repositories.
  • Mid-Training: Synthetic reasoning data covers mathematics, science, and coding competitions, while long-context and agent training extend sequences from 32K to 128K and add synthetic agent trajectories.

3 Post-Training: Expert Model Iteration

Post-training combines expert specialization, unified distillation, and reinforcement-learning techniques to produce models that support both deliberative and direct responses while improving agentic, reasoning, and coding performance.

  • Expert Model Iteration: Stage 1 trains specialized Reasoning, Agent, and General chat experts; Stage 2 uses self-distillation to unify them into one model.
  • Expert Model Iteration: Supervised fine-tuning provides expert cold starts and then distills capabilities from distinct experts using millions of samples across reasoning, chat, agentic, and long-context tasks.
  • Supervised Fine-Tuning: The XML-like function-call template reduces escaping for code-containing arguments without compromising function-call execution performance.
  • Reasoning RL: Filtering hard prompts and scaling responses improves math and science performance by 2%-4% and adds a further 1%-2% gain when four responses are generated per prompt.
  • Reasoning RL: Single-stage RL directly at 64K output length outperforms progressively lengthened multi-stage RL, whose early performance drop is difficult to recover.
  • Reasoning RL: Dynamic sampling temperature increases exploration after reward stabilization while validation limits the next temperature to values causing no more than a 1% performance drop.
  • Code and Science RL: Token-weighted mean loss accelerates code-RL convergence, while expert-verified multiple-choice data produces better science-RL performance than mixed-quality data.

4.1 Evaluation of Base Models

GLM-4.5-Base was evaluated before instruction training and showed stable performance across English, code, mathematics, and Chinese benchmarks.

  • GLM-4.5-Base remained stable across English, Code, Math, and Chinese benchmarks.The evaluation used the last pre-training checkpoint, before instruction-data training.
  • The base-model results support unifying different abilities within one model.

4.2 Evaluation on 12 (ARC) Benchmarks

The full GLM-4.5 models were evaluated across 12 agentic, reasoning, and coding benchmarks, with agentic tests covering tool use, function calling, and web browsing.

  • 4.2 Evaluation on 12 (ARC) Benchmarks: GLM-4.5 was evaluated across 12 benchmarks spanning agentic, reasoning, and coding tasks.The benchmark set includes MMLU-Pro, AIME 24, MATH-500, SciCode, GPQA, HLE, LiveCodeBench, SWE-Bench Verified, Terminal-Bench, TAU-Bench, BFCL V3, and BrowseComp.
  • Agentic: Agentic evaluation measured multi-turn tool interaction, user-defined function calling, and complicated web browsing.TAU-bench covered retail and airline domains, BFCL V3 assessed function calling, and BrowseComp assessed web-agent question answering.
  • Agentic: GLM-4.5 performed better than Gemini 2.5 Pro on TAU-bench and achieved the best overall score among baselines on BFCL V3.
  • Agentic: GLM-4.5’s BrowseComp performance was close to o4-mini and significantly better than Claude Opus 4.
  • Reasoning: Reasoning evaluation covered seven benchmarks, using multi-sample averages for AIME 24 and GPQA and automated answer validation.Humanity’s Last Exam evaluation included only text-based questions.
  • Coding: Coding evaluation used SWE-bench Verified and Terminal-Bench, with GLM-4.5 outperforming GPT-4.1, Gemini-2.5-Pro, and Claude Sonnet 4 on the respective tasks.SWE-bench Verified used 500 human-filtered instances; Terminal-Bench evaluated complex terminal tasks.
  • General abilities: GLM-4.5 performed similarly to leading models on common general-chat benchmarks while surpassing several baselines on instruction following and multi-turn capabilities.It was comparable on MMLU and SimpleQA, exceeded DeepSeek R1 on IFEval, and surpassed GPT-4.1, DeepSeek V3, and Kimi K2 on SysBench.
  • Safety: Safety alignment was evaluated with SafetyBench, which contains 11,435 bilingual multiple-choice questions across seven safety categories.

4.3 Evaluations for Hands-on Experience

Hands-on evaluations combined manual multilingual and logical-reasoning assessments with CC-Bench, a coding-agent benchmark emphasizing task completion and practical tool use.

  • Evaluation setup: Human evaluation used curated real-world prompts across multiple languages and categories, with randomized response order and a consistent 0–10 evaluator.
  • General chat: GLM-4.5 achieved the highest overall score on English prompts, especially in Mathematics, Objective QA, and Text Generation.
  • General chat: GLM-4.5 led on Chinese prompts, with standout performance in Text Generation, Logical Reasoning, and Code Instructions.
  • General chat: GLM-4.5 maintained the lead in other-language prompts, excelling in Text Generation and Subjective QA.
  • Coding agent: CC-Bench contains 52 programming tasks across software-development domains and compares GLM-4.5 with Claude Sonnet 4, Kimi K2, and Qwen3-Coder.Tasks ran in isolated containerized environments with consistent expert interaction strategies.
  • Coding agent: Task completion was the primary CC-Bench metric, while tool-calling success and token efficiency resolved ties.The evaluation prioritized functional correctness and completion over efficiency metrics.
  • Coding agent: GLM-4.5 won 53.9% of head-to-head comparisons against Kimi K2 and 80.8% against Qwen3-Coder.
  • Coding agent: GLM-4.5 achieved a 90.6% tool-calling success rate, higher than Claude Sonnet 4, Kimi-K2, and Qwen3-Coder.

4.4 Evaluation of Translation

The translation evaluation treats modern translation as requiring contextual knowledge and reasoning, then compares GLM-4.5 with specialized translation models on difficult real-world cases.

  • Motivation: Modern translation examples require interpreting internet slang, cultural references, domain nicknames, symbols, and contextual meanings.
  • Evaluation setup: The evaluation compared GLM-4.5 with Qwen-MT-plus, Qwen-MT-turbo, and Seed-X on 100 challenging cases using blind human scores from 0 to 3.
  • Results: GLM-4.5 significantly outperformed the specialized translation models, including on the context-dependent phrase “三花公主驾到”.

5 Conclusion

The report introduces the GLM-4.5 model series, which uses MoE architecture and performs strongly across reasoning, coding, and agentic tasks. GLM-4.5 ranks 3rd globally among open-source and proprietary models, and the model weights are released for applications and research.

  • GLM-4.5 and GLM-4.5-Air comprise the introduced model series.
  • The models adopt MoE architecture to improve computational efficiency compared with previous GLM models.
  • GLM-4.5 excels at reasoning, coding, and agentic tasks.
  • 3rd place globally is GLM-4.5’s reported rank among open-source and proprietary models.
  • The report releases GLM-4.5 and GLM-4.5-Air model weights to advance large-language-model applications and research.

6 Contribution

The contribution section identifies the people involved in the work, grouping them into core contributors, contributors, tech leads, and advisors.

  • Contributor names are ordered alphabetically by first name, and asterisks mark people who have since left the team.
  • Core Contributors: Core contributors are listed as a distinct contributor group.
  • Contributors: Additional contributors are listed separately from the core contributors.
  • Tech Leads: Aohan Zeng, Xin Lv, Qinkai Zheng, and Zhenyu Hou are identified as tech leads.
  • Advisors: Jie Tang, Yuxiao Dong, Juanzi Li, Hongning Wang, Minlie Huang, Bin Xu, Jidong Zhai, and Wenguang Chen are identified as advisors.
Loading 2508.06471v1…