GLM-5: from Vibe Coding to Agentic Engineering

Results

• GLM-5 improves about 20% over GLM-4.7 on average across eight benchmarks, is comparable to Claude Opus 4.5 and GPT-5.2 (xhigh), and outperforms Gemini 3 Pro.The benchmarks cover agentic, reasoning, and coding tasks.Source: §1 · p. 2
• 50 on the Intelligence Index v4.0 makes GLM-5 the open-weights leader, up 8 points from GLM-4.7’s 42.The paper attributes the gain to improvements in agentic performance and knowledge/hallucination.Source: §1 · p. 2
• GLM-5 is the #1 open model in both LMArena Text Arena and Code Arena, and overall on par with Claude Opus 4.5 and Gemini 3 Pro.LMArena uses human judgments across real tasks, unlike static benchmarks.Source: §1 · p. 2, Figure 3 · p. 3
• $4,432 is GLM-5’s final Vending-Bench 2 account balance, ranking it #1 among open-source models and approaching Claude Opus 4.5.The benchmark simulates running a vending-machine business over a year.Source: §1 · p. 2, Figure 4 · p. 3
• On internal CC-Bench-V2, GLM-5 significantly outperforms GLM-4.7 across frontend, backend, and long-horizon tasks, narrowing the gap with Claude Opus 4.5.CC-Bench-V2 is part of the paper’s evaluation of long-horizon task performance.Source: §1 · p. 3, Figure 4 · p. 3

Methods

• GLM-5 uses sequential Reasoning, Agentic, and General RL, with on-policy cross-stage distillation to prevent catastrophic forgetting.Base training used 27 trillion tokens with code and reasoning prioritized early; mid-training extended context from 4K to 200K with long-context agentic data. The model also received full-stack optimization across seven domestic Chinese GPU platforms.Source: §1 · pp. 3–4

Technical contributions

• DSA dynamically allocates attention by token importance to reduce training and inference costs without compromising long-context understanding or reasoning depth.With DSA, GLM-5 scales to 744B parameters and a 28.5T-token training budget.Source: §1 · p. 3
• A new asynchronous RL infrastructure further decouples generation from training to improve GPU utilization and post-training efficiency.It enables large-scale exploration of agent trajectories without synchronization bottlenecks.Source: §1 · p. 3

2.1 Architecture

• GLM-5 uses 744B total and 40B active parameters; Muon Split lets MLA match GQA-8, and MLA-256 reduces decoding computation while matching MLA under Muon Split.Muon Split orthogonalizes per-head projection matrices and stabilizes attention-logit scale without clipping; MLA-256 increases head dimension from 192 to 256 and reduces head count by one-third while keeping training computation and parameter count constant.Source: §2.1 · pp. 4–5, Table 1 · p. 5
• Sharing parameters across three MTP training layers keeps draft-model memory consistent with DeepSeek-V3 while increasing speculative-decoding acceptance length over DeepSeek-V3.2 at four steps.The comparison uses the same four speculative steps on a private prompt set.Source: §2.1 · p. 5, Table 2 · p. 5

2.1.1 Continued Pre-Training with DeepSeek Sparse Attention (DSA)

• DSA replaces dense attention with content-based token selection; after 20B adaptation tokens, its long-context performance is close to MLA’s while reducing long-sequence attention computation by roughly 1.5–2×.The DSA recipe begins from the end-of-mid-training model, uses a 1,000-step dense warm-up, and yields tied SFT loss and evaluation benchmarks against MLA under the same SFT data.Source: §2.1.1 · pp. 5–6, Table 3 · p. 5

2.1.2 Ablation Study of Efficient Attention Variants

• Search-based SWA generalizes across tested context lengths and outperforms fixed interleaving; naive SWA loses 30.35 RULER@128K, while efficient variants lose up to 5.69 on RULER@128K and 7.33 on RepoQA@128K, unlike lossless DSA.The search pattern is found at 16K and applied at other lengths; DSA’s token-level sparsity preserves long-range dependencies without quality degradation.Source: §2.1.2 · pp. 6–7, Table 5 · p. 7, Table 4 · p. 7

2.2 Pre-training Data

• Pre-training data work adds higher-quality web and long-tail knowledge selection, expands fuzzily deduplicated unique code tokens by 28%, and improves math-and-science extraction, scoring, and filtering.Code curation also fixes metadata and language classification issues and adds classifiers for low-resource languages; math-and-science filtering excludes synthetic, AI-generated, and template-based data.Source: §2.2 · p. 8

2.3 Mid-Training

• Mid-training extends context progressively through 32K, 128K, and 200K stages and broadens software-engineering and long-context data to strengthen reasoning, long-context, and agentic capabilities.The stages use 1T, 500B, and 50B tokens, respectively; issue–PR data comprises approximately 160B unique tokens, and the additional 200K stage further improves performance even within 128K contexts.Source: §2.3 · pp. 8–9

2.4.1 Memory Efficiency

• Memory optimizations balance pipeline stages, shard and reuse gradient buffers, reduce redundant Muon communication, offload activations, and chunk output projection and loss computation.Activation offloading overlaps with computation and reduces memory with near-zero overhead; sequence chunking lowers peak memory while maintaining performance comparable to unchunked execution.Source: §2.4.1 · p. 9

2.4.2 Parallelism Efficiency

• Parallelism optimizations defer critical-path weight gradients and rebalance long-sequence workloads; INT4 QAT improves low-precision SFT accuracy with a kernel that behaves identically in training and inference.Long-sequence methods include workload-aware reordering, dynamic attention redistribution, flexible context-parallel group sizes, and hierarchical all-to-all communication.Source: §2.4.2 · pp. 9–10, §2.4.3 · p. 10

3.1 Supervised Fine-Tuning

• GLM-5 expands Agent and Coding SFT data, covering general chat, reasoning, and coding/agent tasks while extending context length to 202,752 tokens.Coding and Agent data emphasizes real-world, long-horizon execution environments; erroneous trajectory segments are retained but masked in the loss so the model can learn error correction without reinforcing incorrect actions.Source: §3.1 · pp. 10–11
• SFT supports interleaved, preserved, and turn-level thinking to improve action consistency while enabling reasoning to be retained or controlled across turns.Interleaved Thinking places thinking before responses and tool calls; Preserved Thinking reuses reasoning across coding-agent turns, while Turn-level Thinking can be disabled for lightweight requests or enabled for complex tasks.Source: §3.1 · p. 11

3.2 Reasoning RL

• Reasoning RL combines mathematics, science, code, and tool-integrated reasoning, using deterministic torch.topk and frozen indexer parameters to stabilize DSA training.The RL backbone builds on GRPO [40] and IcePop to address training-inference mismatch; non-deterministic top-k operators caused sharp entropy drops and performance degradation, while deterministic selection yielded substantial RL gains.Source: §3.2 · pp. 11–12

3.3 Agentic RL

• A decoupled asynchronous RL framework trains coding and search agents through a Multi-Task Rollout Orchestrator, avoiding GPU idle time from long-horizon rollouts.Token-in-Token-out preserves action-level correspondence, and Direct Double-sided Importance Sampling clips rollout log-probabilities to control off-policy bias; verifiable environments span over 10K software-engineering, terminal, and multi-hop search tasks.Source: §3.3 · pp. 12–13

3.4 General RL

• General RL optimizes foundational correctness, emotional intelligence, and task-specific quality using rule-based rewards, ORMs, and GRMs.Human-authored responses provide stylistic and qualitative anchors for more natural, human-aligned patterns; the hybrid reward system balances precision, efficiency, and robustness across complementary signals.Source: §3.4 · p. 13

3.5 On-Policy Cross-Stage Distillation

• On-policy cross-stage distillation uses preceding-stage checkpoints as teachers to recover skills from SFT, Reasoning RL, and General RL after sequential optimization.Prompts are sampled from the corresponding teachers’ RL training sets and mixed in appropriate proportions; the advantage is computed from the gap with teacher models.Source: §3.5 · pp. 13–14

3.6 RL Training Infrastructure: The slime Framework

• The slime framework unifies post-training with customizable rollouts and server-based execution, supporting diverse task logic and heartbeat-driven recovery from failed or degraded servers.External agent frameworks can call rollout servers and inference routers through HTTP APIs, while the optimization backend remains unchanged across short- and long-horizon training.Source: §3.6 · p. 14, §3.6.1 · p. 14, §3.6.3 · p. 15

3.6.2 Scaling Up: Tail-Latency Optimization for RL Rollouts

• RL rollout scheduling prioritizes end-to-end and tail latency; FP8, MTP, and Prefill–Decode disaggregation reduce slow-sample stalls and interference in multi-turn workloads.MTP particularly benefits small-batch stragglers, while dedicated prefill and decode resources keep long-horizon samples progressing without decode disruption.Source: §3.6.2 · pp. 14–15

4.1.1 Asynchronous RL Design for Agentic Training

• Asynchronous RL decouples inference and training to reduce GPU idle time from long-tail rollouts, batching generated trajectories at a threshold and periodically synchronizing weights to limit policy lag.A server-based orchestrator balances task rollout ratios and generation speeds across registered task services, supports over 1k concurrent rollouts, and standardizes trajectories; training uses group-wise policy optimization over K traces and optimizes model-generated tokens, not environment feedback.Source: §4.1.1 · pp. 15–16, §4.1 · p. 15

4.1.2 Optimizing Asynchronous Training Stability

• For asynchronous RL stability, GLM-5 preserves exact sampled tokens, reuses rollout log-probabilities as a behavior proxy, and masks tokens outside a double-sided clipping interval.The system rejects overly stale trajectories and environment-collapse failures, pads incomplete valid groups or drops them when too few samples remain, and routes each rollout to a stable DP rank to reuse KV cache across turns.Source: §4.1.2 · pp. 16–17

4.2.1 Software Engineering (SWE) Environments

• The SWE pipeline creates over 10k executable environments from real-world issues across nine programming languages, while terminal-task synthesis yields thousands of verifiable environments with Docker construction accuracy above 90%.The SWE setup analyzes repositories and generates test commands and log parsers; terminal tasks are drafted, implemented, and refined against rubrics. The pipelines use RepoLaunch [59] and Harbor, respectively.Source: §4.2.1 · p. 18, §4.2.2 · p. 18

4.2.3 Search Tasks

• Search-task synthesis builds challenging multi-hop questions from evidence across web sources, using a Web Knowledge Graph containing over two million high-information pages and filtering for difficulty, answer uniqueness, and evidence consistency.Questions that tool-free models solve in any of eight attempts or that early agents solve in a few steps are removed before bidirectional question-answer verification.Source: §4.2.3 · pp. 18–19

4.2.4 Inference with Context Management for Search Agents

• 75.9 was the final BrowseComp score for Keep-recent-k combined with Discard-all, outperforming all open-source models with context management and consistently beating Discard-all alone across compute budgets.Keep-recent-k alone improved GLM-5 from 55.3% without it to 62.0% with it, using k = 5.Source: Figure 8 · p. 19, §4.2.4 · p. 19

4.2.5 Slide Generation

• Slide generation uses multi-level rewards for static HTML attributes, runtime layout properties, and visual perception, then improves data and models through rejection sampling and masking-based refinement.The pipeline starts with SFT and RL; rejection sampling retains valid, high-quality outputs, while masking corrects defective pages without discarding good content from the same trajectory.Source: §4.2.5 · pp. 20–21
• Generated pages meeting the 16:9 aspect ratio rose from 40% to 92%; versus GLM-4.5, GLM-5 won 60% in content quality, 57.5% in layout rationality, 65% in visual aesthetics, and 67.5% overall.The results are reported as evidence for the multi-level reward design and self-improving framework.Source: §4.2.5 · p. 21

Mixed-Precision W4A8 quantization

• Mixed-precision W4A8 quantization fits the 750B-parameter GLM-5 onto one Atlas 800T A3, using W8A8 for Attention and MLP and W4A8 for MoE experts.QuaRot suppresses outliers, while Flex_AWQ_SSZ calibrates scaling to maintain stability in low-bit deployment.Source: §5 · p. 21

High-Performance fusion kernels

• Three Ascend fusion kernels target sparse-attention bottlenecks: Lightning Indexer combines scoring, ReLU, and TopK; Sparse Flash Attention selects KV-cache tokens while computing sparse attention; MLAPO fuses 13 preprocessing operators.Lightning Indexer overlaps computation with memory access, while MLAPO parallelizes Vector and Cube units.Source: §5 · pp. 21–22

Specialized inference engine optimizations

• Inference-engine adaptations combine asynchronous vLLM scheduling, KV-cache reuse, hybrid attention-data and MoE-expert parallelism, and FlashComm communication overlap.RadixCache and system-RAM Prefix Cache reuse KV entries for long contexts; Multi-Token Prediction increases NPU computation density and reduces sequence-generation time.Source: §5 · p. 22

• On a single Chinese node, GLM-5 achieves performance comparable to dual-GPU international clusters while cutting long-sequence deployment costs by 50%.Source: §5 · p. 22

6.1.1 Evaluation of Reasoning and General Benchmarks

• GLM-5 is comparable to Kimi-K2.5 on reasoning tasks, exceeds Claude Opus 4.5 and Gemini 3 Pro on HLE with tools and HMMT, and ranks second to Gemini 3 Pro on LongBench v2.It also significantly improves over GLM-4.7 on HLE with and without tools.Source: §6.1.1 · p. 22, Table 7 · p. 23

6.1 Evaluation of ARC Benchmarks

• GLM-5 significantly improves over GLM-4.7 and achieves state-of-the-art ARC performance among open-source models, narrowing the gap to proprietary models.The comparison covers agentic, reasoning, and coding benchmarks against open and proprietary models.Source: §6.1 · p. 22, Table 7 · p. 23

6.1.2 Evaluation of Coding Benchmarks

• GLM-5 achieves open-source coding-benchmark SOTA, beats Gemini 3 Pro on SWE-bench Verified and Multilingual, and is competitive with Claude Opus 4.5 on Terminal-Bench 2.0.Its Terminal-Bench performance improves further on the verified benchmark, and results are consistent across Terminus-2 and Claude Code.Source: §6.1.2 · p. 23, Table 7 · p. 23

6.1.3 Evaluation of Agentic Abilities

• GLM-5 improves significantly over GLM-4.7 on agentic benchmarks, leading frontier models on BrowseComp and outperforming Claude Opus 4.5 on GDPval-AA.It also matches Claude Opus 4.5 on three tool-use tasks and records $4,432 on Vending-Bench 2.Source: Table 7 · p. 23, §6.1.3 · p. 24

6.2.1 Frontend Evaluation – Agent-as-a-Judge

• In frontend evaluation, GLM-5 reaches 98.0% BSR and is competitive with Claude Opus 4.5 in CSR, but trails it in ISR across all three stacks.Agent-as-a-Judge first checks whether projects build, then uses GUI interactions to score specification check-items; its verdicts agree with human judgments on 94% of sampled items, with ranking correlation of 85.7%.Source: §6.2.1 · pp. 25–26, Table 8 · p. 25

6.2.2 Backend Evaluation; 6.2.3 Long-horizon Evaluation

• On backend tasks, GLM-5 and Claude Opus 4.5 perform comparably and both significantly outperform GLM-4.7; on chained tasks, GLM-5 improves substantially over GLM-4.7 but remains behind Claude Opus 4.5.Backend Pass@1 requires passing every task unit test; chained evaluation runs tasks sequentially and cumulatively tests for regressions.Source: §6.2.2 · p. 26, §6.2.3 · p. 27

6.2.3 Long-horizon Evaluation; 6.2.4 Evaluation on evolving SWE tasks

• GLM-5 outperforms Claude Opus 4.5 on large-repository exploration and generalizes effectively to fresh SWE-rebench issues.The repo task measures Pass@1 by whether the agent reads the target file, while SWE-rebench continuously mines new GitHub issue-fixing tasks.Source: §6.2.3 · p. 26, §6.2.4 · p. 27, Table 9 · p. 27

6.3 Evaluation of Real-world General Abilities

• GLM-5 consistently improves over GLM-4.7 across machine translation, multilingual dialogue, instruction following, world knowledge, and tool-calling.These real-world abilities are evaluated using a combination of internal and external human and automated assessments.Source: §6.3 · p. 28, Figure 11 · p. 28

• GLM-5 bridges high-performance reasoning and extreme computational efficiency as a next-generation foundation model.Source: §7 · p. 30
• GLM-5 demonstrates that open-weight models can rival top-tier proprietary systems in complex, real-world workflows.The report frames this as a shift from “vibe coding” to “agentic engineering.”Source: §7 · p. 30
• By open-sourcing GLM-5, the authors aim to empower community exploration of efficient agentic general intelligence, where agents plan, implement, and iterate on complex tasks.Source: §7 · p. 30

8 Easter Eggs

• The anonymous OpenRouter release of GLM-5 as “Pony Alpha” was a bold experiment intended to let the model’s capabilities speak without brand influence and gather unbiased feedback.Source: §8 · p. 30
• Developers praised Pony Alpha for complex coding, agentic workflows, and roleplay; preliminary guesses identified it as Claude Sonnet 5 (25%), DeepSeek (20%), or Grok (10%).The remaining guesses identified it as GLM-5.Source: §8 · p. 30
• Confirmation that Pony Alpha was GLM-5 silenced doubts about Chinese LLMs competing at the frontier and signaled a shift toward engineering-level reliability beyond raw benchmarks.Source: §8 · p. 30
• The anonymous release let the community embrace the model beyond geopolitical biases, while the authors say open-weight models are narrowing the proprietary-frontier gap but the race remains unfinished.They remain focused on scalable, efficient, and intelligent systems.Source: §8 · p. 30

A Hyper-Parameters

• Table 10 presents GLM-4.5 and GLM-5 architecture hyperparameters; its parameter counts include MTP layers but exclude word embeddings and the output layer.Source: A Hyper-Parameters · p. 36, Table 10 · p. 36
• GLM-5 training follows GLM-4.5 settings, including the Muon optimizer, cosine decay, and batch-size warmup, with pre-training learning rates warming from 0 to 2e-4 and decaying to 4e-5.Mid-training decreases linearly from 4e-5 to 1e-5; other hyperparameters match GLM-4.5.Source: A Hyper-Parameters · p. 36
• DSA warmup uses a learning rate decreasing from 5e-3 to 2e-4, followed by a constant 1e-5 rate during sparse adaptation.Source: A Hyper-Parameters · p. 36

B.1 Evaluation of Base Models

• GLM-5’s base model is evaluated on English, Chinese, code, and math benchmarks in Table 11.The supplied passage identifies the benchmark domains but provides no results or comparisons.Source: §B.1 · p. 36

B.2 Evaluation of ARC Benchmarks

• HLE and other reasoning tasks use up to 131,072 generation tokens, with text-only results reported by default and GPT-5.2 (medium) as judge.HLE-with-tools uses a maximum context length of 202,752 tokens; results marked with an asterisk are from the full set.Source: §B.2 · p. 36
• SWE-bench and SWE-bench Multilingual are evaluated with OpenHands, a tailored instruction prompt, and a 200K context window.The settings are temperature 0.7, top_p 0.95, and max_new_tokens 16,384.Source: §B.2 · p. 36
• BrowseComp retains the most recent five turns without context management and uses the discard-all strategy with context management.The context-management strategy matches DeepSeek-V3.2 and Kimi K2.5.Source: §B.2 · p. 36

B.3 Optimized User Simulator for τ 2-Bench

• Optimized prompts for Telecom and Retail aim to prevent failures caused by premature user termination.The optimized prompts are shown in Figures 12 and 13 and integrated into the system prompt.Source: §B.3 · p. 37, Figure 12 · p. 38, Figure 13 · p. 39
• The simulated user should provide one line at a time, reveal only information needed for the current step, avoid inventing details, and speak naturally in their instructed personality.When asked about unavailable information, the user should say they do not remember or have it and offer an alternative detail that is provided.Source: §B.3 · p. 38
• The user must follow only explicit constraints, never relax dates, budgets, or specific terms, and treat unspecified attributes as flexible unless the instruction says otherwise.For Retail scenarios, the prompt focuses on product attributes and the specified exchange or return process.Source: §B.3 · p. 38

B.4.1 Frontend Evaluation

• The dataset covers seven scenarios: business management systems, web games, SVG/canvas rendering, creative tools and editors, showcase pages, forms and tables, and data visualization.Source: §B.4.1 · p. 39
• It spans three coding paradigms: the vanilla HTML/CSS/JS web stack, React component-based development, and Vue 3 with Vite.Source: §B.4.1 · p. 39
• Each test case combines a task, a checklist, and a dedicated environment; a representative task asks for a React drawing tool with a brush, eraser, canvas, and save button.Source: §B.4.1 · p. 40
• The drawing-tool checklist requires selectable brush color and thickness and eraser size, mouse-drag drawing and erasing, and saving the image locally.Source: §B.4.1 · p. 40
• The final benchmark contains 220 high-quality frontend coding tasks with corresponding checklists, after expert-led synthesis, checklist refinement, execution-based correction, and iterative removal of trivial tasks.Candidate checklists are initially generated with Claude Sonnet 4.5, then audited by experts; judgments are cross-validated between an Agent-as-a-Judge framework and human experts.Source: §B.4.1 · p. 40
• Expert refinement makes checklist items unambiguous, objective, and comprehensive, while discrepancies between automated and human judgments trigger data re-evaluation and correction.Source: §B.4.1 · p. 40