PaperlayerView exact revision on arXiv

Source-linked AI summary

Pangu Embedded: An Efficient Dual-system LLM Reasoner with Metacognition

Hanting Chen, Yasheng Wang, Kai Han, Dong Li, Lin Li, Zhenni Bi, Jinpeng Li, Haoyu Wang, Fei Mi, Mingjian Zhu, Bin Wang, Kaikai Song, Yifei Fu, Xu He, Yu Luo, Chong Zhu, Quan He, Xueyu Wu, Wei He, Hailin Hu, Yehui Tang, Dacheng Tao, Xinghao Chen, Yunhe Wang

arXiv:2505.22375v2cs.CL

TL;DR

Reasoning-optimized LLMs can be costly and slow because of large models and lengthy thinking processes. Pangu Embedded addresses this with a two-stage training framework combining iterative distillation, Ascend-NPU reinforcement learning, and dual-system fast-slow thinking. The 7B model achieves state-of-the-art reasoning accuracy among similarly sized models while supporting manual and adaptive mode selection.

  • Problem

    Existing LLM reasoners face prohibitive computational costs and latency from excessive model sizes and lengthy thinking processes, limiting time-sensitive deployment.

  • Method

    Pangu Embedded combines iterative distillation with model merging, Ascend-cluster reinforcement learning, and a dual-system framework for manual or complexity-aware fast-slow mode selection.

  • Results

    Pangu Embedded achieves state-of-the-art reasoning accuracy among similarly sized models, with the 7B model outperforming Qwen3-8B.

  • Takeaways & Limitations

    The framework provides a computationally sustainable approach to reasoning models by dynamically balancing reasoning depth with computational efficiency.

  • Takeaways & Limitations

    The observed advantage over Qwen3-8B may be partially attributable to differences in general pre-training or initial fine-tuning data mixtures.

Abstract

from arXiv · show

This work presents Pangu Embedded, an efficient Large Language Model (LLM) reasoner developed on Ascend Neural Processing Units (NPUs), featuring flexible fast and slow thinking capabilities. Pangu Embedded addresses the significant computational costs and inference latency challenges prevalent in existing reasoning-optimized LLMs. We propose a two-stage training framework for its construction. In Stage 1, the model is finetuned via an iterative distillation process, incorporating inter-iteration model merging to effectively aggregate complementary knowledge. This is followed by reinforcement learning on Ascend clusters, optimized by a latency-tolerant scheduler that combines stale synchronous parallelism with prioritized data queues. The RL process is guided by a Multi-source Adaptive Reward System (MARS), which generates dynamic, task-specific reward signals using deterministic metrics and lightweight LLM evaluators for mathematics, coding, and general problem-solving tasks. Stage 2 introduces a dual-system framework, endowing Pangu Embedded with a "fast" mode for routine queries and a deeper "slow" mode for complex inference. This framework offers both manual mode switching for user control and an automatic, complexity-aware mode selection mechanism that dynamically allocates computational resources to balance latency and reasoning depth. Experimental results on benchmarks including AIME 2024, GPQA, and LiveCodeBench demonstrate that Pangu Embedded with 7B parameters, outperforms similar-size models like Qwen3-8B and GLM4-9B. It delivers rapid responses and state-of-the-art reasoning quality within a single, unified model architecture, highlighting a promising direction for developing powerful yet practically deployable LLM reasoners.

1 Introduction

Pangu Embedded addresses the computational and latency costs of reasoning-optimized LLMs with a two-stage framework that combines efficient reasoner construction with flexible fast and slow thinking.

  • Motivation: Reasoning-optimized LLMs face prohibitive deployment costs from excessive model size and lengthy thinking processes.DeepSeek-R1 is cited at 671B parameters, while some complex problems generate over 32,000 thinking tokens.
  • Motivation: Existing efficiency efforts include shortening chain-of-thought, distillation, reinforcement learning, and efficient sampling.
  • Approach: Pangu Embedded uses Stage 1 to construct a capable base reasoner through iterative distillation, model merging, and reinforcement learning on Ascend NPU clusters.The RL phase uses a latency-tolerant scheduler and the Multi-source Adaptive Reward System.
  • Approach: Stage 2 adds fast thinking for routine queries and slow thinking for complex inference within one model, with manual and automatic mode selection.
  • Results: The 7B-parameter Pangu Embedded outperforms similarly sized Qwen3-8B while delivering rapid responses and state-of-the-art reasoning quality.

2 Basic Reasoner Construction

Stage 1 builds Pangu Embedded’s foundational reasoner through data preparation, iterative distillation, and Ascend-NPU-oriented reinforcement learning.

  • 2 Basic Reasoner Construction: Stage 1 constructs a robust foundational reasoner using meticulous data preparation, iterative distillation, and scaled reinforcement learning tailored for Ascend NPUs.

2.1 Preliminaries of Base Model

Pangu Embedded is a 7B language model optimized for edge computing, using Pangu Ultra’s pre-training data and tokenizer foundations.

  • 2.1 Preliminaries of Base Model: Pangu Embedded is a 7-billion-parameter language model specifically optimized for edge computing scenarios.
  • 2.1 Preliminaries of Base Model: Its tokenizer uses a unified vocabulary of 153,376 unique tokens to represent diverse domains while maintaining compression efficiency.
  • 2.1 Preliminaries of Base Model: The pre-training corpus is processed through general, reasoning, and annealing phases.

2.2 Post-training Data

The post-training data pipeline builds a high-quality, diverse pool from varied reasoning and non-reasoning sources, then filters and deduplicates it for RL and distillation.

  • 2.2 Post-training Data: The initial data pool combines general QA, AIGC, text analysis, coding, mathematics, logical reasoning, and tool-use tasks across finance, healthcare, and government domains.Sources include open instructions, industrial queries, and problems synthesized from pre-training corpora.
  • 2.2 Post-training Data: Prior filtering annotates subcategory, question type, answer verifiability, and multidimensional difficulty before removing unqualified prompts.The filtering aims to mitigate reward hacking and reduce undesirable data issues.
  • 2.2 Post-training Data: MinHash-LSH removes near-duplicate samples, while ZIP selects the final set using instruction diversity measured through compression ratios.
  • 2.2 Post-training Data: The resulting initial data pool D serves as a source for both reinforcement learning and distillation.

2.3 SFT Strategy: Model-aware Iterative Distillation

Pangu Embedded’s SFT strategy aligns training difficulty with the evolving student model through iterative, model-aware distillation. Inter-iteration merging consolidates improvements while reducing risks from dynamically changing data distributions.

  • Motivation: 50.42% was achieved by the balanced AIME training-data distribution, exceeding mostly easy data at 45.42% and mostly hard data at 48.75%.The full dataset performed worst, indicating that indiscriminate data selection can hinder reasoning-pattern learning.
  • Model-aware Iterative Distillation: The pipeline dynamically selects samples whose complexity matches the student model’s current capabilities during successive distillation iterations.The previous iteration’s student evaluates sample difficulty, and selected data guides further refinement.
  • Model-aware Iterative Distillation: Model-aware complexity is estimated from the student’s response consistency, with lower scores indicating competence and higher scores indicating difficulty.For non-reasoning data, complexity can instead be estimated from loss values.
  • Model-aware Iterative Distillation: Each iteration uses the previous model as its base, adaptively selects distilled data, and begins with few-shot prompting only for the initial pretrained model.Subsequent iterations remove the initial few-shot examples as the model becomes instruction-tuned.
  • Inter-iteration Model Merging: Inter-iteration model merging applies successive parameter deltas to preserve earlier knowledge, accumulate improvements, and mitigate catastrophic forgetting.The strategy addresses risks including reduced domain coverage and forgetting caused by evolving SFT data distributions.

2.4 Enhancing Generation Quality: Repetition Self-repair

Pangu Embedded mitigates repetitive long-form generation through local n-gram detection and prompt-controlled self-repair. The mechanism reduces detection overhead and steers generation away from detected loops.

  • Local n-gram Detection: Local-window detection reduces repetition-checking complexity from O(N^2) toward O(N·W), or O(N) with efficient implementation.Periodic checks every tdetect decoded tokens further reduce latency impact.
  • Prompt-control Repetition Suppression: When local n-gram similarity exceeds a predefined threshold, the system injects a control prompt that asks the model to revise repetitive output.The detection window is reset or restarted after the intervention.
  • Repetition Self-repair: The repetition self-repair strategy combines local n-gram detection with prompt-controlled suppression.Detection identifies repetition, while an injected control prompt redirects subsequent generation.
  • Prompt-control Repetition Suppression: The self-repair mechanism produces a higher-quality starting point for the subsequent reinforcement-learning phase.It is intended to improve coherence and output diversity after repetitive patterns are detected.

2.5 Scaling Reinforcement Learning on Ascend Clusters

This section describes the reinforcement-learning methodology for Pangu Embedded, combining GRPO optimization, task-specific rewards, curriculum data mixing, and cold-start preparation.

  • Policy Optimization with GRPO: GRPO optimizes the policy using grouped response rewards, a reference policy, clipping, and a KL-divergence penalty.The estimated advantage is computed from group scores, with the reference policy constraining deviation from the initial SFT model.
  • Policy Optimization with GRPO: Zero-Advantage-Mask skips samples whose advantage is zero, preventing them from contributing to PPO-clip or KL-divergence loss terms.This focuses updates on samples with informative advantage signals and preserves exploration when group rewards are identical.
  • Multi-Source Adaptive Reward System (MARS): MARS dynamically routes prompts and responses to task-appropriate evaluators, combining correctness, preference, formatting, and repetition-related rewards.Mathematics uses rule-based and LLM verification, while coding rewards use extraction, syntax checks, execution, and output comparison.
  • Curriculum Data Mixing: Curriculum mixing interleaves queries of varying complexity because extremely easy or difficult samples often produce constant rewards and limited learning signals.Complexity is assessed from the current policy’s pass rate, then used to curate a progressively balanced training mixture.
  • Cold Start: The cold-start SFT model provides instruction following, self-evaluation, error detection, reflection, and iterative correction capabilities for subsequent RL.These capabilities are described as foundational for stabilizing later reinforcement-learning optimization.

2.6 RL Infrastructure on Ascend Clusters

This section presents an Ascend-NPU RL infrastructure that co-schedules training and inference and uses latency-tolerant execution to improve large-scale pipeline efficiency.

  • Infrastructure Overview: The Ascend RL pipeline integrates policy and reference models, MARS, an Ascend-optimized vLLM inferencer, weight reshuffling, and latency-tolerant scheduling.Training and inference are co-located to support scalable reinforcement learning on Ascend clusters.
  • Latency-Tolerant Scheduling: The SSP scheduler combines controlled staleness with distributed prioritized data queues to coordinate parallel RL processing stages.The stages include reference assessment, reward scoring, log-probability extraction, and parameter updates.
  • Training-Inference Co-Scheduling: Training and inference share Ascend hardware through host-device adaptive weight reshuffling and zero-copy parameter transfers where feasible.The deployment uses large-scale tensor, pipeline, and data parallelism for concurrent workloads.
  • vLLM Inference Optimization: Approximately 2x reduction in straggler-induced delays is achieved versus static partitioning while maintaining high throughput during batched decoding.The optimization combines a global request queue, dynamic device scheduling, and proactive prefetching for heterogeneous sequence lengths.
  • Computing Cluster Configuration: The policy training and inference deployment uses 1,024 Ascend NPUs, with another 256 allocated for the reference model.SSP scheduling and parts of reward processing run on host CPUs using Kunpeng NUMA topology.

3 Fast and Slow Thinking: A Dual System Cognitive Architecture for LLMs

Stage 2 equips Pangu Embedded with fast and slow reasoning modes in one model, supporting explicit user control and automatic complexity-aware switching.

  • Dual-System Framework: System 1 generates direct, efficient responses, whereas System 2 performs slower, deliberative reasoning within the dual-system framework.The framework is designed to provide distinct reasoning depths in a unified architecture.
  • Training for Manual Mode Switching: Manual switching lets users select fast or slow thinking through directives such as system prompts or special instruction prefixes.A dedicated fine-tuning dataset pairs both response styles with distinct mode instructions.
  • Training for Manual Mode Switching: Fusion training replays slow-thinking exemplars while adding fast-thinking data, preserving slow reasoning while introducing efficient direct responses.The sequencing addresses interference between the two operational modes in compact models.
  • Adaptive Mode Selection: Adaptive switching selects System 1 or System 2 according to the model’s assessment of query complexity.The mechanism aims to optimize computational efficiency while maintaining performance across tasks with different difficulty levels.
  • Adaptive Mode Selection: Dfusion teaches mode selection using concise fast-mode outputs and reasoning-intensive slow-mode responses, with mathematical queries classified by computation and thinking complexity.Complexity scores use 1-to-5 ratings for computational load and reasoning-step requirements.
  • Natural-Language Controllability: Natural-language control enables users to request a concise answer after receiving a detailed solution, prompting a switch from System 2 to System 1.The example demonstrates soft control over reasoning style through ordinary interaction.

4 Experiments

Experiments show that Pangu Embedded combines competitive dual-mode performance with controllable reasoning depth and substantial token savings through adaptive switching. Ablations further indicate that balanced distillation data, iterative training, and carefully tuned RL settings are important for stable capability development.

  • RL Strategy: More than twice-per-step updates caused rapid reward decreases and less coherent outputs, while excessive or absent KL regularization hindered stable learning.High sampling temperatures also produced low initial format rewards and later readability problems, motivating careful RL hyperparameter selection.
  • Manual Switching: Pangu Embedded achieves leading or highly competitive performance across reasoning benchmarks in both Thinking (system2) and Nothinking (system1) modes.The unified model preserves strong deliberative reasoning while remaining effective in its efficiency-oriented mode.
  • Response Length: Slow-thinking modes generate significantly longer responses than fast-thinking modes across all evaluated benchmarks.Pangu Embedded’s System 1 and System 2 sequence lengths generally mirror those of Fast Only and Slow Only specialists, respectively.
  • Adaptive Thinking: 11% lower average token usage on MATH500 and 88% lower usage on GSM8K were observed with adaptive thinking while accuracy remained nearly unchanged.The larger reduction on GSM8K is consistent with faster, more direct responses being sufficient for simpler tasks.
  • Adaptive Thinking: Adaptive mode activation increases monotonically with problem difficulty, favoring concise reasoning for easier problems and slower reasoning for harder ones.This pattern indicates complexity-aware allocation of reasoning resources while balancing efficiency and reasoning accuracy.

5 Related Work

Related work spans reasoning models, RL infrastructure, knowledge distillation, model-aware training, and adaptive fast–slow thinking. The paper emphasizes unresolved deployment and Ascend-cluster optimization needs.

  • Reasoning Models: Reasoning models use extended chains of thought and reinforcement learning to improve mathematical, logical, and algorithmic problem-solving.
  • RL Infrastructure for LLMs: Most existing RL infrastructures target GPU clusters, leaving Ascend-based environments in need of specialized optimization.
  • Knowledge Distillation: Knowledge distillation transfers capabilities from larger or stronger teacher models to students, with output-distribution alignment requiring shared vocabularies.
  • Model-aware Training: Model-aware training accounts for model-specific characteristics and the difficulty of adapting data distributions that diverge from base-model representations.
  • Fast and Slow Thinking: Fast–slow thinking research dynamically adjusts reasoning depth according to input-question characteristics to balance reasoning quality and efficiency.

6 Conclusion and Discussion

The conclusion presents Pangu Embedded as a compact Ascend-NPU reasoner built through two-stage training and equipped with adaptive fast–slow thinking. It highlights data-complexity scheduling and compact deployment as broader lessons.

  • Pangu Embedded combines iterative distillation, model-aware data selection, checkpoint merging, Ascend-optimized reinforcement learning, and a dual-system fast–slow framework.
  • The model supports manual mode switching and adaptive mode selection to balance reasoning depth with computational efficiency.
  • SFT-based distillation benefits from medium, simpler data, whereas RL-based refinement uses broader, higher-complexity data.
  • The work identifies hybrid SFT–RL frameworks with model-aware complexity scheduling as a future direction for knowledge transfer and policy refinement.

A Contributions and Acknowledgments

The contribution and acknowledgment section lists the paper’s core contributors and additional contributors. It contains attribution information rather than technical findings.

  • Core Contributors: The core-contributor list includes Hanting Chen, Yasheng Wang, Kai Han, Dong Li, Lin Li, and other named researchers.
  • Contributors: The contributor list recognizes Binwei Yan, Can Chen, Chan Tsz Ho, Chen Zhong, Chenyi Pan, and many other named researchers.

B Response Examples

The response examples show a geometry problem, repetition self-repair on a mathematics task, and legal reasoning outputs. They also contrast correct and incorrect reasoning across models under Chinese law.

  • Response Examples: The geometry example asks for the number of integer-coordinate points strictly inside a rectangle with vertices (5, 4), (-5, 4), (-5, -4), and (5, -4).
  • Response Examples: Repetition Self-repair detects repetitive sentences, inserts a guiding prompt, and leads the model to reflect on and correct the repetition.
  • Response Examples: The legal example requires selecting one answer and returning it in the exact specified format between [正确答案] and <eoa>.
  • Response Examples: The China-law comparison labels incorrect reasoning red and correct reasoning green across different reasoning models.
Loading 2505.22375v2…