Hierarchical Reasoning Model

Guan Wang; Jin Li; Yuhao Sun; Xing Chen; Changling Liu; Yue Wu; Meng Lu; Sen Song; Yasin Abbasi Yadkori

Source-linked AI summary

Hierarchical Reasoning Model

Guan Wang, Jin Li, Yuhao Sun, Xing Chen, Changling Liu, Yue Wu, Meng Lu, Sen Song, Yasin Abbasi Yadkori

arXiv:2506.21734v3Jun 26, 2025cs.AIcs.LG

TL;DR

AI reasoning remains limited by shallow architectures and brittle, data-intensive Chain-of-Thought decomposition. The paper proposes HRM, a hierarchical recurrent model with slow planning and fast computation, and reports strong results using few examples without pre-training or CoT supervision. HRM achieves near-perfect performance on difficult Sudoku and maze tasks and outperforms larger CoT-based models on ARC.

Problem
Shallow Transformers and brittle Chain-of-Thought decomposition limit efficient complex reasoning, requiring substantial data and long token sequences.
Method
HRM uses coupled high-level and low-level recurrent modules with hierarchical convergence and one-step gradient training to increase effective depth without BPTT.
Results
With 27M parameters and 1,000 examples, HRM achieves near-perfect Sudoku and maze performance and 40.3% on ARC, exceeding reported CoT-model results.
Takeaways & Limitations
Hierarchical multi-timescale recurrence is presented as a viable alternative to dominant Chain-of-Thought reasoning for challenging long-horizon tasks.
Takeaways & Limitations
Evidence that the emergent hierarchy is necessary remains correlational, and establishing causal necessity is left for future investigation.

Abstract

from arXiv · show

Reasoning, the process of devising and executing complex goal-oriented action sequences, remains a critical challenge in AI. Current large language models (LLMs) primarily employ Chain-of-Thought (CoT) techniques, which suffer from brittle task decomposition, extensive data requirements, and high latency. Inspired by the hierarchical and multi-timescale processing in the human brain, we propose the Hierarchical Reasoning Model (HRM), a novel recurrent architecture that attains significant computational depth while maintaining both training stability and efficiency. HRM executes sequential reasoning tasks in a single forward pass without explicit supervision of the intermediate process, through two interdependent recurrent modules: a high-level module responsible for slow, abstract planning, and a low-level module handling rapid, detailed computations. With only 27 million parameters, HRM achieves exceptional performance on complex reasoning tasks using only 1000 training samples. The model operates without pre-training or CoT data, yet achieves nearly perfect performance on challenging tasks including complex Sudoku puzzles and optimal path finding in large mazes. Furthermore, HRM outperforms much larger models with significantly longer context windows on the Abstraction and Reasoning Corpus (ARC), a key benchmark for measuring artificial general intelligence capabilities. These results underscore HRM's potential as a transformative advancement toward universal computation and general-purpose reasoning systems.

1 Introduction

The paper identifies limits in shallow Transformer and Chain-of-Thought reasoning, then proposes HRM, a hierarchical recurrent architecture designed to increase effective computational depth efficiently. HRM combines slow abstract planning with fast detailed computation and achieves strong performance on demanding search and reasoning tasks.

Motivation: Standard Transformers’ fixed or ineffective depth limits complex algorithmic reasoning, while simply increasing width provides no benefit on Sudoku-Extreme Full.Even very deep Transformers remain far from optimal on Sudoku, whereas HRM uses computational depth to achieve near-perfect accuracy.
Motivation: Chain-of-Thought reasoning uses brittle human-defined intermediate steps, often requiring substantial training data and generating many tokens for complex tasks.A single misstep or misordering can derail the reasoning process, motivating a more efficient approach.
Motivation: Latent reasoning performs computation in hidden-state space, but its power remains constrained by effective depth and training difficulties in naively stacked or recurrent architectures.Vanishing gradients, early recurrent convergence, and BPTT costs hinder stable long-horizon computation.
Approach: HRM uses two coupled recurrent modules: a high-level module for abstract deliberate reasoning and a low-level module for fast detailed computation.The architecture is inspired by hierarchical, multi-timescale processing in the brain, where slower higher-level areas guide faster lower-level processing.
Approach: Hierarchical convergence lets the slow module update after multiple fast-module steps reach local equilibrium, while a one-step gradient approximation avoids BPTT and keeps memory at O(1) rather than O(T).The low-level module is reset for each new computational phase.
Results: Using 1,000 examples without pre-training or CoT supervision, HRM achieves near-perfect Sudoku and maze performance and reaches 40.3% on ARC with 27M parameters.On ARC, this surpasses o3-mini-high at 34.5% and Claude 3.7 with 8K context at 21.2%.

2 Hierarchical Reasoning Model

HRM uses hierarchical recurrent computation, with a slow high-level module coordinating rapid low-level updates across nested cycles. It combines hierarchical convergence, one-step gradient approximation, adaptive halting, and inference-time scaling to support deeper and more efficient reasoning.

Architecture: HRM comprises input, low-level recurrent, high-level recurrent, and output components arranged into N high-level cycles of T low-level timesteps.The model indexes one forward pass over N × T total timesteps, with separate hidden states for the recurrent modules.
Hierarchical processing: The L-module updates at every timestep while the H-module remains fixed within a cycle and updates once using the L-module’s final state.This coupling lets high-level context condition low-level computation before the next high-level update.
Hierarchical convergence: Hierarchical convergence yields nested computations with an effective depth of NT steps, while standard RNN activity rapidly decays after early convergence.The L-module converges within each cycle, then the H-module establishes fresh context for subsequent low-level computation.
Approximate gradient: The one-step gradient approximation avoids BPTT and maintains O(1) memory instead of O(T) memory for T timesteps.It uses the gradient of each module’s last state while treating other states as constant, and is grounded in fixed-point gradient reasoning.
Adaptive computation: Adaptive halting selects halt or continue using Q-values and minimum and maximum segment thresholds, allowing computation to vary across inputs.The halt action requires either exceeding Mmax or favoring halt after reaching Mmin.
Inference-time scaling: HRM supports inference-time scaling by increasing Mmax without retraining or architectural changes, with stronger gains on Sudoku than on ARC-AGI.The paper attributes the difference to Sudoku requiring longer-term planning, whereas ARC-AGI solutions generally require only a few transformations.

3 Results

The evaluation spans inductive reasoning, logical puzzles, and optimal pathfinding, with HRM trained in small-sample settings without pretraining or CoT labels. Results compare HRM with Transformer and CoT-based baselines while also examining its intermediate reasoning trajectories.

Benchmarks: ARC-AGI tasks require extracting abstract rules from a few input–output demonstrations and generalizing them to a test input.ARC-AGI-2 further emphasizes compositional reasoning, multi-step logic, contextual rule application, and symbolic abstraction.
Benchmarks: Sudoku-Extreme evaluates exact solutions to 9×9 puzzles whose rows, columns, and 3×3 blocks contain digits 1–9 exactly once.The dataset combines easy sources with puzzles recognized as exceptionally difficult for human players.
Benchmarks: Maze-Hard evaluates valid and optimal paths through 30×30 mazes, retaining instances whose shortest-path difficulty exceeds 110.Both training and test sets contain 1000 examples.
Evaluation setup: HRM attains its reported results with ~1000 training examples per task, random initialization, and no pretraining or CoT labels.The two-dimensional grids are flattened and padded in a sequence-to-sequence setup; Sudoku uses augmentation, while Maze uses a single inference pass without augmentation.
Results: On ARC-AGI-1, replacing the Transformer with HRM’s hierarchical framework and implementing ACT produces more than a twofold performance improvement.The direct-prediction baseline uses the same training setup as HRM but swaps in a Transformer architecture.
Results: On Sudoku-Extreme and Maze-Hard, the 8-layer Transformer baseline fails with 1000 examples, while a 175M-parameter Transformer trained on 1 million maze examples remains below 20% pass@64 accuracy.The comparison highlights the difficulty of these tasks for baselines requiring lengthy reasoning traces.

4 Brain Correspondence

HRM develops a dimensionality hierarchy that parallels hierarchical organization observed in the mouse cortex. Training expands the high-level module’s representational capacity while leaving the low-level module more compact, although the causal necessity of this hierarchy remains unresolved.

Dimensionality measure: The Participation Ratio (PR) measures the effective dimensionality of neural representations from covariance-matrix eigenvalues.Higher PR indicates variance distributed across more dimensions; lower PR indicates a more compact representation.
Biological correspondence: Mouse cortical areas show higher PR at higher hierarchical positions, linking representational dimensionality with functional complexity.The reported correlation is Spearman ρ = 0.79 (P = 0.0003).
Trained HRM: After training, HRM’s high-level module has PR = 89.95 versus PR = 30.22 for its low-level module during 100 Sudoku-solving trajectories.Increasing task diversity expands zH dimensionality while zL dimensionality remains stable.
Training control: An identical randomly initialized HRM shows low, nearly indistinguishable PR values in its two modules, unlike the trained model.This control supports the interpretation that the dimensionality hierarchy emerges through training rather than from architecture alone.
Cross-system comparison: The trained HRM’s high-to-low PR ratio is approximately 2.98, compared with approximately 2.25 in mouse cortex, while conventional deep networks often exhibit neural collapse.The comparison is presented as a departure from low-dimensional last-layer representations.
Limitation: The observed representational hierarchy is correlational, so its causal necessity for HRM’s reasoning performance remains an open question.The paper notes that intervention tests, such as constraining H-module dimensionality, may be difficult to interpret because of training confounds.

5 Related Work

HRM extends prior neural algorithm-learning and brain-inspired architectures with a brain-inspired, hierarchical recurrent design. Unlike approaches limited to simpler tasks or hand-designed procedures, it is presented as learning complex algorithms from few examples, while hierarchical memory remains a future direction.

Algorithm learning: Earlier algorithm-learning systems such as NTM, DNC, and Neural GPUs use iterative neural architectures trained to execute learned algorithms.These approaches explicitly mimic computational hardware for algorithm execution.
Recurrent Transformers: Universal Transformers and related work add recurrent loops, adaptive halting, or recurrent reasoning tokens to Transformer-based architectures.These methods explore additional inference steps and continuous latent computation.
CoT and search supervision: Some CoT-based methods fine-tune models on reasoning paths generated by search algorithms such as A*.These approaches use search-derived trajectories as supervised fine-tuning targets.
HRM contribution: HRM is presented as extending algorithm learning through a brain-inspired architecture that discovers complex and diverse algorithms from 1000 training examples.The claim emphasizes data efficiency and model expressiveness.
Brain-inspired architectures: Prior brain-inspired models such as Spaun and TEM use specialized modules or structural representations, but some rely on hand-designed algorithms or rules for task solving.The cited comparison frames HRM as targeting more complex learned reasoning tasks.
Hierarchical memory: Hierarchical Sequential Models and Clockwork RNN use multiple timescales to capture long-range dependencies and mitigate forgetting in recurrent processing.The discussion distinguishes hierarchical memory from HRM’s primary focus on reasoning.
Future direction: HRM currently uses full attention for simplicity, leaving integration of hierarchical memory for long-context processing as a future direction.The paper identifies this as a promising extension rather than a demonstrated capability.

6 Discussions

The discussion situates HRM among recurrent and universal-computation approaches. It distinguishes HRM’s computational-universality claim from CoT and linear-attention methods, which address reasoning or context efficiency but retain important limitations.

Turing completeness: HRM is described as computationally universal under sufficient memory and time, meaning it can simulate any Turing machine.The discussion positions this as overcoming fixed-depth Transformer limitations.
CoT reinforcement learning: Reinforcement learning with CoT is reported to mainly unlock existing CoT-like capabilities rather than discover fundamentally new reasoning mechanisms.The cited discussion also characterizes CoT-based RL as unstable and data-inefficient.
Linear attention: Recurrent alternatives to self-attention can reduce quadratic time and memory costs by processing tokens sequentially.This efficiency motivation concerns attention replacement rather than computational depth alone.
Context efficiency: Some linear-attention variants propagate multi-timescale summary statistics to retain long-range context with reduced memory growth.The discussion notes that replacing attention alone does not remove fixed-depth limitations or the need for CoT.

7 Conclusion

The paper presents HRM as a brain-inspired architecture that combines hierarchy and multiple timescales to obtain substantial computational depth efficiently. With 27M parameters and 1000 training examples, it solves challenging reasoning tasks and is framed as an alternative to dominant CoT approaches.

Conclusion: HRM combines hierarchical structure and multi-timescale processing to achieve substantial computational depth while maintaining training stability and efficiency.The architecture is presented as brain-inspired.
Conclusion: 27M parameters and 1000 training examples suffice for HRM to solve challenging ARC, Sudoku, and complex maze-navigation tasks.These tasks are described as difficult for contemporary LLM and CoT models.
Conclusion: The authors frame HRM as a viable alternative to current CoT reasoning methods and as progress toward Turing-complete universal computation.This conclusion is stated as a suggested direction rather than a demonstrated universal system.