PaperlayerView exact revision on arXiv

Source-linked AI summary

Generative Recursive Reasoning

Junyeob Baek, Mingyu Jo, Minsu Kim, Mengye Ren, Yoshua Bengio, Sungjin Ahn

arXiv:2605.19376v2cs.AI

TL;DR

Existing recursive reasoning models refine a single latent trajectory, limiting their ability to represent uncertainty and alternative solutions. GRAM makes recursive reasoning stochastic and multi-trajectory, and reports improvements over deterministic baselines on structured reasoning, multi-solution constraint satisfaction, and unconditional generation.

  • Problem

    Existing recursive reasoning models typically follow a single deterministic latent trajectory, limiting support for uncertainty, alternative hypotheses, and multiple solution strategies.

  • Method

    GRAM models reasoning as stochastic latent trajectories, enabling probabilistic multi-trajectory computation and inference-time scaling through recursive depth and parallel sampling.

  • Results

    GRAM improves structured reasoning, multi-solution constraint satisfaction, and unconditional generation over deterministic recurrent and recursive baselines.

  • Takeaways & Limitations

    Probabilistic multi-trajectory recursion provides a design principle for extending recurrent and recursive reasoning beyond depth-only computation.

  • Takeaways & Limitations

    Sequential deep supervision limits training efficiency compared with Transformers, creating a barrier to scaling GRAM toward larger foundation models.

Abstract

from arXiv · show

How should future neural reasoning systems implement extended computation? Recursive Reasoning Models (RRMs) offer a promising alternative to autoregressive sequence extension by performing iterative latent-state refinement with shared transition functions. Yet existing RRMs are largely deterministic, following a single latent trajectory and converging to a single prediction. We introduce Generative Recursive reAsoning Models (GRAM), a framework that turns recursive latent reasoning into probabilistic multi-trajectory computation. GRAM models reasoning as a stochastic latent trajectory, enabling multiple hypotheses, alternative solution strategies, and inference-time scaling through both recursive depth and parallel trajectory sampling. This yields a latent-variable generative model supporting conditional reasoning via $p_θ(y \mid x)$ and, with fixed or absent inputs, unconditional generation via $p_θ(x)$. Trained with amortized variational inference, GRAM improves over deterministic recurrent and recursive baselines on structured reasoning and multi-solution constraint satisfaction tasks, while demonstrating an unconditional generation capability. https://ahn-ml.github.io/gram-website

1 Introduction

The introduction argues that recursive latent refinement can increase reasoning depth, but deterministic RRMs cannot maintain uncertainty or explore alternative solution paths. GRAM addresses this gap by modeling recursive reasoning as stochastic multi-trajectory computation with inference-time scaling across sampled trajectories and depth.

  • Motivation: Recursive latent-state refinement offers an alternative to extending autoregressive sequences for implementing extended computation in neural reasoning systems.RRMs repeatedly refine a persistent latent state, while recurrent Transformer designs increase computational depth through shared blocks without increasing parameter count.
  • Motivation: Depth alone is insufficient when reasoning requires uncertainty, alternative hypotheses, or multiple valid solution strategies.These capabilities matter especially for ambiguous problems and tasks with intrinsically multiple solutions.
  • Problem: Existing RRMs are deterministic, following one latent trajectory from the same input and initialization and converging to a single prediction.This collapses plausible reasoning paths into a single attractor and leaves probabilistic multi-hypothesis latent reasoning unexplored.
  • GRAM: GRAM turns recursive latent reasoning into probabilistic multi-trajectory computation by sampling each transition conditioned on the input and current reasoning state.Repeated stochastic transitions define a distribution over possible reasoning trajectories.
  • Contributions: GRAM formulates reasoning as a latent-variable generative process and introduces width-based inference-time scaling over sampled trajectories in addition to recursive depth.The framework is evaluated on controlled reasoning and generation tasks targeting recursive refinement, stochastic exploration, multi-solution coverage, and inference-time scaling.

2 Generative Recursive Reasoning Models

GRAM turns recursive reasoning into probabilistic latent computation by sampling stochastic trajectories within nested inner and outer loops, enabling multiple reasoning paths rather than a single deterministic trajectory. It is trained with amortized variational inference and supports both conditional and unconditional generation.

  • Overview: GRAM models p_θ(y | x) by marginalizing over stochastic latent reasoning trajectories generated from a fixed initial state and shared input embedding.The latent state evolves through learned stochastic transitions across the recursive computation.
  • Overview: Inner transitions sample z_t from p_θ(z_t | z_{t−1}, e_x), and the decoder produces a prediction after T transitions, defining one supervision step.The decoder is invoked and the training objective is applied at the end of each supervision step.
  • Overview: N_sup supervision steps form the outer recursion by using each step’s terminal state as the next step’s initial state.This nested structure can use flat or hierarchical recurrent Transformer backbones.
  • Stochastic Latent Transitions: Unlike deterministic RRMs, GRAM adds learned stochastic residual guidance around deterministic updates, producing a distribution over latent trajectories and supporting exploration of the solution space.The guidance variance controls exploration while retaining deterministic refinement, helping capture uncertainty and avoid local minima.
  • Training: GRAM jointly trains generative and variational parameters by maximizing an ELBO, using target-conditioned posterior trajectories during training and the learned prior during inference.The practical objective is a biased, memory-efficient truncated surrogate because gradients are stopped through preceding states.

3 Related Work

Related work reduces reasoning overhead through latent representations, develops recursive architectures for scaling inference-time computation, and models uncertainty with stochastic latent state-space dynamics. GRAM combines these lines by treating recursive reasoning as probabilistic latent computation rather than deterministic refinement or temporal observation modeling.

  • Latent Reasoning: Latent reasoning shifts part or all of Chain-of-Thought into latent or continuous representations, producing more compact traces and reducing generation overhead.Existing approaches use hidden states, latent or soft tokens, continuous thoughts, internal reasoning traces, and recursive state updates, often within autoregressive sequence generation.
  • Recursive Architectures: Recursive architectures iteratively update states, evolving from RNNs to weight-sharing Transformers with adaptive computation and enabling greater inference-time depth.Recent recursive reasoning models can outperform larger static models by increasing inference-time depth; GRAM instead uses stochastic latent trajectories rather than one deterministic refinement path.
  • Probabilistic Latent State-Space Models: Probabilistic latent state-space models use stochastic transitions to represent uncertainty and multimodal dynamics, commonly trained with variational inference.GRAM shares this latent state-space perspective but interprets stochastic dynamics as computation rather than temporal observation modeling.

4 Experiments

GRAM is evaluated on structured reasoning, multi-solution constraint satisfaction, unconditional generation, and ablation tasks. Across these settings, stochastic latent trajectories, parallel sampling, recursive refinement, and inference-time scaling improve performance and extend GRAM beyond conditional reasoning.

  • Experimental Scope: The experiments cover Sudoku-Extreme and ARC-AGI for structured reasoning, N-Queens and Graph Coloring for multi-solution behavior, binarized MNIST and Sudoku generation, and design ablations.The evaluation is organized into Sections 4.1–4.4 and targets iterative latent refinement, diverse solutions, unconditional generation, and key architectural choices.
  • Structured Reasoning: GRAM consistently outperforms prior recursive models by exploring diverse stochastic reasoning trajectories instead of learning from a single deterministic path.The compared recursive baselines are Looped TF, HRM, and TRM, reproduced under identical settings for scaling analysis.
  • Inference-Time Scaling: 97.0% vs 90.5%: GRAM with N = 20 samples at 16 iterations outperforms TRM at 320 iterations despite comparable computational budget.Increasing parallel samples consistently improves performance across iteration counts, adding a test-time scaling axis beyond sequential refinement.
  • Multi-Solution Tasks: Deterministic recursive models achieve at most 36.1% coverage across multi-solution tasks and suffer sharp accuracy degradation as solution counts increase.GRAM maintains consistent performance across solution counts, addressing the mode-collapse limitation of deterministic latent updates.
  • Multi-Solution Tasks: 99.7%: GRAM reaches this N-Queens accuracy versus 96.3% for AR and 96.1% for MDLM, while reducing Graph Coloring conflict edges to 2.7 and 3.3.The corresponding AR conflict-edge values are 19.0 and 61.3 on 8- and 10-vertex tasks, showing sharper constraint satisfaction with comparable diversity.
  • Unconditional Generation: 99.05% validity: GRAM generates valid Sudoku boards from empty inputs using 10.9M parameters and 16 supervision steps, surpassing D3PM baselines using up to 55.1M parameters and 1000 denoising steps.Inference-time scaling also improves MNIST generation monotonically, with IS increasing from 1.85 to 2.04 and FID decreasing from 84.08 to 73.34 over 8 to 256 steps.
  • Ablations: Stochastic guidance improves performance across architectures, while the full GRAM achieves 93.96% / 99.69%; removing stochasticity yields 0.0% on both tasks.Stochasticity alone preserves Sudoku performance at 94.88% but collapses on N-Queens at 50.27%, where structured guidance is necessary.

5 Conclusions and Limitations

GRAM transforms deterministic recursive architectures into probabilistic generative models for conditional and unconditional generation, while stochastic latent transitions support diverse solution discovery and width-based inference-time scaling. It also shows depth-improving unconditional generation, but deep supervision’s sequential nature limits training efficiency relative to Transformers.

  • Contributions: GRAM models both p(y | x) and p(x) through recursive amortized variational inference, converting deterministic recursive architectures into probabilistic generative models.The framework introduces stochasticity into latent transitions for reasoning problems.
  • Contributions: Stochastic latent transitions enable diverse solution discovery and improved exploration compared to deterministic counterparts, complemented by width-based inference-time scaling.GRAM uses width as a complement to recursive depth during inference-time scaling.
  • Contributions: GRAM demonstrates unconditional generation over inputs, with generation quality improving monotonically with recursive depth beyond training-time steps.This suggests generative modeling directions based on hierarchical recursion.
  • Limitations: Deep supervision’s sequential nature limits training efficiency compared to Transformers, creating a significant barrier to scaling GRAM.The passage identifies sequential deep supervision as a training-efficiency limitation relative to Transformers.

Broader Impacts … A.2 Latent Process Reward Model (LPRM).

GRAM’s broader impacts arise from probabilistic multi-trajectory reasoning for problems with uncertainty and multiple valid solutions, while its adaptive computation and latent reward mechanisms support flexible inference and candidate selection. These capabilities also create risks when plausible but invalid generations are mistaken for verified solutions.

  • Broader Impacts: GRAM’s multiple latent trajectories may benefit constraint satisfaction and scientific problem solving, where uncertainty and multiple valid solutions are common.It suggests improving reasoning through inference-time computation rather than parameter scaling alone.
  • Broader Impacts: GRAM’s generality entails risks because plausible but invalid generations may be mistaken for verified solutions.
  • A.1 Adaptive Computation Time: GRAM optionally uses adaptive computation time, allowing each trajectory to terminate at a learned halting depth instead of a fixed supervision-step count.The formulation follows Q-learning introduced by HRM and adopted in TRM.
  • A.1 Adaptive Computation Time: The halt head maps the high-level state h to qhalt and qcontinue, interpreted as Q-values for halting or continuing computation.
  • A.1 Adaptive Computation Time: The halt head is jointly trained with the main objective through temporal-difference regression, while its auxiliary loss does not propagate gradients into the recursive core.The training targets include whether decoding the current state would yield a correct prediction and the bootstrapped value of one additional supervision step.
  • A.1 Adaptive Computation Time: At inference, computation proceeds stepwise, halting when qhalt meets the criterion or continuing up to a maximum budget of N max supervision steps.Parallel trajectories may terminate at different depths; the released code uses only qhalt with σ(qhalt) > 0.5.
  • A.2 Latent Process Reward Model (LPRM).: The Latent Process Reward Model trains a value head vψ(zt) to predict final-output accuracy from the current latent state, enabling ranking or selection among sampled candidates.It is trained jointly with the main objective via regression, with r ∈ [0, 1] denoting trajectory prediction accuracy.

A.3 Empirical Validation of the Surrogate Objective · B Training and Architecture Details

GRAM’s surrogate objective is treated as a truncated gradient approximation for recursive reasoning trajectories, trading gradient bias for constant-memory training. On Sudoku-Extreme and N-Queens 8 × 8, its monotonic improvement tracks the full ELBO despite a structural objective gap.

  • A.3 Empirical Validation of the Surrogate Objective: LGRAM approximates the full trajectory-level ELBO gradient by backpropagating only through final transitions, enabling constant-memory training with a small gradient bias.The exact gradient requires backpropagation through all TTotal transitions, whereas local truncation favors training stability.
  • A.3 Empirical Validation of the Surrogate Objective: GRAM applies this truncation at the level of recursive reasoning trajectories rather than temporal sequences.The passage identifies this application as what is specific to GRAM, while calling the approximation well-precedented more broadly.
  • A.3 Empirical Validation of the Surrogate Objective: Validation compares the full ELBO with the surrogate objective throughout training on Sudoku-Extreme and N-Queens 8 × 8.The full ELBO sums reconstruction and KL contributions across all TTotal transitions, while LGRAM averages L(n)GRAM over Nsup supervision steps.
  • A.3 Empirical Validation of the Surrogate Objective: Both LELBO and LGRAM decrease monotonically during training on both tasks, indicating that surrogate updates improve the full variational bound.Figure 8 plots both quantities as −ELBO, where smaller is better.
  • A.3 Empirical Validation of the Surrogate Objective: The objective gap reflects different KL accounting— all transitions for LELBO versus final-step terms for LGRAM—rather than optimization failure.The passage characterizes LGRAM as a biased but useful surrogate for LELBO.
  • A.3 Empirical Validation of the Surrogate Objective: The full ELBO’s consistent improvement provides evidence that GRAM optimizes a well-defined data likelihood despite training with the surrogate.LELBO serves as an indirect estimate of negative log-likelihood.

B.1 Architecture Details · B.2 Training Details

GRAM uses an Encoder, Recursive Core, and Decoder, with hierarchical latent updates, task-specific tokenization, and auxiliary halt/correctness prediction. Training uses AdamW with EMA, KL balancing, deep supervision, and task-specific configurations for discrete reasoning and image generation.

  • B.1 Architecture Details: GRAM comprises an Encoder, Recursive Core, and Decoder; its encoder combines token embeddings, optional puzzle embeddings, and RoPE positional encodings.The Recursive Core maintains high-level h and low-level l states, refining l K times before updating h.
  • B.1 Architecture Details: The Recursive Core injects the input embedding during each low-level update, then applies shared attention-and-SwiGLU architectures to both latent-state transitions.For Sudoku, the Recursive Core instead uses a [SwiGLU + SwiGLU] network.
  • B.1 Architecture Details: The decoder maps content tokens from h to logits and uses an auxiliary first-token head for halt decisions and correctness values.Puzzle embedding positions are excluded when extracting content tokens.
  • B.1 Architecture Details: For MNIST generation, images are binarized by scaling pixels from 0–255 to [0, 1], thresholding at 0.5, and encoding patches convolutionally.The encoder first normalizes binary tokens to [−1, 1], then applies two convolutional blocks with 5 × 5 kernels, SiLU, and Group Normalization.
  • B.1 Architecture Details: The Transformer backbone uses embedding dimension D = 512, Nhead=8 attention heads, and FFN hidden dimension Dh=512.Inputs and outputs use sequences shaped [B, L], while latent states and decoder outputs use [B, L, D].
  • B.2 Training Details: Tasks use discrete token sequences, including 81 Sudoku tokens, 900 padded ARC-AGI tokens, and N 2 N-Queens tokens with task-specific vocabularies.ARC-AGI grids use a fixed 30×30 canvas and prepend puzzle embeddings to distinguish tasks.
  • B.2 Training Details: Training uses AdamW with learning rate 10^-4, weight decay 1.0, gradient clipping 1.0, global batch size 768, and EMA decay 0.9999.KL balance uses coefficient 0.8, deep supervision uses Nsup = 16, and β varies by task, including 0.1 for Sudoku and 0.04/0.1 for ARC-AGI-1/2.

C Additional Details of Experiment Setup … D Additional Experiment Results

The additional experiments detail why Looped Transformer is omitted on ARC-AGI and describe multi-solution N-Queens and graph-coloring datasets. These setups emphasize computational feasibility, leakage-free evaluation, and recovery of structurally distinct valid solutions.

  • C.1.1 Looped TF on ARC-AGI: Looped Transformer training takes 19 hours on Sudoku-Extreme but is estimated at approximately 97 days on ARC-AGI, so ARC-AGI results are omitted.The estimate corresponds to approximately 776 GPU-days using 8× NVIDIA RTX Pro 6000 GPUs.
  • C.1.1 Looped TF on ARC-AGI: Looped Transformer converges more slowly because it lacks deep supervision, performing one gradient update after the full trajectory instead of Nsup updates per trajectory.HRM, TRM, and GRAM perform one update per segment, whereas Looped Transformer updates only at trajectory end.
  • C.1.1 Looped TF on ARC-AGI: Without adaptive halting, Looped Transformer processes every input to maximum recursion depth, making full ARC-AGI training impractical.The sequential-compute inefficiency compounds with the lack of deep supervision at ARC-AGI scale.
  • C.2.1 N-Queens Problem: N-Queens instances remove 5–7 queens for 8 × 8 boards and 7–9 queens for 10 × 10 boards from complete solutions to create partial-input puzzles.Targets are the original complete boards, and the construction supports diverse valid completions.
  • C.2.1 N-Queens Problem: N-Queens evaluation uses an 85:15 train-test split by unique input configurations, preventing test patterns from appearing in training.Inputs are flattened into sequences x ∈ {0, 1, 2}^L with L = N^2 and zero-padded puzzle embedding tokens.
  • C.2.2 Graph Coloring Problem: The graph-coloring datasets contain 7,002 training and 255 test instances for N = 8, plus 13,465 training and 192 test instances for N = 10.The input is the flattened upper-triangular adjacency matrix, and outputs encode one color per node.
  • C.2.2 Graph Coloring Problem: Graph-coloring outputs use vocabulary codes for padding, non-edges, edges, and red, blue, and green node colors.The output sequence has length N, matching the number of graph nodes.

D.1 Additional Results on Challenging Puzzle Benchmarks · D.2 Scales with Parallel Sampling on ARC-AGI Challenge · D.3 Solution Coverage Analysis

The appendix shows that GRAM advances recursive reasoning on challenging puzzles, scales with parallel sampling, and covers more solutions than deterministic recursive models. These results link GRAM’s stochastic latent trajectories to improved accuracy, inference-time exploration, and multi-solution diversity.

  • D.1 Additional Results on Challenging Puzzle Benchmarks: GRAM consistently outperforms Looped TF, HRM, and TRM across three challenging puzzle benchmarks while using 10M parameters versus HRM’s 27M.All recursive model scores were obtained at 16 supervision steps.
  • D.1 Additional Results on Challenging Puzzle Benchmarks: 0% accuracy on both Sudoku and ARC-AGI-2 for direct prediction indicates that recursive computation is essential for these tasks.The passage attributes these failures to single-pass models being unable to solve the tasks regardless of capacity.
  • D.1 Additional Results on Challenging Puzzle Benchmarks: 0% on Sudoku-Extreme for all tested large reasoning models, including Deepseek-R1 (671B), shows that pretrained capacity alone does not transfer to constraint-propagation reasoning.Gemini 3 Pro reaches 75.0% on ARC-1 and 31.1% on ARC-2, serving as benchmark-difficulty reference points rather than controlled baselines.
  • D.2 Scales with Parallel Sampling on ARC-AGI Challenge: Without data augmentation, GRAM accuracy consistently improves as the number of parallel samples increases, outperforming TRM through generative sampling.This width scaling lets GRAM explore more plausible reasoning trajectories and recover from initial errors.
  • D.2 Scales with Parallel Sampling on ARC-AGI Challenge: At Aug=0, increasing parallel samples consistently improves accuracy, whereas at Aug=50 performance saturates regardless of sample count.The results suggest augmentation and sampling play complementary roles in capturing solution diversity.
  • D.3 Solution Coverage Analysis: Deterministic recursive models HRM and TRM show sharply declining solution coverage as valid solutions increase, reflecting single-trajectory mode collapse.Figure 15 evaluates coverage on 8 × 8 and 10 × 10 N-Queens tasks against the total number of valid ground-truth solutions.
  • D.3 Solution Coverage Analysis: GRAM’s solution coverage improves monotonically from N=1 to N=20 across both 8 × 8 and 10 × 10 N-Queens settings.The passage attributes this broader coverage to generative latent transitions and stochastic guidance across multiple valid reasoning paths.

D.4 Additional Generated Image Samples

GRAM’s unconditional binarized-MNIST samples show progressive latent refinement from black initialization to coherent digits. The recursive feedback loop can correct ambiguous or incorrect early formations by t = 16.

  • Additional Generated Image Samples: GRAM progressively adds details and sharpens digit structure, starting from a black initialization.This behavior demonstrates iterative refinement during unconditional image generation.
  • Additional Generated Image Samples: By t = 16, GRAM corrects disjointed artifacts or incorrect shapes observed at t = 1 and t = 2, producing coherent, high-quality digits.The examples include the second row generating ’2’ and the last row generating ’1’.
  • Additional Generated Image Samples: Figure 16 shows 8 unconditional GRAM samples on binarized MNIST across recursive refinement steps t = 0 to t = 16.Each row represents one generated sample visualized throughout its recursive refinement process.

D.5 Additional Experiment Results on Unconditional Sudoku Generation · D.6 Visualizing Latent Recursion Process · Licenses

The paper reports unconditional Sudoku generation from blank boards, where GRAM achieves high validity with fewer parameters and inference steps than D3PM baselines. Latent-trajectory visualizations contrast TRM’s single deterministic path with GRAM’s diverse stochastic exploration, while the licenses section catalogs external assets and sources.

  • D.5 Additional Experiment Results on Unconditional Sudoku Generation: GRAM generates complete 9 × 9 Sudoku boards from an entirely blank input and evaluates validity over 100K samples using row, column, and box constraints.Every row, column, and 3 × 3 box must contain digits 1 through 9 exactly once.
  • D.5 Additional Experiment Results on Unconditional Sudoku Generation: GRAM is trained on 50K complete solutions derived from Sudoku-Extreme, with clue patterns discarded and all-blank boards used as inputs.No data augmentation is used; training runs for 200 epochs with learning rate 10^-4, EMA decay 0.999, and KL coefficient 0.05.
  • D.5 Additional Experiment Results on Unconditional Sudoku Generation: 99.05% validity: GRAM outperforms all D3PM baselines, while D3PM-Uniform (Big) reaches 91.33% using 55.1M parameters and 1000 denoising steps.GRAM uses fewer parameters and only 16 inference steps.
  • D.5 Additional Experiment Results on Unconditional Sudoku Generation: All valid generated boards are unique under exact board matching across methods, indicating validity is not caused by repeating a small set of boards.The evaluation reports the ratio of generated boards satisfying Sudoku constraints over 100K samples.
  • D.6 Visualizing Latent Recursion Process: Latent trajectories are visualized by tracking h at each supervision step, projecting states into 2D with PCA, and interpolating unobserved states via a K-D tree.The resulting visualization forms a continuous loss landscape for recursive computation.
  • D.6 Visualizing Latent Recursion Process: TRM follows one deterministic path, whereas GRAM samples diverse trajectories that explore different latent-space regions before converging; some reach the global optimum while others remain in local minima.The comparison uses the same Sudoku puzzle, and GRAM’s visualization includes 50 sampled trajectories.
  • Licenses: The licenses section lists existing datasets, benchmarks, and public reference implementations used or cited in the experiments, while author-generated Synthetic N-Queens and Graph Coloring instances are not external assets.Table 10 provides the corresponding licenses and source links.
Loading 2605.19376v2…