PaperlayerView exact revision on arXiv

Source-linked AI summary

Latent Recurrent Thoughts: Recurrent Refinement of Proposed Latents for Reasoning with Frozen LLMs

Zhaoliang Chen, Jie Fu

arXiv:2609.01117v1cs.AIcs.CL

TL;DR

Discrete chain-of-thought requires verbalized, error-propagating traces, while latent reasoning leaves open how useful continuous states should be computed. LRT keeps the decoder frozen, uses a task-dedicated proposer and recurrent residual refiner, and substantially outperforms prior frozen-decoder latent methods across symbolic and natural-language tasks under matched conditions.

  • Problem

    Discrete chain-of-thought reasoning commits intermediate steps to tokens, propagates errors, and presupposes reasoning traces to imitate; continuous reasoning therefore needs a way to compute useful latent states.

  • Method

    LRT trains a task-dedicated proposer and recurrent reasoner to supply and iteratively refine bounded latent residuals for a frozen LLM decoder.

  • Results

    Across Countdown-4, Sudoku, HumanEval, MBPP, and StrategyQA, LRT substantially outperforms prior frozen-decoder latent methods under identical decoder, prompt, data, and budget.

  • Takeaways & Limitations

    A small recurrent latent module can extend a frozen decoder’s reasoning at fixed, small inference cost, including tasks lacking recorded reasoning traces.

  • Takeaways & Limitations

    LRT requires newly trained proposer and refiner modules per task family and is not a single zero-shot general model.

Abstract

from arXiv · show

Chain-of-thought reasoning unfolds in discrete token space: each step is committed as text, errors propagate, and eliciting good traces presupposes traces to imitate. Reasoning instead in a model's continuous representation space - where intermediate states are vectors rather than words - sidesteps these constraints, but leaves open how those latent states should be computed. We approach this along two axes. First, we keep a large language model (LLM) frozen and use it for what it is already good at - modeling and decoding sequences - while a small auxiliary network supplies continuous latent thoughts as input. Second, we produce those latents by recurrence: a tiny recurrent reasoner refines them over many steps, decoupling the depth of computation from the size of the model, so that the latents are a product of iterative processing rather than a single forward pass. We instantiate this as Latent Recurrent Thoughts (LRT): a task-dedicated proposer supplies base latents, a recurrent reasoner refines them through bounded residual corrections, and the frozen LLM decodes the answer. On symbolic reasoning with answer supervision but no reasoning traces (Countdown-4, Sudoku) and on natural-language reasoning (HumanEval, MBPP, StrategyQA), LRT substantially outperforms prior frozen-decoder continuous-space reasoning methods under an identical decoder, prompt, data, and training budget, and outperforms non-thinking-mode chain-of-thought prompting on the same backbone at a small fraction of its inference compute.

1 Introduction

LRT addresses limitations of discrete chain-of-thought and fragile frozen-decoder latent methods by combining task-dedicated proposals with recurrent refinement. Across symbolic and natural-language tasks, it improves substantially over prior latent methods under controlled conditions, while remaining scoped to per-task modules and small inference cost.

  • Motivation: Discrete chain-of-thought commits intermediate steps to tokens, propagates errors autoregressively, and requires reasoning traces to imitate.Continuous-space reasoning instead feeds latent vectors into the model.
  • Approach: SoftCoT and EBM-CoT use a generic proposer, while LRT replaces it with a task-dedicated proposer and recurrent refiner.The recurrent component supplies iterative computation rather than relying only on a single proposal or energy-based adjustment.
  • Results: 56.7% on Countdown-4: LRT exceeds SoftCoT at 5.9% and EBM-CoT at 8.4%, while zero-shot CoT reaches 30.0%.The result is reported for answer-supervised symbolic reasoning without reasoning traces.
  • Results: LRT improves further on natural-language tasks after SoftCoT and EBM-CoT provide modest gains over zero-shot CoT.The evaluated tasks are HumanEval, MBPP, and StrategyQA.
  • Scope: LRT is a per-task-trained latent augmentation of a frozen decoder, compared with trained frozen-decoder modules rather than a single zero-shot general model.Thinking-mode prompting beats LRT on three of five benchmarks while using roughly 5–450× more inference compute.

2 Related Work

Related work spans discrete chain-of-thought, continuous latent reasoning, frozen-decoder latent conditioning, recurrent reasoning, and iterative refinement in specialized models. LRT combines frozen-decoder latent injection with task-dedicated proposals and recurrent multi-step refinement.

  • Chain-of-thought: Chain-of-thought methods spend inference computation through sampling, tree or graph search, compute-optimal scaling, and reinforcement learning.These methods retain reasoning in discrete token space.
  • Continuous-space reasoning: Continuous-space methods feed vectors back into models instead of decoded tokens, including hidden-state recurrence such as Coconut.Coconut can represent alternative continuations in a single continuous thought.
  • Frozen-decoder latent conditioning: Frozen-decoder latent methods train a small module that supplies soft-token latents while leaving the language-model decoder unchanged.SoftCoT and related methods obtain latents from single-pass proposals, optionally adjusted by an energy gradient.
  • LRT distinction: LRT replaces generic single-pass proposals with a task-dedicated encoder and recurrent reasoner that performs many bounded residual updates.This recurrent computation is the paper’s distinguishing axis within frozen-decoder latent conditioning.
  • Recurrent and specialized reasoning: Recursive reasoners decouple computation depth from parameter count, while energy-based and diffusion models perform iterative refinement outside the language-model paradigm.LRT brings recurrent refinement into a frozen pretrained LLM.

3 Method

LRT trains small auxiliary modules to produce and recurrently refine latent vectors for a frozen autoregressive decoder. Its proposer is task-dedicated, while its refiner performs repeated anchored updates and returns a bounded residual for answer decoding.

  • Problem Setup: A small model maps question x to K latent vectors, inserts them after the instruction and question, and trains only through the frozen decoder’s answer cross-entropy.The decoder remains unchanged throughout training.
  • Problem Setup: LRT replaces generic proposal and energy refinement with a task-dedicated encoder and recurrent reasoner.The proposer and refiner are the only variables in the controlled comparison.
  • Task-Dedicated Proposer: The proposer encodes the question with a bidirectional Transformer, uses learnable query vectors, and emits K base latents in the decoder’s embedding space.It shares the frozen decoder’s input embedding table and uses trainable down- and up-projections.
  • Recurrent Refiner: The refiner iteratively updates fast and slow working states while reinjecting the proposed latents at every fast update.The input signal keeps refinement anchored to the problem as the slow state integrates scratch updates.
  • Recurrent Refiner: The refined latents equal the proposed base plus a bounded residual, with an L2 penalty discouraging drift toward generic instance-agnostic latents.The correction is predicted rather than regenerating the refined latent.
  • Training and Inference: The refiner is deeply unrolled but cheaply trained with stop-gradient through preceding cycles, so backpropagation traverses only the final cycle.At inference, the full recursion runs before greedy decoding.
  • Training: Training uses two stages: first optimize the proposer with base latents, then freeze it and optimize the refiner using answer loss plus residual regularization.Only final answers, or StrategyQA answers with reference rationales, supervise the process; reasoning searches are not recorded.

4 Experiments

Across symbolic and natural-language benchmarks, LRT combines task-dedicated latent proposals with recurrent refinement and a frozen decoder to improve reasoning performance under controlled comparisons. Ablations and scaling studies indicate that both components matter, recurrent computation benefits from greater depth and capacity, and the decoder remains part of the computation.

  • Symbolic reasoning: LRT reaches 56.7 on Countdown-4 and 49.2 on Sudoku, improving over zero-shot CoT by 26.7 and 25.3 points, respectively.It is the only frozen-decoder method to improve over zero-shot CoT on either symbolic task.
  • Natural-language reasoning: LRT is strongest on HumanEval, MBPP, and StrategyQA; on HumanEval it reaches 37.8 versus 25.0 for EBM-CoT.These are the natural-language tasks where soft-thought methods were designed to operate.
  • Aggregate results: Across five benchmarks, LRT averages 54.1 versus 33.5 for EBM-CoT and 35.0 for zero-shot CoT.SoftCoT averages 29.5, below the Direct baseline at 30.9, because its symbolic failures offset natural-language gains.
  • Scaling the recurrent refiner: 56.7 to 66.3 to 69.1 on Countdown-4 and 37.8 to 44.0 to 46.1 on HumanEval as the refiner grows from 7M to 14M to 28M parameters.The frozen 8B decoder and proposer remain fixed, and the largest gains occur at the first doubling.
  • Mechanistic ablations: Replacing a generic assistant with a task-dedicated proposer raises average performance from 12.3 to 37.2 without refinement, while recurrent refinement adds +10.7 versus +3.4 for energy refinement.The ablation shows that neither component alone accounts for LRT and that generic-proposer cells remain below Direct on Countdown-4.
  • Depth versus capacity: At fixed 7M capacity, LRT reaches 56.7/37.8 on Countdown-4/HumanEval versus 47.5/33.0 for a non-recurrent refiner.A refiner trained at S=H=3 continues improving when unrolled deeper at test time.

5 Conclusion

LRT combines a task-dedicated proposer with a TRM-based recurrent refiner before a frozen LLM, training fewer than 0.2% of the model’s parameters. Across symbolic and natural-language reasoning, it improves over prior frozen-decoder latent methods and budget-matched zero-shot CoT, while remaining per-task and weaker than large test-time budgets on some tasks.

  • LRT pairs a task-dedicated proposer with a TRM-based recurrent refiner that emits bounded residual corrections for a frozen decoder.The proposer maps problems to usable base latents, while the refiner computes over them.
  • LRT trains fewer than 0.2% of the parameters involved while using the frozen LLM for sequence modeling and decoding.
  • Across symbolic and natural-language reasoning, LRT improves substantially over prior frozen-decoder latent-injection methods under identical decoder, prompt, data, and budget.
  • LRT also improves over budget-matched zero-shot CoT, but large test-time budgets remain stronger on some tasks.

Limitations

The evidence supports latent computation distributed across the recurrent refiner and frozen decoder, but the mechanistic interpretation is correlational and the evaluation scope is constrained. LRT is trained per task family, lacks demonstrated cross-task transfer, and trades peak Sudoku accuracy and inference compute for broader coverage.

  • Mechanistic evidence is correlational: intermediate decoding, contracting residuals, and linear probes do not prove where computation occurs.The linear probe bounds only linearly decodable information.
  • LRT requires newly trained proposer and refiner modules for each task family rather than operating as one zero-shot general model.
  • The evaluation demonstrates within-task generalization but does not test transfer across tasks or difficulty levels.Examples such as harder Countdown variants or Sudoku-to-Kakuro transfer remain untested.
  • The study uses one 8B decoder and five benchmarks, leaving substantially larger scales and broader task distributions open.
  • LRT trades peak symbolic accuracy on Sudoku for a single method that also handles natural language, while recurrent unrolling increases inference compute with depth.

A Implementation and Training Details

LRT uses a small proposer and recurrent refiner around a frozen decoder, trained with staged optimization and evaluated using task-specific data and inference procedures. The implementation adapts TRM to sequence-to-sequence Countdown-4 and reproduces controlled baselines and thinking-mode references.

  • Architecture: The proposer uses a bidirectional encoder with two transformer blocks and K=32 learned queries, while the refiner reuses a transition block for recursive latent updates.Both operate through projections around working width d′=256.
  • Architecture: LRT trains 11.2M parameters, about 0.14% of the frozen Qwen3-8B decoder.
  • Optimization: Both stages use AdamW with cosine decay, 5% warmup, weight decay 0.01, gradient clipping at 1.0, 30 epochs, and batch size 64.Stage peak learning rates are 3×10^-4 for the proposer and 2×10^-4 for the refiner.
  • Datasets: Countdown-4 instances and Sudoku puzzles are generated, while HumanEval, MBPP, and StrategyQA use public splits; HumanEval is evaluation-only.
  • Supervision: Countdown-4 and Sudoku use final-answer supervision without recorded reasoning traces, while main-result StrategyQA includes its reference rationale.
  • Supervision: Answer-only StrategyQA supervision reaches 76.9 versus 75.1 ± 2.3 with rationale supervision.
  • Inference: At inference, four of 32 proposer latents are retained, the refiner runs the full recursion, and the frozen decoder greedily produces the answer.
  • Baselines: The reproduced SoftCoT and EBM-CoT baselines share LRT’s frozen decoder, generic proposer, and stage-wise training budget.

C Parameter-Efficient Baselines and Data Efficiency

Matched-budget controls show that LRT’s gain comes from instance-conditioned latent computation rather than merely adding trainable prompt parameters or using more data.

  • Parameter-efficient baselines: 42.0 on Countdown-4 from the task-dedicated proposer alone rises to 56.7 when recurrent refinement is added.Prefix-tuning reaches 12.5 and P-tuning v2 28.4 under the same approximately 11M trainable-parameter budget.
  • Parameter-efficient baselines: Instance-independent prompt adaptation reaches 12.5 with Prefix-tuning and 28.4 with P-tuning v2 on Countdown-4.P-tuning v2 essentially matches the no-latent Direct baseline at 27.8.
  • Scope: The parameter-efficient prompt controls were run only on Countdown-4, not the remaining four benchmarks.The paper identifies Countdown-4 as the benchmark that separates methods most sharply.
  • Data efficiency: 52.7 at 1,000 Countdown-4 examples reaches 56.7 at 100,000, a 4.0-point increase despite two additional data orders of magnitude.The curve is described as close to flat, with small non-monotonicity within seed noise.
  • Data efficiency: 5.9 for SoftCoT and 8.4 for EBM-CoT at 100,000 examples show that access to the same data does not produce comparable symbolic performance.LRT at 1,000 examples already exceeds every other frozen-decoder method trained on 100,000 examples.

D Refiner Design Ablations

Refiner ablations show that LRT depends on residual anchoring, repeated proposal injection, two-timescale recurrence, and a carefully calibrated residual penalty.

  • Refiner design: 3.9 average points are lost when residual parameterization is replaced by direct regeneration of the refined latent.The residual form preserves an anchor to the instance-conditioned proposal.
  • Refiner design: 3.2 points are lost when the proposed base latent is injected only at initialization rather than at every fast update.The fast state can drift away from the problem when repeated injection is removed.
  • Refiner design: 2.6 points are lost when the two-timescale state is collapsed into a single state.This ablation tests the separate z_L and z_H components.
  • Residual penalty: λ=0.01 is optimal: λ=0 permits large residual drift, whereas λ=1.0 suppresses residuals and collapses toward the proposer-only result.The residual penalty is reported relative to the λ=0.01 setting and evaluated by five-benchmark average.

E Scaling and Depth

Scaling experiments indicate that recurrent refinement benefits from deeper test-time unrolling, recurrence itself matters beyond parameter count, and latent count and width have useful operating points.

  • Test-time depth extrapolation: Accuracy continues to improve when a refiner trained at 9 cycles is unrolled for more inference cycles without retraining.The paper notes that no refiner trained at those greater depths was used for comparison.
  • Recurrence: 6.1 average points separate the recurrent refiner from an equal-7M-parameter non-recurrent feed-forward refiner.The matched-capacity comparison isolates recurrence rather than parameter count as the source of the gain.
  • Latent count and width: K_infer=4 is best, while accuracy saturates at d′=256 and d′=512 adds parameters for negligible gain.The sweep fixes K=32 during training and varies inference latent count and working width.
  • Latent-count asymmetry: Training with K=4 collapses Countdown-4 to 9.5, whereas five-benchmark averages decline mildly from 54.1 at K_infer=4 to 51.8 at K_infer=32.The 32 training query positions serve as proposer scratch space, while retaining more inference latents degrades performance mildly.
  • Latent-count asymmetry: Every tested inference-latent setting remains above SoftCoT at 29.5 and EBM-CoT at 33.5.The authors describe the effect as robust rather than a knife-edge tuning choice and use the inherited protocol across methods.

F Training Protocol Ablations

Training-protocol ablations favor staged optimization, cached frozen proposer outputs, and retaining the task instruction alongside the refined latents.

  • Training schedule: 3.1 average points are lost and training becomes less stable when proposer and refiner are trained jointly instead of in two stages.The refiner otherwise adapts to a proposer that is still moving.
  • Proposer freezing: Freezing the proposer during Stage 2 changes performance by only −0.5 points, allowing its base latents to be cached at little cost.Leaving the proposer unfrozen during Stage 2 changes little.
  • Inference inputs: Removing the task instruction at inference costs 3.4 points because the latents carry instance-specific computation but not the decoder’s instruction-following prior.The comparison leaves the decoder to work from L⋆ alone.

G Mechanistic Analyses

LRT refinement improves decoded accuracy incrementally while latent updates contract toward a fixed point. Analyses indicate that refined latents contain weakly linearly readable outcome information and task-relevant computational fragments.

  • Refinement dynamics: Accuracy rises smoothly across refinement cycles while residual updates contract toward a fixed point.The refiner improves intermediate latents incrementally rather than producing a final-step jump.
  • Latent content: Refined latents contain task-relevant fragments, including partial arithmetic for Countdown-4 and candidate digits for Sudoku, rather than fluent text.These fragments are consistent with computation consumed by the decoder without being verbalized.
  • Latent readout: 57.5% probe accuracy versus a 52.5% majority-class baseline shows weak linear predictability of whether the decoder solves Countdown-4 instances.The same probe on unrefined L(0) is at chance, and the probe does not predict answer strings directly.
  • Proposer quality: Generic proposer latents raise Countdown-4 negative log-likelihood above the no-latent baseline, while task-dedicated and refined latents lower it.Nearest-embedding cosine similarity indicates that generic latents lie farther from the decoder’s input manifold.
  • Decoder dependence: LRT remains above frozen-decoder baselines across every tested backbone, while stronger decoders produce stronger LRT results.This pattern is consistent with the decoder supplying sequence priors and trained modules supplying iterative computation.

I Compute and Efficiency

LRT reduces inference latency by replacing long chain-of-thought generation with compact latent inputs and inexpensive recurrent refinement. Training remains dominated by frozen-decoder passes, while truncated unrolling avoids the cost of full backpropagation.

  • Inference cost: Forty-five refiner passes add little cost relative to one forward pass of the frozen 8B decoder because they operate on a 7M-parameter block.The recurrent computation therefore remains cheaper than long autoregressive chain-of-thought decoding in the reported setup.
  • Training cost: The frozen 8B decoder dominates training cost, not the 11M trainable modules.Both stages run for 30 epochs at batch size 64 on a single 96 GB GPU, and Stage 2 caches base latents while freezing the proposer.
  • Inference cost: LRT takes approximately 1.1 seconds per example versus approximately 3.2 seconds for zero-shot CoT on the same hardware.LRT injects four latent vectors and decodes directly, while zero-shot CoT generates approximately 210 tokens.
  • Training cost: Full backpropagation through all 45 passes provides no accuracy benefit while requiring several times the memory and step time.The reported configuration therefore uses truncated-gradient unrolling for the refiner.
  • Scope and risk: The work notes that stronger reasoning capabilities could also amplify the capacity to generate sophisticated harmful content.This is presented as a potential misuse risk of techniques that enhance LLM reasoning.
Loading 2609.01117v1…