PaperlayerView exact revision on arXiv

Source-linked AI summary

Geometric Latent Reasoning Induces Shorter Generations in LLMs

Shashi Kumar, Yacouba Kaloga, Petr Motlicek, Ina Kodrasi, Andrea Cavallaro

arXiv:2606.02248v1cs.CL

TL;DR

Explicit reasoning traces are costly and discrete, while useful structure for continuous latent reasoning remains unresolved. GLR approximates chain-of-thought trajectories in pretrained token-embedding space with learned continuous updates, and evaluations show shorter generations with a controllable accuracy–length tradeoff.

  • Problem

    Explicit chain-of-thought reasoning creates lengthy, computationally costly discrete traces, while determining useful structures for continuous latent states remains an open challenge.

  • Method

    GLR trains a lightweight transition head to predict continuous embedding-space direction updates anchored to textual chain-of-thought trajectories.

  • Results

    GLR induces shorter generations on mathematical reasoning benchmarks, often producing correct answers with substantially fewer generated tokens without an explicit length objective.

  • Takeaways & Limitations

    The number of latent steps provides inference-time control over the tradeoff between latent computation, output length, and accuracy.

  • Takeaways & Limitations

    At the largest evaluated generation budgets, chain-of-thought supervised fine-tuning often recovers and outperforms GLR in final accuracy, limiting GLR’s main advantage to constrained budgets.

Abstract

from arXiv · show

Large language models solve complex problems by generating lengthy chains of explicit reasoning tokens. While effective, this makes reasoning expensive, length-sensitive, and constrained to (discrete) natural language. While latent reasoning offers a continuous alternative, determining useful structures for intermediate latent states is an open challenge. In this paper, we formulate latent reasoning as a geometric path-approximation problem within the model's pretrained token-embedding space. We introduce Geometric Latent Reasoning (GLR), which uses a lightweight transition head to predict iterative direction updates in embedding space. Using textual chain-of-thought traces as anchors, GLR learns to approximate discrete reasoning trajectories while permitting continuous deviations from exact token embeddings. Evaluations on mathematical reasoning benchmarks using Qwen3 models reveal an emergent phenomenon: geometric latent reasoning induces substantially shorter generations without an explicit length objective. By replacing early explicit reasoning with continuous latent steps, models often reach correct answers using substantially fewer total generation steps. These findings suggest that continuous trajectories act as compact intermediate reasoning states, exposing a new tradeoff between latent computation budget, output length, and accuracy.

1 Introduction

GLR reframes latent reasoning as continuous trajectory approximation in pretrained embedding space, replacing some discrete chain-of-thought steps with learned latent updates. On mathematical benchmarks, it produces shorter generations and exposes a controllable accuracy–length tradeoff.

  • Motivation: Explicit chain-of-thought creates computational overhead by expressing intermediate reasoning in discrete natural-language tokens.It also requires premature commitment to specific tokens at each step.
  • Method: GLR treats textual chain-of-thought as a discrete embedding-space trajectory and approximates it with continuous displacement vectors.The method assumes useful intermediate states can exist in local neighborhoods around token-induced trajectories.
  • Method: A lightweight transition head learns continuous embedding updates from textual chain-of-thought trajectories using a position-discounted transition objective.At inference, GLR replaces an initial explicit-reasoning segment with a fixed number of latent steps before standard decoding resumes.
  • Results: GLR induces shorter generations and often reaches correct answers with substantially fewer generated tokens than chain-of-thought supervised fine-tuning.The effect is especially pronounced under constrained generation budgets and requires no explicit length objective.
  • Results: The number of latent steps controls the allocation between continuous computation and explicit text generation, creating an accuracy–length tradeoff.This provides an inference-time control over the cost and form of reasoning.

2 Related Work

Prior work establishes that continuous states can support non-verbalized reasoning, but their structure remains an open challenge. GLR connects explicit chain-of-thought and latent reasoning by approximating textual paths within pretrained embedding geometry.

  • Discrete reasoning in LLMs: Chain-of-thought improves multi-step reasoning but can produce long traces as models allocate more test-time computation to accuracy.Discrete text may also fail to faithfully reflect the model’s internal computation.
  • Latent reasoning in LLMs: Latent-reasoning methods replace parts of explicit chain-of-thought with continuous states, using hidden-state feedback, distillation, or external latent modules.Structuring these continuous states remains a fundamental challenge.
  • Soft tokens and embedding-space explorations: Soft-token and hybrid methods show that language models can process continuous embedding inputs while preserving useful computational structure.Prior work mainly uses soft tokens for decoding or prompting rather than geometric approximation of reasoning trajectories.
  • Positioning: GLR bridges text-only and unconstrained latent reasoning by learning continuous approximations of explicit chain-of-thought paths in pretrained input geometry.This yields a lightweight way to trade explicit token generation for latent computation.

3 Method

GLR models chain-of-thought as a trajectory through pretrained token-embedding space and learns continuous local transitions around it. During training, latent replacements approximate these transitions, while inference uses continuous steps before resuming standard token decoding.

  • Geometric formulation: GLR interprets textual chain-of-thought as a trajectory through the model’s pretrained token-embedding space.This reframes improved reasoning as searching over continuous input-space trajectories rather than only discrete token sequences.
  • Geometric formulation: GLR relies on meaningful neighborhoods around token-induced trajectories, supported by continuity of the model’s input-to-hidden mapping and evidence that continuous inputs can support coherent reasoning.The paper does not claim that arbitrary embedding-space points are useful; it focuses on local deviations around token trajectories.
  • Latent transition mechanism: A lightweight transition head predicts local embedding-space displacements from the hidden state of the current reasoning context.The predicted displacement approximates the difference between consecutive reasoning-token embeddings rather than an arbitrary next embedding.
  • Training procedure: The transition objective uses a position-discounted penalty, reducing the transition penalty for later reasoning positions.The discount factor satisfies 0 < γ ≤ 1.
  • Training procedure: Training uses two forward passes: the first predicts transitions from the discrete sequence, and the second replaces thought-token embeddings with continuous latent states.The modified sequence is used to compute the latent transition objective, while answer-token generation behavior remains supervised by cross-entropy.
  • Inference procedure: At inference, GLR performs a chosen number of continuous latent steps, feeds each resulting embedding directly back into the model, then resumes token-level reasoning and answer generation.This avoids requiring every intermediate reasoning step to correspond to a vocabulary token and may bypass redundant explicit transitions.

4 Experiments

Experiments across mathematical benchmarks test whether GLR improves accuracy under constrained budgets, reduces successful generation length, and exposes a latent-step tradeoff. Results show shorter correct trajectories and broader benchmark consistency, while large latent budgets and long decoding regimes remain boundary conditions.

  • 4.1 Setup: GLR is evaluated against COT-SFT across six mathematical benchmarks using Qwen3-0.6B and Qwen3-1.7B models fine-tuned on the same 10K-example dataset.Evaluation uses greedy pass@1 accuracy and counts latent plus text steps for GLR.
  • 4.2 Results: Accuracy and Generation Length: At constrained budgets, GLR substantially improves accuracy over COT-SFT, including over 40% versus nearly 0% on MATH500 at 512 steps for Qwen3-1.7B.The comparison uses an equal-step budget, with latent steps replacing part of the explicit reasoning prefix.
  • 4.2 Results: Accuracy and Generation Length: On MATH500, correct Qwen3-1.7B generations fall from approximately 2,000 COT-SFT tokens to roughly 350 total steps with GLR-10 or GLR-20.GLR has no explicit length penalty; the shorter successful trajectories are attributed to latent reasoning carrying part of the reasoning state.
  • 4.2 Results: Accuracy and Generation Length: Moderate latent-step budgets provide the strongest accuracy–length tradeoff, whereas large settings such as GLR-80 or GLR-100 reduce accuracy.The reported effect is non-monotonic and suggests a stability limit for uninterrupted latent reasoning.
  • 4.2 Results: Accuracy and Generation Length: At the largest tested budgets, COT-SFT recovers and often outperforms GLR in final accuracy, limiting GLR’s main advantage to constrained-budget regimes.The paper connects this boundary to possible accumulated geometric drift and limited training data for aligning the transition head.
  • 4.3 Generalization to other benchmarks: Across SVAMP, MultiArith, AMC23, and OlympiadBench, GLR generally improves constrained-budget accuracy and reduces successful generation steps.On SVAMP, COT-SFT uses roughly 500–700 tokens for correct solutions, while GLR sharply reduces this redundant reasoning overhead.
  • 4.4 Understanding Latent Dynamics: With the same GLR-trained backbone, K = 0 yields a median GSM8K length of approximately 1,000 tokens, while K ∈{5, 10} reduces it to under 200 tokens.The ablation indicates that the reduction appears when learned continuous displacements are used at inference, not merely from the training recipe.
  • 4.4 Understanding Latent Dynamics: After latent updates, decoding can resume mid-reasoning with problem-specific quantities that would normally appear earlier in an explicit CoT trace.These examples support the interpretation that latent prefixes move the model into a partially advanced reasoning state.

5 Conclusions

GLR frames reasoning as geometric path approximation in pretrained token-embedding space and replaces part of explicit reasoning with continuous transitions. The paper concludes that this can shorten mathematical reasoning trajectories while exposing a tradeoff between latent computation, output length, and accuracy.

  • 5 Conclusions: GLR formulates latent reasoning as continuous path approximation within the pretrained token-embedding space.The method is presented as a structured alternative to unconstrained latent reasoning.
  • 5 Conclusions: GLR uses a lightweight transition head to learn CoT-anchored directional updates, shifting part of computation from discrete text to continuous representations.The transition objective uses local, CoT-anchored updates.
  • 5 Conclusions: Across mathematical benchmarks, replacing early explicit reasoning with latent steps reaches correct answers using substantially fewer generated tokens without an explicit length penalty.The conclusion presents shorter generations as an emergent outcome of the method.
  • 5 Conclusions: GLR provides an inference-time tradeoff among latent transitions, output length, and accuracy.The conclusion describes this tradeoff as a controllable consequence of the latent-step design.

B.2 Evaluation Benchmarks

The evaluation spans six mathematical reasoning benchmarks, covering foundational arithmetic through advanced and competition mathematics. The suite includes datasets designed to test both multi-step reasoning and over-generation on simple problems.

  • The study evaluates models across six distinct mathematical reasoning benchmarks under greedy decoding.The benchmarks assess foundational arithmetic and highly complex mathematical reasoning.
  • Foundational Arithmetic: GSM8K contains grade-school word problems requiring 2 to 8 steps of basic arithmetic.
  • Foundational Arithmetic: SVAMP varies simple word-problem structures and is used to observe over-generation on simple logic.
  • Foundational Arithmetic: MultiArith focuses on arithmetic word problems requiring multiple reasoning steps.
  • Advanced and Competition Mathematics: MATH500 is a 500-problem subset spanning structurally complex algebra, geometry, and calculus tasks.
  • Advanced and Competition Mathematics: AMC23 tests advanced logical deduction and theorem application, while OlympiadBench represents the highest difficulty tier with IMO-level problems requiring rigorous derivations.

C Results on other benchmarks

On AMC23, OlympiadBench, and MultiArith, GLR shifts the accuracy–length frontier and reduces the median generation steps needed for correct solutions. The results are consistent with the paper’s findings on the primary benchmarks.

  • GLR shifts the accuracy–length frontier and reduces the median generated steps required to solve problems across the remaining benchmarks.This pattern is reported for AMC23, OlympiadBench, and MultiArith and is consistent with results on GSM8K, MATH500, and SVAMP.
  • AMC23: On AMC23, GLR reduces median generation length on competition mathematics.
  • OlympiadBench: On OlympiadBench, GLR shifts the accuracy–length frontier for Olympiad-level mathematical reasoning.
  • MultiArith: On MultiArith, GLR reduces the generated steps required for foundational multi-step arithmetic problems.

D Generated token distribution over full benchmark

Full-dataset generation-length distributions show that GLR produces shorter generations across correct and incorrect answers, including outputs that reach the decoding cap. This indicates an overall reduction in test-time generation length rather than a pattern limited to successful solutions.

  • GLR produces shorter generations across the entire evaluation set, including incorrect answers and generations reaching the decoding cap.
  • The full distributions extend the correct-answer analysis by including all evaluated problems rather than only successful solutions.
  • Primary Benchmarks: Figures 10 and 11 show full generation-length distributions for the primary benchmarks.These figures cover GSM8K and MATH500.
  • Remaining Datasets: Figures 12, 13, and 14 show full generation-length distributions for the remaining datasets.These figures cover AMC23, OlympiadBench, and MultiArith.

E Compute Accounting

GLR’s compute accounting counts latent steps and decoded tokens equally, providing a hardware-independent sequential-step comparison while conservatively measuring its savings. Wall-clock latency is not reported because GLR and COT-SFT use different serving implementations.

  • The primary efficiency metric counts latent steps and decoded tokens equally to compare sequential model steps independently of hardware.This accounting is conservative because latent steps still require Transformer forward passes but avoid the vocabulary head.
  • Latent-Step Cost: A latent step applies the transition head after the Transformer forward pass instead of performing the full vocabulary projection and token selection.
  • Latent-Step Cost: For Qwen3-1.7B, the vocabulary head has roughly 300M parameters, compared with roughly 4M parameters in the GLR transition head.
  • Latency Measurement: Wall-clock latency is not reported because the custom GLR decoder and vLLM-based COT-SFT baseline use different serving stacks and optimization levels.A fair end-to-end speed comparison requires an optimized GLR serving implementation.

F Qualitative Examples of Latent-to-Text Transitions

GLR uses a continuous latent prefix before resuming explicit token decoding, enabling problem-specific intermediate states in qualitative examples. For instance, a Qwen3-1.7B GSM8K run with 20 latent steps resumes with the relevant operation and emits the final answer.

  • Latent-to-text transition: GLR performs K continuous latent steps inside the reasoning span before standard explicit token decoding resumes.These latent steps are not decoded into text; the subsequent tokens are the first explicit outputs.
  • Latent-to-text transition: After the latent prefix, the model often continues from a problem-specific intermediate state rather than restarting a full explicit chain-of-thought trace.
  • Concrete example: 20 latent steps in a Qwen3-1.7B GSM8K example are followed by the relevant operation, 54 −20 = 34, and then the final answer.
  • Visualization: Figure 17 displays 80 continuous updates, with special tokens in teal and latent-step placeholders in orange.

H Limitations

The study is constrained by small-scale training and mathematics-focused evaluation, while continuous latent states reduce interpretability and leave optimized serving latency unmeasured.

  • Scope and training scale: Compute constraints limited fine-tuning to relatively small Qwen3 models and a 10K-example CoT dataset.The authors note this may understate GLR’s potential and contribute to drift at large latent budgets.
  • Evaluation scope: Evaluation focuses on mathematical reasoning, so it remains unclear whether the accuracy–length tradeoff holds in science, code, or general reasoning.
  • Efficiency measurement: The paper does not report wall-clock latency because GLR and the CoT-SFT baseline use different inference implementations.Efficiency claims are limited to hardware-independent generation and projection-count accounting rather than optimized serving latency.
  • Interpretability: Continuous latent states hide part of the reasoning trajectory, making model behavior harder to inspect than standard CoT decoding.
Loading 2606.02248v1…