Answer-Distribution Trajectories: A Stochastic-Dynamics View of LLM Reasoning

Mar Gonzàlez I Català; Haitz Sáez de Ocáriz Borde; Davide Murari; Carola-Bibiane Schönlieb; Pietro Liò; George Montañez

Source-linked AI summary

Answer-Distribution Trajectories: A Stochastic-Dynamics View of LLM Reasoning

Mar Gonzàlez I Català, Haitz Sáez de Ocáriz Borde, Davide Murari, Carola-Bibiane Schönlieb, Pietro Liò, George Montañez

arXiv:2609.09030v1Sep 8, 2026cs.AIcs.CLcs.ITcs.LG

TL;DR

Endpoint accuracy and entropy profiles can miss how competing answer hypotheses evolve during chain-of-thought reasoning. The paper introduces answer-distribution trajectories to capture this evolution, finding distinct dynamics among similarly summarized traces and systematic changes from objectives, training, and inference choices.

Problem
Endpoint accuracy evaluates reasoning outcomes without accounting for the path taken, while entropy profiles track uncertainty without identifying the competing hypotheses behind it.
Method
Answer-distribution trajectories track the model’s predictive distribution over possible final answers across reasoning prefixes and summarize exploration, revision, motion, commitment, and success or failure mechanisms.
Results
Across sixteen open-weight language models and four reasoning benchmarks, traces with similar endpoints and entropy profiles showed different dynamics, while objectives, training, and inference choices systematically reshaped those profiles.
Takeaways & Limitations
Answer-distribution trajectories provide a richer framework for analyzing and evaluating how language-model reasoning unfolds than endpoint or entropy summaries alone.
Takeaways & Limitations
The analysis is computationally expensive, its training and inference comparisons are descriptive, and practical gains from the trajectories remain an open question.

Abstract

from arXiv · show

Chain-of-thought reasoning provides a structured computation between a model's input and final answer. Yet it is often evaluated through endpoint accuracy, which ignores the path taken to reach that answer. An emerging line of work addresses this limitation using entropy profiles, which track how uncertainty evolves over the reasoning process but do not reveal which competing hypotheses account for that uncertainty. We introduce answer-distribution trajectories, a stochastic-dynamics-inspired representation that tracks the model's full predictive distribution over answers as reasoning unfolds. As a strictly finer representation than endpoint and entropy summaries, answer-distribution trajectories enable us to characterize a trace through a dynamical reasoning profile spanning exploration, revision, motion, and commitment, and to distinguish different dynamical mechanisms of reasoning success and failure. Across sixteen open-weight language models and four reasoning benchmarks, we show that traces with the same endpoint and similar entropy profiles can exhibit substantially different reasoning dynamics. We further find substantial variation in these dynamics both within and across models and tasks, with different objectives favoring different dynamical profiles. Additionally, we show that training and inference choices systematically reshape these profiles. Our results suggest that answer-distribution trajectories provide a rich framework for analysing and evaluating the dynamics of LLM reasoning.

1 Introduction

Conventional endpoint and entropy summaries overlook how competing answer hypotheses evolve during chain-of-thought reasoning. Answer-distribution trajectories preserve these dynamics and support analyses of reasoning profiles, mechanisms, and the effects of objectives and model choices.

Motivation: Chain-of-thought traces expose intermediate computation, but reasoning models are still mostly evaluated by final-answer correctness.This endpoint focus does not account for the reasoning path.
Motivation: Entropy profiles track uncertainty over reasoning or final answers but do not identify which hypotheses carry the probability mass.They therefore provide an incomplete view of competing-answer dynamics.
Approach: Answer-distribution trajectories track predictive distributions over possible final answers as reasoning unfolds, preserving supported hypotheses and their evolving support.The framework estimates distributions at selected prefixes by sampling independent continuations conditioned on generated reasoning.
Framework: The framework organizes trajectory metrics into exploration, revision, motion, and commitment, while tracking correct-answer dominance to distinguish success and failure mechanisms.These profiles are applied across open-weight models and four discrete-answer benchmarks.
Research questions: The study asks whether conventional summaries miss distinct trajectories, whether models differ systematically, and how training scale and decoding temperature reshape dynamics.It reports differences within and across model families and tasks, with objectives favoring different profiles.

2 Related work

Related work studies reasoning through observable chain-of-thought traces, internal representations, and information-theoretic signals. This paper complements those approaches by tracking the evolution of predictive distributions over candidate final answers.

Observable reasoning traces: Chain-of-thought exposes intermediate natural-language steps that can be monitored for behavior, length, cross-chain agreement, and local reasoning validity.These signals extend evaluation beyond the final answer.
Internal computations: Chain-of-thought may not reflect the actual reasons determining a model’s final answer, motivating analysis of internal computations and representations.Prior work probes or intervenes on hidden representations related to truthfulness, hallucination, future outputs, and latent reasoning concepts.
Information-theoretic views: Information-theoretic methods use information gain, mutual information, token entropy, and temporal uncertainty patterns to identify important reasoning points and guide exploration or stopping.Entropy-based signals have also been used to compress redundant reasoning steps.

3 Answer-distribution trajectories: a stochastic-dynamics view of reasoning

The paper defines answer-distribution trajectories as sequences of prefix-conditioned predictive distributions over discrete final answers. It proves that these trajectories retain strictly more information than endpoint predictions or entropy trajectories because those summaries cannot uniquely recover the underlying dynamics.

Formal setup: The framework formalizes reasoning models as generating intermediate reasoning tokens followed by answer sequences, then parses answers into discrete labels.The empirical parser deterministically maps each generated answer sequence to one label in the discrete answer space.
Prefix-conditioned distributions: For a reasoning prefix, the prefix-conditioned predictive distribution gives probabilities for discrete answer labels while marginalizing over all future reasoning continuations.This distribution conditions on the observed prefix and averages over possible continuations.
Answer-distribution trajectories: An answer-distribution trajectory is the sequence of prefix-conditioned answer distributions formed as a realized reasoning trace unfolds.The trajectory includes distributions associated with successive reasoning prefixes.
Representation relationships: Endpoint prediction and entropy trajectory are deterministic functions of the full answer-distribution trajectory.Thus, the full trajectory can generate both conventional summaries.
Representation relationships: Neither endpoint prediction nor entropy trajectory identifies the underlying predictive dynamics, because distinct trajectories can share either summary.Endpoint non-injectivity concerns the sequence of predictive states, while entropy non-injectivity holds when the answer space has at least two labels.
Representation relationships: Answer-distribution trajectories are strictly finer because they retain both hypothesis identity and the movement of probability mass between hypotheses.Endpoint summaries preserve destination, whereas entropy preserves concentration over time without hypothesis identity.

4 Trajectory-level diagnostics

Trajectory-level diagnostics represent reasoning as evolving predictive distributions, measuring exploration, revision, motion, and commitment alongside gold-answer dominance. These profiles distinguish success and failure mechanisms that endpoint accuracy alone cannot identify, including cases where continued reasoning moves away from a temporarily correct state.

Dynamical profiles: Dynamical profiles provide a common representation for comparing reasoning behavior across models and tasks.They combine trajectory-level metrics with the evolution of the gold answer to characterize reasoning dynamics and success or failure mechanisms.
Exploration: Exploration measures how many answer hypotheses remain plausible, using effective support derived from the predictive distribution.Seff is interpreted as the effective number of plausible answers after each number of CoT tokens; larger values indicate broader competition.
Revision: Revision measures changes in the dominant answer set and whether previously considered hypotheses recur.Dominant answers are defined as probability-maximizing answers, while switching and recurrence capture changes and returns during reasoning.
Motion: Motion measures how much the predictive distribution changes, including cumulative movement, temporal concentration, and directness versus backtracking.Total variation quantifies consecutive distributional change; higher Rdirect indicates more direct trajectories, while lower values indicate greater backtracking.
Commitment: Commitment measures when a hypothesis becomes confident and stable, with small tcommit indicating early lock-in and large values indicating persistent competition.Commitment is undefined when no hypothesis remains above the selected threshold through the trajectory’s end.
Success and failure mechanisms: Gold-answer dominance separates correct traces into stable success, rescue, detour success, and fragile success, and incorrect traces into four complementary failure mechanisms.Escape and failed rescue capture cases where continued reasoning moves away from a state in which the correct answer was dominant; detour success captures temporary degradation.

5 Results

Across four datasets and sixteen models, the experiments show that endpoint accuracy and entropy profiles can conceal distinct answer-distribution dynamics. These dynamics vary within and across models and tasks, different objectives favor different profiles, and training or inference choices systematically reshape them.

The evaluation tests whether endpoint or entropy summaries can hide different answer-distribution dynamics, how dynamics vary, and how interventions reshape them.
5.1 Coarse summaries collapse distinct reasoning dynamics: Similar endpoint accuracy can accompany markedly different mixtures of reasoning success and failure mechanisms across models.
5.1 Coarse summaries collapse distinct reasoning dynamics: Similar entropy profiles can coexist with disagreement over dominant answers, gold-answer dominance, and success or failure mechanisms.
5.2 Reasoning dynamics vary within and across models and tasks: Trajectory-level metrics show substantial variation within individual model-task conditions, with within-condition variance largest for every metric.
5.2 Reasoning dynamics vary within and across models and tasks: High-accuracy pairs favor narrower support, fewer switches, less movement, and earlier commitment, whereas shortest-CoT traces show the opposite pattern.
5.2 Reasoning dynamics vary within and across models and tasks: Dynamical profiles turn qualitative reasoning behaviors into measurable coordinates for evaluating suitability under a particular model, task, and objective.
5.3 Training and inference choices reshape reasoning dynamics: Instruction tuning shifts profiles toward narrower support, fewer switches, less movement, earlier commitment, and higher accuracy; temperature broadens support, increases movement, and delays commitment.

6 Conclusion and Open Questions

The paper presents answer-distribution trajectories as a richer framework for characterizing how predictive distributions evolve during chain-of-thought reasoning. It finds variation across traces, models, and tasks, while noting computational, interpretive, and practical open questions.

Answer-distribution trajectories preserve which final-answer hypotheses are supported and how probability mass moves during reasoning.
The framework combines exploration, revision, motion, and commitment measures into dynamical profiles across sixteen models and four benchmarks.
The analysis finds variation in how answers are reached across and within model-task pairs, with objectives favoring different profiles and interventions reshaping them.
Open questions: Estimating answer-distribution trajectories is computationally expensive, and the training and inference comparisons are descriptive rather than controlled.
Open questions: Whether trajectory information yields practical gains remains open, including for error prediction, adaptive test-time compute, and reasoning-model optimization.

A.1 Tasks and datasets

The evaluation covers four discrete-answer reasoning datasets and sixteen open-weight language models spanning base, instruction-tuned, specialized, chat, and reinforcement-learning regimes. Questions are sampled into stochastic reasoning trajectories, whose answer distributions are estimated from independent continuations.

Tasks and datasets: The task suite comprises GSM8K, ARC, SVAMP, and algebra-track MATH, all evaluated on official test splits with deterministic answer parsing.
Models: The model set contains sixteen open-weight LLMs spanning base, instruction-tuned, SFT-trained, chat, and reasoning-specialized RL regimes.
Trajectory sampling: For each question, the study samples M independent reasoning trajectories under fixed stochastic decoding and treats each trajectory as one realization of reasoning.
Answer-distribution estimation: Prefix-conditioned answer distributions are approximated by Monte Carlo continuations mapped to discrete labels, including a null category for invalid outputs.
Answer-distribution estimation: Continuation rollouts use fixed sampling parameters and typically N = 16 independent continuations per prefix.

A.5 Checkpointed prefix evaluation

Because evaluating conditional answer entropy at every token is expensive, the analysis uses uniformly spaced prefix checkpoints and always includes the trajectory’s final prefix.

Conditional answer entropy is evaluated at checkpoint positions rather than every token to reduce computational cost.
The checkpoint set J contains uniformly spaced positions with stride s = 16.
The evaluation always includes the final prefix length K, while position 0 represents the empty prefix.

B Reproduction details for figures and tables

The reproduction details define the datasets, matched comparisons, metric standardization, aggregation, and figure conventions used to analyze trajectory dynamics across models and tasks.

Common trace processing: The analyses use ARC, GSM8K, MATH, and SVAMP crossed with sixteen paper models, with answer parsing and correctness evaluation following Section A.1.
Trajectory metrics: Trajectory metrics are computed from observed answer-distribution states at checkpoints, with undefined path-length and commitment cases omitted from summaries.Commitment uses threshold τ = 0.8.
Success and failure mechanisms: Figure 2 separates traces by endpoint correctness and assigns success or failure mechanisms using gold-answer dominance over the trajectory.Gold-answer dominance includes ties among probability-maximizing answers.
Trace disagreement: Cross-model comparisons record disagreement in dominant sets, gold-dominance indicators, and success or failure mechanisms, with mechanism disagreement restricted to matched realized outcomes.
Group definitions: The High accuracy group is the 25% of model-task pairs with the largest within-task standardized accuracy, while profile entries average standardized trajectory metrics within each selected group.The final two columns retain unstandardized mean endpoint accuracy and CoT length for interpretability.
Matched comparisons: Matched comparisons cover instruction tuning and scale across four tasks and temperature on GSM8K, with raw changes standardized by default-decoding metric variability before aggregation.Instruction-tuning and scale effects average across matched model pairs; temperature comparisons use T = 0.2 versus T = 1.0.
Figure conventions: Diamonds show arithmetic means, error bars show two-sided 95% Student-t confidence intervals, and the three panels share a symmetric x-axis for visual effect comparison.The panels use n = 6 for instruction tuning and scale, and n = 4 for temperature.

C Proofs

The proofs establish that endpoint predictions and entropy profiles are determined by trajectories but are not sufficient to identify the underlying distribution dynamics.

Proposition 1: The endpoint prediction depends only on the terminal state, while the entropy profile depends on the corresponding state at each checkpoint.
Endpoint insufficiency: Two binary-answer trajectories can share the same endpoint while differing in whether the dominant hypothesis remains fixed or is revised.The example has E(T1) = E(T2) = a, although T2 switches from b to a.
Entropy insufficiency: Two constant trajectories can have identical entropy profiles while assigning predictive mass to different hypotheses, because entropy is invariant under coordinate permutation.Thus H(T1) = H(T2) even though T1 ≠ T2.