PaperlayerView exact revision on arXiv

Source-linked AI summary

A*-Thought-V2: Efficient Latent Reasoning via Geometric Dynamics of LLM

Xiaoang Xu, Siyuan Liu, Shuo Wang, Junlan Feng, Fanyu Meng, Zhu Zhang, Jixun Wang, Xiaorong Wang, Zihan Zhou, Xin Li, Chaojun Xiao, Yiming Zhang, Huijia Wu, Liuyu Xiang, Peipei Li, Zhaofeng He

arXiv:2609.07821v1cs.CLcs.AIcs.LG

TL;DR

Chain-of-Thought can improve LLM reasoning but incurs computation and context costs, while existing compression lacks an effective way to select explicit steps and preserve the rest. A*-Thought-V2 uses hidden-state trajectory geometry to retain aligned steps as text and encode deviating steps as latent representations. Across six benchmarks, it improves average accuracy by up to 2.6 percentage points over same-data SFT; on Qwen3.6-27B, its 90° variant nearly halves response length and raises ACU from 0.35 to 0.80 (2.29×).

  • Problem

    CoT compression lacks an effective way to select steps for explicit retention while preserving the information in other steps, despite CoT’s computational and context costs.

    Source: §1 · p. 2

  • Method

    A*-Thought-V2 models CoT as a hidden-state trajectory, retaining solution-aligned steps as text and encoding deviating steps in an interleaved latent representation.

    Source: §1 · p. 2, §2 · p. 3

  • Results

    Across six benchmarks, average accuracy improves by up to 2.6 percentage points over same-data SFT; on Qwen3.6-27B, the 90° variant nearly halves response length and raises ACU from 0.35 to 0.80 (2.29×).

    Source: §4.2 · pp. 6–7, §7 · p. 9

  • Takeaways & Limitations

    Directional-angle patterns distinguish recurring reasoning stages and provide a basis for controllable data selection and compression.

    Source: §1 · p. 2

Abstract

from arXiv · show

Chain-of-Thought (CoT) improves the reasoning ability of Large Language Models (LLMs) but incurs substantial computation and context costs. Existing methods either lose intermediate information through hard pruning or lack a principled criterion for continuous compression. We present A*-Thought-V2, a geometric dynamics of LLM guided framework that models CoT as a hidden-state trajectory and replaces hard deletion with an explicit-implicit interleaved latent architecture. After projecting question, step, and solution representations into a 3D PCA space, it measures alignment between each local transition and global question-to-solution direction. Aligned steps remain explicit text, whereas deviating steps are compressed into continuous latent tokens. Directional angles capture both local semantics and reasoning dynamics: small angles indicate direct execution and answer formation, while large angles more frequently involve checking, correction, and branch exploration; their temporal variation reveals exploration, convergence, and refinement stages. To train this architecture, we introduce stepwise embedding forcing, which pools each redundant step into a single latent embedding, and label forcing, which supervises that latent token with a soft multi-modal vocabulary distribution instead of a hard one-hot label. Experiments on Qwen3.5-9B and Qwen3.6-27B across six in-domain and out-of-domain benchmarks show that A*-Thought-V2 improves average accuracy by up to 2.6% while reducing response length by up to half, increasing Accuracy per Computation Unit by 2.29$\times$, and reducing preprocessing and training time by 94.6% and up to 80.3%, respectively. Representation analyses suggest that latent states form a compact region distinct from textual states, while higher entropy at latent-token positions reflects broader soft targets that encourage richer step-level feature learning.

1 INTRODUCTION

A*-Thought-V2 uses hidden-state geometry to allocate explicit text and latent tokens, aiming to compress Chain-of-Thought while preserving reasoning information. Its introduction reports improved accuracy and computation efficiency alongside reduced compression time.

1 INTRODUCTION

  • CoT improves LLM reasoning but carries computation and context costs; hard compression may discard useful steps, while latent reasoning lacks an effective mechanism for selecting explicit steps and preserving remaining information.A*-Thought-V2 models reasoning as a hidden-state trajectory and encodes redundant steps in latent space, unlike A*-Thought, which directly discards them Xu et al. (2025).Source: §1 · p. 2
  • Six 30° directional-angle intervals capture semantic tendencies from direct execution to correction and branch reconsideration, while temporal variation identifies exploration, convergence, and refinement.These geometric dynamics guide controllable allocation of explicit and latent steps.Source: §1 · p. 2
  • Embedding Forcing pools compressed spans into latent embeddings, while Label Forcing supervises latent positions with soft vocabulary distributions alongside standard text loss.The interleaved architecture replaces hard deletion with continuous latent representations intended to preserve compressed reasoning semantics.Source: §1 · p. 2, Figure 2 · p. 2
  • A*-Thought-V2 improves average accuracy by up to 2.6% and ACU by 2.29× over the baseline, while reducing compression time by 94.6% over A*-Thought Xu et al. (2025).The experiments span two model scales and six benchmarks.Source: §1 · p. 2

2 PRELIMINARIES

A*-Thought-V2 replaces hard deletion of selected CoT steps with latent representations and models reasoning as a trajectory in PCA-projected hidden-state space. It uses local-to-global directional alignment to characterize the semantics and temporal stages of reasoning.

2 PRELIMINARIES

  • Unlike hard-compression methods, A*-Thought-V2 encodes steps with flag(n) = 0 as latent representations to retain more reasoning information at higher density, modeling CoT as a PCA-projected trajectory.A*-Thought Xu et al. (2025) searches for a binary flag sequence in a two-dimensional tree structure.Source: §2 · p. 3
  • The method projects question, step, and solution hidden states into 3D, then measures each local transition’s angle to the global question-to-solution direction.The global direction is z0 = hs − hq; local directions are z1 = ht(1) − hq and zn = ht(n) − ht(n−1) for n = 2, . . . , T.Source: §2 · p. 3
  • Small-angle transitions tend toward direct reduction and answer formation, while large-angle transitions more often involve checking, correction, reinterpretation, or branch reconsideration; angle variation marks exploration, convergence, and refinement.Figure 3 illustrates the trajectory, semantic tendencies across angle intervals, and the three stages reflected by step-wise angle variation.Source: §2 · p. 3, Figure 3 · p. 3

3 METHODOLOGY

A*-Thought-V2 compresses redundant reasoning steps into continuous latent representations, supervises them with step-level soft targets, and uses a corresponding latent-generation procedure at inference.

3.1 LATENT ARCHITECTURE

  • A*-Thought-V2 maps each redundant reasoning step to one segmented-mean-pooled latent vector, concatenates consecutive vectors, and interleaves them into the reasoning path.This replaces lengthy redundant text with dense latent representations intended to preserve heuristic information while reducing context-window occupancy.Source: §3.1 · p. 4
  • For each redundant step, label forcing averages its tokens’ one-hot targets into a multi-modal soft vocabulary distribution for the corresponding latent token.The overall objective combines hard-label loss for text tokens with soft-label loss for latent tokens, scaled by a structural weight λ.Source: §3.1 · pp. 4–5

3.2 INFERENCE

  • At inference, discrete placeholders serialize latent positions that use continuous states; generation retains the preceding KV cache, starts from the boundary-tag hidden state, and resumes text after the latent span.During training, the latent input instead uses the pooled embedding of the corresponding compressed step.Source: §3.2 · p. 5

4 EXPERIMENTS

Across two Qwen model scales and six in- and out-of-domain benchmarks, A*-Thought-V2 improves accuracy and efficiency over same-data SFT. The experiments compare 60° and 90° compression thresholds, with 90° offering the best overall trade-off.

4.1 SETUP

  • Experiments use Qwen3.5-9B and Qwen3.6-27B, trainable variants use OpenR1-Math-3k, and evaluation covers four math and two out-of-domain benchmarks.Metrics are accuracy, response length, and Accuracy per Computation Unit (ACU).Source: §4.1 · pp. 5–6

4.2 MAIN RESULTS

  • A*-Thought-V2 gains up to 2.6 percentage points average accuracy over same-data SFT; its Qwen3.6-27B 90° variant raises ACU from 0.35 to 0.80 (2.29× vs. backbone).Similar gains are observed on Qwen3.5-9B.Source: §4.2 · p. 6, Table 1 · p. 6
  • On Qwen3.6-27B, the 90° variant gains 1.0 average-accuracy points and cuts response length by 16.0% versus SFT, while surpassing SwiReasoning, CopT, and A*-Thought in average accuracy and ACU.Across in- and out-of-domain benchmarks, both 60° and 90° improve average accuracy and shorten responses relative to same-data SFT.Source: §4.2 · p. 7
  • The 90° threshold preserves forward-aligned steps and compresses checking, correction, and exploration, while 60° additionally compresses mixed execution and transition.The 90° threshold provides the best overall trade-off across evaluated benchmarks.Source: §4.2 · p. 7

5 ANALYSIS

The analysis supports geometry-guided explicit–latent allocation and the combined use of Embedding Forcing and Label Forcing. It also finds that latent tokens form distinct, structured representations, and examines their entropy, length, training dynamics, and efficiency.

5.1 ABLATION ON STEP SELECTION AND LATENT ARCHITECTURE

  • Geometry-guided selection achieves higher average accuracy and ACU than random-angle and reversed selection while reducing average generation length by 7.54% and 12.91%, respectively.Compression rates were 48.09% for geometry-guided selection, 48.16% for random-angle selection, and 47.15% for reversed selection. Removing EF, LF, or both reduces average accuracy to 92.8%, 72.5%, and 61.7%, respectively, supporting their combined use.Source: §5.1 · p. 7, Table 2 · p. 7

5.2 PCA VISUALIZATION OF LATENT TOKENS

  • Latent tokens form a compact cluster distinct from text-token states in 2D and a coherent intermediate region along the reasoning trajectory in 3D.The authors suggest this structure reflects compressed reasoning information.Source: §5.2 · p. 7, Figure 4 · p. 7

5.3 TOKEN ENTROPY ANALYSIS UNDER LABEL FORCING

  • Predictive entropy is higher at latent-token than explicit-text positions across both model scales, primarily because latent tokens are supervised with broader soft vocabulary targets.Explicit tokens use one-hot targets, while latent-token targets are soft vocabulary distributions aggregated from compressed-step tokens.Source: §5.3 · p. 8, Figure 5 · p. 8

5.4 EFFECT OF MAXIMUM LATENT LENGTH

  • A larger maximum latent-length cap generally yields higher accuracy with shorter responses, and decoded latent-length distributions closely match the training data.These findings suggest compact encoding of intermediate steps and learning of the latent reasoning format.Source: §5.4 · p. 8, Figure 6 · p. 8

5.5 TRAINING DYNAMICS

  • Although A*-Thought-V2 begins with higher training loss, it converges rapidly to the lowest final loss at both model scales.The authors attribute the initial loss to learning latent reasoning segments and suggest solution-oriented latent compression provides a more effective post-convergence training signal.Source: §5.5 · p. 8, Figure 7 · p. 8

5.6 COMPRESSION AND TRAINING EFFICIENCY

  • A*-Thought-V2 reduces compression time from 5:16:22 to 0:16:57 (94.6%); the 60° variant reaches a 31.67% compression rate and cuts training time by 80.3% and 68.7% across the two model scales.The 90° variant retains more reasoning tokens while still providing substantial training acceleration.Source: §5.6 · p. 9, Table 3 · p. 9

6 RELATED WORK

The related work frames A*-Thought-V2 as combining information-preserving compression with explicit–latent reasoning, using trajectory-based step allocation and stepwise embedding and label forcing.

Efficient Reasoning

  • A*-Thought-V2 encodes redundant steps as dense latent representations interleaved with retained text, addressing useful information potentially lost by hard-pruning methods such as TokenSkip Xia et al. (2025) and A*-Thought Xu et al. (2025).TokenSkip prunes low-importance tokens; A*-Thought combines bidirectional importance scoring with A* search to identify compact reasoning paths.Source: §6 · p. 9

Latent Reasoning

  • Recent latent-reasoning approaches include SwiReasoning Shi et al. (2026a), which switches between textual and latent reasoning, and CopT Shi et al. (2026b), which uses continuous-space verifiers for on-policy reflection and correction.These methods are presented among work exploring continuous and implicit reasoning to address the computational cost of lengthy discrete CoT sequences.Source: §6 · p. 9
  • A*-Thought-V2 complements latent-reasoning paradigms with a trajectory-based criterion for explicit–latent step allocation and step-wise embedding and label forcing.The approach is described as enabling dynamic, variable-length, information-preserving compression.Source: §6 · p. 9

7 CONCLUSION · A A*-THOUGHT-V2 ALGORITHM DETAIL · B GEOMETRIC AND DIRECTIONAL DYNAMICS ANALYSIS OF THE LLM REASONING REPRESENTATION TRAJECTORY

A*-Thought-V2 compresses chain-of-thought through explicit–implicit interleaving guided by a PCA-projected reasoning trajectory, retaining solution-aligned steps as text and encoding deviating steps as latent representations. Its reported results combine improved accuracy and computation efficiency with analyses of directional dynamics and latent-state learning.

7 CONCLUSION

  • Across six benchmarks, A*-Thought-V2 improves average accuracy by up to 2.6% over same-data SFT; the 90° variant nearly halves response length, raises ACU by up to 2.29×, cuts preprocessing time by 94.6% over A*-Thought, and reduces training time by up to 80.3% over standard SFT.The conclusion also reports that ablations support complementary roles for step-wise embedding forcing and label forcing.Source: §7 · p. 9
  • The method models reasoning as a PCA-projected hidden-state trajectory, keeping solution-aligned steps explicit and compressing deviating steps into latent representations.Step-wise embedding forcing and label forcing supervise latent learning with continuous embeddings and soft vocabulary targets.Source: §7 · p. 9
  • Directional-angle analysis links six intervals to recurring lexical patterns and characterizes reasoning stages as exploration, convergence, and refinement.The conclusion reports compact latent states distinct from textual states and higher predictive entropy associated with broader soft targets and richer step-level feature learning.Source: §7 · p. 9

A A*-THOUGHT-V2 ALGORITHM DETAIL

  • The algorithm extracts question, step, and solution hidden states, projects them into three dimensions with PCA, and defines the global direction from question to solution.It also specifies an angle threshold τ and numerical tolerance ϵ as inputs.Source: A A*-THOUGHT-V2 ALGORITHM DETAIL · p. 14
  • For each step, the algorithm compares its local transition with the global direction and retains the step when the angle is at most τ; it retains the step if the angle is undefined.The first transition runs from the question representation to the first step, while later transitions connect consecutive steps.Source: A A*-THOUGHT-V2 ALGORITHM DETAIL · p. 14
  • The algorithm constructs an explicit–implicit sequence by bracketing compressed step spans with latent boundary tags and retaining uncompressed steps as tokenized text.It returns the resulting embeddings, aligned hard or soft targets, and the step-wise compression decisions.Source: A A*-THOUGHT-V2 ALGORITHM DETAIL · p. 14

B.1 ANGLE THRESHOLD AND REASONING SEMANTIC ANALYSIS · B.2 CASE STUDY OF STEP-WISE DIRECTIONAL DYNAMICS

The appendix describes how lexical patterns and step content vary across directional-angle intervals, contrasting lower-angle reduction with higher-angle checking and exploration. A case study shows this contrast across six consecutive intervals in one reasoning trajectory.

B.1 ANGLE THRESHOLD AND REASONING SEMANTIC ANALYSIS

  • Lower-angle intervals contain execution and reduction markers such as “hence,” “simplify,” “thus,” and “compute.”The paper presents these recurring lexical markers as a descriptive pattern across six predefined 30° intervals.Source: §B.1 · p. 15, Table 4 · p. 15
  • Higher-angle intervals contain checking, revision, and branch-exploration markers such as “wait,” “however,” “maybe,” and “wait is there.”These lexical patterns are presented descriptively, not as a causal relationship.Source: §B.1 · p. 15, Table 4 · p. 15
  • Figure 8 displays the six predefined 30° angle bins and their within-bin mean angles.The intervals are used to characterize how lexical patterns vary with trajectory direction.Source: §B.1 · p. 15, Figure 8 · p. 16

B.2 CASE STUDY OF STEP-WISE DIRECTIONAL DYNAMICS

  • Figure 9 expands six marked steps from one trajectory, with the steps covering consecutive 30° directional-angle intervals.The examples are selected from the same trajectory.Source: §B.2 · p. 16, Figure 9 · p. 16
  • In the case study, smaller-angle steps primarily perform direct algebraic reduction, while larger-angle steps more often check conditions, reinterpret, or explore alternatives.The comparison describes the content of these examples, not a universal rule for all reasoning steps.Source: §B.2 · p. 16, Figure 9 · p. 16
  • The case-study examples are ordered by angle interval rather than generation order, while their step indices retain their original trajectory positions.This preserves the distinction between angle-based presentation and the steps’ original order.Source: §B.2 · p. 16, Figure 9 · p. 16

C ANALYSIS · C.1 EFFECT OF COT-PCA EXTRACTOR SIZE

CoT-PCA is not highly sensitive to extractor scale: four Qwen3.5 extractor sizes yield no clear downstream performance gap, despite slightly different angle distributions. Qwen3.5-0.8B is the default for effectiveness and computational efficiency.

C.1 EFFECT OF COT-PCA EXTRACTOR SIZE

  • Downstream performance shows no clear gap across Qwen3.5 extractors from 0.8B to 9B parameters, indicating CoT-PCA is not highly sensitive to extractor scale.The comparison included 0.8B, 2B, 4B, and 9B extractors.Source: §C.1 · p. 17
  • Qwen3.5-0.8B is the default extractor, selected for effectiveness and computational efficiency.Source: §C.1 · p. 17
  • Figure 10 shows that extractor size produces slightly different angle distributions, affecting interpretation of the angle threshold.The figure compares directional-angle distributions from CoT-PCA extractors of different sizes.Source: Figure 10 · p. 17, §C.1 · p. 17

D HYPERPARAMETERS · E CASE STUDY OF THE A*-THOUGHT-V2 TRAJECTORIES · F ETHICS STATEMENT

The appendices report training and inference configurations, illustrate A*-Thought-V2 trajectories on two AIME examples, and state that the work introduces no new user-facing deployment while inheriting risks from its models and data.

D HYPERPARAMETERS

  • Tables 5 and 6 report the training and inference configurations; backbone models share optimization settings, while method-specific hyperparameters are configured separately.Source: D HYPERPARAMETERS · p. 18, Table 5 · p. 18, Table 6 · p. 18
  • The latent weight λ is tuned by grid search in 0.1 increments while monitoring latent-token generation across benchmarks.Source: D HYPERPARAMETERS · p. 18
  • A small λ may fail to activate latent generation, while a large λ may yield excessive latent tokens and overfitting, so the smallest reliably activating value is selected.Source: D HYPERPARAMETERS · p. 18

E CASE STUDY OF THE A*-THOUGHT-V2 TRAJECTORIES

  • Figure 11 presents A*-Thought-V2-Qwen3.5-9B on AIME2026-ID25, retaining representative explicit reasoning before ellipses and highlighting all original latent markers in purple.Source: Figure 11 · p. 19
  • Figure 12 presents A*-Thought-V2-Qwen3.6-27B on AIME2024-ID68, retaining representative explicit reasoning before ellipses and highlighting all original latent markers in purple.Source: Figure 12 · p. 20

F ETHICS STATEMENT

  • The work studies computational efficiency of language-model reasoning without introducing a new user-facing deployment.The method inherits factuality, bias, privacy, and misuse risks from its underlying models and training data.Source: F ETHICS STATEMENT · p. 21

G REPRODUCIBILITY STATEMENT · H THE USE OF LARGE LANGUAGE MODELS

The paper reports its experimental configurations and measurement coverage, and states that an LLM assisted only with language polishing.

G REPRODUCIBILITY STATEMENT

  • The reproducibility statement specifies the method, inference, evaluation, hardware, and hyperparameter settings, with Tables 5 and 6 reporting experimental configurations.Main results and ablations report accuracy, response length, ACU, preprocessing time, and training time.Source: G REPRODUCIBILITY STATEMENT · p. 21

H THE USE OF LARGE LANGUAGE MODELS

Related papers

Loading 2609.07821v1…