PaperlayerView exact revision on arXiv

Source-linked AI summary

A*-Thought-V2: Efficient Latent Reasoning via Geometric Dynamics of LLM

Xiaoang Xu, Siyuan Liu, Shuo Wang, Junlan Feng, Fanyu Meng, Zhu Zhang, Jixun Wang, Xiaorong Wang, Zihan Zhou, Xin Li, Chaojun Xiao, Yiming Zhang, Huijia Wu, Liuyu Xiang, Peipei Li, Zhaofeng He

arXiv:2609.07821v1cs.CLcs.AIcs.LG

TL;DR

CoT improves LLM reasoning but is costly, and existing compression methods either discard useful intermediate steps or lack a principled continuous-compression criterion. A*-Thought-V2 models reasoning geometrically and interleaves explicit text with latent tokens, using embedding and label forcing to preserve compressed reasoning. Across six benchmarks and two model scales, it improves accuracy while shortening responses and reducing computation and training costs.

  • Problem

    Existing CoT compression either hard-prunes intermediate steps that may contain useful information or lacks an effective mechanism for selecting explicit steps while preserving the rest.

  • Method

    A*-Thought-V2 models CoT as a PCA-projected hidden-state trajectory, retains aligned steps as text, compresses deviating steps into latent representations, and trains them with embedding and label forcing.

  • Results

    Across six benchmarks and two model scales, A*-Thought-V2 improves average accuracy by up to 2.6% and ACU by up to 2.29× while reducing compression time by 94.6%.

  • Takeaways & Limitations

    The framework provides controllable explicit–latent CoT compression that preserves reasoning information while reducing context use and improving efficiency.

Abstract

from arXiv · show

Chain-of-Thought (CoT) improves the reasoning ability of Large Language Models (LLMs) but incurs substantial computation and context costs. Existing methods either lose intermediate information through hard pruning or lack a principled criterion for continuous compression. We present A*-Thought-V2, a geometric dynamics of LLM guided framework that models CoT as a hidden-state trajectory and replaces hard deletion with an explicit-implicit interleaved latent architecture. After projecting question, step, and solution representations into a 3D PCA space, it measures alignment between each local transition and global question-to-solution direction. Aligned steps remain explicit text, whereas deviating steps are compressed into continuous latent tokens. Directional angles capture both local semantics and reasoning dynamics: small angles indicate direct execution and answer formation, while large angles more frequently involve checking, correction, and branch exploration; their temporal variation reveals exploration, convergence, and refinement stages. To train this architecture, we introduce stepwise embedding forcing, which pools each redundant step into a single latent embedding, and label forcing, which supervises that latent token with a soft multi-modal vocabulary distribution instead of a hard one-hot label. Experiments on Qwen3.5-9B and Qwen3.6-27B across six in-domain and out-of-domain benchmarks show that A*-Thought-V2 improves average accuracy by up to 2.6% while reducing response length by up to half, increasing Accuracy per Computation Unit by 2.29$\times$, and reducing preprocessing and training time by 94.6% and up to 80.3%, respectively. Representation analyses suggest that latent states form a compact region distinct from textual states, while higher entropy at latent-token positions reflects broader soft targets that encourage richer step-level feature learning.

1 INTRODUCTION

A*-Thought-V2 addresses information loss in hard CoT compression by using geometric dynamics to allocate reasoning steps between explicit text and continuous latent representations. Its interleaved architecture combines embedding and label forcing, improving accuracy and efficiency across models and benchmarks.

  • CoT improves LLM reasoning but incurs substantial computation and context costs, while hard compression may discard useful intermediate information.
  • A*-Thought-V2 models reasoning as a hidden-state trajectory and uses geometric alignment to guide explicit–latent step allocation.Question, intermediate steps, and solution representations are projected into three-dimensional PCA space.
  • Embedding Forcing converts compressed spans into pooled latent embeddings, while Label Forcing supervises latent positions with soft vocabulary distributions.The latent and standard text losses are jointly optimized.
  • Across two model scales and six benchmarks, A*-Thought-V2 improves average accuracy by up to 2.6% and ACU by 2.29× over the baseline.It also reduces compression time by 94.6% over A*-Thought.

2 PRELIMINARIES

A*-Thought-V2 represents CoT as a PCA-projected hidden-state trajectory and measures how each local transition aligns with the global question-to-solution direction. Angle ranges capture semantic tendencies, while temporal angle variation identifies exploration, convergence, and refinement.

  • A*-Thought-V2 defines the global direction from question to solution and measures each local transition’s included angle against it.The trajectory includes the question, reasoning steps, and solution after three-dimensional PCA projection.
  • Small-angle transitions mainly indicate direct reduction and answer formation, whereas large-angle transitions more frequently involve checking, correction, reinterpretation, and branch reconsideration.
  • Temporal variation in directional angles reveals exploration with strong oscillations, convergence with decreasing variation, and refinement stages.

3 METHODOLOGY

A*-Thought-V2 compresses redundant reasoning into interleaved continuous latent representations while retaining explicit text, using embedding and label forcing to train the mixed sequence.

  • Latent architecture: Redundant reasoning steps are pooled into single continuous latent embeddings and interleaved with retained text instead of being discarded.The latent span replaces the original embeddings of redundant tokens while preserving the reasoning path structure.
  • Latent architecture: Embedding forcing concatenates the question, interleaved reasoning trajectory, and solution into a teacher-forced training sequence.The resulting sequence contains both standard text embeddings and step-level latent representations.
  • Training objectives: Label forcing maps every redundant step to a multimodal soft vocabulary target rather than a deterministic one-hot label.The target averages one-hot vectors across all tokens in the compressed step, allowing one latent token to represent its macro-semantics.
  • Training objectives: The mixed objective combines hard-label loss for text tokens with structurally weighted soft-label loss for latent tokens.The structural weight λ emphasizes dense conceptual compression and adherence to latent boundaries.
  • Inference: During latent sampling, training uses pooled step embeddings while inference uses the preceding boundary hidden state and retains the preceding context through the KV cache.After the latent span ends, generation resumes with standard text tokens.

4 EXPERIMENTS

Experiments evaluate A*-Thought-V2 across two model scales and six in-domain and out-of-domain benchmarks, using accuracy, response length, and ACU to assess effectiveness and efficiency. The method improves accuracy and shortens responses relative to supervised fine-tuning, with the 90° variant offering a strong overall trade-off.

  • 4.1 SETUP: Experiments use Qwen3.5-9B and Qwen3.6-27B with OpenR1-Math-3k training data and six in-domain and out-of-domain benchmarks.The benchmarks include Math500, AIME 2024–2026, ARC-Challenge, and GPQA-Diamond.
  • 4.1 SETUP: Accuracy measures correctness, Length measures generated tokens, and ACU measures the accuracy–efficiency trade-off as 100×Accuracy/Length.Higher ACU indicates better performance and efficiency trade-offs.
  • 4.2 MAIN RESULTS: Up to 2.6 percentage points higher average accuracy than same-data SFT is achieved across both model scales.The reported gain is averaged across the evaluated benchmarks.
  • 4.2 MAIN RESULTS: The 90° variant nearly halves response length and raises ACU from 0.35 to 0.80, a 2.29× improvement over the original Qwen3.6-27B backbone.On Qwen3.6-27B, it also improves average accuracy by 1.0 points and reduces response length by 16.0%.
  • 4.2 MAIN RESULTS: Across in-domain and out-of-domain benchmarks, 90° and 60° improve average accuracy and shorten responses relative to same-data SFT, with 90° providing the best overall trade-off.The 90° threshold retains forward-aligned steps while compressing checking, correction, and exploration.

5 ANALYSIS

Analysis studies step-selection ablations, latent representations, entropy, latent-length caps, training dynamics, and computational efficiency. The findings support geometry-guided allocation, complementary forcing mechanisms, compact latent states, and substantial preprocessing and training savings.

  • 5.1 ABLATION ON STEP SELECTION AND LATENT ARCHITECTURE: Geometry-guided selection achieves higher average accuracy and ACU than random-angle and reversed-selection variants while reducing average generation length by 7.54% and 12.91%, respectively.The three compression rates are 48.09%, 48.16%, and 47.15%.
  • 5.1 ABLATION ON STEP SELECTION AND LATENT ARCHITECTURE: Removing EF, LF, or both reduces average accuracy to 92.8%, 72.5%, and 61.7%, respectively, supporting complementary roles for the two mechanisms.The ablation particularly highlights Label Forcing’s importance.
  • 5.2 PCA VISUALIZATION OF LATENT TOKENS: Latent tokens form a compact cluster distinct from broader text-token states and occupy a coherent intermediate region along the reasoning trajectory.This pattern suggests structured compressed reasoning information in the latent states.
  • 5.3 TOKEN ENTROPY ANALYSIS UNDER LABEL FORCING: Latent-token positions show higher predictive entropy than explicit-text positions because latent tokens use broader soft vocabulary targets.Explicit tokens use one-hot targets, whereas latent targets aggregate compressed-step tokens.
  • 5.4 EFFECT OF MAXIMUM LATENT LENGTH: A larger maximum latent-length cap generally yields higher accuracy with shorter responses, while decoded latent-length distributions closely match training distributions.The distributional match indicates that the model learned the latent reasoning format.
  • 5.5 TRAINING DYNAMICS: A*-Thought-V2 starts with higher training loss but converges rapidly to the lowest final loss at both model scales.The result is attributed to learning latent reasoning segments and a more effective post-convergence training signal.
  • 5.6 COMPRESSION AND TRAINING EFFICIENCY: 94.6% lower compression time is achieved, falling from 5:16:22 to 0:16:57 relative to A*-Thought.The 60° variant also reduces training time by 80.3% for Qwen3.5-9B and 68.7% for Qwen3.6-27B.

6 RELATED WORK

Related work addresses reasoning efficiency through shorter explicit CoT trajectories, hard token or step pruning, and continuous latent reasoning. A*-Thought-V2 is situated at the intersection of these efficiency and latent-reasoning directions.

  • Efficient Reasoning: Recent efficient-reasoning methods shorten explicit CoT trajectories, while TokenSkip prunes low-importance tokens and A*-Thought searches for compact paths through hard compression.A*-Thought combines bidirectional importance scoring with A* search.
  • Latent Reasoning: Latent-reasoning research explores continuous and implicit alternatives to computationally expensive discrete token sequences.The cited work includes methods spanning continuous-space and implicit reasoning paradigms.

7 CONCLUSION

A*-Thought-V2 replaces hard CoT pruning with explicit–implicit interleaving guided by hidden-state geometry. Its angle-based decisions retain aligned steps as text, compress deviating steps into latent representations, and support efficient reasoning through embedding and label forcing.

  • 7 CONCLUSION: The algorithm extracts question, CoT-step, and solution hidden states, projects them with PCA, and computes angle-based compression decisions against a threshold.Undefined angles are retained, while steps with θ_n ≤ τ are retained explicitly.
  • 7 CONCLUSION: The explicit–implicit sequence construction inserts latent boundary tags around compressed spans and interleaves retained text tokens with latent representations.The construction returns interleaved embeddings, aligned hard or soft targets, and step-wise compression decisions.

B.1 ANGLE THRESHOLD AND REASONING SEMANTIC ANALYSIS

Directional-angle intervals provide a descriptive link between reasoning trajectory direction and lexical or semantic tendencies. Smaller-angle steps emphasize execution and reduction, while larger-angle steps more often involve checking, reinterpretation, and alternative branches.

  • Angle intervals: Six predefined 30° intervals organize recurring lexical markers by reasoning direction.Lower-angle bins contain execution and reduction markers such as “hence” and “compute,” whereas higher-angle bins contain checking and branch-exploration markers such as “wait” and “however.”
  • Step-wise dynamics: Smaller-angle steps primarily perform direct algebraic reduction, while larger-angle steps more frequently involve condition checking, reinterpretation, and alternative branches.The case study orders examples by angle interval while preserving their original trajectory step indices.
  • Examples: A direct-reduction example occupies the [0°, 30°) interval, while a routine derivation and checking example occupies [30°, 60°).The examples illustrate how semantic tendencies vary across consecutive directional-angle intervals.

C ANALYSIS

The analysis compares CoT-PCA extractors of different sizes and finds that extractor scale slightly changes angle distributions without producing a clear downstream performance gap.

  • Extractor comparison: Four Qwen3.5 extractors ranging from 0.8B to 9B parameters produce slightly different directional-angle distributions.These differences affect interpretation of the angle threshold.
  • Extractor comparison: Downstream performance shows no clear gap across extractor sizes, indicating limited sensitivity to extractor scale.The comparison concerns Qwen3.5 extractors with 0.8B, 2B, 4B, and 9B parameters.

D HYPERPARAMETERS

The paper reports shared backbone optimization settings alongside method-specific training and inference configurations, including a tuned latent-loss weight. The study also states that it targets computational efficiency rather than a new user-facing deployment and inherits risks from its underlying models and data.

  • Configurations: Training and inference configurations are reported separately, with shared backbone optimization settings and method-specific hyperparameters.Tables 5 and 6 provide the training and inference configurations.
  • Hyperparameters: The latent weight λ is selected as the smallest grid-searched value that reliably activates latent generation without excessive outputs.The grid-search step size is 0.1; small values may fail to activate latent generation, while large values may cause excessive latent tokens and overfitting.
  • Case studies: The method is evaluated through case studies on Qwen3.5-9B and Qwen3.6-27B outputs, including AIME examples with highlighted latent markers.The cited case studies show representative explicit reasoning and retained latent markers.
  • Scope and risks: The work studies computational efficiency and does not introduce a new user-facing deployment.It inherits factuality, bias, privacy, and misuse risks from its underlying models and training data.

G REPRODUCIBILITY STATEMENT

The paper specifies the configurations, evaluation measurements, and reproducibility-relevant components used in its experiments. It also states that language-model assistance was limited to language polishing and did not alter experimental content.

  • Reproducibility: The reproducibility statement identifies the trajectory criterion, compression threshold, objectives, inference switch, metrics, hardware, and hyperparameters.Tables 5 and 6 report configurations, while results and ablations provide accuracy, length, ACU, preprocessing-time, and training-time measurements.
  • Reproducibility: Language-model assistance was restricted to language polishing and did not generate or modify measurements, citations, figures, or tables.The statement distinguishes editorial assistance from experimental and reporting content.
Loading 2609.07821v1…