Source-linked AI summary
The Molecular Structure of Thought: Mapping the Topology of Long Chain-of-Thought Reasoning
Qiguang Chen, Yantao Du, Ziniu Li, Jinhao Liu, Songyao Duan, Jiarui Guo, Minghao Liu, Jiaheng Liu, Tong Yang, Ge Zhang, Libo Qin, Wanxiang Che, Wenhao Huang
TL;DR
LLMs can struggle to acquire effective Long CoT reasoning from human or non-Long-CoT model imitation. The paper models learnable trajectories as molecular-like structures and introduces Mole-Syn to synthesize them through behavioral distribution transfer. It reports improved Long CoT performance and RL stability across six benchmarks.
Problem
The paper asks how LLMs learn effective Long CoT, given that cold-starting from instruction-tuned or base models remains difficult.
Source: §1 · p. 2Method
The authors model Long CoT as three interacting behavior types and use Mole-Syn to transfer behavior-transition structure through controlled trajectory synthesis.
Source: §1 · pp. 2–3Results
Mole-Syn improves Long CoT performance and RL stability across six benchmarks.
Source: §1 · p. 3, §10 · p. 13Takeaways & Limitations
Task-equivalent traces can differ in behavior distributions and transitions, and mixing stable isomers from different teachers can destabilize learning.
Source: §1 · p. 2The analysis uses a limited set of teacher models and student backbones and focuses on offline distillation and supervised fine-tuning, leaving realistic online or interactive RL-like settings open.
Source: Limitations · p. 14
Abstract
from arXiv · showhide
Large language models (LLMs) often fail to learn effective long chain-of-thought (Long CoT) reasoning from human or non-Long-CoT LLMs imitation. To understand this, we propose that effective and learnable Long CoT trajectories feature stable molecular-like structures in unified view, which are formed by three interaction types: Deep-Reasoning (covalent-like), Self-Reflection (hydrogen-bond-like), and Self-Exploration (van der Waals-like). Analysis of distilled trajectories reveals these structures emerge from Long CoT fine-tuning, not keyword imitation. We introduce Effective Semantic Isomers and show that only bonds promoting fast entropy convergence support stable Long CoT learning, while structural competition impairs training. Drawing on these findings, we present Mole-Syn, a distribution-transfer-graph method that guides synthesis of effective Long CoT structures, boosting performance and RL stability across benchmarks.
1 Introduction
The introduction asks how LLMs learn effective Long CoT, noting that human or weak-model imitation may not reliably instill it. It frames reasoning as a stable molecular-like structure and proposes Mole-Syn to transfer behavioral structure for synthesis.
1 Introduction
- The authors ask how LLMs learn and represent effective Long CoT after reporting that human and instruction-model imitation may fail to instill it reliably.They note that such models can lose coherence in long trajectories or fail to transfer patterns to novel tasks.Source: §1 · p. 2
- Three recurring behaviors—Deep-Reasoning, Self-Reflection, and Self-Exploration—form bonds whose global arrangement is proposed to stabilize long-horizon reasoning.Deep-Reasoning forms local deduction clusters, Self-Reflection links back to prior steps, and Self-Exploration bridges distant clusters.Source: §1 · p. 2
- Long CoT trajectories solving the same tasks and visiting similar semantic regions can differ in behavior distributions and transitions; mixing stable isomers from different teachers can destabilize learning.The paper says this explains why heterogeneous traces can fail despite matched token statistics.Source: §1 · p. 2
- Mole-Syn estimates behavior-transition graphs from strong reasoning models, then transfers behavioral structure through controlled synthesis rather than copying teacher outputs.The paper reports gains in Long CoT performance and RL stability across six benchmarks.Source: §1 · p. 2
- The introduction positions structural synthesis as an alternative to methods such as summarization or reasoning compression, which can disrupt Long CoT structure.The supplied passage connects that disruption to limits on replicating internal reasoning processes.Source: §1 · p. 3
2 Preliminary: Cold-start LLMs for Long Chain-of-Thought
The section compares strong-reasoning-model distillation with weak-instruction-model ICL and human-trace fine-tuning as routes to Long CoT learning. Only the strong-model traces reliably support learning and retaining Long CoT structure.
2 Preliminary: Cold-start LLMs for Long Chain-of-Thought
- The comparison tests three sources: strong reasoning LLMs, weak instruction LLMs prompted with ICL, and human reasoning traces.The study fine-tunes models on human solutions for complex reasoning tasks.Source: §2 · p. 3
- Weak instruction-model ICL distillation sharply reduces performance and produces traces of only about 6–8 steps, without extended exploration and coherence.The authors characterize this as a limitation of ICL-based distillation rather than robust long-chain imitation.Source: §2 · p. 3
- Human-trace fine-tuning does not reproduce the Long CoT gains from strong-reasoning-model distillation.The authors suggest human solutions may support local problem solving without reliably encoding long-horizon reasoning distributions.Source: Figure 4 · p. 3, §2 · p. 4
- Only strong-reasoning-model distillation imparts and retains Long CoT structure, whereas weak-model ICL and human-trace fine-tuning yield limited gains.The authors identify high-quality reasoning exemplars as important for robust long-chain learning.Source: §2 · pp. 3–4
3 Hypothesis: Stable ‘‘Molecular Structure’’ in Long CoT
The section proposes that effective Long CoT is organized as a molecular-like structure of behavior-labeled reasoning steps and edges. Three interaction types describe local logical dependencies, long-range checks, and exploratory associations.
3 Hypothesis: Stable ‘‘Molecular Structure’’ in Long CoT
- The framework represents a Long CoT trace as a behavior-directed graph, with reasoning steps as nodes and behavior-annotated edges.For a corpus, it estimates behavior transitions between consecutive edges and the marginal behavior distribution.Source: §3 · p. 4
- Deep Reasoning acts like a covalent bond, encoding strong logical dependencies that preserve direction and continuity between steps.Breaking this logical backbone can destabilize subsequent steps and the answer.Source: §3 · p. 4
- Self-Reflection acts like a hydrogen bond: later steps test, revise, or reinforce earlier premises, linking distant points in a trace.These links constrain drift and hallucination and support a more self-consistent structure.Source: §3 · p. 4
- Self-Exploration resembles transient van der Waals interactions, enabling low-commitment associations that probe or combine concepts before stronger constraints apply.The passage describes this as supporting abductive and inductive moves in semantic space.Source: §3 · p. 4
4 Verification: Molecular Structure
Long CoT trajectories exhibit stable, shared bond distributions, and fine-tuning internalizes reasoning behavior rather than relying on keywords. Their folded topology and behavior-specific attention energies characterize how the reasoning structure is organized.
4.1 Stable Bond Distribution in Long CoT
- Transfer graphs correlate above 0.9 across models (p<0.001) for over 2,000 samples and exceed 0.95 across sampling sizes.The reported stability suggests effective Long CoT traces share robust reasoning topologies.Source: §4.1 · p. 4, Figure 5 · p. 5
4.2 SFT actually learns these bond structures rather than keywords.
- Without keywords or with arbitrary alternatives, models reach comparable performance given sufficient training, provided the underlying reasoning behaviors remain intact.Explicit keywords accelerate learning but are not essential.Source: §4.2 · pp. 5–6
4.3 ‘‘Logical Bonding-Folding’’ Structure
- 72.56% of deep-reasoning steps remain within distance 3, 81.72% of reflection steps reconnect to prior similar clusters, and exploration spans 5.32 on average.These patterns support a folded, domain-structured topology rather than a simple linear chain.Source: §4.3 · pp. 6–7
4.4 Attention ⇔Energy Level of Bonds
- Attention-energy distributions rank deep reasoning strongest, reflection intermediate, and exploration weakest, consistently across models.The analysis treats attention weights as a Gibbs–Boltzmann distribution over negative logits and uses reparameterized logits as energy.Source: §4.4 · p. 7
5 Feature: Effective Semantic Isomers
Semantic isomers are reasoning chains with similar concepts but different behavior distributions and transitions. Their effectiveness depends on bond structure and distribution alignment: small shifts or co-activation of distinct structures can destabilize learning and performance.
5 Feature: Effective Semantic Isomers
- Semantic isomers are trajectories that solve the same tasks and visit similar semantic regions but differ in behavior distributions and transitions.Multiple near-optimal isomers can exist for a task family, while mixing stable isomers from different teachers can destabilize learning despite matched token statistics.Source: §5 · p. 8
5.1 Bond structure of Semantic Isomers is the key to Long-CoT learning
- ICL gains emerge when demonstrations match a target reasoning-key distribution, yet chains correlated at 0.95 can still yield performance drops over 10%.Well-structured variants with distribution correlation around 0.9 show consistent gains, but small structural differences can make near-optimal isomers fragile.Source: §5.1 · p. 8
5.1.1 Not all bonds in isomers are effective.
- R1 models show accelerating informativeness in 76.1% of cases, while humans show forward information-gain changes below 0.1 in 81.3% of cases.R1 reasoning alternates between divergent exploration (slope > 0.6, Δentropy > 0.05) and convergent validation; human entropy profiles are more uniform.Source: §5.1.1 · p. 9
5.2 Conflict between Two Stable Structures
- Co-activating highly correlated (r ≈0.9) R1 and OSS chains produces fluctuating bond distributions, self-correlation no higher than 0.8, and lower performance than either chain alone.The result indicates that statistical similarity alone does not ensure structural compatibility.Source: §5.2 · pp. 9–10
6 Synthetic Chemistry: Synthesis Long CoT Molecules from Scratch
The section proposes generating Long CoT traces by transferring a teacher-derived behavior-transition structure through instruction LLMs rather than distilling teacher outputs. Mole-Syn-generated data approaches QwQ distillation performance, and Mole-Syn initialization is associated with steadier, sustained RL gains.
6 Synthetic Chemistry: Synthesis Long CoT Molecules from Scratch
- Mole-Syn uses a random walk over a transition-probability graph of four reasoning behaviors from stronger reasoning LLMs to synthesize targeted traces with instruction LLMs.The framework is presented as an alternative to obtaining traces through Long CoT distillation.Source: §6 · p. 10
- Mole-Syn-generated data achieves reasoning performance close to QwQ distillation, suggesting instruction-driven synthesis can induce useful structural regularities.The passage frames this as lower-cost behavior transfer.Source: §6 · p. 11
- Mole-Syn-initialized models outperform base-LLM-initialized models and retain gains over extended RL training.The reported fine-tuning gains are steadier, indicating stronger immediate reasoning and more reliable RL adaptation.Source: §6 · p. 11
- The paper reports that transition-based datasets match capable teachers’ behavior distributions without Long CoT distillation data and achieve stable convergence near distillation performance.The authors describe this as instruction-level synthesis at lower cost.Source: §6 · p. 11
7 Function: Shaping Function of Each Bond in Long CoT Structure
The section characterizes three reasoning behaviors as having distinct effects on semantic structure: deep reasoning densifies a backbone, exploration broadens the search space, and reflection folds and stabilizes the logic. The reported measurements associate these behaviors with changes in semantic-space volume or exploration extent.
7 Function: Shaping Function of Each Bond in Long CoT Structure
- 22%: Deep Reasoning decreases the smallest covering-ball volume in semantic space versus baseline, densifying the core logical structure.The authors say this builds the answer’s skeleton but does not yet ensure global stability or correctness.Source: §7 · p. 11
- 35.2 to 31.2: Self-Reflection reduces system volume while cross-linking distant nodes to test consistency and suppress inconsistent branches.The passage characterizes reflection as folding the logical chain rather than adding new deductive steps.Source: §7 · p. 12
- 23.95 to 29.22: Self-Exploration expands exploration behavior in semantic space, broadening feasible solutions while potentially adding less consistent branches.The passage describes this as increased coverage at the cost of reduced immediate stability.Source: §7 · p. 12
- Long CoT structure combines backbone densification by Deep Reasoning, search-space expansion by Self-Exploration, and convergence toward a stable solution through Self-Reflection.The paper compares this progression to protein folding toward an optimized state in semantic space.Source: §7 · p. 12
8 Deteriorated Molecular Structure Cannot Be Easily Restored
The section reports that compressing or summarizing Long CoT traces weakens distillation outcomes and shifts reasoning-behavior distributions. It argues that this structural disruption can protect Long CoT processes from behavioral cloning.
8 Deteriorated Molecular Structure Cannot Be Easily Restored
- With ∼45% token reduction versus QwQ-32B rationales, distillation from Gemini-2.5-Pro-Thinking and Claude-4-Sonnet causes accuracy drops, showing compression can disrupt Long CoT structure.Table 3 is identified as reporting distillation results from Gemini and Claude.Source: §8 · p. 12
- Training on compressed trajectories yields weaker performance than training on full traces and reduces distillation effectiveness by 2%.Table 4 is identified as reporting results from summarized Long CoT trajectories based on OpenAI-OSS and QwQ distilled trajectories.Source: §8 · p. 12
- Summarization shifts reasoning-behavior distributions and creates a gap between observable outputs and internal error-bounded transitions, limiting trace inversion and behavioral cloning.The paper presents disruption of structural coherence as a way to protect Long CoT structures from distillation.Source: §8 · pp. 12–13, Figure 14 · p. 13
9 Related Work
The related-work section traces CoT research from stepwise imitation through tree- and graph-structured traces, while noting that these representations do not capture distributions of logical behaviors. The paper’s molecular-like view instead treats edges as interactions among behavior distributions.
9 Related Work
- Long CoT cold-starting remains difficult because coherence can degrade, and gains often require targeted training or high-quality trajectories; distillation outcomes depend on reasoning quality [11] [13] [14] [15].The passage contrasts useful behavior transfer from strong reasoning models with weaker instruction models’ format imitation.Source: §9 · p. 13
- Prior work studied reflection for revising earlier steps and combined reflection with exploration for robustness [18].Earlier CoT work emphasized step-level imitation and local coherence.Source: §9 · p. 13
- Later work represented branching and revisitation with tree- or graph-structured reasoning, modeling behaviors as nodes in individual traces.The paper says node-centric views do not capture the overall distribution of logical behaviors.Source: §9 · p. 13
- The paper models Long CoT as a molecular-like structure whose edges encode stable distributions of logical behaviors, to examine how their arrangement supports effective learning.This contrasts with representing individual traces through behavior nodes.Source: §9 · p. 13
10 Conclusion
The study models Long CoT learning as molecular-like structures formed by Self-Reflection, Deep-Reasoning, and Self-Exploration bonds. It uses behavior-transition distributions to analyze semantic isomers and presents Mole-Syn as a way to construct robust Long CoT structures, improving performance and RL stability.
10 Conclusion
- Mole-Syn uses distribution-transfer graphs to construct robust Long CoT structures, improving performance and RL stability.Source: §10 · p. 13
- The semantic-isomer framework uses behavior-transition distributions to provide insight into Long CoT stability and failure modes.Source: §10 · p. 13
- The study conceptualizes Long CoT learning as molecular-like structures emerging through Self-Reflection, Deep-Reasoning, and Self-Exploration bonds.Source: §10 · p. 13
Limitations
This section outlines limitations in model coverage, training settings, geometric visualization, and automatic behavior labels, while describing analyses of attention-based reasoning bonds and Long CoT trajectory structure. It also presents methods for annotating reasoning bonds, analyzing transfer distributions, and visualizing trajectory folding.
Limitations
- The analysis uses few teacher models and backbones, offline distillation and SFT, approximate bond geometry, and automatically labeled behavior data, leaving architecture bias, online/RL transfer, universal structure, and label noise open.The authors note an initial robustness check for labeling but say noise or bias will remain.Source: Limitations · p. 14
C.1.1 Reasoning Behavior Definition.
- Long CoT is modeled as step transitions labeled Deep Reasoning, Self-Reflection, Self-Exploration, or Normal Operation, yielding behavior-labeled edges whose distributions can be compared and transferred across models.Source: §C.1.1 · pp. 18–19
C.1.2 Attention Energy Definition of Reasoning Bonds.
- Attention energy reparameterizes attention logits as a Gibbs/Boltzmann distribution: lower energy means higher attention and stronger dependency, with edge energies aggregated by labeled behavior.Source: §C.1.2 · p. 19
C.1.3 Attention-Energy Ordering of Reasoning Bonds
- Under RoPE's stated distance-decaying covariance and positive-alignment assumptions, expected bond energies order Deep Reasoning < Self-Reflection < Self-Exploration.Source: §C.1.3 · p. 20
C.1.4 Low-energy edges dominate path aggregation.
- Replacing one edge with a lower-energy edge increases that equal-length path's relative influence by at least exp(δ); consistently low-energy Deep Reasoning edges can stabilize multi-hop dependencies.Source: §C.1.4 · p. 22
C.1.5 Long CoT process is looking for a more stable reasoning structure.
- The authors hypothesize Long CoT learning seeks stable configurations minimizing attention energy; under ergodic behavior frequencies and conditional energy assumptions, trajectory-average energy decomposes into behavior-frequency-weighted means.Source: §C.1.5 · pp. 22–23
C.1.5 Long CoT process is looking for a more stable reasoning structure
- Attention exponentially favors lower-energy behaviors, biasing routing toward stable, low-energy Long CoT reasoning structures.Under bounded energy deviations, Deep Reasoning and Self-Reflection dominate Exploration when their mean-energy margin γ exceeds 2∆; the passage also describes the process as reaching an ergodic low-energy equilibrium.Source: §C.1.5 · p. 23
C.2 Stable Reasoning Bond Distribution in Long CoT
- N > 2k produced transfer-graph correlations exceeding 0.9, and comparisons above that threshold exceeded 0.95 across five trials.Traces were generated from OpenThoughts-3 tasks using DeepSeek-R1-671B, OpenAI-o1, and QwQ-32B; bond annotation achieved macro-F1 above 0.85 against human labels on 200 examples.Source: §C.2 · pp. 24–25
C.3 Logical Bonding-Folding Structure in Long CoT
- Embedding reasoning steps in a unified semantic space enables visualization of logical bonds and geometric metrics for quantifying trajectory folding.The geometric analysis represents complete trajectories as ordered sequences of step embeddings.Source: §C.3 · p. 25
C.3.1 Geometric Embedding and Visualization of Bonding Structure
- Qwen3-8B step embeddings and t-SNE projections visualize selected bonding behaviors while preserving local neighborhood structure.Logic clusters are formed by iteratively merging nearby instances using an adaptive distance threshold scaled to the spread of embedded points.Source: §C.3.1 · p. 25
C.3.2 Geometric metrics for folding
This block quantifies how reasoning behaviors move through semantic space, describes bond-attention and SFT analyses, and outlines semantic-isomer construction and ICL simulation.
C.3.2 Geometric metrics for folding
- 81.72% of reflective transitions reconnect to prior states within the predefined cutoff, while 72.56% of deep-reasoning steps satisfy dt < gt < 3 and exploration steps average a trajectory length of 5.32.The results indicate reflection returns to consolidated clusters, deep reasoning stays near logical neighborhoods, and exploration involves extended traversals.Source: §C.3.2 · pp. 25–26
C.4 Attention Energy Levels of Different Logical Bonds
- Bond-attention analysis records preceding-token attention for annotated reasoning behaviors, using final-layer head averages by default and behavior-specific step links.Reflection links to the nearest prior step in hidden space, deep reasoning to the immediately preceding step, and exploration from the previous step toward the exploration step.Source: §C.4 · p. 26
C.5 SFT Learning of Bond Structures
- The SFT analysis compares a pretrained Llama-3.1-8B-Base model with a Think-SFT version trained on R1-distilled Long-CoT traces.The study uses a cross-coder SAE with aligned hidden states and filters features by activation-probability differences before selecting those enriched on reasoning tokens.Source: §C.5 · p. 26, §C.5.1 · p. 26
C.5.1 Setting: How does SFT actually learn these bond structures?
- The keyword-manipulation setup replaces reasoning cues with meaning-preserving alternatives while aiming to preserve trajectories, syntax, problem distributions, answer types, and trajectory lengths.Two replacement variants use distinct alternatives to test sensitivity to surface wording.Source: §C.5.1 · pp. 27–28
D Semantic Isomer Construction Details
- The section presents semantic-isomer construction as distillation of well-structured traces, ICL simulation, metacognitive-oscillation analysis, and conflict learning between stable structures.It states that the construction setting follows Appendix B.Source: D Semantic Isomer Construction Details · p. 28, §D.1 · p. 29
D.1 Distillation of Well-structured Semantic Isomers
- The well-structured-isomer distillation setting uses eight base and instruction LLMs and distills from three advanced reasoning LLMs.Source: §D.1 · p. 29
D.2 ICL Simulation of Semantic Isomer Structures
- ICL simulation selects one demonstration by Pearson correlation between candidate and Qwen2.5-32B teacher reasoning-key distributions, then trains a student on generated traces.The selection strategies are random, high-correlation alignment (r ≳ 0.9), and low-correlation mismatch (r < 0.8); the demonstration is fixed for each question.Source: §D.2 · pp. 29–30
D.3.1 Setting: Information Flow Analysis and Metacognitive Oscillation Quantification
This section describes analyses connecting reasoning structure and behavior to performance, including methods for information-flow measurement, Mole-Syn synthesis, reinforcement learning, and geometric analysis. It also specifies the training setup, using summarized 20K samples and otherwise following the preceding sections’ configuration, with differences in model and reported outputs.
D.3.1 Setting: Information Flow Analysis and Metacognitive Oscillation Quantification
- Reasoning steps were encoded as semantic-probability vectors and mapped to trajectories of cumulative information and stepwise change; exploration and validation states were classified by slope and entropy-change thresholds.High-entropy exploration uses slope mt > 0.6 and ∆entropy > 0.05; low-entropy validation uses mt ≈ 0 and |∆entropy| < 0.05, with transition frequency and periodicity measured.Source: §D.3.1 · p. 30
E.1 Supervised-Finetuning with Mole-Syn
- Mole-Syn synthesis worked poorly with Llama-3.1-8B-Instruct, while Qwen models combined with Llama-70B performed robustly except on deep-reasoning tasks, where Qwen-32B outperformed Qwen-7B and Llama-70B.The analysis varied synthetic backbones to test how their self-reflection, exploration, and intrinsic reasoning capabilities affected distilled data.Source: §E.1 · p. 33
E.2 Reinforcement Learning with Mole-Syn Initialization
- The RL comparison isolates initialization by holding tasks, rewards, and hyperparameters constant between QwQ-distilled and Mole-Syn-SFT checkpoints; the supplied passage gives no performance values.Models were trained for 1,000 steps on MATH and AIME 1989–2023; Table 8 is identified as reporting results across three initializations and six benchmarks.Source: §E.2 · p. 33, Table 8 · p. 34
F.1 Deep Reasoning: Densing the Primary Structure.
- Deep Reasoning was assessed by comparing semantic-space volumes from baseline short-CoT and Long-CoT training with a 50% Deep Reasoning transfer probability; positive ∆Deep indicates core-structure contraction.Step representations were reduced to three dimensions with t-SNE, then measured using an approximate minimum enclosing ball.Source: §F.1 · p. 34
F.2 Self-Exploration: Expanding the Logical Space.
- Self-exploration was measured by comparing minimum-enclosing-ball volumes for baseline short-CoT or direct-answering and Long-CoT training with self-exploration transfer probability above 50%; positive ∆Exp means logical-space expansion.Other behavior probabilities were unchanged in the comparison.Source: §F.2 · p. 35
F.3 Self-Reflection: Densifying and Stabilizing the Logical Results.
- Self-Reflection reduced overall semantic volume from 35.2 to 31.2, with tighter intra-cluster distances and stable or slightly increased inter-cluster distances.The auxiliary clustering analysis suggests the dominant semantic cluster becomes more compact while inconsistent branches are pruned or weakened.Source: §F.3 · p. 35
G.1 Impact of bond-enhanced behaviors on performance
- Performance peaks with balanced reasoning behaviors; depth matters on complex queries, while excess reflection can hurt simple-task accuracy and excess exploration can impede complex-task convergence.62.7% of AIME cases showed extended reasoning without a clear conclusion under excessive exploration; simple queries generally benefited from more direct exploration.Source: §G.1 · p. 36
G.2 Effect of bond length on reasoning quality
- Increasing reasoning-step counts and bond lengths generally improved performance, especially for deep reasoning, but overly long exploration bonds caused semantic drift; shorter bonds favored simple tasks and longer bonds complex ones.The analysis varied bond lengths while holding behavioral ratios constant.Source: §G.2 · p. 36
H.2 Summarize Reasoning Process Analysis.
- The model was trained on summarized 20K samples, with other analytical experiments following the prior configuration except for the model and reported outputs.Results are reported in Table 4.Source: §H.2 · p. 36