PaperlayerView exact revision on arXiv

Source-linked AI summary

ChainPrune: Evaluating and Reducing Redundancy in Long Chain-of-Thought Reasoning

Weihang Pan, Zhengxu Yu, Yuxiang Zhang, Wenzhi Li, Zhongming Jin, Binbin Lin, Xiaofei He, Jieping Ye

arXiv:2608.21860v1cs.LGcs.AI

TL;DR

Long-chain reasoning models can reduce token counts while retaining redundant, fragmented reasoning. ChainPrune compresses reasoning paths using semantic structural pruning and reduces reasoning steps by 26.8% and tokens by 28.1% without accuracy degradation.

  • Problem

    Token-length optimization can produce pseudo-concise reasoning with fragmented, redundant steps and inefficient chains.

  • Method

    ChainPrune merges semantically equivalent reasoning nodes and selects paths that jointly minimize reasoning steps and token length while preserving logical consistency.

  • Results

    26.8% fewer reasoning steps and 28.1% fewer tokens were achieved across five reasoning tasks without accuracy degradation.

  • Takeaways & Limitations

    ChainPrune achieves genuine reasoning conciseness while preserving logical integrity and reasoning accuracy.

  • Takeaways & Limitations

    Randomly sampling sufficiently short correct responses remains costly because current DPO and PPO methods are sample-inefficient.

Abstract

from arXiv · show

Chain-of-Thought (CoT) reasoning has significantly enhanced the multi-step problem-solving capabilities of large language models (LLMs) by introducing explicit intermediate reasoning. However, advanced Large Reasoning Models (LRMs) often exhibit overthinking behaviors, including excessively long reasoning steps, redundant steps, and high computational overhead. Existing token-length reward strategies aim to promote concise outputs, but often result in pseudo-conciseness, where token count is reduced, yet redundant reasoning persists, leading to longer and less structurally efficient chains. To address these limitations, we propose ChainPrune, a novel reasoning path semantic structural optimization method to efficiently and controllably synthesize self-generated high-quality training data. We initially consolidate self-generated reasoning paths into a tree-based structure, followed by a multi-criteria dominant path selection process for preference data construction that formulates shallow reasoning trajectories while preserving essential reasoning steps. To further enhance the quality of reasoning, we incorporate a DPO-based preference learning method combined with supervised loss, effectively mitigating false reward suppression. This innovative integration significantly enhances both the efficiency and effectiveness of our reasoning framework. Comprehensive experimental results demonstrate significant reductions in step length and computational overhead, while maintaining or even enhancing accuracy.

Introduction

ChainPrune addresses overthinking and pseudo-conciseness in Large Reasoning Models by structurally merging redundant reasoning steps while preserving correctness and completeness. It constructs preference data from Pareto-optimal paths and reports shorter reasoning with no accuracy drop across five benchmarks.

  • Introduction: The method targets overthinking caused by deliberative LRM reasoning, including multiple solution strategies, self-reflection, and iterative error correction.These behaviors can produce excessively long and redundant reasoning sequences.
  • Introduction: Token-length rewards can create pseudo-conciseness, reducing token count while leaving reasoning paths redundant, and guide-based methods can propagate reasoning errors through reward bias.ChainPrune is motivated by limitations in both token-length optimization and manually guided tree-search approaches.
  • Introduction: ChainPrune merges semantically equivalent nodes across sampled reasoning paths to reduce redundant steps while preserving core logic.The method consolidates redundant reasoning into a compact, logically complete chain that is shorter in both tokens and steps.
  • Introduction: 26.8% chain lengths and 28.1% token lengths reduction are achieved across five reasoning benchmarks without an accuracy drop.The experiments evaluate ChainPrune’s reasoning-efficiency gains across five benchmarks.
  • Introduction: ChainPrune dynamically searches merged reasoning trees for Pareto-optimal paths to generate high-quality preference data without manual annotation.Among tested preference objectives, DPO + NLL Loss significantly shortens reasoning steps while maintaining reasoning quality.

Formal Analysis of Reasoning

The analysis formalizes LLM inference as exponentially expanding tree search and identifies pseudo-conciseness as a mismatch between token-level rewards and reasoning-chain structure. ChainPrune addresses this mismatch by combining Pareto-dominant path selection with semantic tree merging to reduce both steps and tokens while preserving correctness.

  • Formal Modeling of Autoregressive Inference: LLM inference generates tokens autoregressively from an initial context, forming a tree whose size grows exponentially with depth, O(b^d).Each branch represents a candidate vocabulary-token expansion.
  • Formal Modeling of Autoregressive Inference: Reasoning-focused LLMs extend autoregressive prediction to structured chains, using implicit or explicitly tagged reasoning and, in some methods, self-verification.Explicit-format approaches separate reasoning from final answers with semantic tags and format-constrained rewards.
  • Challenges in RL-Optimized Reasoning Paths: RL reward designs create pseudo-conciseness by shortening phrases per step while fragmenting logic across many steps, producing inefficient reasoning chains.The analysis identifies token-level conciseness as insufficient because it neglects structural chain length.
  • Challenges in RL-Optimized Reasoning Paths: Only 33.95% of shortest-token correct paths are also shortest in reasoning steps, exposing misalignment in length-based preference selection.Existing pipelines such as DPO often choose among correct outputs solely by minimum token length.
  • Challenges in RL-Optimized Reasoning Paths: ChainPrune defines domination over correct paths using both fewer reasoning steps and shorter token length, selecting a Pareto-dominant path or the non-dominated Pareto frontier.This dual-objective criterion directly targets concise expression and concise structure.
  • Cross-Path Semantic Reusability in Tree Search: Semantically equivalent intermediate steps recur across sampled paths, often at similar structural positions, enabling cross-path semantic reuse.Their recurrence appears in both chosen and rejected candidates.
  • Cross-Path Semantic Reusability in Tree Search: Merging semantically equivalent nodes and reusing their strongest continuations removes duplicated segments, yielding shorter composite paths in steps and tokens without compromising correctness.Attention concentrated on recent tokens after equivalent nodes supports preserving the subsequent conditional distribution during merging.

Methodology

ChainPrune generates concise, reliable reasoning chains by merging semantically redundant paths, selecting efficient preference pairs, and jointly optimizing preferences with supervised likelihood. Its safeguards preserve essential reasoning while constraining both token length and reasoning-step length.

  • Framework Overview: ChainPrune combines semantic path merging, dual-objective path selection, and enhanced preference optimization in three synergistic stages.The stages are reasoning-path merging, multi-criteria dominant-path selection with preference-data construction, and direct preference learning with NLL loss.
  • Reasoning Paths Merging: Candidate reasoning paths are consolidated into a compact tree that preserves essential trajectories while removing redundant steps through embedding-based semantic matching.A new step merges with an existing node when cosine similarity reaches θsim; otherwise, it creates a new branch.
  • Reasoning Paths Merging: Merges are accepted only when semantic similarity is accompanied by limited entropy change, preventing lexically similar but logically different reasoning steps from being combined.The entropy criterion requires ΔH = Hafter − Hbefore ≤ ϵ, so predictive uncertainty does not increase after replacement.
  • Preference Data Construction: Preference selection prioritizes correct paths that minimize both token length and chain length, then rejects responses that are unambiguously longer and inferior in correctness or efficiency.A unique Pareto-dominant path is selected when available; otherwise, the shortest-token path on the non-dominated front is chosen, with explicit thresholds for rejected samples.
  • Preference Optimization: DPO and supervised fine-tuning jointly optimize conciseness and correctness, with the supervised term preventing overly terse or incomplete reasoning.The SFT term maximizes preferred-output likelihood, while DPO aligns the model with preference signals; later experiments validate the NLL term’s necessity.

Experiments

Across two long-COT models, four math benchmarks, and LiveCodeBench, ChainPrune consistently improves accuracy while reducing token and reasoning-chain length. Its efficiency gains reflect lower redundancy and invalid reasoning, as confirmed by LLM-as-a-Judge evaluation with human checks.

  • Experimental Setup: Experiments use 9,967 cleaned MATH training pairs with 16 sampled paths per problem and evaluate MATH500, AMC23, AIME24/25, and LiveCodeBench.Accuracy, token length, and chain length are averaged over 16 runs; token and chain metrics are computed only for correctly answered questions.
  • Overall Results: ChainPrune consistently outperforms all baselines in accuracy while reducing both token and chain length across four math benchmarks and LiveCodeBench.It matches or surpasses base-model accuracy, whereas SFT, DAST (reproduce), and Kimi-1.5 show accuracy degradation or drops.
  • Token Efficiency: 28.1% average token reduction over base models accompanies maintained or improved accuracy, including 22.6% on AIME24 and 23.0% on LiveCodeBench.Token averages are computed only over correct answers, and ChainPrune remains best-performing in most datasets under the stricter evaluation including more difficult solved problems.
  • Chain Efficiency: ChainPrune produces substantially shorter reasoning chains while retaining comparable or lower token usage, reducing redundancy across both dimensions without harming reasoning quality.This contrasts with Kimi-1.5, which explicitly optimizes shortest token usage but does not achieve the same joint token–step efficiency.
  • Reasoning Quality: LLM-as-a-Judge evaluation reports 43 faulty reasoning steps, 47 invalid reflections, and 45 redundant steps, reductions of 57.8%, 62.1%, and 65.9% versus the base model.The evaluation uses GPT-o1, DeepSeek-R1, and Qwen-QwQ, with reliability further checked by humans.

Analysis and Discussion

The analysis shows that adding NLL to DPO mitigates reward synchronization collapse while preserving or improving accuracy and reducing token use, although ChainPrune remains more token-efficient. It also finds that shortest-correct-response selection adapts to difficulty and that semantic path merging reduces the cost of obtaining high-quality training samples.

  • Preference Optimization: DPO and SimPO can adopt biased strategies because short-chosen preference datasets have narrow distributions and small chosen-rejected edit distances, causing reward synchronization collapse.The preference pairs were sampled from a reference model with the shorter response designated as preferred.
  • Preference Optimization: DPO+NLL consistently maintains or improves mathematical-reasoning accuracy while reducing token usage, but ChainPrune remains more token-efficient through semantic path merging and Pareto-optimal chain selection.DPO+NLL improves both Kimi-1.5 and DAST on AIME24 and MATH500.
  • Difficulty Analysis: Token length for the shortest correct response grows proportionally with problem difficulty, so shortest-response selection naturally adapts optimization to question difficulty.Figure 4 analyzes shortest correct responses across varying difficulty levels.
  • Difficulty Analysis: Ablations show that shorter chosen responses produce the most concise equally accurate outputs, whereas medium and long chosen responses produce progressively longer outputs.The analysis reports a clear linear relationship between chosen-response length and resulting output length.
  • Data Construction Efficiency: Current RL methods such as DPO and PPO are sample-inefficient, while ChainPrune merges reasoning paths into trees to generate high-quality training samples with minimal overhead.Hundreds of random samples provide only marginal token-length gains; the resulting low-cost pipeline could integrate with online PPO to improve exploration efficiency.

Related Works

Prior work on shortening long chain-of-thought reasoning spans length-penalized reinforcement learning, implicit reasoning distillation, and guide-based inference acceleration. These approaches seek to reduce reasoning-path length or inference cost while preserving model accuracy or speed.

  • Make Long CoT Short: Length-penalized reinforcement learning encourages concise reasoning while targeting accuracy, with O1-Pruner optimizing both objectives (Team et al. 2025; Luo et al. 2025; Shen et al. 2025a; Hou et al. 2025; Aggarwal and Welleck 2025; Li et al. 2024; Yang, Lin, and Yu 2025).The supplied passage identifies RL with length penalties as an effective strategy for concise reasoning.
  • Make Long CoT Short: Implicit-reasoning methods eliminate explicit CoT generation by distilling reasoning into hidden states or performing it in a continuous latent space, as in Implicit-KD and Coconut (Deng et al. 2023).The passage lists related approaches including Implicit-KD and Coconut alongside other cited works.
  • Guide-based Methods: Guide-based reasoning methods aim to accelerate inference, including speculative-decoding approaches that integrate conventional speculation with reasoning and coordinate parallel small models (Yao et al. 2023; Hao et al. 2023; Wang et al. 2025; Xie et al. 2023).Gao et al. pioneered integrating speculative decoding with reasoning, while SEED introduced scheduled speculative decoding with multiple parallel small models (Gao et al. 2024; Wang et al. 2024).

Conclusion · Appendix

ChainPrune addresses pseudo-conciseness in LLM reasoning optimization through semantics-driven compression that merges equivalent reasoning-tree nodes while preserving logical integrity. Across five reasoning tasks, it reduces reasoning steps by 26.8%.

  • Conclusion: ChainPrune targets pseudo-conciseness, where superficial token reduction leaves redundant reasoning paths intact.
  • Conclusion: The method uses semantics-driven compression to merge equivalent nodes in reasoning trees.
  • Conclusion: Merging equivalent reasoning-tree nodes achieves genuine conciseness while preserving logical integrity.
  • Conclusion: Experiments evaluate ChainPrune across five reasoning tasks.
  • Conclusion: 26.8% fewer reasoning steps are reported for ChainPrune across the five reasoning tasks.
  • Conclusion: The conclusion frames ChainPrune as a systematic response to redundancy in LLM reasoning optimization.

A Statistical Analysis of Reasoning Path Efficiency · Evaluation Metrics and Key Findings · Pareto Frontier Visualization

The analysis shows that token-level conciseness and structural conciseness frequently diverge: only 33.95% of training questions share the same shortest-token and shortest-step correct path. A representative visualization further shows that no correct path jointly minimizes token and chain lengths.

  • A Statistical Analysis of Reasoning Path Efficiency: The min distance metric measures the normalized discrepancy between the shortest-token and shortest-chain correct reasoning paths.A value of min distance = 0 indicates that both minima are achieved by the same path.
  • Evaluation Metrics and Key Findings: The analysis evaluates sampled correct paths by token length, total output tokens, and chain length, the number of reasoning steps.Paths are ranked separately to identify τ min-tok and τ min-step.
  • Evaluation Metrics and Key Findings: Only 33.95% of questions have min distance = 0, meaning the shortest-token and shortest-step correct paths are the same.The result comes from analyzing sampled correct reasoning paths across the entire math-question training set.
  • A Statistical Analysis of Reasoning Path Efficiency: The 33.95% overlap quantitatively demonstrates substantial misalignment between expression-level and structural conciseness in current optimization schemes.The analysis was designed to test whether token-level optimization overlooks compact reasoning structures.
  • Pareto Frontier Visualization: Among 64 sampled paths for a representative math question, no correct path appears in the plot’s bottom-left corner, so joint minimality is not achieved.The visualization places chain length on the x-axis and token length on the y-axis; blue nodes indicate wrong responses.
  • Pareto Frontier Visualization: The representative visualization reinforces that minimizing token length alone fails to produce paths simultaneously compact in structure and concise in expression.Each plotted point represents a unique reasoning path, and no single path optimally balances both dimensions.

B Examples of Semantically Equivalent Reasoning Steps in Chosen and Rejected Paths · C Multi-Reasoning Paths Merging Algorithm

The examples show that chosen and rejected paths can contain semantically equivalent reasoning, while rejected paths place those steps earlier because token-length preferences favor delayed, fragmented reasoning. The merging algorithm instead builds a compact reasoning tree that preserves critical logic and diversity while pruning redundancy.

  • B Examples of Semantically Equivalent Reasoning Steps in Chosen and Rejected Paths: Semantically equivalent reasoning steps appear in both chosen and rejected paths, but rejected paths systematically place them earlier, revealing an ordering bias.This bias occurs across preference pairs rather than reflecting different underlying reasoning content.
  • B Examples of Semantically Equivalent Reasoning Steps in Chosen and Rejected Paths: Figure 8 shows both proportional-reasoning paths computing meat per hamburger and scaling to 24 hamburgers, while the rejected path performs these shared steps earlier.The example illustrates that ordering, rather than semantic content, distinguishes the paths.
  • B Examples of Semantically Equivalent Reasoning Steps in Chosen and Rejected Paths: Token-minimizing dataset construction favors paths that delay semantic content, producing syntactically fragmented reasoning despite correct answers and nearly identical intermediate steps.The resulting preference sacrifices structural clarity and efficiency for lower token count per step.
  • C Multi-Reasoning Paths Merging Algorithm: Given K candidate paths, the merging algorithm hierarchically constructs a compact step-level reasoning tree that retains critical logic while pruning redundancy.Each candidate path is represented as a sequence of reasoning steps before merging.
  • C Multi-Reasoning Paths Merging Algorithm: The algorithm selects the shortest candidate path as a backbone and builds an initial linear tree whose nodes represent sequential reasoning steps.This backbone provides a minimal but coherent reasoning trajectory for subsequent merging.
  • C Multi-Reasoning Paths Merging Algorithm: Remaining paths are incrementally merged by comparing step embeddings with tree nodes using cosine similarity and checking logical consistency through predictive-entropy change.A step becomes a merge candidate when similarity exceeds θsim, subject to the entropy-based consistency check.
  • C Multi-Reasoning Paths Merging Algorithm: The resulting tree represents shared reasoning as internal nodes, divergent strategies as branches, and unique answers as leaves, preserving diversity while eliminating redundant computation.This structure provides the compact representation used by ChainPrune to organize multiple reasoning paths.

D Theoretical Analysis of different variants of DPO in Short-Chosen Preference Optimization

The analysis attributes DPO and SimPO’s reasoning-performance decline to gradient entanglement, where chosen and rejected response probabilities move together instead of diverging. SimPO partially mitigates this issue through length normalization, while adding NLL loss is proposed to break the entanglement.

  • Gradient Entanglement Mechanism: DPO’s objective widens the chosen–rejected log-probability gap, but synchronized increases fail to suppress rejected responses while synchronized decreases cause the model to forget both response types.The analysis identifies these as two problematic gradient-alignment scenarios.
  • Gradient Entanglement Mechanism: Gradient entanglement occurs when the chosen and rejected log-probability gradients are strongly aligned, producing synchronized increases or decreases rather than ideal divergence.The ideal optimization pattern is log πw ↑ and log πl ↓.
  • Empirical Analysis: 85.6% combined prevalence of synchronized decrease and increase conditions remains for SimPO, showing that length normalization does not eliminate gradient entanglement.These conditions respectively cause forgetting or fail to suppress rejected responses.
  • NLL-Regularized DPO: Combining DPO with NLL loss increases the chosen-gradient coefficient while leaving the rejected-gradient coefficient unaffected, explicitly regularizing optimization to break gradient entanglement.This relaxes the condition for increasing the chosen response’s log probability while preserving contrastive learning’s advantage.

E LLM-as-a-Judge for Evaluation of Reasoning Model

The framework evaluates reasoning quality through dynamic semantic step segmentation and multi-model voting, replacing brittle fixed chunking and direct long-context assessment. It classifies faulty reasoning, invalid reflection, and redundant steps using context-aware judgments with explanations.

  • Framework Overview: The framework combines GPT-4o semantic chain splitting with three-model voting for scalable, fine-grained evaluation of reasoning steps.GPT-4o segments full reasoning paths into coherent, minimal units, which three expert models classify independently; majority voting resolves labels, with human review for high disagreement.
  • Step Segmentation: GPT-4o dynamically segments reasoning paths into semantically meaningful steps, avoiding brittle rule-based chunking and fixed heuristic segmentation.The approach replaces newline-based segmentation with an LLM-as-a-Chainsplitter strategy designed to make each unit coherent and minimal.
  • Step Classification: Each step is categorized as faulty reasoning, invalid reflection, or redundant steps to attribute logical, metacognitive, and repetitive weaknesses.Faulty reasoning is logically incorrect or mathematically invalid; invalid reflection is ineffective or incoherent; redundant steps add no new information.
  • Context-Aware Evaluation: Context-aware prompts provide each step with up to three preceding and three following steps, improving distinction between locally novel, redundant, and flawed reasoning.Evaluators return a JSON tag and justification, enabling judgments that account for neighboring steps rather than isolated text.

F More Experiment Results

Ablation results indicate that selecting shorter reasoning samples produces more concise model outputs, while a structured judge prompt classifies individual reasoning steps using local context.

  • Reasoning Step Classification: The step-level classifier assigns each atomic reasoning step to one of four categories using the problem, target step, and up to three neighboring steps on each side.The structured prompt requires a JSON output containing the classification tag and an explanation.
Loading 2608.21860v1…