PaperlayerView exact revision on arXiv

Source-linked AI summary

Diagnosing Harmful Continuation in Answer-Correct Long-CoT Training Traces

Chen He, Yuhao Wu, Lei Wang, Wenxuan Zhang, Fumin Shen

arXiv:2605.29288v1cs.AI

TL;DR

Answer-correct long-CoT traces can contain post-conclusion reasoning whose training value is unclear. The paper uses answer-preserving suffix removal, uncertainty and hidden-state analyses, and HCC to study this continuation; removal improves SFT outcomes in the reported setting, while HCC approximates its boundary.

  • Problem

    Existing methods do not explain the internal failure mode of answer-correct traces or characterize where useful reasoning ends and post-conclusion continuation begins.

  • Method

    The paper uses a delete-only editor for answer-preserving suffix removal, analyzes uncertainty and hidden-state progress, and develops HCC as a lightweight boundary proxy.

  • Results

    Answer-preserving removal improves downstream SFT outcomes in the reported settings, and the removed continuation shows an uncertainty–geometry mismatch that HCC approximates.

  • Takeaways & Limitations

    The study identifies harmful continuation as an empirically supported, setting-specific training-unfavorable phenomenon rather than assuming that all long reasoning tails are harmful.

  • Takeaways & Limitations

    The editor is an operational intervention rather than a ground-truth oracle, the measurements are diagnostic rather than causal proof, and HCC approximates the editor boundary rather than intrinsic harmfulness.

Abstract

from arXiv · show

Long chain-of-thought (CoT) traces are widely used as supervision for reasoning-oriented LLM SFT, yet answer-correct traces can still lead to markedly different fine-tuning outcomes. We study post-conclusion continuation in answer-correct long-CoT data: a continuation where the answer appears sufficiently supported, but the trace continues with additional reasoning that remains in the supervised target. To test its training effect, we use a delete-only editor to construct answer-preserving suffix removal and compare CoT-based SFT on the original and processed traces. We observe improved SFT outcomes after removing the editor-identified post-conclusion continuation, suggesting that this continuation is harmful to training in our setting. We therefore refer to this empirically supported phenomenon as harmful continuation. Beyond this intervention, we further characterize the removed post-conclusion continuation through uncertainty and hidden-state progress. We observe persistent local uncertainty together with weakened terminal-directional progress, forming an uncertainty--geometry mismatch. Finally, we instantiate Harmful Continuation Cut (HCC), a lightweight boundary proxy that approximates the editor-identified post-conclusion continuation boundary.

1 Introduction

Answer correctness does not ensure that every part of a long-CoT trace is useful supervision. The paper studies post-conclusion continuation, tests answer-preserving removal, and proposes HCC as a boundary proxy.

  • Answer-correct traces can differ substantially in training utility, but prior methods do not explain where useful reasoning ends or why later reasoning may weaken SFT.
  • The paper examines a low-value post-conclusion continuation that begins after the answer is sufficiently supported and may show local cost and weak hidden-state progress.
  • The paper calls the empirically supported training-unfavorable case harmful continuation when removing the continuation improves SFT outcomes.
  • Answer-preserving suffix removal tests whether editor-identified continuation affects SFT without rewriting the retained prefix or final answer.
  • Harmful Continuation Cut (HCC) is proposed as a lightweight proxy for the editor-identified continuation boundary.

2 Related Work

Prior work shows that answer correctness alone does not determine the training value of long-CoT traces. Existing research studies selection, compatibility, reasoning patterns, and trajectory properties but leaves the boundary between useful and less useful reasoning underexplained.

  • Long-CoT traces are increasingly used as structured reasoning trajectories in post-training pipelines, not merely as final-answer supervision.
  • Data selection, trace compatibility, and informative alignment studies suggest that only a subset of answer-correct traces provides beneficial supervision.
  • Existing approaches do not directly characterize where useful reasoning ends or why later reasoning may be associated with weaker SFT despite answer correctness.
  • Studies of overthinking report repeated verification or continued reasoning without meaningful gain, alongside analyses of trajectory geometry and step-level progress.

3 Operational Partition and Diagnostics of Post-Conclusion Continuation

The paper operationally partitions answer-correct traces into retained reasoning and editor-removed post-conclusion continuation, then diagnoses the latter using uncertainty and hidden-state progress. The continuation shows rising local uncertainty without stable answer-support gains and weaker geometric progress than retained reasoning.

  • Operational Partition: The delete-only editor partitions each trajectory into retained reasoning that supports the final answer and an editor-removed post-conclusion continuation.These are operational groups for diagnosis, not predefined theoretical labels.
  • Uncertainty View: Evaluator-based diagnostics compare answer recoverability as reasoning segments are progressively appended to retained reasoning and editor-removed continuation.The analysis uses prefix-conditioned final-answer entropy and NLL at the answer level, alongside sentence-level uncertainty measures.
  • Uncertainty View: Retained reasoning generally lowers answer NLL, whereas editor-removed continuation increases answer entropy and answer NLL after the answer is sufficiently supported.The continuation therefore does not consistently improve evaluator-based answer recoverability.
  • Uncertainty View: At the boundary, local uncertainty rises in editor-removed continuation without a corresponding stable increase in answer support.Within retained reasoning, local uncertainty can still accompany stronger answer-NLL reduction; from C1 to CT, that reduction is not stable.
  • Geometric View: Retained reasoning more often exhibits larger token-normalized hidden displacement and stronger forward progress than editor-removed continuation.The distributions overlap substantially, so geometry is a distributional signal rather than a pointwise deletion rule.
  • Geometric View: Editor-removed continuation has lower hidden displacement than retained reasoning, with means of 21.91 versus 44.92 and lower values in 79% of samples.Forward progress and their token-normalized variants show the same trend, while curvature has only a small absolute gap.
  • Uncertainty–Geometry Mismatch: The combined pattern is an uncertainty–geometry mismatch: persistent local cost or instability accompanies weaker hidden displacement and terminal-directional progress.This mismatch characterizes editor-removable continuation but does not itself define harmfulness or prove causal training harm.

4 Method

The method diagnoses editor-identified post-conclusion continuation through uncertainty and hidden-state progress, then trains HCC to predict a boundary for removing that continuation.

  • Boundary formulation: HCC predicts a boundary that retains the prefix r≤c∗ and removes the post-conclusion continuation r>c∗.The boundary c∗ is supplied by a delete-only editor and serves as supervision for the lightweight proxy.
  • State encoding: Sentence-level hidden representations are extracted from a frozen causal language model and processed by a shared sequence encoder.The contextualized states provide the common input for latent regularization and diagnostic estimation.
  • Latent regularization: HCC uses sequential variational latent regularization to compact boundary states without imposing a hard information bottleneck or answer-prediction objective.Posterior and sequential-prior distributions regularize the sampled latent representation before projection into boundary-prediction space.
  • Uncertainty–geometry diagnostics: HCC jointly estimates local uncertainty and operational hidden-state progress to represent their mismatch during post-conclusion reasoning.Uncertainty targets derive from entropy, NLL, or log-perplexity, while progress targets derive from hidden-state movement statistics.
  • Boundary prediction: The boundary predictor fuses diagnostic context vectors and scalar estimates, while cut and deletion heads learn editor-derived boundary and sentence-deletion labels.The cut head scores the last retained sentence, and the deletion head predicts whether each sentence should be removed.

5 Experiments

Experiments compare original, editor-processed, HCC-processed, and heuristic or random-cut traces across SFT, diagnostics, reinforcement learning, and selected MMLU subjects.

  • Experimental setup: HCC is trained across source-model splits to test transfer, then used to process trajectories before baseline-model SFT.The setup uses Qwen2.5-0.5B-Instruct as a frozen backbone and lightweight prediction heads.
  • Main results: Answer-preserving removal of editor-identified continuation improves downstream SFT outcomes across training traces and models.The authors interpret this as interventional evidence that the continuation is training-unfavorable in these settings.
  • Main results: 45.2 and 49.3 are HCC average scores on T_Q and T_R in the LLaMA3.2-3B setting, closely matching editor-processed scores of 45.7 and 49.8.HCC also exceeds the editor reference on MATH500 under T_Q and GSM8K under T_R.
  • Uncertainty dynamics: HCC and editor processing stabilize answer NLL and continue reducing later segment entropy, unlike Vanilla’s late-stage uncertainty rebound.Figure 4 provides post-training diagnostic evidence consistent with the proposed continuation pattern.
  • Geometric dynamics: HCC and editor processing produce larger token-normalized hidden displacement and reduce the positive entropy-progress mismatch near the end.Figure 5 indicates stronger state movement per generated token and a mismatch closer to zero than Vanilla.
  • Reinforcement learning: HCC-based SFT initializes GRPO models that consistently outperform Vanilla-based initialization at each evaluated reinforcement-learning step.The result suggests that SFT data processing effects can persist into subsequent RL in this setting.
  • Random-cut baseline: Random cut scores 29.0 on average versus HCC on MATH500, AMC23, and GSM8K, despite matching HCC’s average truncation length.This comparison tests whether improvement is explained mainly by shorter responses.
  • MMLU transfer: HCC-based SFT outperforms Vanilla across six selected MMLU subjects and is comparable to the editor reference.These results do not establish that the same continuation pattern occurs in non-mathematical training traces.

6 Conclusion

The paper studies answer-correct traces that continue after the answer appears sufficiently supported and identifies an empirically training-unfavorable continuation in its setting.

  • Conclusion: Delete-only, answer-preserving suffix removal improves downstream SFT outcomes, motivating the term harmful continuation.The removed continuation is characterized by persistent uncertainty and weak hidden-state progress, forming an uncertainty–geometry mismatch.
  • Conclusion: HCC is introduced as a lightweight boundary proxy that approximates editor-identified post-conclusion continuation removal.The proxy is based on the observed uncertainty–geometry characterization.

Limitations

The study’s intervention and diagnostics identify editor-defined boundaries and proxy measurements rather than proving intrinsic harmfulness.

  • Scope and interpretation: The delete-only editor is an operational intervention, not a ground-truth oracle of harmfulness.Its labels represent editor-identified post-conclusion continuation boundaries, while measurements are diagnostic proxies rather than causal proof.
  • Scope and interpretation: HCC approximates the editor-identified removal boundary rather than intrinsic harmfulness, leaving finer component-level attribution for future work.

A Implementation Details

The paper specifies evaluation, uncertainty, and information-bottleneck procedures for identifying answer-sufficient prefixes and characterizing post-conclusion suffixes. It also defines an ideal boundary where the remaining suffix becomes irrelevant to answer prediction, while emphasizing that answer sufficiency alone does not establish training benefit.

  • Implementation: All compared SFT variants use the same AdamW configuration and deterministic temperature-0 decoding, differing only in their supervision traces.The learning rate is 1 × 10−5.
  • Sequential conditional information bottleneck: Each sentence-level prefix receives a stochastic bottleneck representation induced from its contextualized reasoning state.The trace is represented as sentence units with prefix R≤t and suffix R>t.
  • Sequential conditional information bottleneck: The ideal bottleneck objective preserves answer-relevant information while minimizing unnecessary dependence on the reasoning prefix.Its constraint requires I(Zt; A | Q) to remain sufficiently large.
  • Ideal answer-sufficiency boundary: At an answer-sufficient boundary, conditioning on the remaining suffix does not change the Bayes-optimal answer decision under the bottleneck representation.This follows from the stated conditional-independence relation.
  • Ideal answer-sufficiency boundary: The ideal answer-sufficiency boundary is the earliest step where the suffix has zero conditional mutual information with the answer given the question and bottleneck state.If no such step exists, the trace has no boundary satisfying the ideal condition.
  • Scope condition: The paper cautions that answer sufficiency does not imply deletion improves SFT, because empirical benefit additionally depends on local uncertainty and geometric progress.The answer-sufficiency proposition is therefore narrower than the harmful-continuation training claim.
  • Uncertainty metrics: Answer-level uncertainty measures how each appended reasoning prefix changes recovery of the boxed answer, with answer-NLL reduction increasing when recovery becomes easier.The evaluator scores the final answer after appending each sentence.

C.2 Geometric Metrics

The geometric analysis treats sentence-boundary hidden states as an operational proxy for reasoning-state evolution. It measures update size, terminal-directional progress, efficiency, curvature, and paired differences between retained and removed reasoning.

  • Geometric metrics: Hidden-state geometry is computed at sentence boundaries using the evaluator state after each sentence.The local state update is the difference between consecutive sentence-boundary hidden states.
  • Geometric metrics: Hidden displacement measures the Euclidean size of each local hidden-state update.The metric is Dt = ∥∆ht∥2.
  • Geometric metrics: Forward progress measures the projection of each local update onto the remaining direction toward the analyzed trace’s terminal hidden state.The resulting quantity is an operational proxy for terminal-directional progress.
  • Scope condition: The forward-progress measure is not a direct measurement of the true reasoning process.It should be interpreted as an operational hidden-state proxy.
  • Geometric metrics: Progress efficiency is defined by combining forward progress with hidden displacement.The paper separately reports token-normalized variants to control for sentence length.
  • Geometric metrics: Curvature is used only as an auxiliary diagnostic of hidden-state direction changes.It compares consecutive local updates through their normalized cosine similarity.
  • Comparison protocol: Paired comparisons average each metric within retained reasoning and editor-removed continuation for every example.The analysis reports group means, directional fractions, and 95% confidence intervals for the paired difference.

D.1 Additional Analysis of Harmful Continuation Diagnosis

Additional diagnostics distinguish editor-removed continuation from retained reasoning through answer-level uncertainty and hidden-state geometry. The removed continuation produces stronger answer perturbations and lower terminal-directional progress, consistent with a low-progress phase.

  • Uncertainty view: Editor-removed continuation induces larger answer-level NLL perturbations than retained reasoning.Its average log-probability perturbation distribution is also shifted toward larger values.
  • Combined diagnosis: The combined uncertainty and geometry results characterize the removed continuation as answer-affecting but less effective at moving representations toward the terminal reasoning state.This interpretation is presented as consistent with a low-progress phase rather than as a direct measurement of reasoning itself.
  • Geometric view: Editor-removed continuation is more concentrated in low forward-progress regions under the operational geometric proxy.Retained reasoning instead induces larger token-normalized hidden displacement.

D.2 Additional analysis of Test Datasets

The paper supplements its diagnosis with a qualitative SFT case study, detector-based self-consistency analysis, and computational comparison of HCC with the offline editor. These analyses illustrate HCC’s behavioral and computational scope without treating detector matches as causal proof.

  • Case study: The HCC-trained model reaches the correct reasoning path without entering the long verification loop shown for the Vanilla-trained model.The Vanilla example reaches the correct answer, derives a conflicting alternative result, and eventually exhausts its token budget.
  • Case study: The case study qualitatively links HCC-processed supervision with reduced uncertain, low-progress verification after sufficient answer support.This is presented as an illustration consistent with the harmful-continuation pattern.
  • Self-consistency diagnostic: Table 6 uses HCC to detect whether post-SFT outputs match the removable-continuation pattern across training sources.The analysis is explicitly a detector-based self-consistency diagnostic, not independent causal evidence of harmfulness.
  • Computational costs: HCC uses 498M parameters, about 1.8% of Qwen3.5-27B, and requires 2.5T MACs versus 137.1T for the 27B editor.The reported comparison corresponds to roughly 54.2× lower computation.
  • Computational costs: The computational comparison positions HCC as a cheaper proxy for approximating editor-identified harmful-continuation boundaries.The comparison uses similar input lengths for Qwen3.5-27B and HCC.
Loading 2605.29288v1…