Source-linked AI summary
Semantic Reasoning Denoising: Correcting Language Model Reasoning with Semantic Operators
Yujiao Yang
TL;DR
Existing approaches often lack an explicit mechanism for correcting local semantic errors in reasoning, while token-level diffusion primarily models textual corruption. SRD uses executable semantic-error operators and iterative applicability-checked inverse transitions, improving same-backbone performance across six in-domain benchmarks and transferring competitively across datasets.
Problem
Language models can produce reasoning traces with local semantic errors, while existing diffusion methods mainly define noise through token masking or replacement rather than reasoning-process errors.
Method
SRD composes executable semantic-error operators into progressively noisy states and trains a denoiser to predict operators and reconstruct adjacent lower-noise reasoning states.
Results
SRD improves the strongest same-backbone baseline by 3.2 points on average across six in-domain benchmarks and remains competitive on Llama transfer while achieving the best Qwen3 transfer average.
Takeaways & Limitations
Structured semantic-noise prediction and iterative operator execution are central to SRD's reported improvement, with analyses supporting joint operator and adjacent-state supervision.
Takeaways & Limitations
Evaluation is limited to 8B backbones and fixed operators, motivating larger models and adaptive operators for long or ambiguous error chains.
Abstract
from arXiv · showhide
Large language models can produce fluent reasoning traces whose local semantic errors propagate to an incorrect conclusion, while unconstrained self-correction may preserve, amplify, or introduce errors. Existing diffusion language models provide iterative refinement, but usually define noise as token masking or replacement rather than as errors in the reasoning process. We present Semantic Reasoning Denoising (SRD), an operatorized Markov denoising method for natural-language reasoning trajectories. SRD represents semantic noise with executable error operators that describe the error type, its location, and the corrupted and repaired propositions. Composing these operators constructs progressively noisier states. During training, the model learns to identify the semantic noise active in the current trajectory and to reconstruct the paired adjacent lower-noise state. During inference, noise-level-aware denoising repeatedly predicts an inverse operator and checks whether it is applicable, so each executed update makes a localized move toward a stable trajectory. Across six in-domain benchmarks spanning mathematics, code, knowledge, and commonsense, SRD improves the strongest same backbone baseline by 3.2 points on average. On seven cross-dataset transfer targets, it remains competitive with Llama-3-8B-Instruct and improves the strongest Qwen3-8B baseline average by 2.9 points. Analyses of noise sources, objectives, and denoising depth further show that structured semantic-noise prediction and iterative operator execution are central to the improvement.
1 Introduction
SRD reframes reasoning refinement as executable semantic denoising, addressing the limits of token-level diffusion and unconstrained correction. It trains and applies localized inverse transitions, improving performance across in-domain tasks.
- Token-level diffusion mainly recovers textual form, not reasoning errors such as incorrect intermediate conclusions, misused conditions, or broken dependencies.
- SRD represents semantic noise with executable operators and composes them into coherent reasoning states at different noise levels.Operators specify error structure and support ordered transitions between adjacent states.
- Training combines real, synthetic, and interpolated failures to teach semantic-noise identification and lower-noise state reconstruction.
- 3.2 points: SRD averages 67.5 across six in-domain benchmarks, outperforming the strongest same-backbone baseline and leading on all six tasks.Experiments span mathematical reasoning, code generation, and general understanding.
- Each inference round predicts an inverse operator and checks applicability, producing localized transitions instead of unconstrained regeneration.
2 Related Work
Prior work develops reasoning generation, self-correction, and token-level diffusion, while SRD is situated within efforts to refine reasoning trajectories more directly.
- Reasoning generation and correction: Chain-of-thought prompting elicits intermediate steps, while STaR bootstraps training from model-generated rationales.
- Reasoning generation and correction: Self-consistency aggregates sampled chains, and Self-Refine revises answers using self-feedback.
- Reasoning generation and correction: RISE and SCoRe extend correction through recursive self-improvement and reinforcement learning.
- Diffusion language models: Discrete diffusion methods typically corrupt text through token masking or replacement, including applications to chain-of-thought generation.
- Diffusion language models: LLaDA-8B and Dream-7B scale discrete diffusion to general-purpose language models, while later work broadens token corruption through interpolating diffusion.
3 Method
SRD models reasoning refinement as an operatorized semantic denoising process over coherent trajectories. It trains a model to diagnose executable errors and recover adjacent lower-noise states, then iteratively applies predicted inverse operators during inference.
- 3.1 Preliminaries: SRD defines a discrete Markov process whose transitions are induced by executable semantic error operators.The forward process composes operators into corrupted states, while the reverse process predicts and inverts the active operator.
- 3.1 Preliminaries: Each operator records an error type, step location, original proposition, and corrupted proposition, covering several typed reasoning failures.The library includes premise insertion or deletion, relation reversal, intermediate-value substitution, step reordering, and conclusion substitution.
- 3.1 Preliminaries: SRD measures semantic noise by the shortest executable corruption path rather than by surface-token differences.Each primitive operator counts as one semantic corruption step, and the normalized coordinate τt = t/T marks position along the constructed chain.
- 3.2 Forward Process: Training combines real failures, synthetic corruptions, and interpolated traces to cover different semantic-noise levels.The construction retains executable, well-formed transitions and includes clean identity pairs for preservation.
- 3.3 Reverse Process: The denoiser jointly identifies the active operator and reconstructs the paired adjacent lower-noise state.Adjacent-state prediction supplies transition-reconstruction supervision, while the predicted inverse operator determines the next inference state.
- 3.3 Reverse Process: At inference, SRD starts from a frozen-backbone trajectory and repeatedly applies predicted repairs under a reverse process-coordinate schedule.It stops on an empty operator, an unchanged state, or the maximum denoising budget, then extracts the answer from the resulting trajectory.
4 Experiments
SRD is evaluated across in-domain benchmarks, cross-dataset transfer settings, semantic-noise analyses, inference budgets, and core ablations. Results indicate strong accuracy, sample efficiency, transfer, and robustness, while exposing higher costs and localization challenges for deeper denoising.
- 4.1 Main In-Domain Results: SRD achieves a 67.5 average across six in-domain benchmarks, improving over DiffCoT by 3.2 points on average and leading on all six tasks.It also outperforms SCoRe by 3.8 points on average; all six improvements over DiffCoT remain significant after Holm correction.
- 4.2 Cross-Dataset Transfer: SRD improves cross-dataset transfer averages from 67.7 to 76.4 on Llama-3 and from 72.9 to 80.2 on Qwen3-8B.It leads every reported Qwen3 target and is best on five reported Llama targets, supporting transfer beyond the training dataset and across backbones.
- 4.3 Semantic-Noise Analysis and Sample Efficiency: Real failures produce the most diverse semantic corruptions, while interpolated and synthetic trajectories provide a gradual severity continuum for reverse transitions.Interpolated trajectories are mildest, synthetic corruptions have moderate controllable severity, and real failures extend toward larger lexical and semantic distances.
- 4.3 Semantic-Noise Analysis and Sample Efficiency: SRD is more sample-efficient than correction and self-training baselines from 50 through 2,000 training examples.Its early advantage suggests structured semantic reverse transitions provide a useful inductive bias rather than merely benefiting from additional data.
- 4.4 Inference Efficiency: Increasing the denoising budget from K=1 to K=16 grows generated tokens by 7.8× but total latency by only 2.0×.Self-speculative decoding parallelizes verification and reuses shared-prefix KV caches, while accuracy gains are mostly realized in the first few steps.
- 4.5 Core Ablations: Full SRD is the most robust ablation configuration, whereas removing noise sources or separating its objectives weakens performance.Synthetic noise contributes most overall, and operator-only, adjacent-state-only, answer-only, and full-CoT supervision underperform semantic trajectory denoising.
- 4.5 Core Ablations: With three-operator corruptions, execution reaches 81.4% on GSM8K and 85.7% on PIQA, while answer recovery reaches 77.0% and 84.6%, respectively.Performance declines more on MATH and code, identifying localization as the main bottleneck for longer compositions.
5 Conclusion
SRD is an operatorized Markov denoising method that improves reasoning across backbones while exposing localization as a bottleneck for longer error chains. Its evaluation remains limited to 8B backbones and fixed operators, motivating larger models and adaptive operators.
- SRD improves the strongest same-backbone baseline by 3.2 points and transfers across Llama-3 and Qwen3.
- Ablations and compositional diagnostics support joint operator and adjacent-state supervision.
- Localization is identified as the bottleneck for longer reasoning chains.
- Evaluation is limited to 8B backbones and fixed operators.The authors motivate larger models and adaptive operators for long or ambiguous error chains.