PaperlayerView exact revision on arXiv

Source-linked AI summary

DCGC: Draft-Conditioned Global Correction for Complex Reasoning with Masked Diffusion Models

Minhae Oh, Nakyung Lee, Jungwoo Lee

arXiv:2608.25428v1cs.CLcs.AI

TL;DR

Flawed reasoning traces remain difficult to correct because autoregressive generation can propagate early errors and masked diffusion correction is underexplored. DCGC trains masked diffusion models for problem-only and draft-conditioned solving, then uses Dynamic Dual-CFG to scale draft residuals by relative confidence. Across math, code, and knowledge benchmarks, it improves correction performance, including 24.8% average accuracy on solver-failure hard sets, while remaining effective without oracle feedback.

  • Problem

    Autoregressive self-refinement can propagate early mistakes, while masked diffusion models remain underexplored for global correction of long reasoning traces.

  • Method

    DCGC combines dual-capability SFT with Dynamic Dual-CFG, separating problem-only and joint problem-draft branches and scaling draft-conditioned residual guidance by relative confidence.

  • Results

    24.8% average accuracy was achieved on solver-failure hard sets across math, code, and knowledge reasoning benchmarks, with DCGC outperforming strong autoregressive and diffusion-based baselines.

  • Takeaways & Limitations

    DCGC supports verifier-free global correction by extracting constructive information from imperfect drafts through relative dual guidance.

  • Takeaways & Limitations

    Training discarded samples exceeding 1,028 tokens, limiting the current DCGC instantiation in extremely long-context scenarios.

Abstract

from arXiv · show

Correcting flawed reasoning traces remains a significant challenge for Large Language Models (LLMs), whose autoregressive generation can propagate early mistakes into subsequent reasoning. We introduce DCGC, a Masked Diffusion Model (MDM) framework for global correction that uses an imperfect solution draft from an upstream solver as auxiliary context. DCGC combines task-specific Supervised Fine-Tuning (SFT) with a novel inference-time mechanism called Dynamic Dual-CFG. This mechanism separates problem-only and joint problem-draft branches and scales the draft-conditioned residual using a relative confidence gap. Across math, code, and knowledge reasoning benchmarks, DCGC outperforms standard sampling and simpler CFG variants, with additional results suggesting transfer to different diffusion backbones. In test-time setting where ground-truth failure labels are unavailable, DCGC improves full test set accuracy by correcting low-consensus upstream outputs, highlighting its utility as a verifier-free global correction module for difficult reasoning instances.

1 Introduction

DCGC addresses error propagation in autoregressive self-refinement by using masked diffusion for draft-conditioned global correction. It combines dual-capability SFT with Dynamic Dual-CFG to selectively reuse imperfect drafts without external tools.

  • Autoregressive self-refinement can propagate early mistakes because each revised token conditions subsequent tokens.
  • Masked diffusion revisits masked positions during denoising, making it suitable for globally correcting long reasoning traces conditioned on an imperfect draft.
  • DCGC performs iterative denoising under separate problem-only and joint problem-draft contexts, with the draft supplying auxiliary reasoning information.
  • DCGC operates tool-free using internal model signals rather than external tools, memory buffers, or tree searches.
  • Dual-capability SFT trains both problem-only solving and draft-conditioned correction, supporting the two conditioning modes used at inference.
  • 24.8% average accuracy was achieved on solver-failure hard sets across math, code, and knowledge reasoning benchmarks.

2 Related Works

Related work identifies sequential error propagation and limited global editing in autoregressive refinement, while diffusion models offer parallel alternatives. DCGC extends adaptive guidance to multi-context correction by using relative confidence to filter auxiliary signals.

  • Autoregressive refinement methods inherently suffer from sequential error propagation and lack global editing flexibility.
  • Diffusion language models provide parallel, non-autoregressive alternatives, but prior work predominantly used them for generation rather than global correction.
  • Classifier-free guidance can steer global denoising for non-sequential, fine-grained correction while preserving valid logic.
  • Existing adaptive CFG methods target single-condition trajectories, whereas DCGC modulates guidance using the relative certainty gain from a secondary context.

3 Preliminaries

Masked diffusion corrupts token sequences with masks and reconstructs them through iterative denoising. Classifier-free guidance combines conditional and unconditional predictions to increase adherence to the conditioning context.

  • Masked Diffusion Model: The forward masked-diffusion process independently corrupts clean tokens according to a monotonically decreasing noise schedule.
  • Masked Diffusion Model: The reverse process reconstructs the clean sequence from a corrupted state using a neural denoiser and posterior-substitution transitions.
  • Training objective: Training minimizes a variational lower bound that simplifies to cross-entropy on masked positions.
  • Classifier-Free Guidance: Classifier-free guidance trains conditional and unconditional distributions by randomly replacing the condition with a null token or empty sequence.
  • Classifier-Free Guidance: A higher guidance scale w emphasizes condition c, reducing diversity while improving prompt relevance.

4 DCGC

DCGC trains MDMs for both problem-only solving and draft-conditioned correction, then combines unconditional, problem, and joint predictions through Dynamic Dual-CFG. Relative confidence controls when draft-derived residual guidance is amplified.

  • Task Formulation: DCGC models problem-only generation as P(G | Q) and draft-conditioned correction as P(G | Q, W).
  • Supervised Fine-Tuning: Mixed-format SFT interleaves problem-solution pairs with problem-draft-solution triples under a masked denoising objective.
  • Dynamic Dual-CFG: A single concatenated condition makes it difficult to control the problem statement and draft separately because they play different roles.
  • Dynamic Dual-CFG: Dynamic Dual-CFG evaluates unconditional, problem-conditioned, and joint problem-draft contexts before combining their logits.
  • Dynamic Dual-CFG: The guided-logit decomposition anchors generation to the problem while adding a scaled relative residual, S2 ⊙(sjoint − sprob), from the draft-conditioned branch.
  • Confidence-Modulated Scaling: Token-level confidence uses the maximum predicted probability, and the draft residual is amplified only when joint confidence exceeds problem-only confidence.

5 Experimental Setups

The evaluation compares DCGC with controlled diffusion and autoregressive self-refinement baselines across mathematics, code, and general reasoning benchmarks. It also examines full-test-set correction and guidance-scaling strategies under fixed evaluation protocols.

  • Benchmarks and Evaluation Protocol: DCGC is evaluated on GSM8K, MATH-500, MBPP-test, HumanEval, MMLU-STEM, and MMLU-Pro across mathematics, code generation, and general reasoning.
  • Benchmarks and Evaluation Protocol: The main experiments use solver-failure hard sets to measure conditional correction of failed reasoning attempts.
  • Benchmarks and Evaluation Protocol: Full-test-set evaluation measures gold-agnostic selective refinement without using ground-truth failure labels to determine whether refinement is needed.
  • Baselines and Variants: Comparisons include controlled MDM variants, tool-free autoregressive self-refinement systems, and guidance variants using problem-only or joint problem-draft conditioning.
  • Overall Performance: Table 1 reports that DCGC achieves the highest average accuracy and the best score on five of six benchmarks.
  • Guidance Scaling: DCGC’s relative dynamic ReLU-based scaling consistently outperforms static and independent guidance variants across all benchmarks.

6 Results

Across math, code, and knowledge benchmarks, DCGC delivers the strongest reported performance among evaluated approaches while ablations show that SFT, separated conditioning, and relative scaling each matter. Gold-agnostic experiments further indicate reliable correction of low-consensus and initially incorrect outputs.

  • Overall performance: DCGC achieves 44.9% on GSM8K, exceeding the strongest autoregressive Self-Refine baseline at 26.4% and the strongest non-DCGC LLaDA variant at 43.9%.
  • Overall performance: DCGC reaches 22.3% on MATH, surpassing the strongest autoregressive baseline at 11.0% and the best non-DCGC LLaDA variant at 18.4%.
  • Overall performance: DCGC obtains 13.1% on HumanEval and top scores of 35.7% on MMLU-STEM and 22.5% on MMLU-Pro, while MBPP remains comparable to the strongest baseline.The authors attribute larger HumanEval gains to its richer problem specifications and more explicit correction constraints.
  • Effect of SFT: Dual-capability SFT raises standard-sampling average accuracy from 4.7 to 18.9, including GSM8K improvement from 7.4% to 32.4%.The SFT stage trains both problem-only solving and draft-conditioned correction capabilities.
  • Limits of Single-Condition Guidance: Dynamic Dual-CFG raises the base LLaDA model’s average accuracy from 4.7 to 11.2 and GSM8K accuracy from 7.4% to 16.7% before task-specific adaptation.
  • Limits of Single-Condition Guidance: Single-CFG (Problem) improves SFT standard sampling from 18.9 to 22.4, but DCGC improves over it on all six benchmarks by adding controlled draft-conditioned residual guidance.
  • Effect of relative scaling: Static dual guidance reaches 20.2 average accuracy below the problem-only baseline at 22.4, while independent scaling reaches 21.5 and still underperforms DCGC’s 24.8.DCGC scales the residual using the relative confidence gain of the joint branch over the problem-only branch.
  • Gold-agnostic evaluation: In gold-agnostic evaluation, DCGC consistently improves final accuracy across GSM8K, MATH, and MMLU-STEM, including correction-subset gains over Maj@5.The evaluation selects low-consensus examples without using ground-truth failure labels at test time.

7 Conclusion

DCGC combines Supervised Fine-Tuning with Dynamic Dual-CFG to correct complex reasoning traces using problem-only and joint problem-draft contexts. Experiments show strong benchmark performance and effectiveness without oracle feedback.

  • DCGC is a masked-diffusion framework for draft-conditioned global correction of complex reasoning traces.
  • The method combines Supervised Fine-Tuning with Dynamic Dual-CFG to adaptively modulate the draft-conditioned residual signal.
  • DCGC generates under both problem-only and joint problem-draft contexts during correction.
  • Extensive experiments across diverse benchmarks show that DCGC significantly outperforms strong baselines on reasoning tasks.
  • DCGC remains effective in realistic settings without oracle feedback.

Limitations

DCGC’s current instantiation is constrained by training-time length filtering and uses self-consistency as a simple uncertainty signal.

  • Samples exceeding 1,028 tokens were discarded during supervised fine-tuning because of computational resource constraints.
  • The 1,028-token limit may restrict DCGC in extremely long-context scenarios.
  • The gold-agnostic correction strategy uses self-consistency as a simple uncertainty signal.
  • More sophisticated triggering criteria may further improve correction efficiency and reliability.

A Derivation of Dual-Guide CFG

The Dual-Guide CFG derivation constructs a guided distribution that combines problem adherence with draft-based refinement in logit space. Its update separates unconditional problem guidance from the joint-context refinement signal.

  • The derivation grounds the linear logit combination in the intersection of two guided probability distributions.
  • Because diffusion-model outputs are unnormalized logits, softmax probabilities and log-probabilities permit equivalent logit-space manipulation up to normalization.
  • The final guided distribution is formulated as a product of two classifier-free guidance distributions for problem adherence and draft refinement.
  • Problem guidance applies standard CFG to anchor generation to the problem statement relative to the unconditional baseline.
  • Refinement guidance captures the corrected logic in the joint context relative to the problem-only context.
  • The derived update adds problem guidance and refinement guidance to the problem-only and joint logits, respectively.

B.1 SFT Dataset Construction Details

The experiments construct balanced multi-domain training data and hard evaluation sets, then test DCGC across source models, scaling functions, and diffusion backbones. Results support refinement beyond the training solver’s error distribution, while transfer is not claimed to be universal.

  • SFT Dataset Construction Details: The unified dataset targets approximately 10,000 unique problems per domain across Mathematics, Coding, and General Reasoning.
  • SFT Dataset Construction Details: The final dataset contains 55,440 training samples and 2,913 validation samples after constructing solution pairs and refinement triplets.
  • SFT Dataset Construction Details: Mathematics supervision uses incorrect Math-Shepherd solutions as drafts paired with ground-truth solutions, yielding approximately 5,100 GSM8K and 14,600 MATH samples.
  • SFT Dataset Construction Details: Coding drafts are generated heuristically without explicit execution verification, while MMLU drafts come from selectively collected Qwen3-8B failure cases.
  • Evaluation and Generalizability: The Hard Set contains only instances initially failed by Llama-3.1-8B-Instruct, focusing evaluation on error correction rather than solving easy problems from scratch.
  • Generalizability to Diverse Initial Solvers: On Mistral-7B-v1 hard sets, DCGC consistently outperforms all baselines across GSM8K, MATH, and MBPP, achieving 42.0% accuracy on GSM8K.
  • Generalizability to Diverse Initial Solvers: On Qwen-2.5-32B hard sets, DCGC achieves 28.8% on GSM8K and 18.6% on MATH, exceeding Qwen-32B self-refinement results of 12.1% and 0.6%.
  • Generalization to the DREAM Backbone: The authors do not claim universal transfer across all masked diffusion models, only instantiation on more than one backbone.

G Empirical Analysis of Confidence-Guided Draft Reuse

DCGC uses a relative confidence gap between problem-only and joint conditioning to selectively control draft reuse during denoising. Analyses show stronger gaps and reuse where the branches disagree, while the gap is not a correctness verifier.

  • Confidence-guided draft reuse: DCGC computes a relative confidence gap between problem-only and joint conditioning to determine the residual scale.Additional draft-conditioned amplification is activated when joint conditioning increases confidence relative to problem-only conditioning.
  • Token-level analysis: Token-level analysis compares branch agreement and disagreement positions using mean confidence gaps and draft-reuse rates.Gate-open positions are those where S2 > 0, and draft reuse is measured by 8-gram overlap with the input draft.
  • Token-level analysis: 0.214 vs. 0.026: the mean confidence gap is approximately eight times larger at disagreement than agreement positions.Draft reuse is also higher at disagreement positions, while the gate scale increases monotonically with branch-disagreement probability (ρ = 0.167; 95% confidence interval [0.150, 0.182]).
  • Example-level analysis: 0.195 and 0.247: confidence-gap correlations with draft reuse remain positive before and after controlling for generation length.The shuffled control weakens the relationship and has a confidence interval including zero, suggesting sensitivity to semantic problem-draft correspondence.
  • Interpretation and limitation: The token- and example-level analyses support the gap’s role as a selective signal for where and how strongly draft information is reused.They do not establish that the gap verifies logical correctness or that a larger gap guarantees successful correction.

I.2 Case Studies: Error Correction across Benchmarks

DCGC’s qualitative case studies show global correction of flawed drafts across GSM8K and MATH. It filters misleading logic in one example and generalizes beyond incomplete examples in another.

  • Case-study mechanism: DCGC evaluates and updates the entire reasoning sequence simultaneously, enabling it to filter logical noise while injecting corrections.This contrasts with autoregressive refinement methods, where errors can propagate through sequential reasoning.
  • GSM8K: On GSM8K, DCGC interprets the final state volume correctly instead of summing misleading past states from the draft.Standard Sampling copies the draft’s flawed logic, although it corrects the arithmetic from 16000 to 18000.
  • MATH: On MATH, DCGC generalizes from n = 6, 8, 12 to the rule that n is a divisor of 24, finding all 7 solutions.The draft, Self-Refine, and Standard Sampling remain limited to the specific examples or exhibit tunnel vision.
Loading 2608.25428v1…