PaperlayerView exact revision on arXiv

Source-linked AI summary

From Denoising to Refining: A Corrective Framework for Vision-Language Diffusion Model

Yatai Ji, Teng Wang, Yuying Ge, Zhiheng Liu, Sidi Yang, Ying Shan, Ping Luo

arXiv:2510.19871v1cs.CL

TL;DR

Parallel decoding in vision-language diffusion can compound token errors into syntactic mistakes and hallucinations. ReDiff trains models to refine synthetic errors and their own drafts; it improves caption quality and stability, including a 11.2-point CLAIR gain over LLaDA-V on CapMas.

  • Problem

    Discrete diffusion models are trained on clean data but generate from noisy intermediate outputs, allowing parallel token errors to compound into syntactic errors and hallucinations.

    Source: §1 · p. 1

  • Method

    ReDiff first learns to revise synthetic errors, then uses an online self-correction loop to learn from expert revisions of its own flawed drafts.

    Source: §1 · p. 2

  • Results

    ReDiff improves coherence, factual accuracy, and few-step stability over traditional denoising; on CapMas, CLAIR improves 11.2 points over LLaDA-V.

    Source: §1 · p. 3, §4.2 · p. 7, §5 · p. 10

  • Takeaways & Limitations

    ReDiff can revisit and refine generated content, including previously unmasked tokens, supporting robust parallel generation.

    Source: §1 · p. 2

    Stage I uses synthetic errors that may not reflect the characteristic mistakes the model itself makes.

    Source: §3.2 · p. 5

Abstract

from arXiv · show

Discrete diffusion models have emerged as a promising direction for vision-language tasks, offering bidirectional context modeling and theoretical parallelization. However, their practical application is severely hindered by a train-inference discrepancy, which leads to catastrophic error cascades: initial token errors during parallel decoding pollute the generation context, triggering a chain reaction of compounding errors and leading to syntactic errors and semantic hallucinations. To address this fundamental challenge, we reframe the generation process from passive denoising to active refining. We introduce ReDiff, a refining-enhanced diffusion framework that teaches the model to identify and correct its own errors. Our approach features a two-stage training process: first, we instill a foundational revision capability by training the model to revise synthetic errors; second, we implement a novel online self-correction loop where the model is explicitly trained to revise its own flawed drafts by learning from an expert's corrections. This mistake-driven learning endows the model with the crucial ability to revisit and refine its already generated output, effectively breaking the error cascade. Extensive experiments demonstrate that ReDiff significantly improves the coherence and factual accuracy of generated content, enabling stable and efficient parallel generation far superior to traditional denoising methods. Our codes and models are available at https://rediff-hku.github.io/.

1 INTRODUCTION

Discrete diffusion offers bidirectional context modeling and potential parallel speedups, but clean-data training leaves models vulnerable to compounding errors during parallel generation. ReDiff shifts from passive denoising to active refining, training models to correct synthetic errors and their own flawed drafts; it improves coherence, factual accuracy, and few-step stability.

1 INTRODUCTION

  • In parallel decoding, errors in a few tokens pollute the context for other tokens, compounding into incoherent artifacts and fabricated descriptions.The train-inference discrepancy arises because models train on clean ground truth but generate from noisy intermediate outputs.Source: §1 · p. 1
  • ReDiff uses two-stage training: it first learns to correct synthetic errors, then learns from expert revisions of its own flawed drafts.The online self-correction loop is intended to address the mismatch between clean training data and noisy inference outputs.Source: §1 · p. 2
  • Active refining lets the model revisit and refine generated content, including previously unmasked tokens, rather than only filling masked tokens.The paper presents this ability as a way to break error cascades and support robust parallel generation.Source: §1 · p. 2
  • ReDiff improves coherence and factual accuracy and is more stable than traditional denoising in challenging few-step parallel generation.Figure 1 also reports superior generation quality and more stable results with fewer inference steps.Source: Figure 1 · p. 2, §1 · p. 3

2 RELATED WORK

Related work describes discrete diffusion's parallel-generation potential and the difficulties of maintaining output quality, alongside established vision-language architectures and hallucination challenges. ReDiff is distinguished by learning to correct its own characteristic errors through refinement.

2.1 LARGE LANGUAGE DIFFUSION MODELS

  • Discrete text diffusion corrupts tokens rather than adding Gaussian noise; mask-and-predict models progressively unmask tokens, as in LLaDA (Nie et al., 2025).Discrete Markov-chain approaches can corrupt tokens toward a uniform distribution or an absorbing state such as [MASK].Source: §2.1 · p. 3
  • SEED-Diffusion adds edit-specific noise, while FUDOKI revises random sentences; ReDiff learns from its own characteristic errors.The paper frames its approach as high-level refinement rather than another form of noise.Source: §2.1 · p. 3

2.2 LARGE VISION LANGUAGE MODELS

  • LVLMs commonly connect a pretrained vision encoder to an autoregressive language model, then use visual instruction tuning for vision-centric tasks.The connection can use a lightweight module such as an MLP or Q-Former.Source: §2.2 · p. 3
  • LLaDA-V, LaViDa, and MMaDA fix tokens after unmasking, whereas ReDiff can revisit generated content.The paper links this refinement capability to mitigating hallucination.Source: §2.2 · p. 3
  • Figure 2 depicts ReDiff-Base learning to revise synthetic caption errors, followed by ReDiff learning from expert revisions of its own drafts.The two stages target general correction and then the model's characteristic mistakes.Source: Figure 2 · p. 4

3 METHODOLOGY

ReDiff shifts discrete diffusion from recovering masked tokens to refining generated text, enabling correction of both prior errors and new tokens during generation. Its training combines foundational revision of synthetic errors with iterative learning from expert-corrected drafts.

3 METHODOLOGY

  • ReDiff trains the model to refine generated text while predicting new tokens, rather than only recovering masked text.The framework is designed to mitigate error cascades in parallel generation.Source: §3 · p. 4, §3.1 · p. 5

3.1 PRELIMINARIES OF DISCRETE DIFFUSION MODELS

  • Discrete diffusion predicts masked tokens using bidirectional context, but treating already-unmasked tokens as fixed can let an incorrect prediction derail later steps.ReDiff uses the models’ bidirectional attention to update previously generated tokens.Source: §3.1 · pp. 4–5

3.2 STAGE I: FOUNDATIONAL REVISION TRAINING

  • Stage I teaches revision of syntactic chaos and factual hallucinations using synthetic corruptions of ground-truth text and human-corrupted captions.The resulting ReDiff-Base can correct both error types, but synthetic errors may not reflect the model’s characteristic mistakes.Source: §3.2 · p. 5

3.3 STAHE II: ONLINE SELF-CORRECTION LEARNING

  • Online self-correction trains on the model’s drafts after an expert corrects grammatical and hallucinatory errors, focusing loss on the corrected segments.The cycle can be repeated with newly generated drafts, while a small amount of Stage I data helps preserve foundational capabilities.Source: §3.3 · pp. 5–6

3.4 INFERENCE PROCESS

  • At each inference step, ReDiff unmasks the top-n most confident masked tokens and replaces previously unmasked tokens with new predictions.This simultaneously adds content and revises existing tokens as context grows.Source: §3.4 · p. 6

4 EXPERIMENTS

Experiments evaluate ReDiff on detailed image captioning, where it outperforms diffusion baselines across benchmarks and remains more stable as parallel generation speeds up. Ablations and qualitative examples examine the training stages and the model’s refinement behavior.

4.1 EXPERIMENT SETTINGS

  • Experiments use LLaDA-V for detailed image captioning, with LLaVA-1.5, ShareGPT4V, and ViCrit training data and CapMAS, CapArena, and DetailCaps-4870 evaluation.Stage I uses 260k image-text pairs; each Stage II round generates about 10k draft-refined pairs, and o4-mini provides expert revisions.Source: §4.1 · pp. 6–7

4.2 MAIN RESULTS

  • 61.88 CAPTURE surpasses Qwen2.5-VL, while ReDiff achieves state-of-the-art results among diffusion models across CapMAS metrics and outperforms LLaDA-V on CapArena.On CapMAS, CLAIR improves by 11.2 points over LLaDA-V; Coverage and Factuality rise by 5.85 and 2.23 points, respectively.Source: §4.2 · p. 7, Table 2 · p. 7, Table 3 · p. 8
  • With identical datasets and base model, ReDiff degrades more gracefully than mask-pred training as generation accelerates; its CLAIR score falls 9.30 points from 1 to 8 tokens/step, versus 28.15.At 4 tokens/step, ReDiff also exceeds both LLaDA-V and the mask-trained baseline at 1 token/step; CAPTURE drops only 0.65 points from 1 to 4 tokens/step.Source: §4.2 · p. 7, Table 2 · p. 7, Table 3 · p. 8

4.3 ABLATION STUDIES

  • Both training stages improve performance and stability over LLaDA-V, with their combination yielding the largest gains; Stage II alone contributes more than Stage I.Stage I alone maintains stable parallel generation, while the combined stages add 5.25 Factuality points and 17.67 CapArena points.Source: §4.3 · pp. 7–8, Table 4 · p. 8
  • Syntactic-error revision mainly boosts CLAIR and Coverage, hallucination revision raises Factuality, and combining error types performs best; fixed token replacement outperforms a dynamic rate.The first online self-correction round substantially improves ReDiff-Base, while subsequent rounds yield no further significant gains across most metrics.Source: §4.3 · p. 8, Table 5 · p. 8, Table 6 · p. 9

4.4 QUALITATIVE ANALYSIS

  • During 32-step generation, ReDiff both unmasks new tokens and revises earlier ones, and it can correct user-provided captions containing syntactic chaos or factual hallucinations.Examples include changing “rocks painted rocks” to “colorful painted rocks” and revising “a tall green plant” to “a small green plant.”Source: §4.4 · pp. 9–10, Figure 4 · p. 9, Figure 6 · p. 10
  • Without inference refinement, errors accumulate as repeated words or symbols and incoherent sentences; qualitative comparisons show ReDiff captions are more fluent, coherent, and factually grounded.The comparison is illustrated at 4 tokens/step.Source: §4.4 · pp. 9–10, Figure 3 · p. 9, Figure 5 · p. 10

5 CONCLUSION

ReDiff reframes vision-language diffusion generation as active refining, using mistake-driven online self-correction to revisit and revise generated output. Experiments report state-of-the-art performance and greater stability and factual accuracy in few-step generation regimes.

5 CONCLUSION

  • ReDiff addresses error cascades in vision-language diffusion, particularly during efficient parallel generation, through a mistake-driven online self-correction loop.The model learns from its own characteristic errors and can revisit and refine its generated output.Source: §5 · p. 10
  • Experiments report state-of-the-art performance and greater stability and factual accuracy in challenging few-step generation regimes than traditional denoising models.The conclusion describes traditional denoising models as catastrophically failing in these regimes.Source: §5 · p. 10
  • The work presents ReDiff as a promising path toward more robust, efficient, and controllable generative systems.Source: §5 · p. 10

A MORE VISUALIZATION

The visual comparisons show ReDiff producing image-consistent captions where LLaDA-V hallucinates at 2 tokens per step, and more fluent text with fewer grammar errors at 8 tokens per step.

A MORE VISUALIZATION

B PROMPT FOR STAGE II DATA CONSTRUCTION

Stage II online self-correction uses model-generated caption drafts and expert revision to construct training examples.

B PROMPT FOR STAGE II DATA CONSTRUCTION

C ETHICS STATEMENT

The ethics statement reports that the study uses public, open-source materials and no private or personally identifiable information.

C ETHICS STATEMENT

Loading 2510.19871v1…