PaperlayerView exact revision on arXiv

Source-linked AI summary

DODO: Discrete OCR Diffusion Models

Sean Man, Gilad Deutch, Roy Ganz, Roi Ronen, Shahar Tsiper, Shai Mazor, Niv Nayman

arXiv:2602.16872v2cs.CV

TL;DR

OCR transcription is costly with sequential autoregressive decoding, yet global masked diffusion is unstable under OCR’s exact, rigid output requirements. DODO addresses this gap with block discrete diffusion that anchors generation locally while retaining parallel decoding. It achieves competitive accuracy with state-of-the-art autoregressive models and up to 5× faster inference.

  • Problem

    Autoregressive OCR decoding is computationally expensive for long documents, while global masked diffusion suffers from length and positional instabilities that exact transcription cannot tolerate.

  • Method

    DODO decomposes OCR generation into sequentially conditioned blocks using block discrete diffusion, local causal anchoring, dynamic length adaptation, and cached committed prefixes.

  • Results

    DODO achieves transcription accuracy competitive with state-of-the-art autoregressive models and up to 5× faster inference.

  • Takeaways & Limitations

    Structured block diffusion reconciles causal consistency with parallel generation for OCR and makes masked-diffusion decoding effective for document transcription.

  • Takeaways & Limitations

    Exact KV-caching fixes the history and limits accuracy at larger block sizes compared with the bidirectional variant.

Abstract

from arXiv · show

Optical Character Recognition (OCR) is a fundamental task for digitizing information, serving as a critical bridge between visual data and textual understanding. While modern Vision-Language Models (VLM) have achieved high accuracy in this domain, they predominantly rely on autoregressive decoding, which becomes computationally expensive and slow for long documents as it requires a sequential forward pass for every generated token. We identify a key opportunity to overcome this bottleneck: unlike open-ended generation, OCR is a highly deterministic task where the visual input strictly dictates a unique output sequence, theoretically enabling efficient, parallel decoding via diffusion models. However, we show that existing masked diffusion models fail to harness this potential; those introduce structural instabilities that are benign in flexible tasks, like captioning, but catastrophic for the rigid, exact-match requirements of OCR. To bridge this gap, we introduce DODO, the first VLM to utilize block discrete diffusion and unlock its speedup potential for OCR. By decomposing generation into blocks, DODO mitigates the synchronization errors of global diffusion. Empirically, our method achieves near state-of-the-art accuracy while enabling up to 5x faster inference compared to autoregressive baselines.

1. Introduction

OCR is deterministic but costly under sequential autoregressive decoding, while global masked diffusion introduces structural errors that exact transcription cannot tolerate. DODO uses block discrete diffusion to preserve alignment and length flexibility, achieving competitive accuracy with faster inference.

  • DODO: DODO is presented as the first VLM using block discrete diffusion for OCR.This contribution targets the efficiency bottleneck created by sequential autoregressive decoding in large-document transcription.
  • Motivation: OCR’s uniquely rigid, image-determined output makes parallel decoding theoretically suitable but leaves no room for positional or length errors.Unlike captioning, OCR permits essentially one valid sequence, so incorrect token placement or sequence length cannot be repaired by alternative phrasing.
  • Motivation: Standard masked diffusion can truncate valid text or hallucinate padding when its fixed canvas or positional alignment is wrong.These structural instabilities are recoverable in flexible generation but catastrophic for exact OCR transcription.
  • DODO: DODO introduces block discrete diffusion, decomposing generation into sequentially conditioned blocks to enforce local alignment and dynamic length adaptation.The block structure addresses the rigidities of global diffusion while retaining parallel generation within blocks.
  • Results: 5× faster inference is achieved while DODO matches the accuracy of state-of-the-art autoregressive baselines.The result supports parallel decoding for dense text recognition without sacrificing reported transcription accuracy.

2. Related Work

Prior multimodal diffusion models struggle with precise OCR, while block diffusion combines autoregressive conditioning across blocks with parallel masked-diffusion decoding within them. DODO applies this strategy to VLM-based OCR, addressing the gap between diffusion efficiency and autoregressive accuracy.

  • Specialized OCR: Specialized OCR systems predominantly use autoregressive decoding, making DODO the first reported successful MDM application to OCR.The related work positions DODO against systems such as MonkeyOCR, MinerU, dots.ocr, DeepSeek-OCR, and Mistral-OCR.
  • Discrete Diffusion: Discrete diffusion models reverse corruption over discrete tokens, with masked diffusion simplifying training through a tighter evidence lower bound.The related work identifies a remaining perplexity gap with autoregressive language models and frames this paper as narrowing it for OCR.
  • Block Diffusion: Block diffusion generates token blocks autoregressively during inference while decoding each block with masked diffusion, enabling KV-caching across blocks.This design preserves parallel decoding within blocks and bridges autoregressive and diffusion approaches.
  • Multimodal Diffusion Models: Prior multimodal diffusion models such as LaViDa and LLaDA-V struggle with OCR tasks requiring precise text reproduction.Dimple does not evaluate on OCR benchmarks, while these systems are discussed as limited for exact transcription.

3. Preliminaries

This section formulates OCR as conditional sequence generation and introduces masked diffusion models, whose parallel decoding depends on conditional independence but whose carry-over unmasking prevents revision of revealed tokens.

  • OCR as Conditional Sequence Modeling: OCR maps a document image I to a serialized, tokenized target sequence x1:L, modeled conditionally on the image and optional text context.The serialization may be plain text, LATEX, or HTML.
  • OCR as Conditional Sequence Modeling: Autoregressive decoding factorizes the conditional distribution left to right, requiring one sequential step for each token.The prefix x<l denotes tokens generated before step l.
  • Masked Diffusion Models: Masked diffusion corrupts discrete tokens by replacing them with a dedicated [MASK] token at a noise level t, then trains a denoiser to recover original tokens.The training objective is an ELBO-derived weighted masked cross-entropy.
  • Masked Diffusion Models: MDM sampling begins from a fully masked sequence and iteratively reveals selected positions while sampling their token values from denoiser estimates.Selection may be random, top-k, confidence-based, or deterministic.
  • Conditional Independence Assumption: Conditional independence permits parallel token decoding when tokens are independently sampled at each step, but incorrect dependence can produce erroneous results.The assumption creates substantial parallelization potential when valid.
  • Carry-Over Unmasking: Carry-over unmasking allows sampling only at masked positions, so previously revealed tokens cannot be revised.This constraint can be detrimental when only one output sequence is correct.

4. Method

The method analyzes why global masked diffusion is brittle for rigid OCR and uses block discrete diffusion to combine sequential prefix anchoring with parallel decoding inside bounded blocks.

  • 4.1. Parallel Decoding Potential: OCR’s highly peaked posterior can effectively decouple token predictions, allowing large spans to be decoded simultaneously.The visual input strongly constrains document transcription toward a single valid sequence.
  • 4.2. Brittleness of Parallel Decoding in OCR: Global masked diffusion is brittle because fixed-length canvases and immutable early predictions expose OCR to length mismatch and positional misalignment.These errors are tolerable in flexible generation but hazardous for exact transcription.
  • 4.2. Brittleness of Parallel Decoding in OCR: Incorrect initial length or premature [EOS] can truncate valid text when the canvas is too short or force hallucinated content when it is too long.OCR cannot freely paraphrase to fit an imposed canvas.
  • 4.2. Brittleness of Parallel Decoding in OCR: Parallel absolute-position predictions can place segments at incorrect offsets, and carry-over unmasking locks those errors into fractured outputs.Because revealed tokens are not revised, disjoint segments may collide.
  • 4.3. Block Diffusion as a Structural Remedy: Block diffusion replaces one length-L denoising problem with bounded-span problems conditioned on a prefix of previously decoded blocks.Blocks are processed sequentially while tokens within each block are decoded in parallel.
  • 4.3. Block Diffusion as a Structural Remedy: Block factorization anchors indices at boundaries, reduces length sensitivity, supports block-level stopping, and reuses committed-prefix KV caches.This retains parallel token updates within each block while enabling variable-length generation.
  • 4.3. Block Diffusion as a Structural Remedy: DODO scales block size from 32 to 256 tokens using OCR properties and bidirectional cross-block attention, applying block-causal masking during training and inference.The paper presents this as the first multimodal VLM application of block-causal masking in both phases.

5. Experiments

DODO is evaluated on diverse OCR benchmarks against specialized, autoregressive, and diffusion baselines. It achieves strong transcription quality while substantially improving inference throughput through parallel decoding, block-causal attention, and KV-caching.

  • Setup: DODO is trained on approximately 270K PDF-derived document-text pairs and evaluated on OmniDocBench and Fox-Page-EN.OmniDocBench covers 290 layout-sensitive documents, while Fox-Page-EN contains 112 text-only pages.
  • Main results: DODO achieves an NED of 0.069 on OmniDocBench, substantially outperforming prior diffusion-based VLMs that exceed 0.5 NED.Lower NED indicates higher transcription fidelity.
  • Main results: DODO surpasses its Qwen2.5-VL autoregressive backbone across all model scales and reaches near-parity with MonkeyOCR and Mistral OCR.It also outperforms various specialized OCR models.
  • Throughput analysis: DODO achieves approximately 105 tokens per second, delivering a 5× speedup over the autoregressive Qwen 2.5 VL baseline.The speedup combines parallel decoding with block-causal attention and exact KV-caching.
  • Throughput analysis: DODO’s block-wise computation with cached prefixes yields higher throughput than diffusion VLMs that repeatedly attend over the full sequence.Global diffusion baselines incur computation proportional to the total sequence length at every denoising step.

6. Ablation and Empirical Analysis

Ablations show that block-wise training is central to DODO’s accuracy and efficiency. Exact caching and carefully chosen block structure avoid the synchronization failures and accuracy collapse observed with vanilla or approximate alternatives.

  • 6.1. Vanilla vs. Block Training: Vanilla masked diffusion retains high error rates even with oracle sequence length, indicating positional anchoring—not only length estimation—causes fractured outputs.Global parallel decoding cannot adjust offsets across disjoint text segments.
  • 6.1. Vanilla vs. Block Training: Inference-time block decoding alone performs poorly without a matching block-based training objective.This differs from results reported for math and coding benchmarks.
  • 6.1. Vanilla vs. Block Training: DODO’s global baseline has lower throughput because it computes attention over up to 8192 tokens at every denoising step.Block decomposition distributes computational cost more efficiently.
  • 6.2. Block Size and Caching Strategies: Approximate KV-caching causes a sharp accuracy collapse, while block-causal training enables exact KV-caching.The contrast supports training-time alignment between the attention structure and caching strategy.
  • 6.2. Block Size and Caching Strategies: Bidirectional attention lets DODO scale its usable block size from 32 to 256 tokens, whereas larger blocks in standard bidirectional decoding degrade accuracy.DODO performs best with smaller blocks, but bidirectional attention improves accuracy at B = 256.
  • 6.1. Vanilla vs. Block Training: Fewer than 0.1 inference steps per token compresses DODO’s sequential workload by an order of magnitude relative to autoregressive decoding.DODO generates 10 to 20 tokens per autoregressive step, offsetting the higher cost of an uncached bidirectional pass.

7. Conclusion

The paper concludes that DODO makes discrete diffusion a practical alternative for latency-sensitive OCR by combining block-structured parallel generation with causal stability. Its main remaining limitation is the accuracy constraint imposed by exact caching at larger block sizes.

  • Conclusion: DODO combines semiautoregressive blocks, block-causal attention, and exact KV-caching to achieve competitive OCR performance with a 5× inference speedup.The conclusion positions discrete diffusion as a practical alternative to autoregressive OCR systems.
  • Conclusion: DODO’s diffusion architecture also supports bidirectional attention across blocks, increasing the utilizable block size from 32 to 256 tokens.This capability further improves accuracy.
  • Conclusion: Exact KV-caching fixes history representations, which limits accuracy at larger block sizes compared with the bidirectional variant.Future work targets this gap and OCR-specific diffusion samplers.

A. Attention Structure Visualization

The attention visualization contrasts full bidirectional attention with block-causal masking. Block-causal masking preserves committed-prefix representations, enabling exact KV-caching for faster inference.

  • A. Attention Structure Visualization: Full bidirectional attention lets prior blocks attend to the current block, while block-causal masking prevents that backward interaction.The two masking structures differ in whether prior-block representations can adapt during the forward pass.
  • A. Attention Structure Visualization: Block-causal masking keeps history representations invariant, enabling exact KV-caching and faster inference.Only the active block must be recomputed at each step.

B.1. Sampling Strategies

The evaluation compares adaptive confidence thresholding with fixed-rate top-k decoding, prioritizing the fastest strategy that preserves OCR fidelity. Confidence thresholding at p = 0.99 offers the best reported balance between accuracy and speed.

  • Sampling strategies: Confidence Thresholding dynamically unmasks tokens whose prediction probability exceeds a fixed threshold p, producing adaptive step sizes.The model accelerates through clear text and slows down for ambiguous regions.
  • Sampling strategies: p = 0.99 is the default thresholding setting used in the main experiments.
  • Sampling strategies: Confidence Top-K unmasks exactly K tokens per step, maintaining a fixed generation rate regardless of model confidence.
  • Sampling strategies: Confidence Thresholding at p = 0.99 provides the optimal balance in the high-accuracy regime required for OCR.Aggressive Top-K can increase raw throughput but incurs unacceptable error rates.

C. Fine-Grained Evaluation Breakdown

The fine-grained evaluation shows that DODO’s advantage increases on longer documents and remains consistent across document types. Qualitative examples further show accurate transcription of dense content and complex layouts with parallel decoding.

  • By document length: 0.079 vs. Qwen 7B’s 0.185 is DODO’s edit distance at 4096+ tokens, where longer sequences amplify parallel-decoding benefits.Lower edit distance is better; at 0–128 tokens, methods perform similarly because sequential overhead is minimal.
  • By document type: 0.014 on PPT slides and 0.050 on academic papers are particularly strong DODO edit distances across document types.The bidirectional variant further improves results on structured Academic and Book layouts.
  • Sampling strategy comparison: Confidence Thresholding achieves the optimal speed–accuracy balance at p = 0.99.Figure B2 compares edit distance and speed across sampling strategies in the OCR high-accuracy regime.
  • Qualitative results: DODO’s qualitative results demonstrate transcription of dense text while preserving complex layout structures.The examples are drawn from OmniDocBench and include accompanying visualizations of the parallel decoding process.
  • Qualitative results: DODO recovers multi-column text, tables, and mathematical formulas while maintaining high parallel efficiency.The decoding heatmap visualizes token commitment order alongside the generated transcript rendered to PDF.
  • Qualitative results: Figures D3 and D4 provide additional qualitative results for DODO’s document transcription behavior.
Loading 2602.16872v2…