PaperlayerView exact revision on arXiv

Source-linked AI summary

OCR-EDR: Rendering-Aware Diagnosis and Repair for Closed-Loop OCR Improvement

Linnan Zhao, Kang Liu, Hao Yu, Jiabo Zhan, Chong Sun, Chen Li

arXiv:2609.03445v1cs.CV

TL;DR

Complex document OCR remains error-prone, and aggregate evaluation offers limited insight into individual errors. OCR-EDR jointly evaluates images, editable predictions, and renderings, then supports iterative repair; DocEDR reports diagnostic and repair results across benchmarks and OCR systems.

  • Problem

    Complex formulas, structured text, and long-tail formats remain error-prone, while aggregate scores provide limited insight into individual OCR error patterns.

    Source: Introduction · p. 1

  • Method

    OCR-EDR jointly assesses a source image, editable OCR prediction, and rendering, then supports executable edits and rendering-based reassessment; DocEDR instantiates this loop.

    Source: Introduction · p. 2, DocEDR · p. 4

  • Results

    Across benchmarks, DocEDR achieves 94.78% diagnostic accuracy, repairs 86.23% of erroneous inputs to visual consistency, and improves formula CDM by up to 4.62 points on identified Bad subsets of four OCR systems.

    Source: Introduction · p. 2

  • Takeaways & Limitations

    OCR-EDR turns difficult OCR cases into actionable feedback and verified corrections across benchmarks and OCR systems.

    Source: Introduction · p. 2, Conclusion · p. 7

Abstract

from arXiv · show

Although document OCR systems perform increasingly well on routine documents, complex formulas, structured text, and long-tail formats remain error-prone. OCR predictions may omit fine-grained content or hallucinate unsupported outputs, while equivalent encodings of the same visible content must be accommodated. Existing OCR evaluation methods mostly report aggregate metrics, offering limited support for analyzing case-level errors and improving OCR performance. We propose OCR-EDR (OCR Error Diagnosis and Repair), a rendering-aware framework that advances from fine-grained diagnosis to iterative repair. Given a source image, an editable OCR prediction, and its rendered image, OCR-EDR first jointly assesses whether the prediction and its rendering are consistent with the source, preserving valid predictions, including rendering-equivalent ones, while diagnosing and localizing genuine errors. It then applies executable edits and may request an updated rendering for iterative reassessment. We construct OCRErrBench from diverse real OCR predictions, covering text and formulas, exact and rendering-equivalent positives, and genuine errors, and develop the DocEDR model to execute the diagnosis--repair loop. On OCRErrBench, DocEDR achieves 94.78% diagnostic accuracy. It repairs 86.23% of erroneous inputs to visual consistency, raises formula Case-F1 by 30.99 percentage points over DOCR-Inspector-7B on DOCRcaseBench, and improves formula CDM by up to 4.62 percentage points on the identified Bad subsets of four OCR systems on UniMER-Test. These results show that OCR-EDR turns fine-grained OCR analysis into verified corrections and performance gains.

Introduction

OCR-EDR addresses the difficulty of diagnosing individual OCR errors while preserving different encodings that render identically. It combines diagnosis, executable repair, and rendering-based reassessment, with DocEDR evaluated on a new benchmark and across external OCR predictions.

Introduction

  • Complex formulas, structured text, and long-tail formats remain error-prone, while aggregate scores and error counts reveal little about individual failure patterns.Fine-grained diagnosis and repair can clarify failure modes and directly improve predictions.Source: Introduction · p. 1
  • OCR-EDR jointly assesses a source image, editable OCR prediction, and rendered image, preserving visually consistent predictions while diagnosing, localizing, and repairing genuine errors.The prediction can be revised, rerendered, and visually reassessed rather than treated as a fixed output.Source: Introduction · pp. 1–2
  • OCRErrBench contains 900 balanced text and formula cases: 457 valid and 443 erroneous predictions, including rendering-equivalent positives and genuine errors.Cases are annotated for validity, error type, location, and target correction.Source: Introduction · p. 2
  • DocEDR achieves 94.78% diagnostic accuracy and repairs 86.23% of erroneous inputs to visual consistency on OCRErrBench; formula CDM improves by up to 4.62 points on identified Bad subsets of four OCR systems.It also improves formula Case-F1 by 30.99 percentage points over DOCR-Inspector-7B on DOCRcaseBench.Source: Figure 1 · p. 1, Introduction · p. 2
  • DocEDR instantiates OCR-EDR as a unified policy, learning validity assessment, classification, and localization before staged repair training.Repair training covers preservation, local or global editing, optional rerendering, reassessment, and stopping.Source: Introduction · p. 2

Related Work

Prior OCR and post-correction work improves recognition or diagnoses errors, but the cited approaches do not combine visual verification, preservation of equivalent outputs, and repair in one process.

Related Work

  • Document OCR has advanced toward end-to-end generation and broader recognition-and-reasoning evaluation, but residual errors are not thereby made diagnosable, repairable, and reusable as supervision.The passage surveys modular and OCR-free systems as well as realistic document and formula benchmarks.Source: Document OCR · p. 2
  • Text-based post-correction improves noisy OCR sequences but does not verify an edit’s visual consequence; DOCR-Inspector diagnoses errors, while LATTE iteratively repairs formulas and tables in task-specific formats.Neither cited approach unifies correct-result preservation, diagnosis across text and formulas, and reuse of verified corrections (Zhang et al. 2025).Source: OCR Post-Correction · pp. 2–3
  • OCR-EDR jointly compares the source region, editable prediction, and rendering, preserving non-canonical predictions when they are rendering-equivalent and correcting inconsistent ones.This task framing incorporates preservation alongside diagnosis and correction.Source: Figure 2 · p. 3, OCR Post-Correction · p. 3

Method

OCR-EDR treats OCR correction as a stateful interaction over the source image, current prediction, and rendering. Its policy diagnoses error scope, applies localized or global edits, and can refresh the rendering before continuing or stopping.

Method

  • At each turn, the policy observes the source image, current OCR result, rendering, and prior trajectory, then may diagnose, edit, request a rendering, or stop.After an edit, the prior rendering is stale until a new rendering is requested; the final result is valid when content and visual structure agree with the source, even if its string differs from a canonical reference.Source: Task Formulation · p. 3
  • OCR-EDR distinguishes ⊥stale, a rendering made outdated by an edit, from ⊥fail, an explicit rendering failure.The rendering function returns either a rendered image or ⊥fail.Source: Task Formulation · p. 3
  • The stateful correction algorithm initializes the prediction and rendering, updates the prediction after patch actions, marks its rendering stale, and refreshes it on request_render.The loop appends each transition to the trajectory and returns when the policy selects stop.Source: Task Formulation · p. 3
  • The error taxonomy separates global failures from local errors; local errors require an erroneous span or region and corrected content.Completeness covers missing or redundant content, content errors cover misrecognized characters or tokens, and structure errors cover reading order, grouping, hierarchy, or spatial and markup relations.Source: Task Formulation · pp. 3–4
  • Localized insert, delete, or replace operations describe textual changes, while global rewrite is reserved for outputs not reliably repairable through localized edits.The action space distinguishes what is wrong, the textual repair operation, and the policy action controlling diagnosis and repair.Source: Task Formulation · pp. 3–4, Table 1 · p. 4

DocEDR

DocEDR evolves one editable OCR hypothesis through diagnosis, selective rendering, and repair. Its staged training builds diagnostic, editing, feedback, and stopping capabilities before reward optimization.

Policy

  • DocEDR evolves a single OCR hypothesis, using diagnostic actions before local or global edits and requesting updated renderings selectively before stopping.Edits make the previous rendering stale; the policy may stop without another rendering when reassessment is unnecessary.Source: DocEDR · p. 4

Training Strategy

  • A verifier-trained diagnostic checkpoint provides both a visual-consistency signal and an initialization encoding validity, error type, location, and correction cues.The verifier is trained from source crops, OCR candidates, and their renderings.Source: Training Strategy · p. 4

Curriculum Repair SFT

  • Repair SFT progresses from direct edits to selective rendering decisions, then to renderer feedback, residual repair, and recovery from unsuccessful edits.The curriculum introduces each expanded decision space after establishing the preceding behavior.Source: Training Strategy · p. 4

Reward Optimization

  • Label-conditioned rewards favor preserving valid inputs or repairing erroneous ones, while trajectory costs encourage efficient recovery and rendering only when useful.Equivalent positives remain in the preservation branch rather than being forced toward a canonical transcription.Source: Training Strategy · pp. 4–5

Experiments

Experiments evaluate DocEDR’s diagnosis and repair on OCRErrBench, then test transfer to external OCR benchmarks and ablate input modalities and rendering-based iteration.

Benchmark Construction

  • OCRErrBench uses source-isolated real OCR outputs and includes exact positives, rendering-equivalent positives, and genuine errors, with difficult annotations reviewed by experts.Automatic acceptance requires complete agreement among three model branches; rendering-equivalent positives receive an additional manual audit.Source: OCRErrBench Construction · p. 5

Diagnostic Results on OCRErrBench

Boundary and Cross-Benchmark Diagnosis

  • 25.81% equivalent-positive FP: DocEDR lowers false positives while reducing completeness, content, and structure FN to 1.85%, 4.35%, and 5.49%.Edit distance falsely rejects 57.14% of equivalent text, while CDM has 2.94% FP on equivalent formulas; both are nearly insensitive to subtle local errors in this subset.Source: Table 3 · p. 6, Boundary and Cross-Benchmark Diagnosis · p. 6
  • 56.69% to 87.68% formula Case-F1: DocEDR improves over DOCR-Inspector-7B on DOCRcaseBench, adding 61 correct formula cases.The gain corresponds to 31.77% of the formula subset.Source: Table 4 · p. 6, Boundary and Cross-Benchmark Diagnosis · p. 6

Repair Results

  • 82.17% ExactFix and 86.23% VisFix exceed the strongest direct baseline by 55.98 and 14.00 points, respectively; Preserve reaches 93.44%.VisFix gain has a paired 95% CI of 9.48–18.74 (p = 6.44 × 10−9).Source: Table 5 · p. 6, Repair Results · p. 6

Formula Recognition Refinement

  • 2.24–4.62 CDM points: repair improves every fixed Bad partition on UniMER-Test, with the largest gain on GLM-4.6V-Flash.Good partitions cover 62.82–77.81% of predictions; their CDM remains high for MinerU2.5 and PaddleOCR-VL-1.5.Source: Table 6 · p. 7, Formula Recognition Refinement · p. 7

Further Ablations

  • 94.78% accuracy, 94.59% Bad-F1, and 74.88% Type-F1 are reached with I + p + R; removing p or R weakens localization or equivalent-positive preservation.Without the OCR sequence, text/formula localization falls to 80.54/55.25%; without rendering, equivalent-positive FP doubles from 4.75% to 8.86%.Source: Further Ablations · p. 7
  • Rendering adds 3.61 ExactFix and 3.39 VisFix points in one step; iteration without rendering lowers ExactFix to 66.82%, versus 82.17% ExactFix and 86.23% VisFix for the complete loop.Initial rendering establishes the diagnostic boundary, and updated rendering closes the repair loop.Source: Further Ablations · p. 7

Conclusion

OCR-EDR unifies rendering-aware diagnosis, repair, and reuse for difficult OCR outputs. OCRErrBench and DocEDR support evaluation and execution of that loop, with reported cross-benchmark diagnosis and correction gains.

Conclusion

  • OCR-EDR brings rendering-aware diagnosis, repair, and reuse of difficult OCR outputs into one framework.The paper presents it as a unified approach across text and formulas.Source: Conclusion · p. 7
  • OCRErrBench evaluates preservation, diagnosis, localization, and correction, while DocEDR executes an edit–render–reassess loop.Source: Conclusion · p. 7
  • Cross-benchmark diagnosis, high repair accuracy, and consistent gains on external OCR predictions support treating difficult cases as actionable feedback rather than terminal failures.Source: Conclusion · p. 7

A OCRErrBench Details

OCRErrBench is built from real OCR predictions, with document-level source isolation from model training. Its annotation protocol verifies equivalence and repairs through rendering and review.

A OCRErrBench Details

  • OCRErrBench contains 900 real OCR outputs, balanced between text and formula crops, and is isolated from diagnostic and repair training at the source-document level.Predictions are collected from DeepSeek-OCR, PaddleOCR-VL-1.5, and Qwen3.5-27B on OmniDocBench and PureDocBench.Source: A OCRErrBench Details · p. 10
  • Rendering-equivalent positives differ from the reference in source form but produce identical visible content, and are admitted only after rendering comparison and manual audit.Equivalence checks use a fixed local rendering environment; rendering failure is not accepted as evidence of equivalence.Source: A OCRErrBench Details · p. 10
  • Three-model agreement can automatically accept annotations; difficult cases receive independent expert review, and residual disputes are retained only when reconciled repairs recover the reference.Source: A OCRErrBench Details · p. 10

B Baseline Protocol and Qualitative Examples

The protocol standardizes VLM and DOCR-Inspector evaluation, including structured error outputs and localization scoring. Qualitative examples show DocEDR preserving visually equivalent OCR that DOCR-Inspector flags as erroneous.

Baseline Protocol

  • VLM baselines receive the ordered source-image, rendered-OCR, and OCR-source triplet under fixed decoding, without reference labels or corrections at inference.The protocol caps OCR input at 6,000 characters and output at 2,048 tokens.Source: Unified VLM Protocol · p. 10
  • The required JSON output records verdicts, atomic error types, edit operations, and source-grounded contexts; global rewrites are reserved for invalid outputs or global mismatches.The error types are invalid_output, global_mismatch, completeness, content, and structure.Source: Unified VLM Protocol · p. 10
  • DOCR-Inspector-7B keeps its native single-image decision rule, while a frozen, ground-truth-blind adapter maps its 28 labels to the OCR-EDR taxonomy.Quoted erroneous strings are resolved against the OCR source; no native label directly represents a globally unrelated prediction.Source: DOCR-Inspector Alignment · p. 10, Table B1 · p. 11
  • Localization scoring uses character offsets for text and tokenized LaTeX spans for formulas, excludes global errors, and matches local events within a boundary gap of three.Loc-Text offsets begin at the first erroneous character and end immediately after the last; formula tokenization covers commands, escaped symbols, delimiters, and non-whitespace characters.Source: Localization Metrics · pp. 10–11

Qualitative Examples

  • DocEDR preserves valid OCR across varied rendering-equivalent formula and text cases, while DOCR-Inspector reports nonexistent errors.Examples span delimiter sizing, wrapper and punctuation variation, command aliases, Unicode-equivalent symbols, and benign text typesetting differences.Source: Qualitative Examples · p. 11

C Training Curriculum Details

The curriculum progressively teaches diagnosis, executable correction, selective rendering, and repair from visual feedback. Stage 3 supplies the largest repair-quality gain, while GRPO further improves outcomes and reduces interaction.

C Training Curriculum Details

  • GRPO starts from the selected Stage-3 checkpoint and groups eight on-policy trajectories per prompt; OCRErrBench remains isolated from training and validation splits until checkpoint identities are fixed..Source: C Training Curriculum Details · p. 11
  • GRPO improves ExactFix, VisFix, and Preserve by 1.81, 1.13, and 1.76 points, while reducing average turns from 2.594 to 2.313 and Bad renders from 2.104 to 1.598.These reductions correspond to 10.8% fewer turns and 24.0% fewer rendering calls.Source: Table C1 · p. 11, C Training Curriculum Details · p. 11
  • Training progresses from diagnosis and direct corrections to a render-or-stop decision, then to updated-rendering feedback for residual-error repair and recovery from ineffective edits.Diagnostic SFT teaches validity, error type, operation, and localization; repair stages add increasingly complex actions and feedback.Source: C Training Curriculum Details · p. 11
  • An updated rendering can reveal that an edit failed, prompting a second targeted repair before the policy stops.Figure C1 illustrates this recovery trajectory.Source: C Training Curriculum Details · p. 11, Figure C1 · p. 12
  • Stage 3 adds the largest curriculum gains over Stage 2: 15.80 ExactFix and 13.54 VisFix points, while Preserve rises from 91.03% to 91.68%.Stage 1 to Stage 2 raises ExactFix and VisFix by 2.93 and 3.61 points, respectively.Source: C Training Curriculum Details · p. 12
  • The optimization schedule lengthens context from diagnosis through repair SFT to GRPO rollouts, while reducing learning rates after the first repair stage and again during GRPO.All stages use full-parameter BF16 optimization with ZeRO-2, gradient checkpointing, and the same random seed.Source: C Training Curriculum Details · p. 12

D Reward and Optimization Objective

The reward design separates preservation of valid inputs from recovery of erroneous ones, then evaluates terminal quality, trajectory progress, and interaction costs. GRPO optimizes grouped trajectories with a clipped group-relative advantage and a reference-policy constraint.

D Reward and Optimization Objective

  • The two label-conditioned reward branches optimize preservation and recovery independently, sharing only the structural output contract.For valid inputs, the preservation term rewards retaining the normalized source and structurally valid output.Source: Label-Conditioned Reward · pp. 12–13
  • Terminal repair quality combines normalized edit distance, exact normalized recovery, and a rendering-aware Verifier score grounded in the source image and terminal rendering.The signals provide progressively stricter supervision, from character-level proximity to visual consistency.Source: Label-Conditioned Reward · pp. 12–13
  • Interaction costs penalize unnecessary changes to Good inputs, repeated edits, excessive rendering, multi-turn reasoning without new visual evidence, and inefficient termination.For Bad inputs, the cost design also accounts for turns, rendering calls, repeated edits, and budget exhaustion with a Bad terminal state.Source: Label-Conditioned Reward · p. 13
  • Process rewards distinguish single-evaluation recovery, recovery after Bad states, remaining Good states, regression, and trajectories that stay Bad.They credit observable state changes rather than longer trajectories alone and complement terminal scoring of where a rollout ends.Source: Label-Conditioned Reward · p. 13
  • GRPO samples and normalizes grouped trajectory rewards, then combines clipped group-relative advantage with a KL constraint to a frozen reference repair policy.The reference policy is separate from the frozen reward Verifier.Source: GRPO Objective · p. 14

E Additional Ablation Studies

The ablations distinguish the contribution of rendering and iteration: the complete edit–render–reassess loop outperforms iteration without feedback. The full input triplet also balances localization with validity and rendering-equivalence robustness.

Input-Modality Ablation

  • The editable OCR source most strongly supports exact localization, while rendering improves validity and equivalence robustness; the full triplet gives the strongest validity, type, and equivalence results.Table E1 evaluates input-modality variants and defines Eq-FP as false positives on rendering-equivalent positives.Source: Table E1 · p. 14, Input-Modality Ablation · p. 14

Iteration and Updated-Rendering Ablation

  • 9.71, 9.03, and 1.97-point gains in ExactFix, VisFix, and Preserve over the one-step no-feedback baseline accompany the complete loop; iteration without rendering lowers these metrics.The combined setting reaches 82.17% ExactFix and 86.23% VisFix, while iteration alone falls to 66.82% ExactFix.Source: Table E2 · p. 14, Iteration and Updated-Rendering Ablation · p. 14

Qualitative Examples

  • For formula cases with alternative delimiter sizing or wrapper and punctuation variation, DocEDR preserves valid OCR while DOCR-Inspector reports nonexistent structural or content errors.The examples also cover a Greek-symbol command alias and operator or interval-wrapper variation.Source: Figure B1 · p. 15, Figure B2 · p. 15, Figure B3 · p. 16
  • Chinese and English text examples retain their content despite benign radical-layout, font, spacing, or line-wrapping differences, so DocEDR stops without editing..Source: Figure B5 · p. 17, Figure B6 · p. 17
Loading 2609.03445v1…