Source-linked AI summary
NaviDC-OCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
Peng Cai, Zhaofan Zou, Shifa Liu, Yikun Wang, Jiawei Tang, Kaicheng Yang, Meng Tong, Zhongjiang He, Hao Sun
TL;DR
Complex document parsing remains difficult for structured content and camera-captured pages. NaviDC-OCR unifies deformation-aware and structure-aware learning for diverse documents, achieving strong benchmark and challenge results, including 88.53 on Wild OmniDocBench v1.5 and first place in the ICDAR 2026 Sci-ImageMiner Challenge.
Problem
Structured document tasks require visual understanding, structural modeling, and cross-modal transformation, making learning and optimization more difficult.
Method
NaviDC-OCR unifies deformation-aware learning, adaptive sampling, and content-structure decoupled learning for digital and camera-captured document parsing.
Results
NaviDC-OCR achieves strong performance across multiple benchmarks and scenarios, including an overall score of 88.53 on Wild OmniDocBench v1.5 and first place in the ICDAR 2026 Sci-ImageMiner Challenge.
Takeaways & Limitations
The framework supports unified parsing of text, tables, formulas, code blocks, seals, and scientific figures across digital and camera-captured documents.
Takeaways & Limitations
Multi-model voting can still produce pseudo-labels containing sample bias and residual errors, requiring self-judgement verification.
Abstract
from arXiv · showhide
Document parsing aims to transform unstructured documents into structured and machine-readable representations. Recent advances in Vision-Language Models (VLMs) have significantly advanced document parsing. However, existing approaches still face two major challenges. First, decoupled VLM-based methods heavily rely on accurate layout analysis, where geometric distortions in camera-captured documents can introduce cascading errors. Second, although end-to-end VLM-based methods alleviate the dependence on explicit layout detection, they often suffer from redundant generation, hallucinations, and insufficient structural reasoning in high-resolution scenarios. To address these challenges, we propose NaviDC-OCR, a unified framework for document parsing. NaviDC-OCR introduces deformation-aware learning to incorporate geometric perception into VLMs and proposes an adaptive sampling mechanism for complex layout representation. Furthermore, a content-structure decoupled learning strategy is developed to explicitly model formula grammars and table structures, enabling more effective structured representation learning. Extensive experiments demonstrate that NaviDC-OCR achieves state-of-the-art performance across diverse document parsing benchmarks. It obtains overall scores of 96.87, 88.53 and 78.41 on OmniDocBench v1.6, Wild-OmniDocBench, and PureDocBench, respectively, and ranks first in the ICDAR 2026 Sci-ImageMiner Challenge. These results validate the effectiveness and generalization capability of NaviDC-OCR in complex document parsing scenarios.
1 Introduction
NaviDC-OCR targets unified parsing of digital and camera-captured documents by combining deformation-aware visual processing with explicit structural reasoning. Its evaluations show strong performance across document-parsing benchmarks and competitions.
- Motivation: Document parsing supports training-data pipelines and Retrieval-Augmented Generation systems, requiring accurate text recognition, layout understanding, and structural reconstruction.
- Challenges: Existing systems must handle both digital and camera-captured documents with complex degradations, motivating evaluation across specialized benchmarks.The introduction identifies OmniDocBench V1.6 for digital documents and Wild-OmniDocBench for camera-captured documents.
- Method: NaviDC-OCR unifies parsing across document types through global point-level and region-level deformation-aware learning plus adaptive sampling points.The framework implicitly integrates dewarping into VLMs and replaces conventional detection paradigms with layout segmentation.
- Method: Content-structure decoupled learning explicitly models formula grammars and table structures as intermediate reasoning processes for highly structured parsing tasks.For tables, the model first predicts OTSL structure and then reconstructs cell contents; the strategy targets formula parsing, table parsing, and scientific figure-to-table conversion.
- Results: 96.87 on OmniDocBench v1.6 and 88.53 on Wild OmniDocBench v1.5, while ranking first in the ICDAR 2026 Sci-ImageMiner Challenge.These evaluations cover digital, camera-captured, and scientific document parsing tasks.
2 Related Work
Prior document-parsing methods mainly follow end-to-end or decoupled VLM paradigms, while camera-captured documents motivate deformation-aware modeling. Related segmentation work also informs efficient VLM-based sequence representations and adaptive geometric sampling.
- VLM-based Document Parsing: Document parsing methods are broadly categorized into end-to-end VLM approaches and decoupled VLM approaches.End-to-end methods directly map document images to structured representations, whereas decoupled methods separate layout analysis from content parsing.
- VLM-based Document Parsing: Decoupled VLM methods generally use a two-stage paradigm in which layout analysis guides subsequent content parsing.This paradigm combines traditional pipelines’ controllability with VLMs’ semantic modeling capability.
- VLM-based Document Parsing: Recent methods improve table recognition, OCR efficiency, shared visual representations, parallel decoding, and camera-captured document parsing through varied architectural and data strategies.Examples include MonkeyOCR v1.5, Youtu-Parsing, PaddleOCR-VL-1.5, PaddleOCR-VL-1.6, and MinerU2.5-Pro.
- Document Dewarping: Document dewarping methods correct perspective distortion and geometric deformation to improve subsequent parsing performance.Prior work includes control-point-based mapping and segmentation-supervised approaches, while deformation-aware mechanisms integrate rectification into VLM modeling.
- Region Segmentation and Sampling: MLLM-based region segmentation methods either generate segmentation results end to end or use dedicated segmentation heads, with SAM-based approaches adding deployment overhead.VistaLLM instead represents binary masks as point sequences through gradient-based dynamic sampling.
- Region Segmentation and Sampling: The CGDP method adaptively distributes sampling points according to document geometric deformation characteristics for unified and efficient VLM-based sequence prediction.It reformulates conventional two-point layout analysis as a VLM-based sequence prediction task.
3 Data Engineering
NaviDC-OCR builds large-scale document-parsing data through automated cleaning, construction, and model-assisted validation across layouts, text, tables, and formulas. Its pipeline combines multi-model consensus voting, deformation-aware synthesis and supervision, adaptive contour sampling, and image-to-image self-judgement.
- Pipeline overview: The data engineering pipeline automates cleaning, construction, and model-assisted validation for diverse document elements, including layouts, text, tables, and formulas.Its three stages are multi-node consensus voting, deformation-aware synthesis, and image-to-image consistency evaluation with a Self-Judgement VLM.
- Consensus voting: Multi-node Consensus Voting selects pseudo-labels through agreement among heterogeneous models rather than relying on a single model’s prediction.A task-specific consistency function measures agreement, and predictions meeting threshold τ are accepted while uncertain samples undergo correction or human verification.
- Deformation-aware synthesis: Deformation-aware synthesis generates camera-captured samples and corresponding warped boundaries and control points from undistorted document data.The method provides region-level and point-level deformation supervision, replacing rectangular boxes with distorted boundary-point prediction for curved or folded documents.
- Adaptive sampling: Curvature-Guided Douglas–Peucker Sampling adaptively prioritizes high-curvature structures while preserving global contours under limited sampling budgets.This addresses uniform sampling’s tendency to under-sample corners and folds and preserves critical details such as creases and sharp corners.
- Automatic validation: A Self-Judgement model renders structured predictions into visual representations and evaluates image-to-image consistency against the original document images.This reduces residual pseudo-label errors and sample bias after multi-model voting by replacing cross-modal image-text verification with intra-modal visual comparison.
4 Progressive Training
NaviDC-OCR uses four-stage progressive training that advances from vision-language alignment and deformation-aware perception to content-structure decoupled parsing and task-level reinforcement learning. Each stage targets a distinct capability, progressively strengthening recognition, geometric modeling, structured representation, and downstream parsing optimization.
- Progressive training strategy: Four-stage progressive training equips NaviDC-OCR with document parsing capabilities ranging from basic visual perception to complex structural understanding.The stages target vision-language alignment, deformation-aware supervision, structured parsing, and downstream task optimization.
- Stage 1: Vision-language alignment: Stage 1 aligns the vision encoder and language model through VQA pre-training to establish fundamental OCR recognition and layout understanding.Only the Patch Merger MLPs and vision encoder are optimized, while the language model remains frozen; training uses captioning, interleaved image-text, alignment, and OCR data.
- Stage 3: Content-structure decoupled learning: Stage 3 applies content-structure decoupled learning to formulas and tables, separating formula content from grammar and emphasizing table topology and cell-merging relationships.Formula syntax labels are extracted from LaTeX annotations, while table cell contents are removed and standard OTSL syntax tokens are retained.
- Stage 4: Reinforcement learning: Stage 4 applies GRPO-based reinforcement learning to the Stage 3 model, using task-specific verifiable rewards for text, table, and formula parsing.The rewards use normalized edit similarity for text, TEDS for tables, and CDM for formulas, normalized to [0, 1].
5 Experimental Evaluation
NaviDC-OCR is evaluated across digital, camera-captured, clean, degraded, and specialized scientific-document benchmarks. It consistently demonstrates strong parsing performance, robustness to distortions, and generalization to scientific figure-to-table extraction.
- OmniDocBench v1.6: NaviDC-OCR outperforms pipeline-based methods and OvisOCR2 on OmniDocBench v1.6, leading text recognition, table reconstruction, and reading-order recovery.It obtains the lowest normalized edit distances for text and reading order, and the highest TEDS and TEDS-S scores for table parsing.
- PureDocBench: PureDocBench tests document parsing across Clean, Digital, and Real tracks using 1,475 pages from 10 domains and 66 subcategories.PureDocBench-Clean derives annotations directly from rendered HTML source files, while PureDocBench-Degraded simulates digital degradation and real-world acquisition conditions.
- Wild-OmniDocBench: Wild-OmniDocBench evaluates robustness under real-world capture conditions created through printing, deformation, image acquisition, and diverse illumination.The benchmark targets camera-captured documents rather than clean scans or digital renderings.
- ICDAR 2026 Sci-ImageMiner Challenge: NaviDC-OCR achieves the best TEDS performance on the Sci-ImageMiner Scientific Figure-to-Table task, exceeding the second-best method by over 2 percentage points.The result reflects content-structure decoupled learning for joint structural and semantic modeling of scientific figures.
6 Conclusion · Appendix · A Prompt Design and Task Examples
NaviDC-OCR is a unified framework for robust parsing of digital and camera-captured documents, combining broad data engineering, structure-aware training, and model improvements. Its unified image-plus-instruction interface supports eight parsing tasks spanning layout, text, formulas, tables, code, scientific figures, and seals.
- 6 Conclusion: NaviDC-OCR targets robust understanding of digital and camera-captured documents through improvements in data construction, training strategies, and model capabilities.The framework addresses complex layouts, structured content parsing, and real-world acquisition scenarios.
- 6 Conclusion: Its data pipeline covers text, layout, tables, formulas, code, and scientific figures, using cleaning, synthetic generation, and model-assisted verification to improve generalization.The paper also introduces a structure-aware progressive training strategy.
- 6 Conclusion: NaviDC-OCR unifies parsing of text, tables, formulas, code blocks, seals, and scientific figures, with evaluations validating performance across digital and camera-captured documents.The framework transforms document images into structured information end to end.
- A Prompt Design and Task Examples: All 8 supported tasks use one interface containing only an <image> token and textual instruction, without few-shot examples or structured metadata.The tasks include digital layout detection, camera-captured layout segmentation, text recognition, formula recognition, table recognition, code block recognition, scientific figure analysis, and seal recognition.
- A.8 Seal Recognition: Seal recognition extracts only textual content from irregular seal regions while suppressing surrounding text, lines, textures, and other irrelevant background information.The output excludes irrelevant background text.
- A.1 Digital Layout Detection / A.2 Camera-captured Layout Segmentation: Digital layout detection outputs rectangular boxes, semantic categories, and text orientations, while camera-captured segmentation outputs polygonal boundaries using deformation-aware modeling and adaptive point sampling.Simple regions use fewer points, whereas complex non-rigid deformations receive additional points.
- A.3 Text Recognition / A.4 Formula Recognition: Text recognition transcribes cropped regions while preserving formulas, subscripts, superscripts, and symbols; formula recognition produces compilable LATEX strings and preserves equation numbers with \tag{...}.Camera-captured text recognition retains segmented foreground regions and masks non-text areas.
- A.5 Table Recognition / A.6 Code Block Recognition / A.7 Scientific Figure Analysis: Table recognition converts OTSL token sequences into HTML, code recognition reconstructs formatted source code with its programming language, and scientific figure analysis outputs OTSL table sequences.These tasks respectively represent cell contents and complex structures, preserve indentation and syntax, and convert scientific figures into structured tables.
A.7 Scientific Figure Analysis
NaviDC-OCR analyzes scientific figures by recovering the structured tabular data they imply. It covers five major figure categories and 19 fine-grained classes, producing flattened OTSL sequences that are converted into tables.
- Scientific figure analysis recovers structured tabular data from scientific figures.
- The framework supports five major categories and 19 fine-grained scientific figure classes.Categories include univariate distribution, multivariate comparison, matrix-based, spatial localization, and structural flow figures.
- Outputs use a flattened OTSL token sequence representing the structured table, subsequently converted into a tabular representation.A representative example is shown in Figure 5.
B Data Synthesis Details
The paper develops a scalable synthetic-data pipeline for scientific chart understanding, pairing rendered figures with structured annotations and combining content-rich with structure-only samples. This design addresses limited real-world annotations and supports generalization across diverse scientific figures.
- Synthetic data generation: Rendering engines generate diverse scientific figures that mimic real publications’ visual characteristics and statistical distributions.The pipeline includes radar charts, heatmaps, line plots, scatter plots, and energy spectra.
- Structured supervision: Each synthetic chart is paired with numerical data, axis labels, legends, and data tables for end-to-end supervision.Chart types, layouts, data distributions, and annotation styles are controllable to cover diverse visualization scenarios.
- Complementary samples: The pipeline constructs full-content charts for quantitative extraction and structure-only charts that preserve geometric layouts after masking text and numerical values.These complementary samples target both visual information extraction and structural-relationship learning.
- Generalization: The combined strategy provides scalable supervision, mitigates insufficient real-world annotations, and improves generalization across scientific figures from research literature.The reported benefit comes from jointly leveraging content-rich and structure-aware samples.
C Qualitative Comparison with SOTA Methods
This section qualitatively compares NaviDC-OCR with state-of-the-art methods across native digital, digitally degraded, and real-world captured documents. The comparisons assess layout analysis, text recognition, table parsing, and formula extraction.
- Document Scenarios: Qualitative comparisons span native digital, digitally degraded, and real-world captured document scenarios.NaviDC-OCR is evaluated against state-of-the-art methods across these representative settings.
- Evaluation Dimensions: The visual results evaluate layout analysis, text recognition, table parsing, and formula extraction.These tasks define the qualitative comparison dimensions across the document scenarios.
C.1 Layout Recognition
NaviDC-OCR uses a decoupled parsing framework in which accurate layout recognition supports subsequent content understanding. Its deformation-aware learning improves layout completeness and accuracy on distorted, wrinkled, rotated, and complex captured documents.
- Framework: Accurate layout recognition is essential for NaviDC-OCR’s decoupled parsing framework and subsequent content understanding.
- Deformation-aware learning: NaviDC-OCR uses region-level and point-level deformation-aware learning with deformation augmentation to capture geometric variations.
- Recognition results: On wrinkled, geometrically distorted, and complex layouts, NaviDC-OCR provides more complete and accurate recognition than methods producing missing regions or incorrect categories.
- Qualitative comparisons: NaviDC-OCR more reliably analyzes creased structured tables than other state-of-the-art methods.
- Qualitative comparisons: NaviDC-OCR better covers creased regions, recognizes small text areas, and produces fine-grained layouts for dense, complex, and severely distorted documents.
C.2 Table Parsing · C.3 Formula Extraction
NaviDC-OCR robustly parses tables and formulas under camera-captured distortions, preserving structural relationships that competing methods often disrupt. Its advantages span table topology, cell correspondence, mathematical notation, and global formula structure.
- C.2 Table Parsing: NaviDC-OCR preserves table topology and cell relationships more reliably than competing methods under skew, perspective distortion, curvature, blur, and dense layouts.The comparisons target challenging camera-captured scenarios, with particularly strong performance on distorted or fine-grained tables.
- C.2 Table Parsing: On a distorted handwritten-note table, NaviDC-OCR recovers the four-column structure despite skew, perspective compression, and local blur.MinerU2.5-Pro and Paddle-VL-1.6 show row-column misalignment and content merging in this case.
- C.2 Table Parsing: On a curved newspaper page, NaviDC-OCR restores the three-column layout and preserves correspondences among crop year, deliveries, and producer prices.Competing methods introduce incorrect row-spanning structures or miss columns.
- C.2 Table Parsing: For a dense financial ledger, NaviDC-OCR better recovers hierarchical headers, narrow columns, fine-grained grids, and empty cells than competing methods.The results support content-structure decoupled learning for table topology modeling.
- C.3 Formula Extraction: NaviDC-OCR more robustly preserves mathematical structures, including subscripts, superscripts, radicals, fractions, limits, and bracket scopes, under real-world degradations.The evaluated degradations include wrinkles, shadows, low resolution, and severe rotation.
- C.3 Formula Extraction: In a wrinkled, shadowed, low-resolution formula, NaviDC-OCR more accurately preserves the summation relation inside the radical and subscript/superscript structure.MinerU2.5-Pro and Paddle-VL-1.6 struggle with variable subscripts, exponent positions, and the final numerical magnitude.
- C.3 Formula Extraction: Under severe rotation with an upside-down formula, NaviDC-OCR successfully recovers the complete formula structure, unlike competitors that miss orientation or reconstruct only fragments.The example includes limits, fractions, brackets, and product rule derivations.
D Benchmark Evaluation Details
NaviDC-OCR is evaluated across three document-parsing benchmarks using a consistent OmniDocBench-style end-to-end protocol and fixed matching settings. The evaluation reports element-level metrics and an overall score, while documenting repetitive-generation cases removed from PureDocBench scoring.
- Benchmark protocol: Evaluation covers OmniDocBench v1.6, Wild OmniDocBench v1.5, and PureDocBench using Markdown predictions matched with the quick_match strategy.Wild OmniDocBench v1.5 and PureDocBench follow the OmniDocBench-style end-to-end protocol.
- Evaluation metrics: Reported metrics include TextEdit, FormulaCDM, TableTEDS, TableTEDS-S, and ReadOrderEdit across text blocks, formulas, tables, and reading order.Text blocks and reading order use normalized edit distance; formulas use edit distance and CDM; tables use TEDS and edit distance.
- Evaluation metrics: Overall = (1 −TextEdit) × 100 + FormulaCDM × 100 + TableTEDS × 100.Lower TextEdit and ReadOrderEdit are better, whereas higher FormulaCDM, TableTEDS, TableTEDS-S, and Overall are better.
- Evaluation consistency: Evaluator versions, matching strategies, metric sets, and timeout settings remain fixed across models compared on the same benchmark.Long or complex pages use 1200-second page-matching and truncated-quickmatch timeouts, with timeout_fallback_max_chunk_span=200 and timeout_fallback_order_penalty=0.05.
- Evaluation limitations: PureDocBench is substantially more challenging, and invalid Markdown predictions from severe repetitive-generation cases were removed before scoring.Missing predictions receive a score of zero, preserving evaluation fairness; no predictions were removed from the digital-degraded subset.