PaperlayerView exact revision on arXiv

Source-linked AI summary

PaddleOCR-VL: Boosting Multilingual Document Parsing via a 0.9B Ultra-Compact Vision-Language Model

Cheng Cui, Ting Sun, Suyin Liang, Tingquan Gao, Zelun Zhang, Jiaxuan Liu, Xueqing Wang, Changda Zhou, Hongen Liu, Manhui Lin, Yue Zhang, Yubo Zhang, Handong Zheng, Jing Zhang, Jun Zhang, Yi Liu, Dianhai Yu, Yanjun Ma

arXiv:2510.14528v4cs.CV

TL;DR

Document parsing must handle complex layouts, multilingual content, and diverse elements efficiently. PaddleOCR-VL combines dedicated layout analysis with a compact NaViT–ERNIE-4.5-0.3B vision-language model and systematic data construction. It reports state-of-the-art performance across document-parsing benchmarks with efficient inference, while chart evaluation is limited to an in-house benchmark because public datasets have size, imbalance, and annotation-quality limitations.

  • Problem

    Document parsing must handle complex layouts, multiple languages, handwriting, and diverse elements, while end-to-end approaches face latency, memory, instability, and hallucination challenges.

  • Method

    PaddleOCR-VL separates layout detection and reading-order prediction from element recognition, using PP-DocLayoutV2, a NaViT-style encoder, ERNIE-4.5-0.3B, and a high-quality data-construction pipeline.

  • Results

    PaddleOCR-VL achieves state-of-the-art document-parsing performance, including an overall score of 92.86 versus 90.67 for MinerU2.5-1.2B, while delivering lower latency and higher throughput.

  • Takeaways & Limitations

    The model supports practical document processing across complex elements and more than 100 languages with low resource consumption and fast inference.

  • Takeaways & Limitations

    Chart recognition is evaluated only on an in-house benchmark because public test sets have limited size, imbalanced categories, and poor annotation quality.

Abstract

from arXiv · show

In this report, we propose PaddleOCR-VL, a SOTA and resource-efficient model tailored for document parsing. Its core component is PaddleOCR-VL-0.9B, a compact yet powerful vision-language model (VLM) that integrates a NaViT-style dynamic resolution visual encoder with the ERNIE-4.5-0.3B language model to enable accurate element recognition. This innovative model efficiently supports 109 languages and excels in recognizing complex elements (e.g., text, tables, formulas, and charts), while maintaining minimal resource consumption. Through comprehensive evaluations on widely used public benchmarks and in-house benchmarks, PaddleOCR-VL achieves SOTA performance in both page-level document parsing and element-level recognition. It significantly outperforms existing solutions, exhibits strong competitiveness against top-tier VLMs, and delivers fast inference speeds. These strengths make it highly suitable for practical deployment in real-world scenarios. Code is available at https://github.com/PaddlePaddle/PaddleOCR .

1. Introduction

PaddleOCR-VL addresses the complexity of multilingual document parsing with a resource-efficient multimodal solution combining layout analysis and compact element recognition. Its training-data pipeline and benchmarking support strong accuracy, efficiency, and practical deployment.

  • Modern documents combine dense text, complex tables and charts, mathematical expressions, multiple languages, handwriting, and diverse layouts, challenging document parsing systems.
  • PaddleOCR-VL combines a robust layout analysis model with the compact PaddleOCR-VL-0.9B vision-language model.The system targets high-performance, resource-efficient document parsing.
  • The system detects layout elements and reading order, then segments elements for recognition by PaddleOCR-VL-0.9B.Recognized elements include text blocks, tables, formulas, and charts.
  • Over 30 million training samples support the model through public-data acquisition, synthesis, automated labeling, cleaning, evaluation, hard-sample mining, and limited manual annotation.Manual annotation is reserved for a small number of corner cases.
  • Comprehensive public and in-house benchmarks show state-of-the-art document-parsing performance, competitiveness with leading VLMs, lower latency, and higher throughput.
  • The reported contributions emphasize a compact VLM architecture, systematic high-quality data construction, and state-of-the-art recognition of complex document elements.

2. PaddleOCR-VL

PaddleOCR-VL uses a two-stage architecture that separates layout analysis from fine-grained recognition, combining specialized layout models with a compact dynamic-resolution VLM. This design targets stable parsing, efficient inference, and practical training.

  • PaddleOCR-VL Overview: The pipeline first predicts layout regions and reading order, then recognizes segmented text, tables, formulas, and charts before formatting structured Markdown and JSON.
  • Layout Analysis: End-to-end autoregressive VLM approaches can incur high latency, memory consumption, unstable layout analysis, and hallucinations in complex layouts.
  • Layout Analysis: PP-DocLayoutV2 uses RT-DETR for element detection and classification, followed by a six-layer pointer network for reading-order prediction.
  • Layout Analysis: Separating layout analysis enables stable and accurate processing while using a lower-resolution vision model with fewer parameters.
  • Layout Analysis: The layout model embeds proposals with absolute 2D positions and class labels while modeling pairwise geometry through attention bias and relation heads.
  • Layout Analysis: The model achieves higher performance with fewer parameters than LayoutReader by extending RT-DETR with a pointer network.
  • Element-level Recognition: PaddleOCR-VL-0.9B integrates a NaViT-style native-resolution encoder, a two-layer MLP projector, and ERNIE-4.5-0.3B language model.Dynamic high-resolution preprocessing handles arbitrary-resolution images without distortion; ERNIE-4.5-0.3B supports faster decoding through a smaller decoder.
  • Element-level Recognition: Combining NaViT with ERNIE-4.5-0.3B improves document parsing while reducing memory usage and inference time.

3. Dataset

The training-data methodology combines diverse sources with automated annotation, cleaning, evaluation, and targeted hard-case mining to support broad and robust document recognition.

  • The training-data construction process gathers diverse sources, generates labels automatically with large models, and integrates challenging examples.
  • Data comes from open-source, synthesized, network-accessible, and in-house datasets.
  • Open-source materials include handwriting, mathematical expressions, charts, graphs, papers, newspapers, scientific articles, examinations, and slides.These sources broaden stylistic, structural, and domain diversity.
  • PP-StructureV3 produces preliminary pseudo-labels, which advanced multimodal models refine through prompts containing images and pseudo-labels.
  • An evaluation engine categorizes text and table data into detailed types, enabling targeted hard-sample mining and synthesis of challenging examples.

4. Evaluation

PaddleOCR-VL is evaluated across page-level parsing, element recognition, multilingual and handwritten text, tables, formulas, charts, and inference efficiency. Across these evaluations, it generally achieves leading accuracy while also offering substantially faster processing than competing approaches.

  • Page-level Evaluation: 0.115 average overall edit distance is reported on OmniDocBench v1.0.The model achieves best or comparable results across Chinese and English text, formula, table, and reading-order measurements.
  • Page-level Evaluation: 80.0 ± 1.0 overall score leads olmOCR-Bench, with category-leading results for ArXiv and Headers and Footers.The benchmark contains 1,402 PDF documents and 7,010 test cases covering diverse extraction challenges.
  • Element-level Recognition: PaddleOCR-VL achieves the lowest reported error rates across many document types and all evaluated multilingual scripts.Examples include 0.049 on PPT2PDF, 0.021 on Academic Literature, and edit distances from 0.013 to 0.135 across the listed scripts.
  • Element-level Recognition: PaddleOCR-VL leads handwritten English and Chinese recognition and surpasses expert OCR VLMs and some 72B-level multimodal models for chart recognition.Chart evaluation uses only an in-house benchmark and RMS-F1 because of dataset and annotation limitations.
  • Element-level Recognition: 0.9195 overall TEDS and 0.9543 structural TEDS lead OmniDocBench-Table-block, while formula CDM reaches 0.9453 and 0.9882 on the two reported datasets.The table benchmark also reports 0.0561 overall edit distance.
  • Inference Performance: 53.1% higher page throughput and 50.9% higher token throughput than MinerU2.5 are achieved with the FastDeploy backend.These results position PaddleOCR-VL as the most efficient solution in the reported comparison.

5. Conclusion

PaddleOCR-VL combines page-level and element-level document recognition with multilingual support and efficient inference. The report presents it as practical for real-world document processing across complex content types.

  • 5. Conclusion: PaddleOCR-VL uses a NaViT-style visual encoder with ERNIE-4.5-0.3B to recognize text, tables, formulas, and charts in over 100 languages.The report also describes handling handwriting and historical documents.
  • 5. Conclusion: Fast inference and low resource consumption support practical deployment of PaddleOCR-VL for document parsing.The report states that the model outperforms existing pipeline solutions on many benchmarks.
  • 5. Conclusion: PaddleOCR-VL is reported to have potential to advance multimodal document processing for automated analysis and information retrieval.The report connects this scope to more efficient extraction from complex documents.

A. Training Dataset Details

The training-data strategy uses isolated document elements and their annotations to construct multimodal training data for text, tables, formulas, and charts. This two-stage setup makes element-image collection more feasible than collecting complete pages with varied elements.

  • A. Training Dataset Details: The two-stage approach collects isolated element images and annotations before training the multimodal model.The paper states that this is more feasible than collecting complete document pages containing different elements.
  • A. Training Dataset Details: The training data construction covers text, tables, formulas, and charts.The following sections elaborate on multimodal model training data for these four element types.

A.1. Text

The text dataset is generated through a multi-stage annotation pipeline that combines document-structure models with multimodal language models. It provides hierarchical annotations across 109 languages and diverse document scenes and writing styles.

  • A.1. Text: 20 Million High-Quality Image-Text Pairs are curated through a multi-stage dataset-generation pipeline.Figure A1 presents the construction method and characteristics of the text training data.
  • A.1. Text: Automatic annotation integrates PP-StructureV3 with multimodal models including ERNIE-4.5-VL and Qwen2.5-VL.PP-StructureV3 generates pseudo labels that are converted into prompts, refined, aggregated, and merged at multiple granularities.
  • A.1. Text: Annotations are provided at text-line, text-block, and text-page levels across 109 languages.The dataset includes scenes such as academic papers, newspapers, handwritten texts, ancient books, identity cards, tickets, and seals.

A.2. Table

The appendix describes large-scale table and formula training-data construction for PaddleOCR-VL-0.9B, combining automated annotation, synthesis, and targeted coverage of diverse recognition scenarios.

  • Table: Over 5 million high-quality image-table pairs were constructed for table recognition across languages, border styles, and table types.The dataset uses automatic annotation, potential annotation mining, and high-quality synthesis, with OTSL as the target format.
  • Table: Tables are located with PP-StructureV3, pseudo-labeled by ERNIE-4.5-VL, validated by a larger discriminative model, and refined after rejection.
  • Table: The table dataset covers diverse categories and recognition scenarios, supporting training for table recognition.
  • Formula: The formula dataset combines source-code rendering, automatic annotation, targeted long-tail synthesis, and public data collection across educational, academic, financial, and handwritten scenarios.
  • Formula: Formula processing extracts and deduplicates arXiv LaTeX, normalizes it with KaTeX, and targets Chinese-character formulas using cropped-region recognition.

A.4. Chart

The appendix presents a bilingual chart dataset built from public data, annotation, synthesis, and long-tail augmentation, covering varied chart types and application sources.

  • Chart: Over 0.8 million high-quality Chinese-English image-chart pairs were constructed for chart recognition.
  • Chart: The chart dataset combines public-data collection and cleaning, automatic annotation, data synthesis, and targeted long-tail augmentation.
  • Chart: Sources include ChartQA, PlotQA, Chart2Text, DVQA, Unichart, Beagle, ChartINFO, visText, and ExcelChart.
  • Chart: The final dataset covers diverse application scenarios and chart styles, providing training support for chart models.
  • Chart: PaddleOCR-VL supports 109 languages, with language-specific text-line recognition accuracy reported in Table 6 and language mappings listed in Table A1.

C. Inference Performance on Different Hardware Configurations

The appendix evaluates PaddleOCR-VL’s end-to-end inference across hardware and backend configurations, reporting stable and efficient performance with flexible adaptation to computing environments.

  • Inference Performance: End-to-end inference performance is summarized for different hardware configurations in Table A2.
  • Inference Performance: Using FastDeploy 2.3.0, vLLM 0.10.2, and SGLang 0.5.2, PaddleOCR-VL demonstrates stable and efficient inference across hardware and backend configurations.
  • Inference Performance: The reported results indicate that the system can flexibly adapt to diverse computing environments.

D. Real-world Samples

The appendix illustrates PaddleOCR-VL’s real-world document parsing across document types, layouts, languages, handwriting, vertical text, tables, formulas, and charts.

  • Overall Parsing: Figures A5-A8 show Markdown parsing examples for books, reports, slides, exams, notes, vertical documents, certificates, newspapers, and magazines.
  • Layout Detection: Figures A9-A11 illustrate processing of pages with intricate or challenging layouts.
  • Reading Order: Figures A12-A13 demonstrate excellent reading order on complex reports, textbooks, newspapers, magazines, and vertical documents.
  • Text Recognition: Figures A14-A22 present robust recognition of multilingual, handwritten, and vertical text.
  • Table Recognition: Figures A23-A24 showcase table recognition across academic, financial, watermarked, image-containing, formula-containing, and photographed tables.
  • Formula Recognition: Figures A25-A26 demonstrate recognition of complex, handwritten, screen-captured, vertical, and Chinese-character formulas.
  • Chart Recognition: Figures A27-A29 show parsing of pie, bar, line, bar-line hybrid, and heatmap charts, a capability described as lacking in many expert OCR VLMs.

E. Compare with Others

PaddleOCR-VL outperforms competing solutions across complex-layout pages and diverse recognition tasks, including multilingual, handwritten, vertical, table, formula, and chart content.

  • Complex Layout: PaddleOCR-VL surpasses other solutions on PDF pages with complex layouts and unique elements.Figures A30 and A31 highlight its performance on intricate layouts.
  • Text Recognition: PaddleOCR-VL achieves exceptionally high accuracy in multilingual, handwriting, and vertical text recognition.Competitors misidentify Russian and Hindi, overlook handwritten characters, and struggle with vertical text.
  • Table Recognition: PaddleOCR-VL stands out on complex table parsing, where other models frequently encounter difficulties.Figures A38 and A39 provide the comparison.
  • Formula Recognition: PaddleOCR-VL accurately parses complex formulas, whereas other SOTA models often produce flawed outputs for challenging mathematical notation.Figure A40 demonstrates the comparison.
  • Chart Recognition: PaddleOCR-VL outperforms Qwen2.5VL-72B and GPT-4o in chart recognition by reconstructing chart structure and content accurately.Figures A41 and A42 support this comparison.
Loading 2510.14528v4…