PaperlayerView exact revision on arXiv

Source-linked AI summary

PaddleOCR-VL-1.5: Towards a Multi-Task 0.9B VLM for Robust In-the-Wild Document Parsing

Cheng Cui, Ting Sun, Suyin Liang, Tingquan Gao, Zelun Zhang, Jiaxuan Liu, Xueqing Wang, Changda Zhou, Hongen Liu, Manhui Lin, Yue Zhang, Yubo Zhang, Yi Liu, Dianhai Yu, Yanjun Ma

arXiv:2601.21957v2cs.CV

TL;DR

Document-parsing models still struggle with severe physical distortions in real-world inputs. PaddleOCR-VL-1.5 addresses this gap with upgraded layout analysis, expanded recognition tasks, and a new robustness benchmark, achieving SOTA results while remaining a 0.9B VLM.

  • Problem

    Existing document-parsing models are primarily optimized for clean or digital-born documents and remain challenged by severe skew, warping, screen-photography artifacts, and illumination variation.

  • Method

    PaddleOCR-VL-1.5 upgrades layout analysis with PP-DocLayoutV3 and adds seal recognition and text spotting while retaining a 0.9B architecture, with progressive training for broader task diversity and robustness.

  • Results

    94.5% accuracy on OmniDocBench v1.5 and 92.05% overall accuracy on Real5-OmniDocBench establish SOTA performance, with leading results across text spotting and seal recognition.

  • Takeaways & Limitations

    The model provides a compact document-parsing solution with reported robustness across complex real-world conditions and expanded functionality for downstream document understanding applications.

Abstract

from arXiv · show

We introduce PaddleOCR-VL-1.5, an upgraded model achieving a new state-of-the-art (SOTA) accuracy of 94.5% on OmniDocBench v1.5. To rigorously evaluate robustness against real-world physical distortions, including scanning, skew, warping, screen-photography, and illumination, we propose the Real5-OmniDocBench benchmark. Experimental results demonstrate that this enhanced model attains SOTA performance on the newly curated benchmark. Furthermore, we extend the model's capabilities by incorporating seal recognition and text spotting tasks, while remaining a 0.9B ultra-compact VLM with high efficiency. Code: https://github.com/PaddlePaddle/PaddleOCR

1. Introduction

PaddleOCR-VL-1.5 targets the robustness gap in document parsing caused by severe real-world physical distortions while preserving a compact 0.9B architecture. It upgrades layout analysis, expands task coverage, and achieves SOTA results on clean and distorted-document benchmarks.

  • Existing document-parsing models remain vulnerable to skew, page warping, screen-capture moiré, and erratic lighting in real-world scenarios.
  • PaddleOCR-VL-1.5 is designed to improve both general precision and robustness while retaining its 0.9B ultra-compact architecture.
  • PP-DocLayoutV3 predicts multi-point layout boxes and logical reading orders for skewed and warped surfaces in a single forward pass.The design is intended to reduce cascading errors compared with conventional layout analysis.
  • The model adds seal recognition and text spotting while retaining the NaViT-style dynamic-resolution encoder and ERNIE-4.5-0.3B language backbone.
  • 94.5% accuracy on OmniDocBench v1.5 and 92.05% overall accuracy on Real5-OmniDocBench establish new SOTA results.The model is reported to outperform massive general VLMs despite its compact scale and to achieve leading performance on text spotting and seal recognition.

2. PaddleOCR-VL-1.5

PaddleOCR-VL-1.5 combines unified geometric layout analysis with multimodal recognition and text spotting in an efficient document-processing framework. Its architecture integrates segmentation, reading-order prediction, recognition, post-processing, and progressive training for distorted and complex documents.

  • PaddleOCR-VL-1.5 supports Document Parsing and Text Spotting through a unified framework.
  • PP-DocLayoutV3: PP-DocLayoutV3 uses multi-point localization and instance segmentation to represent document elements under perspective tilt, curvature, skew, and warping.Its mask-based detection head produces pixel-accurate regions rather than only axis-aligned boxes.
  • PP-DocLayoutV3: Reading-order prediction is integrated into the Transformer decoder alongside detection and segmentation instead of using a separate pointer network.The unified model jointly produces classifications, coordinates, segments, and logical reading sequences in one forward pass.
  • PP-DocLayoutV3: A Global Pointer Mechanism computes pairwise precedence relations from refined object queries, and voting-based ranking sorts elements into the final reading order.The relation matrix is constrained to be anti-symmetric, with positive scores indicating that one element precedes another.
  • Element-level Recognition and Text Spotting: The 0.9B model retains its lightweight visual-language architecture and expands coverage to six tasks, including OCR, formulas, tables, charts, seals, and text spotting.Recognition is further optimized for complex tables, mathematical formulas, rare characters, ancient Chinese, multilingual tables, and text decorations.
  • Training: Training uses joint end-to-end optimization for layout modules, distortion-aware augmentation, and a progressive training paradigm that increases data scale, task diversity, and robustness.The layout corpus contains over 38k manually annotated document samples, while the recognition model is trained with PaddleFormers.

3. Dataset

The dataset combines broad document coverage with uncertainty-aware sampling to improve robustness and expand PaddleOCR-VL-1.5’s supported capabilities. It adds targeted data for spotting, seals, OCR, formulas, tables, and charts.

  • Dataset construction: 38k document images span academic papers, textbooks, financial reports, slides, newspapers, examinations, invoices, receipts, and other domains.Manual annotations cover 25 distinct document component categories.
  • Dataset construction: Clustering-based sampling and hard-case mining expand visual diversity while identifying difficult examples for robustness improvement.The workflow uses visual-feature clustering followed by dual-threshold inference with PP-DocLayoutV2.
  • Uncertainty-Aware Cluster Sampling: Uncertainty-Aware Cluster Sampling allocates more training data to visually diverse clusters where stochastic inference indicates higher model uncertainty.The method combines CLIP embeddings, K-Means clustering, repeated stochastic decoding, and uncertainty-based weighting.
  • Uncertainty-Aware Cluster Sampling: The sampling plan uses α=1.0 and β=2.0 to up-sample complex scenarios such as distorted seals and dense tables while retaining simpler cases.N_i is determined from cluster uncertainty, cluster size, and the total sampling budget.
  • New capability data: Capability data expands coverage to text spotting, seals, OCR enhancement, complex tables, formulas, and charts across diverse document types and languages.Additions include simulated physical distortions for formula data, multilingual OCR support, and cell-level formula recognition in dense tables.

4. Evaluation

Evaluation covers document parsing under standard and physical-distortion conditions, plus text spotting, seal recognition, and inference efficiency. PaddleOCR-VL-1.5 achieves leading results across these evaluations while retaining a compact design.

  • Document parsing: OmniDocBench v1.5 evaluates text, reading order, tables, and formulas on 1,355 pages using Edit Distance, TEDS, and CDM-based metrics.The benchmark adds 374 documents over version 1.0 and uses a weighted overall metric.
  • Document parsing: 94.50% overall accuracy makes PaddleOCR-VL-1.5 the top-ranked solution on OmniDocBench v1.5, outperforming pipeline tools, general VLMs, and specialized parsers.The overall score rises from 92.86% for PaddleOCR-VL to 94.50%.
  • Real-world robustness: 92.05% overall accuracy establishes a new SOTA record on Real5-OmniDocBench, covering scanning, warping, screen photography, illumination, and skew.The benchmark is derived from OmniDocBench v1.5 and uses corresponding ground-truth annotations and evaluation protocols.
  • Real-world robustness: 91.66% accuracy in the Skewing category improves 14.19% over the predecessor, indicating stronger performance under extreme geometric distortion.Real5-OmniDocBench uses handheld mobile-device captures for all categories except scanning.
  • Text spotting: The model achieves the highest text-spotting accuracy across all 9 evaluated dimensions, including blur, handwriting, tables, ancient documents, and multilingual recognition.Overall denotes the average accuracy across the nine dimensions.
  • Seal recognition: Seal recognition reaches an NED of 0.138 versus 0.382 for Qwen3-VL with 235B parameters, demonstrating stronger performance on specialized seal elements.The benchmark contains 300 images with varied seal shapes and challenging real-world conditions.
  • Inference performance: 1.4335 pages/s and 2016.6 tokens/s on one NVIDIA A100 with FastDeploy exceed PaddleOCR-VL by 16.9% and 18.6%, respectively.The end-to-end measurement includes PDF rendering and Markdown generation.

5. Conclusion

PaddleOCR-VL-1.5 combines strong document-parsing accuracy with robustness to challenging real-world distortions and adds seal recognition and text spotting.

  • 94.5% accuracy on OmniDocBench v1.5 establishes PaddleOCR-VL-1.5 as a record SOTA document-parsing model.
  • The model handles aggressive skewing, non-rigid page warping, and erratic lighting in unconstrained real-world environments.
  • PaddleOCR-VL-1.5 integrates Seal Recognition and Text Spotting to expand its document-understanding capabilities.

A. Comparison of PaddleOCR-VL-1.5 and 1.0 Models

Table A1 compares PaddleOCR-VL-1.5 with PaddleOCR-VL across functional capabilities and robustness dimensions using relative star ratings rather than absolute accuracy.

  • Table A1 compares PaddleOCR-VL-1.5 and PaddleOCR-VL across functional evolution and robustness.
  • The star ratings indicate relative performance between the two versions, not absolute accuracy.
  • The comparison is intended to summarize version-level improvements rather than provide standalone benchmark scores.

B. Details of the Real5-OmniDocBench Benchmark

Real5-OmniDocBench evaluates document parsing under real-world distortions including scanning, warping, screen photography, illumination variation, and skew. PaddleOCR-VL-1.5 achieves strong results across these scenarios, with reported overall scores above 91%.

  • Benchmark: Real5-OmniDocBench is a new real-world benchmark constructed from OmniDocBench v1.5.
  • Scanning: 93.43% overall score in scanning improves on PaddleOCR-VL’s 92.11% while retaining a 0.9B parameter size.
  • Warping: 91.25% overall score in warping exceeds Qwen3-VL-235B’s 89.99% and preserves formula and table structure under geometric distortion.
  • Screen photography: 91.76% overall score in screen photography accompanies a 90.88% Formula-CDM score, surpassing MinerU2.5 and dots.ocr on that metric.
  • Illumination: 92.16% overall score under illumination variation improves on PaddleOCR-VL’s 89.61% and exceeds Gemini-3 Pro’s 89.53%.
  • Skew: 91.66% overall score under skew exceeds Gemini-3 Pro’s 89.45%, with a 91.00% Table-TEDS score and 0.047 Text-Edit distance.

C. Supported Languages

PaddleOCR-VL-1.5 supports 111 languages and extends PaddleOCR-VL with recognition for Tibetan and Bengali scripts.

  • Supported languages: 111 languages are supported by PaddleOCR-VL-1.5.
  • New additions: Tibetan and Bengali recognition are newly added compared with PaddleOCR-VL.
  • Language mapping: Table A7 maps each language category to its corresponding supported languages and scripts.

E. Real-world Samples

PaddleOCR-VL-1.5 is presented as robust across diverse real-world document conditions and extended to specialized parsing capabilities, including seal recognition and text spotting.

  • Real-world document parsing: The appendix demonstrates PaddleOCR-VL-1.5’s robustness across illumination, skew, screen photography, scanning, and warping conditions.Figures A1–A5 cover real-world document parsing under these physical distortions.
  • Layout analysis: Layout analysis remains robust on skewed or curved geometries, screen-photography noise, and illumination variations.The model also generalizes to comics, CAD drawings, and multi-stamped documents.
  • Text recognition: Text recognition improves for decorated text, special characters, vertical orientations, and character-level ambiguities.The reported examples include underlines, emphasis marks, wavy patterns, and long-tail cases.
  • Table recognition: The model processes complex, multilingual, and cross-page tables, including tables with embedded images or mathematical formulas.The examples include cross-page table detection and merging.
  • Formula recognition: Formula recognition improves for sub/superscripts and multi-line segmentation while reducing overall errors.These results are presented in Figure A17.
  • New capabilities: Seal recognition extracts content from varied seals, while text spotting simultaneously localizes and recognizes text in challenging layouts.The examples include cluttered seal backgrounds, multi-column magazines, complex tables, and irregular handwriting.

E.2.1. Layout Analysis for Real-world Documents

This subsection compares PaddleOCR-VL and PaddleOCR-VL-1.5 on layout analysis under warping, screen photography, skew, and illumination.

  • Warping: Figure A6 compares layout analysis results between PaddleOCR-VL and PaddleOCR-VL-1.5 for warping.
  • Screen photography: Figure A7 compares layout analysis results between PaddleOCR-VL and PaddleOCR-VL-1.5 for screen photography.
  • Skew: Figure A8 compares layout analysis results between PaddleOCR-VL and PaddleOCR-VL-1.5 for skew.
  • Illumination: Figure A9 compares layout analysis results between PaddleOCR-VL and PaddleOCR-VL-1.5 for illumination.

E.2.2. Layout Analysis for New Scenarios

This subsection presents comparisons of PaddleOCR-VL-1.5 outputs across specialized document scenarios, recognition tasks, seals, and text spotting.

  • New scenarios: Figure A10 compares layout analysis results between PaddleOCR-VL and PaddleOCR-VL-1.5 for new scenarios.
  • Text recognition: Figures A11–A13 compare markdown outputs for text decoration, special-character, and long-tail-scenario documents.
  • Table recognition: Figures A14–A16 compare markdown outputs for general, multilingual, and cross-page tables.
  • Formula recognition: Figure A17 compares markdown outputs for various types of formulas.
  • Seal recognition: Figures A18–A20 present markdown outputs for various types of seals.
  • Text spotting: Figure A21 presents text spotting results on various types of documents.
Loading 2601.21957v2…