PaddleOCR-VL: Boosting Multilingual Document Parsing via a 0.9B Ultra-Compact Vision-Language Model

• Pipeline approaches face integration complexity and cumulative errors, while end-to-end multimodal methods can misorder text, hallucinate on complex layouts, and incur substantial computational overhead.Source: §1 · p. 3
• PaddleOCR-VL detects layout and reading order, segments elements, then recognizes them with a NaViT-style dynamic-resolution encoder and ERNIE-4.5-0.3B for efficient dense-text recognition and decoding.Compared with grounding and sequence-output methods such as MinerU2.5 [2] and Dolphin [3], it offers faster inference, lower training costs, and easier extension to new layout categories.Source: §1 · pp. 3–4
• The training-data pipeline collected over 30 million samples and combined automated labeling, data cleaning, element-category evaluation, targeted hard-sample mining, data synthesis, and manual annotation of corner cases.Evaluation results informed targeted mining and synthesis of challenging examples, while manual annotation completed the dataset for a small number of corner cases.Source: §1 · pp. 3–4
• PaddleOCR-VL achieves SOTA on OmniDocBench v1.0 and v1.5 [16], olmOCR-Bench [12], and in-house benchmarks, outperforming pipeline solutions, competing strongly with leading VLMs, and delivering lower latency and higher throughput.Source: §1 · p. 4
• PaddleOCR-VL supports 109 languages and recognizes text, tables, formulas, and charts, including challenging handwritten text and historical documents.Source: §1 · p. 4

2.1. Architecture

• PP-DocLayoutV2 performs layout analysis before PaddleOCR-VL-0.9B recognizes text, tables, formulas, and charts; postprocessing combines both stages into structured Markdown and JSON.The layout model localizes semantic regions and predicts their reading order for the recognition stage.Source: §2.1 · p. 4

2.1.1. Layout Analysis

• PP-DocLayoutV2 separates RT-DETR-based element detection and classification from pointer-network reading-order prediction, yielding stable analysis and outperforming LayoutReader [19] with fewer parameters.The pointer network uses six transformer layers; a deterministic win-accumulation decoder recovers a topologically consistent order.Source: §2.1.1 · pp. 5–6

2.1.2. Element-level Recognition

• PaddleOCR-VL-0.9B combines native-resolution NaViT-style vision encoding, a 2-layer MLP projector, and ERNIE-4.5-0.3B for efficient element recognition.The model also incorporates 3D-RoPE; native-resolution processing handles arbitrary image resolutions without distortion, while the NaViT–ERNIE combination is reported to use minimal memory and infer faster.Source: §2.1.2 · pp. 6–7

2.2.1. Layout Analysis

• PP-DocLayoutV2 trains RT-DETR for detection and classification first, then freezes it while a pointer network learns reading order.Detection starts from PP-DocLayout_Plus-L [25] weights and runs for 100 epochs on over 20,000 samples; pointer training uses Generalized Cross Entropy Loss [26], AdamW, and a constant 2e-4 learning rate for 200 epochs.Source: §2.2.1 · p. 7

2.2. Training Recipe

• The recognition model uses pretrained Keye-VL vision and ERNIE-4.5-0.3B language weights, with training divided into two stages.Training is based on ERNIEKit.Source: §2.2.2 · p. 7

2.2.2. Element-level Recognition

• Stage 1 aligns vision and text on 29 million image-text pairs; Stage 2 instruction-tunes on 2.7 million samples for OCR, table, formula, and chart recognition.Fine-tuning runs for two epochs and covers varied document types, languages, writing systems, and visual complexities.Source: §2.2.2 · pp. 7–8

3. Dataset

• The dataset pipeline gathers diverse data, generates labels with large models, and adds challenging examples to strengthen model performance and robustness.The paper presents these as the core steps in constructing the training dataset.Source: §3 · p. 9

3.1. Data Curation

• Training data combines public, synthetic, Internet-accessible, and in-house sources to broaden document coverage and address imbalanced data and stylistic diversity.Public sources include CASIA-HWDB, UniMER-1M [30], and MathWriting [31]; online documents span papers, newspapers, handwritten scans, exams, and slides. In-house data is included at controlled proportions.Source: §3.1 · pp. 9–10

3.2. Automatic Data Annotation

• Automatic annotation starts with PP-StructureV3 pseudo-labels, refines them using ERNIE-4.5-VL and Qwen2.5VL [24], then filters hallucinated content.The refinement prompts pair original images with their pseudo-labels to produce reliable labels at scale.Source: §3.2 · p. 10

3.3. Hard Cases Mining

• Hard-case mining evaluates manually labeled text, table, formula, and chart categories with task-specific metrics, then synthesizes examples for identified weaknesses.The evaluation set covers 23 text categories, 20 table categories, 4 formula categories, and 11 chart categories. Metrics include EditDist for Text, TEDS for Tables, RMS-F1 [42] for Charts, and BLEU for Formulas.Source: §3.3 · p. 10

4.1. Page-level Evaluation

• 92.86 overall score on OmniDocBench v1.5 surpasses the next-best MinerU2.5-1.2B score of 90.67.PaddleOCR-VL also records a Formula-CDM score of 91.22 and Table-TEDS and Table-TEDS-S scores of 90.89 and 94.76, respectively.Source: Table 2 · p. 11, §4.1 · p. 11
• 0.115 average overall edit distance on OmniDocBench v1.0 accompanies strong text, formula, table, and reading-order results.Formula edit distances are 0.241 EN and 0.316 ZH; Chinese and English text edit distances are 0.062 and 0.041, while Chinese Table TEDS is 92.14. English Table TEDS is 88.0, which the authors attribute largely to typo-related annotation errors.Source: Table 3 · p. 12, §4.1 · p. 12
• 80.0 ± 1.0 is PaddleOCR-VL’s highest overall score on olmOCR-Bench.It leads in ArXiv (85.7) and Headers and Footers (97.0), and places second in Multi-column text (79.9) and Long Tiny Text (85.7).Source: §4.1 · p. 12, Table 4 · p. 13

4.2.1. Text Recognition

• PaddleOCR-VL achieves leading text-recognition results across OmniDocBench-OCR-block, In-house-OCR, and Ocean-OCR-Bench.It records the lowest edit distances in nine OmniDocBench-OCR-block document categories, including Academic Literature (0.021), and across the reported In-house-OCR scripts and text types, including Latin (0.013) and Rare Characters (0.001). On Ocean-OCR-Bench, it leads all reported metrics, with edit distances of 0.118 for English and 0.034 for Chinese handwriting.Source: §4.2.1 · pp. 13–15, Table 5 · p. 14, Table 6 · p. 14

4.2.2. Table Recognition.

• 0.9195 overall TEDS and 0.9543 structural TEDS lead the OmniDocBench-Table-block comparison.The model also records the lowest Overall Edit Distance (0.0561); on the in-house table set, it leads on Overall TEDS (0.8699) and Structural TEDS (0.9066).Source: Table 8 · p. 15, §4.2.2 · pp. 15–16, Table 9 · p. 16

4.2.3. Formula Recognition.

• 0.9453 CDM on OmniDocBench-Formula-block and 0.9882 CDM on In-house-Formula are the reported state-of-the-art and best scores, respectively.The evaluation uses cropped formula images to reduce the influence of layout detection; the in-house set contains 34,816 samples.Source: Table 10 · p. 16, §4.2.3 · pp. 16–17, Table 11 · p. 17

4.2.4. Chart Recognition.

• PaddleOCR-VL outperforms expert OCR VLMs and some 72B-level multimodal language models on the in-house chart benchmark.The authors use only an in-house benchmark because public test sets have limited size, imbalanced chart categories, and poor annotation quality; the set contains 1,801 manually reviewed samples across 11 chart categories.Source: Table 12 · p. 17, §4.2.4 · p. 17

4.3. Inference Performance

• 53.1% higher page throughput and 50.9% higher token throughput than MinerU2.5 are achieved with the FastDeploy1 backend.The inference workflow uses asynchronous, multithreaded data loading, layout processing, and VLM inference, with queued items batched by count or wait-time threshold.Source: §4.3 · pp. 17–18, Table 13 · p. 18

5. Conclusion

• PaddleOCR-VL is presented as an efficient model that excels at both element-level and page-level document recognition.The conclusion says it outperforms existing pipeline solutions on many benchmarks.Source: §5 · p. 18
• Its PaddleOCR-VL-0.9B core combines a NaViT-style visual encoder with the ERNIE-4.5-0.3B language model.The conclusion attributes accurate recognition of complex document elements to this compact architecture.Source: §5 · p. 18
• The model recognizes text, tables, formulas, and charts in over 100 languages while using few resources and running inference quickly.These properties are described as making it practical for real-world deployment.Source: §5 · p. 18
• PaddleOCR-VL effectively handles challenging content, including handwriting and historical documents.Source: §5 · p. 18
• It can convert chart visuals into structured data, extending document parsing beyond recognition of the visual elements themselves.Source: §5 · p. 18
• The conclusion says PaddleOCR-VL could advance multimodal document processing and make complex-document information extraction more efficient for RAG systems.It frames improved RAG performance and stability as a potential benefit.Source: §5 · p. 18

A. Training Dataset Details

• Collecting annotated isolated element images is more feasible than collecting complete document pages containing different elements.The approach applies to training data for text, tables, formulas, and charts.Source: A. Training Dataset Details · p. 25

A.1. Text

• 20 million high-quality image–text training pairs are produced through automated annotation and targeted OCR-data synthesis.PP-StructureV3 layout and recognition outputs are refined by multimodal models and randomly merged at multiple granularities; synthetic data using diverse CSS styles and over 200 fonts addresses messy handwriting and dense, blurry text.Source: §A.1 · p. 25, Figure A1 · p. 25
• Annotations cover text lines, blocks, and pages across 109 languages, diverse document scenes, writing systems, and text styles.Scenes include academic papers, newspapers, handwritten texts, ancient books, IDs, tickets, and seals; styles include printed, handwritten, scanned, and artistic-font text.Source: §A.1 · p. 25

A.2. Table

• The table model uses OTSL [28] rather than conventional HTML as its target format for coding efficiency.The dataset construction overview is shown in Figure A2.Source: §A.2 · p. 26, Figure A2 · p. 26
• The table pipeline produced over 5.5 million training instances through high-quality synthesis, using randomized and targeted configurations of table structures, fonts, styles, and content.The synthesis tool generates 10,000 samples per hour and supports specified parameters for specialized table types.Source: §A.2 · p. 26, Figure A2 · p. 26
• Table data construction combines automatic annotation, mining of public source code, and synthesis to cover diverse languages, border styles, table types, and recognition scenarios.Unlabeled tables are pseudo-labeled and validated, while public HTML is aligned with extracted tables and rule-filtered; annotations also undergo n-gram and HTML checks.Source: §A.2 · pp. 26–27, Figure A2 · p. 26

A.3. Formula

• The formula dataset covers Chinese and English examples across four types: simple and complex printed, screen-captured, and handwritten expressions.Its sources include educational materials, school tests, mathematical papers, courseware, theses, financial reports, and handwritten notes; construction includes source rendering, annotation, long-tail synthesis, and public data collection.Source: §A.3 · p. 27, Figure A3 · p. 27
• Formula source rendering extracts LaTeX from arXiv papers, removes duplicate or similar code with MinHash, normalizes it with KaTeX, and renders images.For real-world formulas, PP-StructureV3 [10] locates regions and ERNIE-4.5-VL-28B-A3B generates LaTeX; rendering and aspect-ratio checks screen candidate pairs, with PP-OCRv5 [10] supporting targeted optimization when Chinese characters are detected.Source: §A.3 · pp. 27–28
• Rule-generated LaTeX and inverse rendering create formula pairs for long-tail cases that multimodal models struggle to recognize, including vertical calculations, strikethroughs, and handwritten explanatory arrows.This targeted synthesis addresses data-distribution issues for these formula structures.Source: §A.3 · p. 28
• Public formula data includes UniMER-1M [30], with one million formulas from multiple sources, and MathWriting [31], with 230,000 real and 400,000 synthetic handwritten samples.UniMER-1M [30] is oriented toward real-document scenarios; MathWriting [31] is described as the largest handwritten mathematical formula dataset.Source: §A.3 · p. 28

A.4. Chart

• The chart training dataset contains over 0.8 million high-quality Chinese-English image-chart pairs from diverse sources and chart types.Its construction combines public data collection and cleaning, automatic annotation, data synthesis, and targeted long-tail augmentation.Source: §A.4 · p. 28, Figure A4 · p. 28
• Public-data cleaning produced 220k high-quality samples, while a two-stage ERNIE-4.5-VL pipeline annotated chart axes and data points with a final consistency check.The source public datasets had noisy annotations and imbalanced distributions; annotation extracts x- and y-axis tick labels, then retrieves corresponding data points using random permutations.Source: §A.4 · p. 29
• A three-stage synthesis pipeline uses ERNIE-X1 and an LLM Persona [77] strategy to diversify table content and rendering styles for chart training.It controls aesthetics including color, font, and layout, and uses matplotlib and seaborn for rendering.Source: §A.4 · p. 29
• Long-tail augmentation selects seed charts by distinctive visual features, replicates their rendering code with ERNIE-4.5-VL, then diversifies data and styles using ERNIE-X1 and personas [77].The modified code generates augmented charts paired with corresponding data tables.Source: §A.4 · p. 29
• The four construction strategies yield chart data spanning varied application scenarios and styles to support chart-model training.The dataset includes charts from academic papers, financial reports, and web pages.Source: §A.4 · pp. 28–29

B. Supported Languages

• PaddleOCR-VL supports 109 languages; Table A1 lists the correspondence between language categories and supported languages.Table 6 in the main text reports text line recognition accuracy for different languages.Source: B. Supported Languages · p. 30, Table A1 · p. 30

C. Inference Performance on Different Hardware Configurations

• PaddleOCR-VL demonstrates stable and efficient inference across a wide range of hardware and backend configurations, supporting flexible use in diverse computing environments.Performance is summarized in Table A2.Source: C. Inference Performance on Different… · p. 31, Table A2 · p. 31
• The inference evaluation used FastDeploy 2.3.0, vLLM 0.10.2, and SGLang 0.5.2.The measurements covered different hardware configurations.Source: C. Inference Performance on Different… · p. 31

D.1. Comprehensive Document Parsing

• Figures A5-A8 show layout and Markdown outputs for books, textbooks, academic papers, reports, financial reports, slides, exams, notes, vertical and ancient books, certificates, newspapers, and magazines.The examples demonstrate parsing across different document types.Source: D. Real-world Samples · p. 32, Figure A5 · p. 33, Figure A6 · p. 34

D.2. Layout Detection

• Figures A9-A11 illustrate PaddleOCR-VL’s ability to process pages with intricate or challenging layouts across various document types.These figures show layout detection results.Source: D. Real-world Samples · p. 32, Figure A9 · p. 37, Figure A10 · p. 38

D.3. Reading Order

• Figures A12-A13 demonstrate reading-order handling in complex reports, textbooks, newspapers, magazines, and vertical documents.The appendix highlights maintaining reading order across these layouts.Source: D. Real-world Samples · p. 32, Figure A12 · p. 40, Figure A13 · p. 41

D.4.1. Multilingual Text Recognition

• Figures A14-A18 present multilingual text recognition examples spanning French, Hindi, Croatian, Spanish, English, Arabic, German, Chinese, Russian, and Japanese documents.The appendix also notes multilingual text recognition as a challenging case.Source: D. Real-world Samples · p. 32, Figure A14 · p. 42, Figure A15 · p. 43

D.4.2. Handwriting Text Recognition; D.4.3. Vertical Text Recognition

• The appendix presents recognition examples for mixed printed and handwritten text, handwritten Chinese and English, and vertical documents.These are identified as challenging text-recognition cases.Source: D. Real-world Samples · p. 32, Figure A20 · p. 48, Figure A21 · p. 49

D.5. Table Recognition

• Figures A23-A24 showcase table recognition across academic and financial reports, watermarked and image-containing tables, tables with formulas, and photographed tables.The appendix describes robust handling of a wide range of table formats.Source: D. Real-world Samples · p. 32, Figure A23 · p. 51, Figure A24 · p. 52

D.6. Formula Recognition

• Figures A25-A26 show formula recognition examples including complex printed, handwritten, screen-captured, and vertical English formulas, as well as formulas containing Chinese characters.The examples cover varied formula forms and scripts.Source: D. Real-world Samples · p. 32, Figure A25 · p. 53, Figure A26 · p. 54

D.7. Chart Recognition

• Figures A27-A29 showcase parsing of pie, bar, line, bar-line hybrid, and heatmap charts, a capability lacking in many expert OCR VLMs such as MinerU2.5, dots.ocr, and MonkeyOCR [1].The cited figures present examples of the chart types recognized.Source: D. Real-world Samples · p. 32, Figure A27 · p. 55, Figure A28 · p. 56

E. Compare with Others

• PaddleOCR-VL consistently outperforms existing SOTA models on PDF pages with complex layouts and unique elements.Figures A30 and A31 illustrate the layout comparisons.Source: E. Compare with Others · p. 58, Figure A30 · p. 59, Figure A31 · p. 60

E.2.1. Multilingual Text Recognition

• PaddleOCR-VL outperforms competitors including MinerU2.5 [2] and MonkeyOCR [1] in multilingual text recognition, avoiding language misidentifications such as Russian and Hindi as English.Figures A32–A34 illustrate multilingual text comparisons.Source: E. Compare with Others · p. 58, Figure A32 · p. 61, Figure A33 · p. 62

E.2.2. Handwriting Text Recognition

• PaddleOCR-VL outperforms competitors including MinerU2.5 [2] and MonkeyOCR [1] in handwriting recognition, where they may overlook handwritten characters.Figures A35 and A36 show handwriting recognition comparisons.Source: E. Compare with Others · p. 58, Figure A35 · p. 64, Figure A36 · p. 65

E.2.3. Vertical Text Recognition

• PaddleOCR-VL outperforms competitors including MinerU2.5 [2] and MonkeyOCR [1] in vertical text recognition, where they struggle.Figure A37 illustrates the vertical text comparison.Source: E. Compare with Others · p. 58, Figure A37 · p. 66

E.3. Table Recognition

• PaddleOCR-VL stands out for parsing complex tables, a task where other models frequently encounter difficulties.Figures A38 and A39 show the table recognition comparisons.Source: E. Compare with Others · p. 58, Figure A38 · p. 67, Figure A39 · p. 68

E.4. Formula Recognition

• PaddleOCR-VL accurately parses complex formulas, while other SOTA models often produce incorrect or flawed outputs on challenging mathematical notations.Figure A40 shows the formula recognition comparison.Source: E. Compare with Others · p. 58, Figure A40 · p. 69

E.5. Chart Recognition

• PaddleOCR-VL accurately reconstructs chart structure and content, outperforming Qwen2.5VL-72B [24] and GPT-4o.Figures A41 and A42 show the chart recognition comparisons.Source: E. Compare with Others · p. 58, Figure A42 · p. 71