PaperlayerView exact revision on arXiv

Source-linked AI summary

DocAtlas: Multilingual Document Understanding Across 80+ Languages

Ahmed Heakl, Youssef Mohamed, Abdullah Sohail, Rania Elbadry, Ahmed Nassar, Peter W. J. Staar, Fahad Shahbaz Khan, Imran Razzak, Salman Khan

arXiv:2605.12623v2cs.CLcs.CVcs.LG

TL;DR

Multilingual document understanding lacks high-quality, broadly multilingual data, especially for low-resource languages and scripts. DocAtlas builds model-free OCR datasets and benchmarks through differential rendering and synthetic RTL generation, then uses rendering-derived ground truth for DPO adaptation. It reports stable cross-lingual gains while preserving base-language capabilities, although native digital document sources are required.

  • Problem

    Scarce high-quality training data and model-based annotation pipelines limit multilingual document understanding and can perpetuate existing biases.

  • Method

    DocAtlas combines differential rendering and synthetic RTL generation to create unified DocTag annotations without learned models for core annotation.

  • Results

    DPO with rendering-derived ground truth achieves stable cross-lingual transfer, with +1.7% accuracy and <3% base-language degradation while supervised methods show up to 21% forgetting.

  • Takeaways & Limitations

    Annotation quality, rather than model scale alone, bounds multilingual document understanding, and QKV-only LoRA balances multilingual gains with capability preservation.

  • Takeaways & Limitations

    Differential rendering requires native DOCX or structured markup, so it cannot annotate scanned or photographed documents lacking digital text layers.

Abstract

from arXiv · show

Multilingual document understanding remains limited for low-resource languages due to scarce training data and model-based annotation pipelines that perpetuate existing biases. We introduce DocAtlas, a framework that constructs high-fidelity OCR datasets and benchmarks covering 82 languages and 9 evaluation tasks. Our dual pipelines, differential rendering of native DOCX documents and synthetic LaTeX-based generation for right-to-left scripts produce precise structural annotations in a unified DocTag format encoding layout, text, and component types, without learned models for core annotation. Evaluating 16 state-of-the-art models reveals persistent gaps in low-resource scripts. We show that Direct Preference Optimization (DPO) using rendering-derived ground truth as positive signal achieves stable multilingual adaptation, improving both in-domain (+1.9%) and out-of-domain (+1.8%) accuracy without measurable base-language degradation, where supervised fine-tuning degrades out-of-domain performance by up to 21%. Our best variant, DocAtlas-DeepSeek, improves +1.7% over the strongest baseline. Code is available at https://github.com/ahmedheakl/DocAtlas .

1 Introduction

DocAtlas addresses multilingual document-understanding gaps by constructing model-free, structurally precise datasets and benchmarks across diverse languages, including RTL scripts. Its evaluation exposes persistent low-resource challenges and studies DPO as a cross-lingual adaptation strategy.

  • Motivation: Limited annotated data and biased model-based annotation pipelines hinder document understanding for low- and medium-resource languages.Existing methods also struggle to scale, represent complex structures, and cover RTL scripts.
  • Approach: DocAtlas combines differential rendering of native documents with synthetic RTL generation to produce model-free structural annotations in unified DocTag format.The pipelines encode component type, geometry, and text while addressing underrepresented bidirectional scripts.
  • Benchmark: 360K training pages and a 5.8K-page benchmark span 82 languages and 9 evaluation tasks, enabling systematic multilingual model comparison.The benchmark covers parsing, recognition, tables, formulas, charts, reading order, and format-specific subtasks.
  • Findings: Low-resource scripts show 40–60% accuracy drops, while structured extraction reaches a language-invariant ceiling of 73% TEDS.Chart parsing also separates OCR-specialized systems from general VLMs.
  • Adaptation: DPO with rendering-derived ground truth outperforms supervised fine-tuning and closed-model distillation for cross-lingual transfer.The study identifies rendering-derived annotations as the key training signal for stable multilingual adaptation.

2 Related Works

Prior OCR datasets trade off scalability, realism, annotation reliability, language coverage, and task breadth. DocAtlas extends rendering-based construction with aligned annotations and a broader multilingual benchmark covering diverse document elements.

  • Dataset construction: Manual annotation provides high-quality labels but scales poorly, whereas synthetic generation misses nested tables and authentic formatting.Synthetic pipelines use predefined text positions, making labels correct but limiting structural realism.
  • Evaluation benchmarks: DocAtlas offers 3× more languages than comparable parsing benchmarks while supporting all major document elements.Its benchmark is designed for comprehensive multilingual parsing evaluation.
  • Rendering-based methods: Rendering-based approaches avoid learned detectors but can suffer rendering drift, weak text-box alignment, opaque chart treatment, and absent RTL coverage.These limitations arise in prior colorization and format-conversion pipelines.
  • DocAtlas: DocAtlas uses lossless rendering, pixel-wise differential subtraction, and IoU-based matching to produce aligned DocTag annotations.The resulting annotations jointly support text, geometry, and component-level supervision.
  • Evaluation benchmarks: Existing benchmarks remain fragmented across languages and document elements, preventing direct comparison across systems.Prior resources range from table-only evaluations to broader benchmarks with limited language diversity.

3 Methods

DocAtlas constructs multilingual OCR supervision through complementary native-DOCX and synthetic-RTL pipelines, using source-level structure and rendering to produce aligned annotations. A unified DocTag representation supports downstream document-understanding tasks.

  • Pipeline A: Native Word Documents: The native pipeline processes Word documents by recovering structure, injecting color codes, and subtracting colorized from uncolorized renderings for model-free bounding boxes.Text is aligned to geometric regions through word-level IoU containment.
  • Quality Control: A quality-control stage uses language identification and language-specific 5-gram perplexity, retaining over 94% of high-quality data while filtering 38% of low-confidence pages.The perplexity threshold is τ = 120.
  • Unified Representation: DocTag serializes component type, geometry, and text content in a unified XML-like schema for layout detection, reading order, and content extraction.Additional outputs include JSON, HTML, Markdown, and visual overlays.
  • Pipeline A: Native Word Documents: Differential rendering addresses WordScape’s false detections and rendering drift by separating injected colors from pre-existing colors and using MS Word rendering.Word-level IoU matching jointly aligns text, geometry, and component type.
  • Pipeline B: Synthetic RTL Pipeline: The synthetic RTL pipeline parses structured inputs, renders them through 205 LuaTeX templates, and logs positions across compilation passes to recover element-level annotations.It generates 52K pages across Arabic, Hebrew, Urdu, and Persian with near-perfect annotation precision.
  • Benchmark and Adaptation: The benchmark samples up to 100 pages per language across 82 languages, stratifying pages into equal easy, medium, and hard splits and adding challenging formula samples.The training strategies compare full-page SFT, component-level SFT, and DPO with rendering-derived positives while varying trainable parameters.

4 Analysis & Experiments

The study evaluates 14 models across general VLMs, expert document models, and OCR-specific systems. Its evaluation spans multilingual document-understanding capabilities represented by these model categories.

  • Dataset Statistics: The dataset statistics describe 1.9M documents across 5.48M pages and 136 languages, with automated PII detection removing 5.15% of documents.The native pipeline sustains 100k+ pages per day on one CPU, while the synthetic pipeline generates 195k pages at 183 pages per minute.
  • Model Selection: The evaluation includes 14 models spanning general VLMs, expert document models, and OCR-specific systems.The categories distinguish multilingual baselines, compact layout-grounded parsers, and cross-lingual OCR systems.

5 Results

Experiments reveal a persistent resource divide and a stronger bottleneck in structured extraction than text recognition. DPO with rendering-derived supervision improves adaptation while preserving capabilities better than supervised alternatives.

  • Leaderboard Comparison: 83.37% overall accuracy makes DocAtlas-Deepseek the strongest system, followed by DeepseekOCR at 81.66%.Text edit distances for top models average 0.068–0.095, while table TEDS plateaus at 71–73%.
  • Leaderboard Comparison: High-resource languages reach 80–95% accuracy, whereas low-resource scripts span 20–85% with median performance often below 40%.The distribution shows substantially wider variation for low-resource scripts.
  • Language Families: Indo-European and Cyrillic families achieve 80–87% accuracy, while Japonic ranges from 26.9–70.5% and Austroasiatic languages remain difficult.The reported contrast associates weaker performance with typologically challenging scripts and language families.
  • Chart Extraction: Gemini-2.5-Flash achieves the highest average chart-extraction performance at 61.82%, while DeepseekOCR falls from 87% on English to 8–17% on Thai, Arabic, and Italian.Multimodal models outperform document-specific architectures across chart types.
  • Training Strategy Analysis: Full SFT gains +13.6 TEDS on new languages but loses −12.1 TEDS on base languages, whereas QKV-only LoRA provides the reported gain-preservation balance.QKV adaptation is described as learning cross-lingual attention routing without biasing output distributions.
  • Training Strategy Analysis: DPO improves in-domain and out-of-domain performance simultaneously, while component-level training suffers catastrophic forgetting up to −21.3%.The authors attribute capability preservation to using base-model predictions as negative examples.
  • Dataset Quality: Rendering-derived positives outperform GPT-4o distillation, whose low-resource-script biases produce marginal in-domain gains of +0.4 but reduce out-of-domain transfer by −0.7.Reported biases include hallucinated diacritics and RTL column misordering.
  • Out-of-Distribution Generalization: DocAtlas-DeepSeek reduces edit distance from 22.1%→20.7% and 0.137→0.122 on unseen photographed or scanned benchmarks.These results concern DocPTBench and OmniDocBench, which are English-dominated and unseen during training.

6 Conclusion

DocAtlas provides model-free multilingual OCR supervision and benchmarking across 82 languages and 9 tasks. Its results identify low-resource gaps and structured-extraction limits, while DPO and QKV-only LoRA support more stable adaptation.

  • Conclusion: DocAtlas produces 360K training pages and a 5.8K-page benchmark across 82 languages and 9 tasks without learned models for core annotation.The corpus and benchmark are constructed through differential rendering and complementary pipelines.
  • Conclusion: Evaluation identifies persistent low-resource gaps and a language-invariant table ceiling of 73% TEDS, indicating spatial reasoning as the primary bottleneck.DPO with rendering-derived ground truth achieves +1.7% accuracy with less than 3% base degradation.
  • Conclusion: The authors conclude that annotation quality, rather than model scale, bounds multilingual document understanding.QKV-only LoRA is reported to balance multilingual gains with capability preservation.

Limitations

The differential rendering pipeline depends on native document source files, limiting annotation to born-digital documents with digital text layers.

  • Native DOCX or structured markup is required for model-free annotation, excluding scanned and photographed documents without digital text layers.The authors identify combining this supervision with OCR-from-scratch methods for scanned documents as a natural extension.

Use of Language Models

Language models assisted only with minor manuscript editing, while the authors produced, verified, and finalized the technical content, experiments, results, and conclusions.

  • Language models were used only for minor editing and polishing, including clarity and grammar improvements.
  • The authors produced, verified, and finalized all technical content, experimental design, results, and conclusions.

7 Data Generation Details

DocAtlas combines large-scale multilingual document collection with native and synthetic RTL generation, model-free structural annotation, filtering, and quality control.

  • Corpus and collection: The raw corpus contains 1,011,501 documents spanning 5.48M pages across 136 languages from native DOCX and synthetic RTL pipelines.Pipeline A contributes 1,002,465 documents and 5.29M pages; Pipeline B contributes 9,036 documents and 195K pages across four RTL languages.
  • Corpus and collection: The corpus covers over 25 primary categories and subcategories, balancing professional, academic, and public-interest domains.
  • Annotation and enrichment: Core structural annotations are fully model-free, while two optional enrichment steps use learned models without affecting DocTag output.
  • Filtering and quality control: PII filtering automatically excluded documents containing three or more PII instances or any government-issued identifier.The supplied passage reports 942,118 removals from 18.3M initially collected documents.
  • Filtering and quality control: Perplexity thresholding at τ = 120 retains over 94% of high-quality data while filtering out 38% of low-confidence pages.
  • Synthetic RTL generation: The synthetic RTL pipeline uses language-specific LaTeX templates supporting page formats, columns, typography, bidirectional text, and mixed LTR/RTL layouts.Persian templates additionally support mixed LTR/RTL layouts for scientific content.
  • Synthetic RTL generation: LuaTeX compilation uses three passes for layout, positional logging, and validated final rendering, reaching 183 pages/minute on one CPU core.
  • Filtering and quality control: Quality filtering removes pages with coordinate drift, template misalignment, or font-rendering failures, yielding 195K filtered RTL pages.

7.6 Dataset Statistics and Quality Control

DocAtlas combines broad multilingual and domain coverage with difficulty-stratified sampling, expert-checked chart generation, attribute annotation, and targeted quality filtering.

  • Dataset statistics: The corpus spans 5.48M pages across 136 languages, with 100k+ native-pipeline pages processed daily and 195K RTL pages generated at 183 pages/minute.
  • Dataset statistics: The corpus covers 25+ domains, including Health, Law & Government, Finance, and Science, with balanced professional, academic, and public-interest categories.
  • Benchmark construction: Difficulty-stratified sampling uses visual clustering and component-weighted difficulty scores to approximate equal easy, medium, and hard splits.
  • Benchmark construction: Chart generation combines expert VLM topic creation, multi-type rendering, GPT-4o filtering, and verification by three domain specialists.Structural integrity, formula alignment, and RTL reading order reached 94.2% inter-annotator agreement (κ=0.89).
  • Benchmark construction: Page-level attributes such as column layout, watermark presence, and background color support controlled evaluations under specified visual and layout conditions.
  • Quality control: Scanned or rasterized PDFs comprise 8.2% of downloads and are excluded because they lack text layers and structured supervision.
  • Quality control: Rendering drift affects less than 0.3% overall; mitigation recovers 92% of affected documents, while residual failures comprise 0.024% of the total corpus.
  • Quality control: 98.9% of retained documents achieve >95% annotation accuracy, with a corpus mean of 97.8% (σ = 2.1%).The 1.1% below 95% are retained with warning flags.

8 Experiments

DocAtlas evaluates multilingual document models with complementary metrics spanning text, tables, formulas, charts, and full pages, while comparing model scale and layout-parsing behavior. The experiments show that differential rendering produces cleaner segmentation and that compact expert systems can compete with larger general-purpose VLMs.

  • 8.1 Metric Definitions: Five complementary metrics cover text, table, formula, chart, and full-page performance.The evaluation includes NED, TEDS, CDM, chart score, and full-page assessment.
  • 8.1 Metric Definitions: NED compares predicted and ground-truth strings using normalized Levenshtein distance, with lower reported TextEdit indicating better performance.NED ranges from 0 for completely dissimilar strings to 1 for identical strings; the paper reports 1 − NED as TextEdit.
  • 8.1 Metric Definitions: TEDS evaluates predicted and ground-truth tables as HTML DOM trees, penalizing both structural and within-cell textual discrepancies.This captures errors such as missing rows or merged cells alongside textual differences.
  • 8.1 Metric Definitions: Chart extraction is scored with TEDS after charts are converted into structured HTML tables.The formulation evaluates both the resulting table layout and its textual content.
  • Qualitative Comparisons: Figure 13 shows DocAtlas producing cleaner segmentation than traditional layout parsers across extra, overlapping, missing, and misclassified detections.The comparison covers dense tables, multilingual pages, rotated elements, and visually complex or RTL documents.
  • Model Scale: Compact expert systems can match or exceed larger general-purpose VLMs on text and table scores.Figure 14 specifically identifies systems with 3B or fewer parameters as examples of this pattern.

9 Results

Results show that model scale alone does not determine document-understanding performance, while training-data composition strongly shapes domain generalization. Models perform best on academic and research documents but substantially worse on newspapers and magazines, and systematic OCR errors remain concentrated in recurring categories.

  • Model Scale: Compact expert systems can rival much larger general-purpose VLMs in overall document-understanding performance.Figure 14 visualizes this relationship between model scale and overall performance.
  • Document Type Performance: 65–70% accuracy on academic literature and research reports contrasts with 30–45% on newspapers and magazines.The paper attributes the latter domain's difficulty to dense multi-column layouts and varied typography.
  • OCR Error Analysis: 15.7% of 88,036 OCR errors are spacing errors, followed by formatting errors at 14.6%.The evaluation covers 11 OCR models, 5,345 documents, and 12 error categories.
  • OCR Error Analysis: 13.2% of errors involve character encoding issues, and another 13.2% involve content omission.Encoding problems particularly affect Unicode normalization, while omissions affect hyphenated words and list separators.
Loading 2605.12623v2…