PaperlayerView exact revision on arXiv

Source-linked AI summary

HunyuanOCR Technical Report

Hunyuan Vision Team, Pengyuan Lyu, Xingyu Wan, Gengluo Li, Shangpin Peng, Weinong Wang, Liang Wu, Huawen Shen, Yu Zhou, Canhui Tang, Qi Yang, Qiming Peng, Bin Luo, Hower Yang, Xinsong Zhang, Jinnian Zhang, Houwen Peng, Hongming Yang, Senhao Xie, Longsha Zhou, Ge Pei, Binghong Wu, Rui Yan, Kan Wu, Jieneng Yang, Bochao Wang, Kai Liu, Jianchen Zhu, Jie Jiang, Linus, Han Hu, Chengquan Zhang

arXiv:2511.19575v2cs.CVcs.AI

TL;DR

Existing OCR pipelines can propagate errors across stages, while large general VLMs may require substantial compute and struggle to meet low-latency needs. HunyuanOCR uses a lightweight, end-to-end VLM for multiple OCR tasks, and reports strong performance across parsing, spotting, and information extraction. The report also identifies translation as a relative weakness of the model.

  • Problem

    Cascaded OCR systems incur error propagation and maintenance overhead, while large VLMs can require substantial compute and miss low-latency needs.

    Source: §1 · p. 2, §2.1 · p. 4

  • Method

    HunyuanOCR combines a compact VLM with end-to-end training and inference, broad OCR task support, high-quality application-oriented data, and task-specific online reinforcement learning.

    Source: §1 · p. 3, §2.2.2 · p. 5

  • Results

    HunyuanOCR reports state-of-the-art results in text spotting, document parsing, and information extraction, and outperforms leading open-source alternatives and commercial APIs on several task-specific comparisons.

    Source: §1 · p. 2, §7 · p. 14

  • Takeaways & Limitations

    The report shows that a 1B-parameter model can unify diverse OCR tasks while achieving competitive performance against larger VLMs and traditional pipeline systems.

    Source: §7 · p. 14

    Translation capability lags behind text detection, recognition, and document parsing, so applications needing higher translation accuracy may require a multilingual parsing cascade or another model.

    Source: §6.4 · p. 13

Abstract

from arXiv · show

This paper presents HunyuanOCR, a commercial-grade, open-source, and lightweight (1B parameters) Vision-Language Model (VLM) dedicated to OCR tasks. The architecture comprises a Native Vision Transformer (ViT) and a lightweight LLM connected via an MLP adapter. HunyuanOCR demonstrates superior performance, outperforming commercial APIs, traditional pipelines, and larger models (e.g., Qwen3-VL-4B). Specifically, it surpasses current public solutions in perception tasks (Text Spotting, Parsing) and excels in semantic tasks (IE, Text Image Translation), securing first place in the ICDAR 2025 DIMT Challenge (Small Model Track). Furthermore, it achieves state-of-the-art (SOTA) results on OCRBench among VLMs with fewer than 3B parameters. HunyuanOCR achieves breakthroughs in three key aspects: 1) Unifying Versatility and Efficiency: We implement comprehensive support for core capabilities including spotting, parsing, IE, VQA, and translation within a lightweight framework. This addresses the limitations of narrow "OCR expert models" and inefficient "General VLMs". 2) Streamlined End-to-End Architecture: Adopting a pure end-to-end paradigm eliminates dependencies on pre-processing modules (e.g., layout analysis). This fundamentally resolves error propagation common in traditional pipelines and simplifies system deployment. 3) Data-Driven and RL Strategies: We confirm the critical role of high-quality data and, for the first time in the industry, demonstrate that Reinforcement Learning (RL) strategies yield significant performance gains in OCR tasks. HunyuanOCR is officially open-sourced on HuggingFace. We also provide a high-performance deployment solution based on vLLM, placing its production efficiency in the top tier. We hope this model will advance frontier research and provide a solid foundation for industrial applications.

1 Introduction

OCR has expanded from text extraction to multilingual, layout-aware understanding, while existing pipelines and VLM approaches retain practical limitations. HunyuanOCR unifies diverse OCR tasks in a compact end-to-end model, with reported gains across benchmarks and task-specific settings.

1 Introduction

  • OCR now covers diverse layouts, casual images, multilingual and handwritten text, and semantic tasks beyond scanned-document text extraction.The report describes applications in education, healthcare, and building corpora for large language models.Source: §1 · p. 2
  • Traditional multi-model pipelines offer modularity and task-specific optimization, but cascading stages bring error propagation and higher development and maintenance overhead.Some OCR VLMs also retain preliminary layout analysis because of limitations with complex layouts and lengthy text.Source: §1 · p. 2
  • HunyuanOCR supports spotting, parsing, receipt information extraction, subtitle recognition, VQA, and translation in a 1B-parameter model, outperforming named alternatives on several tasks.The report cites document parsing, text image translation, information extraction, and text spotting comparisons.Source: §1 · p. 2
  • High-quality application-aligned data and targeted online reinforcement learning are identified as key to gains, especially in complex parsing, VQA, and image translation.The model uses approximately 200 million training samples and task-specific reward mechanisms.Source: §1 · p. 3

2 Related Work

Prior OCR systems evolved from modular recognition pipelines toward unified vision-language models, but practical trade-offs remain. HunyuanOCR is presented as an end-to-end, lightweight model combining broad task coverage with strong reported performance.

2 Related Work

  • Traditional OCR developed from template matching and feature engineering through statistical learning to deep-learning systems and VLM-based unified processing.Source: §2 · p. 3

2.1 Traditional OCR Systems

  • Modular OCR systems combine specialized components, but their complexity, maintenance burden, and error amplification can limit performance on overlapping text and non-standard layouts.A typical document-parsing system may require at least five subsystems.Source: §2.1 · p. 4

2.2 Vision-Language Models

  • VLMs use unified neural architectures for end-to-end cross-modal understanding and generation, with multi-task processing and language-model reasoning among their stated advantages.Source: §2.2 · p. 4

2.2.1 General Vision-Language Models

  • General VLMs can recognize printed and handwritten text across complex layouts and multilingual content, but their size can demand substantial compute and prevent low-latency deployment.Source: §2.2.1 · p. 4

2.2.2 OCR-Specific Vision-Language Models

  • Unlike OCR-specific VLMs that use a layout-detection stage, HunyuanOCR is fully end-to-end at training and inference, avoiding cascaded error accumulation.The report also describes single-pass workflows and support for parsing, spotting, IE, VQA, and cross-lingual translation.Source: §2.2.2 · pp. 4–6
  • HunyuanOCR is presented as achieving exceptional performance across OCR benchmarks, with reported results surpassing state-of-the-art models and matching or exceeding leading commercial OCR APIs.The report highlights end-to-end processing, broad task coverage, and a three-module architecture as distinguishing features.Source: §2.2.2 · p. 5, Figure 2 · p. 5
  • Its Native Resolution Visual Encoder, Adaptive MLP Connector, and Lightweight Language Model form the core architecture; the encoder preserves aspect ratio for arbitrary resolutions.The connector compresses visual tokens while preserving information from text-dense areas, and the language model uses XD-RoPE for text, height, width, and time subspaces.Source: §2.2.2 · pp. 5–6

4 Data Construction

HunyuanOCR brings spotting, parsing, information extraction, VQA, and translation into one task framework. Its data pipeline combines large-scale real and synthetic data with controlled image transformations and cross-task QA generation.

4.1 Task Design / 4.1.1 Spotting

  • Text spotting outputs recognized text with coordinates in a structured format, with coordinates normalized to [0, 1000].Source: §4.1.1 · p. 6

4.1.2 Parsing

  • Document parsing supports both fine-grained extraction of elements and full-page parsing that preserves reading order and structures tables and formulas.The described element types include mathematical and chemical formulas, tables, and charts.Source: §4.1.2 · pp. 6–7

4.1.3 IE & VQA

  • IE supports open-world field extraction, instruction-controlled single or multi-field outputs, and subtitle extraction; VQA handles open-ended questions and visual reasoning.VQA inputs include text lines, formulas, documents, charts, and street-view imagery.Source: §4.1.3 · p. 7

4.1.4 Text Image Translation

  • Image-to-text translation covers over 14 source languages into Chinese or English, plus bidirectional Chinese-English translation across document and general scenes.The system describes complementary prompts for general scenes and structured documents.Source: §4.1.4 · p. 7

4.2 Data Pipelines

  • The training corpus contains over 200 million image-text pairs from nine real-world scenarios and more than 130 languages.It combines public benchmarks, real-world data, and synthetic samples.Source: §4.2 · p. 8

4.2.1 Image data synthesis

  • The synthesis pipeline supports paragraph-level rendering in over 130 languages, including RTL layouts, controllable text attributes, image perturbations, and task-aligned data reuse.It generates aligned data for spotting, long-document parsing, and cross-lingual translation.Source: §4.2.1 · p. 8, Figure 3 · p. 8

4.2.2 Image data augmentation

  • Warping augmentation simulates folds, curves, perspective distortion, blur, noise, compression artifacts, lighting changes, shadows, and reflections.The report says this pipeline improves robustness in spotting, parsing, and VQA.Source: §4.2.2 · p. 9

4.2.3 Question–Answer Pair Generation

  • A QA pipeline retrieves hard samples, generates task-specific question-answer pairs, and verifies them across models, reusing spotting and parsing data for unified training.Some failed cases receive manual verification.Source: §4.2.3 · p. 9

5 Training Recipe

HunyuanOCR’s training recipe combines four pre-training stages with reinforcement-learning design choices tailored to OCR data, tasks, and output constraints.

5.1 Pre-Training

  • Four pre-training stages progress from bridge warm-up and end-to-end multimodal learning to 32K context extension and application-oriented tuning.The first stage trains the visual encoder and adapter while freezing the language model; the second unfreezes all parameters. Later stages use long-context data and curated examples with standardized instructions and outputs.Source: §5.1 · pp. 9–10

5.2.1 Data Curation

  • Data curation filters for image–text alignment, task diversity, and task difficulty, removing exploitable, low-diversity, trivial, and unsolvable samples.Source: §5.2.1 · p. 10

5.2.2 Reward Design

  • Rewards are task-adaptive: spotting combines localization matching with text edit distance, parsing uses normalized edit distance, VQA uses binary semantic matching, and translation scores are normalized to [0,1].For translation, an LLM assigns a raw score from [0, 5], with normalization expanding reward granularity in the mid-range.Source: §5.2.2 · pp. 10–11

5.2.3 Training Strategy

  • GRPO updates the current policy from grouped responses and group-derived advantages with KL regularization; overlength or malformed structured outputs receive zero reward.Source: §5.2.3 · p. 11

6 Evaluation

HunyuanOCR performs strongly across spotting, parsing, information extraction, and document-image translation. The evaluations report strong results across document settings and comparisons with substantially larger models, alongside a relative weakness in translation.

6.1 Spotting

  • HunyuanOCR achieves the best overall spotting performance, outperforming traditional pipelines and general VLMs while using substantially fewer parameters.Source: Table 3 · p. 11, §6.1 · p. 11

6.2 Parsing

  • HunyuanOCR achieves the highest overall performance on digital/scanned and Wild document parsing, and state-of-the-art results across all 14 DocML languages.Source: Table 4 · p. 12, §6.2 · p. 12

6.3 IE & VQA

  • Across 30 card/receipt categories and subtitle extraction, HunyuanOCR significantly outperforms much larger VLMs; on OCRBench, it exceeds DeepseekOCR and is comparable to Qwen3VL-2B-Instruct.Source: Table 5 · p. 13, §6.3 · p. 13

6.4 Text Image Translation

  • On DoTA, HunyuanOCR surpasses VLMs over 8B parameters, places first in the ICDAR 2025 Small Model Track, and outperforms VLMs over 4B parameters on DocML.Source: Table 6 · p. 13, §6.4 · p. 13
  • Despite strong results, HunyuanOCR’s translation capability lags its text detection, recognition, and document parsing performance because its language model is relatively small.Source: §6.4 · p. 13

7 Conclusion

The conclusion presents HunyuanOCR as an open-source, lightweight end-to-end VLM that unifies OCR tasks. It reports strong results and identifies future work on efficiency, document coverage, and edge deployment.

7 Conclusion

  • HunyuanOCR unifies diverse OCR tasks in an open-source, lightweight end-to-end VLM architecture.Source: §7 · p. 14
  • The 1B-parameter model achieves state-of-the-art results in text spotting, document parsing, and information extraction, validating the reported data-centric training and targeted reinforcement learning approach.Source: §7 · p. 14
  • Future work targets inference efficiency, higher-resolution and multi-page documents, and eventual edge-device deployment.Source: §7 · p. 14

Supplementary Material

The supplementary material provides task instructions, IE category coverage, and additional reinforcement-learning details. It also outlines the RL training configuration, dynamics, and performance analyses.

Supplementary Material

  • The supplement covers recommended instructions, common IE categories, RL details, element-level evaluation, and qualitative examples.Source: Supplementary Material · p. 18

A Recommended Instruction

B Common Supported IE Categories

C Reinforcement Learning Details

C.1 Training Configuration

The RL configuration uses a constant learning rate, Adam, long-context settings, no explicit KL penalty, and eight sampled responses per prompt.

C.1 Training Configuration

  • RL uses a constant learning-rate schedule and Adam, with a large global batch and long-context settings.Source: §C.1 · p. 19
  • No explicit KL penalty is applied, allowing the policy to adjust more freely under task-specific rewards.Source: §C.1 · p. 19
  • Rollouts use temperature 0.85 and sample N = 8 responses per prompt for reward evaluation and policy updates.Source: §C.1 · p. 19

C.2 Training Dynamics

The RL training dynamics track the proportion of samples receiving reward 1 and the mean reward. Mean reward rises steadily as training progresses, indicating increasingly reward-aligned outputs.

  • RL training tracks both the proportion of samples receiving reward 1 and the mean reward at each step.Source: §C.2 · p. 19
  • The mean reward increases steadily over the course of training.Source: §C.2 · p. 19
  • The upward reward trend indicates that the policy gradually learns outputs that better satisfy task-specific criteria, supporting the process’s effectiveness and stability.Source: §C.2 · p. 19

C.3 Task-wise Performance Improvements

After RL training, HunyuanOCR improves across multiple OCR-related tasks. The report attributes these gains to diverse, high-quality training data and fine-grained task-specific rewards.

Task-wise results

  • After RL, HunyuanOCR gains across spotting, parsing, IE, OCRBench, and text-image translation.Reported changes include spotting gains above 2 points on Art and Screen, OmniDocBench parsing rising from 92.5 to 94.1, IE improving by about 2 points, and OCRBench average score increasing by 3.3.Source: §C.3 · pp. 19–20

Spotting

  • Spotting scores increase by more than 2 points on Art and Screen after RL training.The report attributes these gains to fine-grained rule-based rewards assessing bounding-box discrepancies and text correctness.Source: §C.3 · p. 19

Parsing

  • OmniDocBench parsing rises from 92.5 to 94.1 after RL training.The report attributes the improvement to rule-based rewards measuring content consistency with reference text.Source: §C.3 · p. 19

Discussion

  • The mean reward rises steadily during RL training, and the RL stage complements supervised training to produce a more capable and reliable model.Source: Figure 4 · p. 20, §C.3 · p. 20

Information Extraction, VQA, and Translation

  • IE improves by about 2 points, OCRBench average score increases by 3.3, and text-image translation shows noticeable gains.The report links higher-level task improvements to LLM-as-a-judge rewards guiding faithful, semantically accurate outputs.Source: §C.3 · p. 20

Discussion

  • RL training data are curated and diverse, while task-specific rewards provide fine-grained feedback across recognition, parsing, and semantic understanding.The reward functions include both rule-based and LLM-as-a-judge designs.Source: §C.3 · p. 20

D Element-level Evaluation

The element-level evaluation isolates formula and table regions from OmniDocBench1.5 to assess fine-grained parsing, with prompts tailored to each element type.

E Qualitative examples

The qualitative examples illustrate HunyuanOCR outputs for text spotting, flowchart parsing, document translation, and structured information extraction.

  • A spotting example returns recognized Chinese text with a quadrilateral coordinate annotation.Source: E Qualitative examples · p. 22
  • A flowchart-parsing prompt requests Mermaid for flowcharts, and the output represents the chart as connected nodes and branches.Source: E Qualitative examples · p. 31
  • A document-translation example requests Chinese translation while specifying that headers and footers be ignored and formulas and tables be formatted separately.The output includes translated document text and its source-paper content.Source: E Qualitative examples · p. 33
  • A receipt-extraction prompt requests eight named fields in JSON, and the example output supplies values for those fields.Source: E Qualitative examples · p. 34
  • The examples also pose visual questions about a factory name, male life expectancy, and Samsung’s largest-market-share platform in 2018.Source: E Qualitative examples · p. 36

Related papers

Loading 2511.19575v2…