Source-linked AI summary
PP-OCRv3: More Attempts for the Improvement of Ultra Lightweight OCR System
Chenxia Li, Weiwei Liu, Ruoyu Guo, Xiaoting Yin, Kaitao Jiang, Yongkun Du, Yuning Du, Lingfeng Zhu, Baohua Lai, Xiaoguang Hu, Dianhai Yu, Yanjun Ma
TL;DR
Practical OCR deployment requires both accuracy and computational efficiency, motivating improvements beyond PP-OCRv2. PP-OCRv3 upgrades detection and recognition with nine strategies, and experiments report 5% higher Hmean than PP-OCRv2 at comparable inference speed.
Problem
Practical OCR systems need to balance accuracy and computational efficiency, while PP-OCRv2 still had missed detections and recognition errors.
Method
PP-OCRv3 upgrades PP-OCRv2’s detection and recognition models through nine strategies, including LK-PAN, RSE-FPN, DML, SVTR-LCNet, and additional training and acceleration methods.
Results
5% higher Hmean is reported for PP-OCRv3 than PP-OCRv2 with comparable inference speed.
Takeaways & Limitations
PP-OCRv3 is a more robust ultra-lightweight OCR system that improves the PP-OCRv2 design while preserving comparable inference speed.
Abstract
from arXiv · showhide
Optical character recognition (OCR) technology has been widely used in various scenes, as shown in Figure 1. Designing a practical OCR system is still a meaningful but challenging task. In previous work, considering the efficiency and accuracy, we proposed a practical ultra lightweight OCR system (PP-OCR), and an optimized version PP-OCRv2. In order to further improve the performance of PP-OCRv2, a more robust OCR system PP-OCRv3 is proposed in this paper. PP-OCRv3 upgrades the text detection model and text recognition model in 9 aspects based on PP-OCRv2. For text detector, we introduce a PAN module with large receptive field named LK-PAN, a FPN module with residual attention mechanism named RSE-FPN, and DML distillation strategy. For text recognizer, the base model is replaced from CRNN to SVTR, and we introduce lightweight text recognition network SVTR LCNet, guided training of CTC by attention, data augmentation strategy TextConAug, better pre-trained model by self-supervised TextRotNet, UDML, and UIM to accelerate the model and improve the effect. Experiments on real data show that the hmean of PP-OCRv3 is 5% higher than PP-OCRv2 under comparable inference speed. All the above mentioned models are open-sourced and the code is available in the GitHub repository PaddleOCR which is powered by PaddlePaddle.
1 Introduction
PP-OCRv3 extends the ultra-lightweight PP-OCR family to improve OCR robustness while balancing accuracy and computational efficiency. It optimizes detection and recognition through new architectural, distillation, training, and compression strategies, achieving higher Hmean than PP-OCRv2 at comparable inference speed.
- Motivation: Practical OCR systems must balance accuracy with limited storage and computing resources across deployment environments.OCR applications include document electronization, identity authentication, digital finance, and license plate recognition.
- System Overview: PP-OCRv3 retains PP-OCRv2’s three-stage framework while further optimizing text detection and recognition, replacing CRNN with SVTR for recognition.The stages are text detection, detected-box rectification, and text recognition.
- Text Detection: The detector adds LK-PAN, RSE-FPN, and DML distillation, improving teacher and student models before Collaborative Mutual Learning.LK-PAN enlarges receptive fields, RSE-FPN is integrated into the student, and DML improves the teacher model.
- Text Recognition: The recognizer introduces SVTR-LCNet and five additional strategies to combine Transformer and CNN advantages while improving accuracy without increasing prediction cost.The strategies include guided CTC training, TextConAug, TextRotNet, U-DML, and UIM.
- Model Compression: PP-OCRv3 tiny applies additional compression and acceleration strategies to produce a smaller model variant.The gray-box strategies in the framework are used for PP-OCRv3 tiny models.
- Results: 5% higher Hmean is achieved by PP-OCRv3 than PP-OCRv2 with comparable inference speed.The paper reports this outcome from experiments and ablations.
2 Improvement Strategies
PP-OCRv3 improves detection and recognition through architectural changes, distillation, attention, augmentation, pretraining, and unlabeled-data mining while targeting practical accuracy–efficiency trade-offs.
- Text Detection: CML combines teacher-guided distillation with mutual learning among student networks in the PP-OCRv3 detection model.The teacher and student networks are optimized separately before the teacher guides student training.
- Text Detection: LK-PAN expands PAN path-augmentation kernels from 3 × 3 to 9 × 9, enlarging receptive fields for large-font and extreme-aspect-ratio text.It remains lightweight while increasing each feature-map pixel’s receptive field.
- Text Detection: DML trains two same-structure teacher models to learn from each other, improving teacher-model text-detection accuracy before distillation.The strategy is used specifically to raise the teacher model’s Hmean.
- Text Detection: RSE-FPN replaces FPN convolutions with residual squeeze-and-excitation convolutions to strengthen feature representation while mitigating suppression of important channels.The residual structure addresses limitations observed when using squeeze-and-excitation blocks alone.
- Text Recognition: The recognizer replaces CRNN with SVTR and applies six strategies to improve practicality, including GTC, TextConAug, TextRotNet, U-DML, and UIM.SVTR uses Transformer structure to mine text-line context without an RNN.
- Text Recognition: SVTR-LCNet fuses SVTR with PP-LCNet by replacing SVTR-Tiny’s first half, reducing Global Mix Blocks from 4 to 2, and moving them after pooling.These changes address SVTR-Tiny’s CPU inference speed while retaining its core design.
- Text Recognition: TextRotNet pretrains SVTR-LCNet on unlabeled text-line data, while U-DML jointly supervises intermediate outputs from PP-LCNet, SVTR, and attention modules.UIM additionally selects high-confidence pseudo-labeled samples from unlabeled images for lightweight-model training.
3 Experiments
Experiments evaluate PP-OCRv3 on expanded detection, recognition, and end-to-end datasets, using ablations to measure its strategies and system performance. The results show improvements in detection, recognition, and overall OCR efficiency compared with PP-OCRv2.
- Datasets: The experiments use expanded datasets for text detection and recognition, plus 800 real application images for overall OCR evaluation.Detection uses 127k training and 200 validation images; recognition uses 18.5M training and 18.7K validation images.
- Text Detection: The text detector combines an optimized teacher using LK-PAN and DML with a student using RSE-FPN under CML distillation.The teacher prioritizes effect, while the student considers both effect and efficiency.
- Text Detection: 86.0% Hmean is reached by the teacher after LK-PAN and DML, while RSE-FPN raises the student’s Hmean from 81.3% to 84.5% with only 6% more inference time.The teacher baseline is DB-R50; the student comparison uses experiments without CML distillation.
- Text Detection: The final detection configuration improves Hmean from 83.2% to 85.4% through CML and RSE-FPN.CML first raises Hmean to 84.3%, after which adding RSE-FPN raises it to 85.4%.
- Text Recognition: 73.98% recognition accuracy is achieved by SVTR-LCNet, close to the PP-OCRv2 recognizer trained with U-DML distillation.SVTR-LCNet progressively trades accuracy and speed through architectural changes, while increasing input height from 32 to 48 improves accuracy.
- Text Recognition: Recognition ablations report gains of 1.82% from GTC, 0.5% from TextConAug, 0.6% from TextRotNet, 1.5% from UDML, and 1.0% from UIM.GTC adds no inference cost because its attention module is removed during prediction.
- System Performance: 5.3% higher Hmean and 22% faster T4 GPU inference are reported for PP-OCRv3 than PP-OCRv2, with the same CPU inference cost.Hmean is used for text detection and end-to-end OCR, while sentence accuracy evaluates recognition.
4 Conclusions
PP-OCRv3 is presented as a more robust ultra-lightweight OCR system with nine improvements across text detection and recognition. Experiments report a 5% Hmean advantage over PP-OCRv2 at the same prediction cost.
- 4 Conclusions: PP-OCRv3 introduces nine improvements, comprising three for text detection and six for text recognition.The improvements target a more robust OCR system while retaining the ultra-lightweight design.
- 4 Conclusions: 5% higher Hmean than PP-OCRv2 is reported for PP-OCRv3 with the same prediction cost.The paper also provides corresponding ablation experiments.