Source-linked AI summary
SVTR: Scene Text Recognition with a Single Visual Model
Yongkun Du, Zhineng Chen, Caiyan Jia, Xiaoting Yin, Tianlun Zheng, Chenxia Li, Yuning Du, Yu-Gang Jiang
TL;DR
SVTR addresses limitations in prior feature extractors for scene text recognition by using a single visual model with patch-wise character components and hierarchical mixing. It achieves competitive or higher accuracy across English and Chinese benchmarks, while SVTR-T offers 6.03M parameters and 4.5ms average inference.
Problem
Prior feature extractors do not adequately model both intra-character local patterns and inter-character long-term dependence, while sequence representations can be noisy for irregular text.
Method
SVTR decomposes text images into 2D character components and applies three-stage multi-scale mixing, merging, and combining operations with local and global mixing blocks.
Results
SVTR-L achieves state-of-the-art performance on English and Chinese scene text recognition, while SVTR-T uses 6.03M parameters and consumes 4.5ms per image text on average.
Takeaways & Limitations
A single visual model can deliver competitive or higher accuracy with accuracy, efficiency, and cross-lingual versatility, and different capacities support diverse application needs.
Abstract
from arXiv · showhide
Dominant scene text recognition models commonly contain two building blocks, a visual model for feature extraction and a sequence model for text transcription. This hybrid architecture, although accurate, is complex and less efficient. In this study, we propose a Single Visual model for Scene Text recognition within the patch-wise image tokenization framework, which dispenses with the sequential modeling entirely. The method, termed SVTR, firstly decomposes an image text into small patches named character components. Afterward, hierarchical stages are recurrently carried out by component-level mixing, merging and/or combining. Global and local mixing blocks are devised to perceive the inter-character and intra-character patterns, leading to a multi-grained character component perception. Thus, characters are recognized by a simple linear prediction. Experimental results on both English and Chinese scene text recognition tasks demonstrate the effectiveness of SVTR. SVTR-L (Large) achieves highly competitive accuracy in English and outperforms existing methods by a large margin in Chinese, while running faster. In addition, SVTR-T (Tiny) is an effective and much smaller model, which shows appealing speed at inference. The code is publicly available at https://github.com/PaddlePaddle/PaddleOCR.
1 Introduction
SVTR replaces the common visual-plus-sequence recognizer with a single visual model that mixes character-component information at local and global scales. It targets accurate, efficient, cross-lingual recognition, with strong English and Chinese benchmark results.
- Motivation: Scene text recognizers commonly combine a visual feature extractor with a sequence model, but complex paradigms can restrict efficiency.CNN-RNN, encoder-decoder, and vision-language approaches differ in how they model sequence context and language information.
- Approach: Local and global mixing blocks capture stroke-like intra-character patterns and inter-character dependence, forming multi-grained character features.The design reinforces the visual model to represent complementary local and long-term recognition clues.
- Approach: SVTR decomposes text images into character components and uses hierarchical mixing, merging, and combining operations instead of sequential modeling.Its three-stage architecture progressively decreases height before linear recognition.
- Contribution: A single visual model can achieve competitive or higher accuracy than advanced vision-language models for scene text recognition.This is presented as a practical alternative to more complex recognition paradigms.
- Results: SVTR-L achieves state-of-the-art performance on English and Chinese scene text benchmarks, while SVTR-T provides an effective, efficient smaller model.SVTR-T has 6.03M parameters and averages 4.5ms per image text on one NVIDIA 1080Ti GPU.
2 Method
SVTR builds a hierarchical visual backbone from overlapping character-component patches, alternating global and local mixing with spatial reduction. It then predicts the character sequence in parallel through a linear classifier.
- 2.1 Overall Architecture: SVTR transforms an H×W×3 text image into overlapping patches called character components, each representing part of a character.Progressive overlapping patch embedding produces patches with feature dimension D0.
- 2.1 Overall Architecture: Three stages repeatedly apply mixing blocks followed by merging or combining operations at different scales.The network progressively decreases height while extracting component features and dependencies.
- 2.2 Mixing Block: Global mixing evaluates dependence among all components, capturing long-term relationships across characters and reducing non-text influence.It operates on a reshaped feature sequence after layer normalization and attention-based mixing.
- 2.2 Mixing Block: Local mixing evaluates components within a sliding window to encode character morphology and stroke-like local patterns.The empirically selected window size is 7 × 11.
- 2.3 Merging and Combining: Merging halves the height while preserving width, reducing computation and building a text-customized multi-scale representation.The height reduction is intended to preserve the horizontal patch layout of text.
- 2.3 Merging and Combining: After height pooling and feature combination, a parallel linear classifier predicts components, and de-duplication produces the final character sequence.Duplicate character outputs are condensed, while non-text components map to a blank symbol.
- Model Variants: SVTR-T, SVTR-S, SVTR-B, and SVTR-L vary in capacity through depth, attention heads, mixing-block count, and block permutation.These configurations are designed to meet different application needs.
3.1 Datasets
The evaluation uses established English benchmarks spanning regular and irregular text plus a large Chinese scene-text dataset. English models train on synthetic data and are tested across six public benchmarks.
- English Recognition: English models are trained on the synthetic MJSynth and SynthText datasets and tested on six public scene-text benchmarks.The listed benchmarks include IC13, SVT, and CUTE80, covering regular, street-view, and curved text settings.
- Chinese Recognition: The Chinese Scene Dataset contains 509,164 training, 63,645 validation, and 63,646 test images.The validation set selects the best model, which is then evaluated on the test set.
3.2 Implementation Details
SVTR uses image rectification, AdamW optimization, cosine learning-rate scheduling with warm-up, and varied geometric and noise augmentations for training.
- Training: Training augmentation randomly applies rotation, perspective distortion, motion blur, and Gaussian noise.The alphabet is case-insensitive alphanumeric with maximum prediction length 25.
3.3 Ablation Study
Ablations show that progressive patch embedding and feature merging improve recognition, while mixing-block arrangements provide additional gains, particularly on irregular text.
- Experimental setting: The controlled experiments use SVTR-T on regular IC13 and irregular IC15 without rectification or data augmentation.This isolates the architectural choices under study.
- Patch embedding: Progressive embedding improves average accuracy by 0.75% and 2.8% across the two datasets compared with the two default strategies.The improvement is especially notable for irregular text.
- Merging: Merging reduces computational cost and increases accuracy on both IC13 and IC15.This supports multi-scale sampling along the height dimension.
- Mixing blocks: Almost every tested global-local mixing permutation improves accuracy, with larger gains on irregular text.The study attributes these gains to more comprehensive character-component features.
- Efficiency comparison: Figure 5 compares different models using accuracy-parameter and accuracy-speed plots on IC15.The figure evaluates both compactness and inference efficiency alongside accuracy.
3.4 Comparison with State-of-the-Art
SVTR performs competitively across English benchmarks and strongly on the Chinese scene dataset, while retaining simplicity and speed advantages over language-aware alternatives.
- English benchmarks: SVTR-T achieves highly competitive accuracy among language-free English recognizers, while larger SVTR variants reach state-of-the-art results on most datasets.SVTR-L obtains the best accuracy on IIIT, IC15, and CUTE among the six English benchmarks.
- Language and efficiency: SVTR-L matches the overall accuracy of recent studies using extra language models while remaining simpler and faster.The comparison covers six English benchmarks and one Chinese scene dataset.
- Chinese benchmark: 5.4% to 9.6% accuracy gains over SAR are observed on the Chinese Scene Dataset.The authors associate this improvement with multi-grained character-component features suited to rich Chinese stroke patterns.
- Efficiency comparison: On IC15, SVTR models rank in the top tier of accuracy-parameter and accuracy-speed plots.This further compares accuracy with model size and inference time.
3.5 Visualization Analysis
Visualization analysis shows that SVTR-T attention maps specialize in different character regions and component relationships, supporting its multi-grained recognition behavior.
- Attention maps: SVTR-T attention maps focus on distinct regions of the same character, including the left side, bottom, and middle of “B”.The visualization indicates that different character regions contribute separately to recognition.
- Practical implication: SVTR-T is described as effective and much smaller, while remaining quite fast for resource-limited scenarios.The conclusion positions SVTR-T as appealing when computational resources are constrained.
4 Conclusion
SVTR uses multi-grained character features to recognize scene text with a single visual model, achieving competitive accuracy and faster operation across English and Chinese benchmarks.
- Conclusion: SVTR extracts stroke-like local patterns and inter-component dependencies at multiple height scales.These features support recognition with a single visual model.
- Conclusion: Experiments on English and Chinese benchmarks show highly competitive or better accuracy than state-of-the-art methods while running faster.The conclusion frames SVTR as accurate, efficient, and cross-lingually versatile.
- Conclusion: SVTR variants with different capacities are designed to meet diverse application needs.The paper specifically characterizes SVTR-L as suitable for accuracy-oriented applications and SVTR-T as appealing in resource-limited scenarios.