Source-linked AI summary
DeepText: A Unified Framework for Text Proposal Generation and Text Detection in Natural Images
Zhuoyao Zhong, Lianwen Jin, Shuye Zhang, Ziyong Feng
TL;DR
Text detection in natural images must locate text amid complex backgrounds and wide variation in appearance and image conditions. DeepText unifies word-region proposal generation and text detection with a fully convolutional network, reporting state-of-the-art F-measures on ICDAR 2011 and 2013.
Problem
Text detection in natural images must locate text despite complex backgrounds and variation in text appearance and image conditions.
Source: §1 · pp. 1–2Method
DeepText combines Inception-RPN word proposals, a text detection network with ambiguous text category information and multi-level ROI pooling, and iterative box voting with filtering.
Source: §1 · p. 3Results
F-measures of 0.83 on ICDAR 2011 and 0.85 on ICDAR 2013 outperform previous state-of-the-art results.
Source: §1 · p. 3Takeaways & Limitations
The reported performance on both datasets is described as highlighting the robustness and effectiveness of DeepText.
Source: §3.4 · p. 9
Abstract
from arXiv · showhide
In this paper, we develop a novel unified framework called DeepText for text region proposal generation and text detection in natural images via a fully convolutional neural network (CNN). First, we propose the inception region proposal network (Inception-RPN) and design a set of text characteristic prior bounding boxes to achieve high word recall with only hundred level candidate proposals. Next, we present a powerful textdetection network that embeds ambiguous text category (ATC) information and multilevel region-of-interest pooling (MLRP) for text and non-text classification and accurate localization. Finally, we apply an iterative bounding box voting scheme to pursue high recall in a complementary manner and introduce a filtering algorithm to retain the most suitable bounding box, while removing redundant inner and outer boxes for each text instance. Our approach achieves an F-measure of 0.83 and 0.85 on the ICDAR 2011 and 2013 robust text detection benchmarks, outperforming previous state-of-the-art results.
1 Introduction
DeepText targets text detection’s challenges in natural images with a unified framework for word-region proposals and text detection. It combines proposal generation, text/non-text discrimination, and box refinement, reporting higher F-measures than previous results on two ICDAR benchmarks.
1 Introduction
- DeepText combines word-proposal generation and text detection to avoid sequential error accumulation in bottom-up character-candidate approaches.Earlier methods also faced complex backgrounds and text variation in color, font, orientation, language, and scale, alongside illumination, exposure, contrast, blur, resolution, and occlusion challenges.Source: §1 · pp. 1–3
- DeepText aims to reduce thousands of word proposals to hundreds while maintaining high word recall, using text-characteristic prior boxes.The proposal design and detection network form the paper’s unified approach.Source: §1 · p. 3
- Inception-RPN uses multi-scale sliding windows and text-characteristic prior boxes to achieve high recall with only hundreds of word proposals.The sliding-window features retain local and contextual information to help filter non-text prior boxes.Source: §1 · p. 3
- The text detection network adds ambiguous text category information and multi-level ROI pooling to learn more discriminative text-versus-background features.Source: §1 · p. 3
- Iterative bounding-box voting uses intermediate models for complementary high word recall, while filtering keeps the most suitable box and removes redundant inner or outer boxes.Source: §1 · p. 3
- F-measures of 0.83 on ICDAR 2011 and 0.85 on ICDAR 2013 outperform previous state-of-the-art results.The paper presents these as results on the robust text detection benchmarks.Source: §1 · p. 3
2 Methodology
DeepText generates word proposals with Inception-RPN, then classifies and localizes them using ambiguous-text labels and multilevel features. End-to-end training and post-processing combine proposals across iterations and filter redundant boxes.
2.1 Text region proposal generation
- Inception-RPN uses text-characteristic priors and multiscale features to generate word proposals, then applies NMS and selects the top 2,000 candidates for detection.It assigns 24 priors per sliding position across four scales and six aspect ratios; the proposal network uses parallel 3×3 convolution, 5×5 convolution, and pooling features.Source: §2.1 · p. 4
2.2 Text Detection
- The detector labels proposals with IoU 0.2–0.5 as ambiguous text and pools Conv4_3 and Conv5_3 features for classification and regression.The ambiguous category addresses proposals that may contain partial or extensive text; multilevel pooled features are concatenated and fused with a 1×1 convolution.Source: §2.2 · p. 5, Figure 2 · p. 5
2.3 End-to-end learning optimization
- The two networks are trained end-to-end with a multitask classification-and-regression loss, using shared VGG16 convolutional layers initialized from ImageNet pretraining.The loss balances classification and bounding-box regression; λ is set to 3 for Inception-RPN and 1 for the text detection network.Source: §2.3 · p. 6
2.4 Heuristic processing
- Iterative bounding-box voting merges detection candidates across training iterations, while filtering removes inner and outer boxes and retains the highest-scoring box per text instance.Voting is followed by NMS at an IoU threshold of 0.3; filtering addresses redundant boxes that may remain after NMS.Source: §2.4 · p. 6
3 Experiments and Analysis
The experiments evaluate proposal recall, text-detection features, and the full DeepText system on the ICDAR benchmarks. They report high recall with hundreds of proposals and stronger overall benchmark results than previous methods.
3.1 Experiments Data
- 4,072 training images were collected, including 1,715 Flickr images and 2,028 images selected from COCO-Text.The authors note that the ICDAR training sets are small for training a reasonable network.Source: §3.1 · p. 7
3.2 Evaluation of Inception-RPN
- Nearly 90% word recall is achieved with hundreds of proposals; 300 proposals approximate 500-proposal recall and outperform standard RPN by 8%-10%.The results also report superior performance to SS and EB, especially as proposals fall from 500 to 100.Source: §3.2 · pp. 7–8, Figure 3 · p. 8
3.3 Analysis of text detection network
- Adding ATC and MLRP increases the TP rate by 3.13% and decreases the FP rate by 0.82%.The comparison uses 300 proposals per ICDAR 2013 test image, with positive and negative labels assigned by IoU overlap thresholds.Source: §3.3 · p. 8
3.4 Experimental results on full text detection
- DeepText achieves 0.83 F-measure on ICDAR 2011 and 0.85 F-measure on ICDAR 2013, outperforming previous results on both benchmarks.Sun et al.'s results are described as not comparable because they used millions of additional training samples, versus DeepText's 4,072.Source: §3.4 · p. 8, Table 2 · p. 9, Table 3 · p. 9
- Qualitative detections include non-uniform illumination, multiple and small regions, and low-contrast text; average runtime is 1.7 s per image on one K40 GPU.Source: §3.4 · p. 9
4 Conclusion
DeepText combines proposal generation and text detection in an end-to-end fully convolutional framework. It achieves state-of-the-art F-measure on two ICDAR benchmarks, while the authors identify recall and precision as areas for further improvement.
4 Conclusion
- DeepText combines Inception-RPN proposals, text classification and localization, iterative bounding-box voting, and filtering to remove redundant boxes.Source: §4 · p. 9
- The system achieves state-of-the-art F-measure performance on the ICDAR 2011 and 2013 robust text detection benchmarks.Source: §4 · p. 9
- The authors identify further improvement in candidate-proposal recall and proposal classification and localization accuracy as future work.Source: §4 · p. 9