Source-linked AI summary
Single Shot Text Detector with Regional Attention
Pan He, Weilin Huang, Tong He, Qile Zhu, Yu Qiao, Xiaolin Li
TL;DR
Text detection in natural images must handle large variation in scale and orientation while resisting complicated backgrounds. The paper proposes a single-shot detector with learned text attention and hierarchical inception features, achieving state-of-the-art performance on ICDAR benchmarks, including an F-measure of 0.87 in reported comparisons.
Problem
Natural-image text detection is challenging because text varies in scale, orientation, illumination, and font against complicated backgrounds.
Method
The detector combines a text attention module with pixel-wise supervision and a hierarchical inception module that aggregates multi-scale features within an SSD framework.
Results
The detector achieves state-of-the-art performance on ICDAR 2013 and ICDAR 2015, with an F-measure of 0.87 reported on ICDAR 2013 word-level evaluation.
Takeaways & Limitations
The single-shot model directly predicts word-level boxes and works reliably on multi-scale and multi-orientation text with single-scale inputs.
Abstract
from arXiv · showhide
We present a novel single-shot text detector that directly outputs word-level bounding boxes in a natural image. We propose an attention mechanism which roughly identifies text regions via an automatically learned attentional map. This substantially suppresses background interference in the convolutional features, which is the key to producing accurate inference of words, particularly at extremely small sizes. This results in a single model that essentially works in a coarse-to-fine manner. It departs from recent FCN- based text detectors which cascade multiple FCN models to achieve an accurate prediction. Furthermore, we develop a hierarchical inception module which efficiently aggregates multi-scale inception features. This enhances local details, and also encodes strong context information, allow- ing the detector to work reliably on multi-scale and multi- orientation text with single-scale images. Our text detector achieves an F-measure of 77% on the ICDAR 2015 bench- mark, advancing the state-of-the-art results in [18, 28]. Demo is available at: http://sstd.whuang.org/.
1. Introduction
Natural-image text detection is difficult because text varies widely in scale, orientation, illumination, and font against complicated backgrounds. The paper addresses these challenges with a single-shot detector combining text attention and hierarchical multi-scale feature aggregation.
- Natural-image text detection remains challenging because text varies in scale, orientation, illumination, and font, often against complicated backgrounds.
- Bottom-up detectors use sequential character detection, filtering, text-line construction, and word splitting, allowing errors to accumulate across stages.
- Pixel-based FCN detectors use semantic text segmentation to exploit regional context, while box-based detectors directly predict text boxes from convolutional features.
- The proposed text attention module learns rough text regions from pixel-wise supervision, suppressing background interference and highlighting challenging text patterns.
- The hierarchical inception module aggregates multi-scale features across layers, enhancing local details and encoding contextual information for word prediction.
- Integrated into SSD, the text-specific modules support fast, accurate, single-shot detection of multi-scale and multi-orientation text using single-scale inputs.The reported running time is about 0.13s per 704 × 704 image.
2. Related Work
Related text detectors include bottom-up pipelines, pixel-based FCNs, and box-based extensions of object detectors. The paper positions its model as a direct, efficient alternative that combines word-box prediction with text-specific feature processing.
- Bottom-up methods detect text components and then group them through multiple stages into characters, character pairs, and text lines.
- FCN-based approaches use shared deep features and pixel-wise text or non-text estimation to exploit regional context, but may provide insufficient localization accuracy from a single model.
- Object-detector-based methods predict text bounding boxes from convolutional features, but bounding-box annotations can provide coarser supervision than text masks.
- FCRN requires three post-processing stages taking about 1.2s/image, compared with 0.07s/image for its bounding-box estimations.
- The proposed detector directly outputs word boxes without bottom-up or heuristic post-processing and is designed to handle multi-orientation text efficiently.
3. Methodology
The detector extends SSD with text-specific attention and hierarchical inception modules to directly predict word-level boxes, combining rough text-region guidance with multi-scale feature aggregation. These components are designed to reduce background interference and improve detection of small, multi-scale, and multi-orientation text.
- Detector architecture: The detector directly outputs word-level bounding boxes through SSD-derived convolutional and box-prediction components, with text-specific modules added between them.It is end-to-end trainable and requires only simple NMS as post-processing.
- Text Attention Module: The text attention module learns a pixel-wise text-probability map from Aggregated Inception Features and encodes the resized map back into those features.An auxiliary loss supervises the attention map with a binary text/non-text mask.
- Text Attention Module: Text attention suppresses background interference in convolutional features, reducing false detections and highlighting challenging text patterns.The paper reports improvements in false detections, ambiguous-text detection, and word-level accuracy relative to a baseline.
- Hierarchical Inception Module: The inception module processes convolutional maps with four operations, including dilated convolutions, to capture multi-scale receptive fields.The operations use 1 × 1, 3 × 3, pooled 3 × 3, and decomposed 5 × 5 convolutions.
- Hierarchical Inception Module: The hierarchical inception module fuses inception features from adjacent layers into Aggregated Inception Features with aligned resolutions.Down-sampling and up-sampling align features before channel concatenation, enriching local details and contextual information.
- Word Prediction Module: The word prediction module uses text/non-text classification and smoothl1 regression for five bounding-box parameters, including orientation.Multiple default-box scales and 45 default boxes per layer accommodate text with varied scales, aspect ratios, and orientations.
4. Experimental Results
The detector is evaluated on three benchmarks, with component studies and comparisons covering near-horizontal, multi-orientation, large-scale, challenging, and runtime settings.
- Experimental setup: The evaluation uses ICDAR 2013, ICDAR 2015, and COCO-Text, with component effectiveness and state-of-the-art comparisons assessed.Training uses 13,090 images from ICDAR 2013, ICDAR 2015, and Internet-harvested images, excluding COCO-Text training data.
- Exploration study: The combined text attention and hierarchical inception modules produce a 0.87 F-measure on ICDAR 2013, with both modules improving recall individually.The combined model further improves recall and precision, while the modules can compensate for each other.
- Benchmark comparisons: 0.87 F-measure is achieved on ICDAR 2013 under the word-level standard, outperforming FCRN, CTPN, and TextBoxes.The detector is also comparable to CTPN under the less strict DetEval standard.
- Benchmark comparisons: 0.77 F-measure is achieved on ICDAR 2015, exceeding CTPN’s 0.61 and the cited 0.71 result.The benchmark evaluates multi-orientation text, including arbitrary orientations, motion blur, and low-resolution text.
- Benchmark comparisons: 0.37 F-score is achieved on COCO-Text, slightly improving the recent state-of-the-art result.The method also obtains significantly higher recall than all compared approaches.
- Runtime: 0.13s/image is reported on a single GPU, slightly faster than CTPN’s 0.14s/image and with substantially improved performance over TextBoxes.FCRN predicts boxes in 0.07s/image but requires 1.2s/image for post-processing.
5. Conclusion
The paper concludes that its text-specific attention and hierarchical inception modules yield a fast, accurate single-shot detector for multi-scale and multi-orientation text.
- Conclusion: The detector predicts word-level bounding boxes in one shot and achieves new state-of-the-art results on ICDAR 2013, ICDAR 2015, and COCO-Text.The attention mechanism learns rough text regions, while hierarchical inception features add local detail and context.