Source-linked AI summary
TextBoxes: A Fast Text Detector with a Single Deep Neural Network
Minghui Liao, Baoguang Shi, Xiang Bai, Xinggang Wang, Wenyu Liu
TL;DR
Scene text detection is challenging and traditionally relies on multi-stage processing, motivating a faster end-to-end approach. TextBoxes directly predicts word boxes in a fully convolutional network, and its combination with CRNN achieves state-of-the-art word spotting and end-to-end recognition performance. The method is efficient, including a fast implementation taking 0.09s per image, but remains challenged by overexposure and large character spacing.
Problem
Scene text varies widely in appearance and lighting, while traditional detectors require multiple tuned processing stages.
Method
TextBoxes is an end-to-end fully convolutional detector that predicts text presence and offsets to default boxes, aggregates outputs, and applies NMS.
Results
TextBoxes consistently outperforms competing methods on text localization, while TextBoxes combined with CRNN achieves state-of-the-art word spotting and end-to-end recognition performance.
Takeaways & Limitations
The detector provides a computationally efficient approach to text detection and robust text reading in the wild when paired with recognition.
Takeaways & Limitations
TextBoxes still fails on some difficult cases, including overexposure and large character spacing.
Abstract
from arXiv · showhide
This paper presents an end-to-end trainable fast scene text detector, named TextBoxes, which detects scene text with both high accuracy and efficiency in a single network forward pass, involving no post-process except for a standard non-maximum suppression. TextBoxes outperforms competing methods in terms of text localization accuracy and is much faster, taking only 0.09s per image in a fast implementation. Furthermore, combined with a text recognizer, TextBoxes significantly outperforms state-of-the-art approaches on word spotting and end-to-end text recognition tasks.
Introduction
TextBoxes addresses the complexity and slowness of multi-stage scene text detection with an end-to-end neural detector. Combined with CRNN, it also supports strong word spotting and end-to-end recognition performance.
- Scene text reading is difficult because foreground text, backgrounds, and lighting vary substantially.
- Traditional detectors use candidate generation, filtering, and grouping, requiring heuristic tuning and slowing detection.
- TextBoxes directly predicts word bounding boxes by jointly estimating text presence and coordinate offsets to default boxes.
- A single forward pass aggregates outputs from novel inception-style layers and applies standard non-maximum suppression.
- Combining TextBoxes with CRNN adds recognition outputs and semantic-level regularization, boosting word spotting and end-to-end recognition.
- The paper targets an end-to-end detector, a combined recognition framework, and competitive accuracy with computational efficiency.
Related Works
Prior scene text systems commonly decompose detection into character or word proposals followed by grouping or classification. TextBoxes instead uses a fully convolutional, SSD-inspired design with text-specific aspect-ratio handling and CRNN recognition.
- Scene text reading separates into detection, which localizes words, and recognition, which transcribes cropped words into character sequences.
- Character-based methods detect individual characters before grouping them into words.
- TextBoxes uses a 28-layer fully convolutional architecture with VGG-16 layers, added convolutions, text-box layers, and aggregated NMS outputs.
- Word-based methods generate word candidates, classify proposals, and perform bounding-box regression.
- Unlike SSD on words with extreme aspect ratios, TextBoxes adds text-box layers to improve detection for this setting.
- CRNN produces character sequences and its confidence scores regularize TextBoxes detection outputs for word spotting and end-to-end recognition.
Detecting text with TextBoxes
TextBoxes is a fully convolutional detector that predicts text boxes across multiple layers using task-specific default boxes and text-shaped filters. It combines direct detection with recognition-based rescoring and multi-scale processing to handle varied word shapes and sizes.
- Text-box layers: Text-box layers predict text presence and bounding-box offsets from feature maps, then aggregate outputs and apply non-maximum suppression.The network is fully convolutional and supports arbitrary-size images during training and testing.
- Default boxes: Six default-box aspect ratios—1, 2, 3, 5, 7, and 10—target words with large aspect ratios, while vertical offsets improve matching coverage.Default boxes of different sizes and aspect ratios let the model learn specialized regression and classification weights.
- Text-box layers: Irregular 1*5 convolutional filters provide rectangular receptive fields that better fit wide words and reduce noisy signals from square receptive fields.
- Learning: TextBoxes uses a joint confidence-and-localization loss with smooth L1 regression and a 2-class softmax confidence loss.The loss is normalized by the number of default boxes matching ground-truth boxes, with α set to 1.
- Multi-scale detection: Multi-scale inputs use five image resolutions to improve localization of words with extreme aspect ratios and sizes.The scales are 300*300, 700*700, 300*700, 500*700, and 1600*1600; additional NMS is applied across multi-scale outputs.
- Word spotting and recognition: CRNN recognition rescoring uses lexicon compatibility to remove unlikely word candidates after TextBoxes generates a high-recall proposal set.With multi-scale ICDAR 2013 inputs, the proposal stage preserves about 35 boxes per image at recall 0.93 before recognition-based re-evaluation.
Experiments
TextBoxes is evaluated on text detection, word spotting, and end-to-end recognition, achieving strong accuracy and speed across these tasks while retaining difficult-case weaknesses.
- Text detection: 0.09s per image is achieved by the fast implementation without much accuracy loss.The multi-scale version takes 0.73s per image, while the fast implementation prioritizes speed with limited accuracy reduction.
- Text detection: TextBoxes handles large word aspect ratios better than SSD, benefiting from text-box layers designed for word-length variation.SSD remains competitive overall but performs poorly on words with large aspect ratios.
- Word spotting: TextBoxes outperforms existing methods on word spotting, by at least 2 percentage points on ICDAR 2011 and ICDAR 2013 and over 8 percentage points on SVT and SVT-50.The evaluation uses detection results refined by recognition and covers ICDAR 2011, SVT, and ICDAR 2013.
- End-to-end recognition: Coupled with a recognition model, TextBoxes achieves state-of-the-art end-to-end recognition performance.On ICDAR 2013, it surpasses Adelaide ConvLSTMs* across all lexicon settings; with a strong lexicon, it retains 0.91 recall and 0.97 precision after reserving 3.8 boxes per image.
- Limitations: TextBoxes still fails on difficult cases involving overexposure and large character spacing.Examples of these failure cases are shown in Figures 3 and 4.
Conclusion
TextBoxes is an end-to-end fully convolutional network that efficiently generates stable word proposals against cluttered backgrounds. Evaluations validate its advantages across text detection, word spotting, and end-to-end recognition.
- TextBoxes uses an end-to-end fully convolutional network for text detection.
- The detector efficiently generates stable word proposals against cluttered backgrounds.
- Benchmark evaluations validate TextBoxes across text detection, word spotting, and end-to-end recognition.