Source-linked AI summary
Multi-Oriented Text Detection with Fully Convolutional Networks
Zheng Zhang, Chengquan Zhang, Wei Shen, Cong Yao, Wenyu Liu, Xiang Bai
TL;DR
Multi-oriented scene-text detection is difficult because existing methods largely focus on horizontal text and character detectors struggle with complex backgrounds and character variation. The paper proposes a coarse-to-fine FCN framework that combines holistic text-block saliency, local character components, orientation estimation, and character-centroid filtering. It reports state-of-the-art performance across horizontal and multi-oriented benchmarks, while remaining short of real-time speed and perfect accuracy.
Problem
Multi-oriented scene-text detection remains challenging because horizontal-text methods leave an accuracy gap, while character detectors struggle with complex backgrounds and varied characters.
Method
A coarse-to-fine framework combines an FCN text-region salient map, character components, component-based orientation estimation, text-line candidate construction, and a second FCN for centroid-based false-candidate removal.
Results
The method achieves state-of-the-art performance on horizontal and multi-oriented scene-text detection benchmarks, including MSRA-TD500 precision 0.83, recall 0.67, and f-measure 0.74.
Takeaways & Limitations
Combining semantic text-block labeling with local MSER components provides a supported direction for multi-oriented text-line localization.
Takeaways & Limitations
Failure cases include low contrast, curvature, strong reflected light, closely spaced text lines, and large gaps between characters, and the method remains far from real-time speed.
Abstract
from arXiv · showhide
In this paper, we propose a novel approach for text detec- tion in natural images. Both local and global cues are taken into account for localizing text lines in a coarse-to-fine pro- cedure. First, a Fully Convolutional Network (FCN) model is trained to predict the salient map of text regions in a holistic manner. Then, text line hypotheses are estimated by combining the salient map and character components. Fi- nally, another FCN classifier is used to predict the centroid of each character, in order to remove the false hypotheses. The framework is general for handling text in multiple ori- entations, languages and fonts. The proposed method con- sistently achieves the state-of-the-art performance on three text detection benchmarks: MSRA-TD500, ICDAR2015 and ICDAR2013.
1. Introduction
Multi-oriented text detection remains difficult because most existing methods target horizontal or near-horizontal text. The proposed coarse-to-fine framework combines global text-block cues with local character components and FCN-based filtering.
- Arbitrary text-line orientations enlarge the search space, leaving a substantial accuracy gap for existing horizontal-text methods.
- The framework first predicts a holistic pixel-wise text/non-text salient map with an FCN to guide text-block localization.
- The Text-Block FCN fuses feature maps across stages so lower levels capture local structures while higher levels capture global information.
- Text-line candidates are generated by extracting character components, estimating orientation through component projections, and combining local components with global text blocks.
- A second FCN predicts character centroids within candidates, using their positions to remove false text-line hypotheses.
- The framework is designed to handle scale, orientation, script, language, and font variation while detecting both horizontal and multi-oriented scene text.
2. Related Work
Earlier scene-text detection mainly focused on horizontal or near-horizontal text, using word boxes, end-to-end recognition, or connected-component pipelines. Multi-oriented methods extended component extraction with orientation estimation and grouping, while CNN-based approaches improved representations but remained focused on horizontal text.
- Most established scene-text methods detect horizontal or near-horizontal word boxes or combine detection with recognition end to end.
- Early multi-oriented systems used connected-component extraction together with character and text-line orientation estimation.
- Graph partitioning and multi-stage clustering were applied to group MSER components for multi-oriented text detection.
- Deep convolutional methods learned robust component representations, reduced false positives, or supported recognition, but the cited approaches focused on horizontal text.
3. Proposed Methodology
The proposed methodology combines global text-block cues with local character components in a coarse-to-fine pipeline for multi-oriented text detection. A second FCN predicts character centroids to eliminate false text-line candidates.
- 3. Proposed Methodology: The pipeline detects text blocks with Text-Block FCN, extracts MSER character components, estimates orientation, constructs text-line candidates, and removes false candidates using character centroids.The framework combines block-level and component-level information across successive stages.
- 3.1. Text Block Detection: Text-Block FCN uses local and global context for efficient pixel-wise text-region labeling and coarse text-block localization.FCN supports end-to-end pixel labeling while combining context from different feature-map stages.
- 3.1. Text Block Detection: During training, pixels inside each annotated text-line or word bounding box are labeled positive to incorporate character gaps and global text structure.The model is trained with cross-entropy loss and stochastic gradient descent.
- 3.2. Multi-Oriented Text Line Candidate Generation: The method estimates text-line orientation by counting character components along candidate orientations and selecting the direction with the maximum count.The component-count function Φ(θ, h) represents the number of components at orientation θ and vertical offset h.
- 3.2. Multi-Oriented Text Line Candidate Generation: Character components are grouped according to height, pair orientation, and estimated block orientation, after which minimum bounding boxes form text-line candidates.Repeating this procedure for each text block extracts all candidates while using both block-level and component-level cues.
- 3.2. Multi-Oriented Text Line Candidate Generation: The candidate-generation process does not require MSER to detect every character because text-block guidance preserves candidate generation when components are missed or partially detected.This combines the coarse global localization with incomplete local character evidence.
4. Experiments
Experiments evaluate the method on three text-detection benchmarks, showing strong performance across horizontal and multi-oriented text while revealing failure cases and non-real-time speed.
- Benchmark results: The method achieves precision 0.83, recall 0.67 and f-measure 0.74 on MSRA-TD500, improving f-measure by 0.03 over.It takes 2.1s per image on average with GPU acceleration.
- Benchmark results: The method achieves the best F-measure among collected competition results on ICDAR2015.ICDAR2015 contains incidental text appearing at arbitrary orientations and locations, including small or low-resolution text.
- Benchmark results: The method achieves precision 0.88, recall 0.78 and f-measure 0.83 on ICDAR2013, outperforming recent methods designed only for horizontal text.ICDAR2013 is a horizontal text database with 229 training and 233 testing images.
- Qualitative analysis: Detection examples cover inner texture, non-uniform illumination, dot fonts, broken strokes, multiple orientations, perspective distortion and mixed languages.These examples are shown on MSRA-TD500 and ICDAR2013.
- Parameter analysis: Recall of text-line candidates is insensitive to T1 and T2 across a large range on MSRA-TD500.T1 and T2 are used to extract MSER components for computing text-line candidates.
- Limitations: False positives and missing characters occur under extremely low contrast, curvature, strong reflected light, closely spaced lines or large gaps between characters.The method remains far from the real-time speed requirement.
5. Conclusion
The paper concludes that combining FCN semantic labeling with MSER offers a promising framework for multi-oriented scene text detection. It identifies end-to-end text recognition as a future extension.
- Conclusion: Combining semantic labeling by FCN with MSER provides a natural solution for handling multi-oriented text.The authors describe combining local and global cues for text-line localization as a direction worthy of further study.
- Conclusion: The framework achieves superior performance over competing methods on horizontal and multi-oriented text-detection benchmarks.The conclusion presents this performance as verification that combining local and global cues is promising.
- Future work: Future work could extend the framework to an end-to-end text-recognition system.This is stated as a possible extension of the presented detection framework.