Source-linked AI summary
Text Flow: A Unified Text Detection System in Natural Scene Images
Shangxuan Tian, Yifeng Pan, Chang Huang, Shijian Lu, Kai Yu, Chew Lim Tan
TL;DR
Sequential scene-text detection can accumulate errors across character and text-line processing, especially when connected components produce false alarms or split non-Latin characters. Text Flow combines cascade-boosted character detection with min-cost flow extraction, outperforming prior methods across three datasets, including multilingual text.
Problem
Sequential detection pipelines propagate errors across character detection, false-alarm removal, text-line extraction, and verification, while connected components can split many non-Latin characters.
Method
Text Flow uses cascade boosting to detect whole character candidates and a min-cost flow network to jointly integrate false-candidate removal, text-line extraction, and verification.
Results
80.25% F-score versus 75.89% for the ICDAR Robust Reading Competition 2013 winner, with up to 10% higher detection recall and 7% higher F-score on the multilingual dataset.
Takeaways & Limitations
The system detects scene text across ICDAR2011, ICDAR2013, and multilingual datasets, supporting detection of texts in different languages.
Abstract
from arXiv · showhide
The prevalent scene text detection approach follows four sequential steps comprising character candidate detection, false character candidate removal, text line extraction, and text line verification. However, errors occur and accumulate throughout each of these sequential steps which often lead to low detection performance. To address these issues, we propose a unified scene text detection system, namely Text Flow, by utilizing the minimum cost (min-cost) flow network model. With character candidates detected by cascade boosting, the min-cost flow network model integrates the last three sequential steps into a single process which solves the error accumulation problem at both character level and text line level effectively. The proposed technique has been tested on three public datasets, i.e, ICDAR2011 dataset, ICDAR2013 dataset and a multilingual dataset and it outperforms the state-of-the-art methods on all three datasets with much higher recall and F-score. The good performance on the multilingual dataset shows that the proposed technique can be used for the detection of texts in different languages.
1. Introduction
Text Flow addresses low recall, language limitations, and error accumulation in sequential scene text detection by combining whole-character detection with unified text-line extraction. It is evaluated across English, Chinese, and multilingual datasets with strong reported performance.
- Connected-component candidate detection admits many false positives and struggles with Chinese and Japanese characters composed of multiple components.
- Sequential detection accumulates errors across character detection, false-candidate removal, text-line extraction, and verification, often reducing recall.
- Cascade boosting detects whole character candidates, avoiding the need to group isolated strokes and supporting non-Latin scripts such as Chinese.
- The min-cost flow model integrates false-candidate removal, text-line extraction, and verification using character confidence and neighboring-candidate smoothness.
- The unified model jointly reduces character- and line-level noise while retaining high recall and remaining simple and efficient to solve.
- The technique is evaluated on ICDAR2011, ICDAR2013, and multilingual datasets containing English and Chinese text.
2. Related Work
Prior scene text detectors typically process candidate detection, false-alarm removal, line extraction, and verification sequentially, creating propagation and adaptability problems. Text Flow instead jointly optimizes these stages with line-level context for Latin and non-Latin scripts.
- Most existing scene text detection systems use four sequential stages: candidate detection, false-candidate removal, text-line extraction, and verification.
- Connected-component methods use bottom-up cues but cannot reliably process characters composed of multiple components.
- Sliding-window methods detect multi-component characters as wholes and use high-level texture and shape information, but processing many windows is computationally expensive.
- False-alarm classifiers can make unreliable hard decisions when they lack text-line-level context.
- Sequential-stage errors propagate and motivate an integrated model, which prior work had not proposed for scene text detection.
- Text Flow jointly optimizes the stages, removes false alarms with line-level context, and addresses both Latin and non-Latin scripts.
3. Our Proposed System
The proposed system detects whole character candidates with cascade boosting and uses a min-cost flow network to jointly extract and verify text lines while removing false candidates. The network combines character confidence with spatial and geometric relationships between neighboring candidates.
- 3.2. Text Line Extraction: Text Flow integrates false candidate removal, text line extraction, and text line verification into one min-cost flow process.Detected character candidates become network inputs, while network flows correspond to extracted text lines.
- 3.1. Character Candidate Detection: The character detector combines sliding windows with cascade boosting and uses six simple intensity and gradient features for efficient processing.The approach also uses speedup strategies including integral feature maps, SIMD extensions, and multithreaded window processing.
- 3.1. Character Candidate Detection: Cascade boosting detects whole character candidates, avoiding the need to group isolated character strokes into complete characters.This supports detection of characters composed of multiple connected components, including Chinese characters.
- 3.1. Character Candidate Detection: The detector achieves 23.1% precision and 89.2% recall under an intersection-over-union threshold of 0.5.A candidate is positive when its overlap with the ground-truth character box exceeds 0.5.
- 3.2.2. Flow Network Costs: The network assigns unary data costs from character confidence and pairwise smoothness costs from the likelihood that neighboring candidates belong to the same text line.The data cost uses a text/non-text CNN confidence, while smoothness uses candidate size and normalized distance.
- 3.2.1. Network Construction: Neighboring candidates are connected only when horizontal distance, vertical distance, and size similarity satisfy the model’s transition constraints.The default thresholds are TH = 2, TV = 0.6, and TS = 0.2, with relaxation possible for text lines that violate these constraints.
4. Experiments
Experiments evaluate Text Flow on two ICDAR datasets and a multilingual English–Chinese dataset, using dataset-specific metrics and comparisons with prior methods. The method achieves strong detection performance across datasets, including higher F-score than the ICDAR2013 competition winner and improved multilingual recall and F-score.
- 4.1. Data and Evaluation Metric: Text Flow is evaluated on ICDAR2011, ICDAR2013, and a multilingual dataset containing English and Chinese text.The datasets use different evaluation procedures, including one-to-one matching for the multilingual dataset and many-to-one and one-to-many matching for the ICDAR datasets.
- 4.1. Data and Evaluation Metric: The training procedure uses dataset-specific cascade boosting and CNN models, with synthetic augmentation producing roughly 600,000 positive and negative samples.Synthetic samples are created through rotation, shifting, blurring, and Gaussian noise.
- 4.2. Experimental Results: 80.25% F-score is obtained on ICDAR2013, exceeding the ICDAR Robust Reading Competition 2013 winner’s 75.89%.The authors attribute this performance to reduced error accumulation from the min-cost flow model.
- 4.2. Experimental Results: On the multilingual dataset, Text Flow exceeds the best prior method by up to 10% in detection recall and 7% in F-score.The comparison is made against the best-performing method reported in Table 3.
- 4.2. Experimental Results: Multilingual recall is 79.1% for Chinese text lines and 76.6% for English text lines, although precision cannot be computed separately by language.The test set contains 669 Chinese and 282 English text lines.
- 4.3. Discussion: The system processes ICDAR2011 images in 1.4 seconds each, while reported MSER-based methods require 0.43 and 1.8 seconds per image.The authors also report that the system is faster than a hybrid multilingual method and comparable with an MSER-based method.
5. Conclusion
Text Flow combines cascade-boosted character candidate detection with min-cost-flow text-line extraction. Across ICDAR2011, ICDAR2013, and multilingual datasets, it greatly outperforms state-of-the-art techniques and supports non-Latin text detection at competitive speed.
- Text Flow uses cascade boosting for whole-character candidate detection and a min-cost flow network for text-line extraction.The system has two steps and integrates the remaining sequential processing into the flow-based extraction stage.
- The min-cost flow model integrates three sequential steps into one process to address error accumulation.It combines character confidence with text layout information during text-line extraction.
- Experiments on ICDAR2011, ICDAR2013, and multilingual datasets show substantially better performance than state-of-the-art techniques.
- Text Flow detects non-Latin texts with competitive speed compared with connected-component-based methods.