PaperlayerView exact revision on arXiv

Source-linked AI summary

Multi-Oriented Text Detection with Fully Convolutional Networks

Zheng Zhang, Chengquan Zhang, Wei Shen, Cong Yao, Wenyu Liu, Xiang Bai

arXiv:1604.04018v2cs.CV

TL;DR

Multi-oriented scene-text detection is difficult because existing methods largely focus on horizontal text and character detectors struggle with complex backgrounds and character variation. The paper proposes a coarse-to-fine FCN framework that combines holistic text-block saliency, local character components, orientation estimation, and character-centroid filtering. It reports state-of-the-art performance across horizontal and multi-oriented benchmarks, while remaining short of real-time speed and perfect accuracy.

  • Problem

    Multi-oriented scene-text detection remains challenging because horizontal-text methods leave an accuracy gap, while character detectors struggle with complex backgrounds and varied characters.

    Source: §1 · p. 1, §3.1 · p. 3

  • Method

    A coarse-to-fine framework combines an FCN text-region salient map, character components, component-based orientation estimation, text-line candidate construction, and a second FCN for centroid-based false-candidate removal.

    Source: §1 · pp. 1–2

  • Results

    The method achieves state-of-the-art performance on horizontal and multi-oriented scene-text detection benchmarks, including MSRA-TD500 precision 0.83, recall 0.67, and f-measure 0.74.

    Source: §1 · p. 2, §4.3 · pp. 7–8

  • Takeaways & Limitations

    Combining semantic text-block labeling with local MSER components provides a supported direction for multi-oriented text-line localization.

    Source: §5 · p. 8

    Failure cases include low contrast, curvature, strong reflected light, closely spaced text lines, and large gaps between characters, and the method remains far from real-time speed.

    Source: §4.5 · p. 8

Abstract

from arXiv · show

In this paper, we propose a novel approach for text detec- tion in natural images. Both local and global cues are taken into account for localizing text lines in a coarse-to-fine pro- cedure. First, a Fully Convolutional Network (FCN) model is trained to predict the salient map of text regions in a holistic manner. Then, text line hypotheses are estimated by combining the salient map and character components. Fi- nally, another FCN classifier is used to predict the centroid of each character, in order to remove the false hypotheses. The framework is general for handling text in multiple ori- entations, languages and fonts. The proposed method con- sistently achieves the state-of-the-art performance on three text detection benchmarks: MSRA-TD500, ICDAR2015 and ICDAR2013.

1. Introduction

Multi-oriented text detection remains difficult because most existing methods target horizontal or near-horizontal text. The proposed coarse-to-fine framework combines global text-block cues with local character components and FCN-based filtering.

  • Arbitrary text-line orientations enlarge the search space, leaving a substantial accuracy gap for existing horizontal-text methods.Source: §1 · p. 1
  • The framework first predicts a holistic pixel-wise text/non-text salient map with an FCN to guide text-block localization.Source: §1 · p. 1
  • The Text-Block FCN fuses feature maps across stages so lower levels capture local structures while higher levels capture global information.Source: Figure 2 · p. 2, Figure 3 · p. 2
  • Text-line candidates are generated by extracting character components, estimating orientation through component projections, and combining local components with global text blocks.Source: §1 · p. 2
  • A second FCN predicts character centroids within candidates, using their positions to remove false text-line hypotheses.Source: §1 · p. 2
  • The framework is designed to handle scale, orientation, script, language, and font variation while detecting both horizontal and multi-oriented scene text.Source: §1 · p. 2

2. Related Work

Earlier scene-text detection mainly focused on horizontal or near-horizontal text, using word boxes, end-to-end recognition, or connected-component pipelines. Multi-oriented methods extended component extraction with orientation estimation and grouping, while CNN-based approaches improved representations but remained focused on horizontal text.

  • Most established scene-text methods detect horizontal or near-horizontal word boxes or combine detection with recognition end to end.Source: §2 · p. 2
  • Early multi-oriented systems used connected-component extraction together with character and text-line orientation estimation.Source: §2 · p. 3
  • Graph partitioning and multi-stage clustering were applied to group MSER components for multi-oriented text detection.Source: §2 · p. 3
  • Deep convolutional methods learned robust component representations, reduced false positives, or supported recognition, but the cited approaches focused on horizontal text.Source: §2 · p. 3

3. Proposed Methodology

The proposed methodology combines global text-block cues with local character components in a coarse-to-fine pipeline for multi-oriented text detection. A second FCN predicts character centroids to eliminate false text-line candidates.

3. Proposed Methodology

  • The pipeline detects text blocks with Text-Block FCN, extracts MSER character components, estimates orientation, constructs text-line candidates, and removes false candidates using character centroids.The framework combines block-level and component-level information across successive stages.Source: §3 · p. 3, §3.3 · p. 5

3.1. Text Block Detection

  • Text-Block FCN uses local and global context for efficient pixel-wise text-region labeling and coarse text-block localization.FCN supports end-to-end pixel labeling while combining context from different feature-map stages.Source: §3.1 · pp. 3–4
  • During training, pixels inside each annotated text-line or word bounding box are labeled positive to incorporate character gaps and global text structure.The model is trained with cross-entropy loss and stochastic gradient descent.Source: §3.1 · p. 3

3.2. Multi-Oriented Text Line Candidate Generation

  • The method estimates text-line orientation by counting character components along candidate orientations and selecting the direction with the maximum count.The component-count function Φ(θ, h) represents the number of components at orientation θ and vertical offset h.Source: §3.2 · p. 4, Figure 6 · p. 4
  • Character components are grouped according to height, pair orientation, and estimated block orientation, after which minimum bounding boxes form text-line candidates.Repeating this procedure for each text block extracts all candidates while using both block-level and component-level cues.Source: §3.2 · pp. 4–5
  • The candidate-generation process does not require MSER to detect every character because text-block guidance preserves candidate generation when components are missed or partially detected.This combines the coarse global localization with incomplete local character evidence.Source: §3.2 · pp. 4–5, Figure 7 · p. 5

4. Experiments

Experiments evaluate the method on three text-detection benchmarks, showing strong performance across horizontal and multi-oriented text while revealing failure cases and non-real-time speed.

Benchmark results

  • The method achieves precision 0.83, recall 0.67 and f-measure 0.74 on MSRA-TD500, improving f-measure by 0.03 over.It takes 2.1s per image on average with GPU acceleration.Source: §4.3 · p. 7
  • The method achieves the best F-measure among collected competition results on ICDAR2015.ICDAR2015 contains incidental text appearing at arbitrary orientations and locations, including small or low-resolution text.Source: §4.3 · p. 7, §4.1 · p. 7
  • The method achieves precision 0.88, recall 0.78 and f-measure 0.83 on ICDAR2013, outperforming recent methods designed only for horizontal text.ICDAR2013 is a horizontal text database with 229 training and 233 testing images.Source: §4.3 · pp. 7–8, §4.1 · p. 7

Qualitative analysis

  • Detection examples cover inner texture, non-uniform illumination, dot fonts, broken strokes, multiple orientations, perspective distortion and mixed languages.These examples are shown on MSRA-TD500 and ICDAR2013.Source: Figure 10 · p. 6, §4.3 · p. 8

Parameter analysis

  • Recall of text-line candidates is insensitive to T1 and T2 across a large range on MSRA-TD500.T1 and T2 are used to extract MSER components for computing text-line candidates.Source: §4.4 · p. 8, Figure 12 · p. 8

Limitations

  • False positives and missing characters occur under extremely low contrast, curvature, strong reflected light, closely spaced lines or large gaps between characters.The method remains far from the real-time speed requirement.Source: Figure 11 · p. 6, §4.5 · p. 8

5. Conclusion

The paper concludes that combining FCN semantic labeling with MSER offers a promising framework for multi-oriented scene text detection. It identifies end-to-end text recognition as a future extension.

Conclusion

  • Combining semantic labeling by FCN with MSER provides a natural solution for handling multi-oriented text.The authors describe combining local and global cues for text-line localization as a direction worthy of further study.Source: §5 · p. 8
  • The framework achieves superior performance over competing methods on horizontal and multi-oriented text-detection benchmarks.The conclusion presents this performance as verification that combining local and global cues is promising.Source: §5 · p. 8

Future work

  • Future work could extend the framework to an end-to-end text-recognition system.This is stated as a possible extension of the presented detection framework.Source: §5 · p. 8

Related papers

Loading 1604.04018v2…