Source-linked AI summary
Deep Direct Regression for Multi-Oriented Scene Text Detection
Wenhao He, Xu-Yao Zhang, Fei Yin, Cheng-Lin Liu
TL;DR
Multi-oriented scene text challenges indirect regression because proposal and anchor generation can fail for varied text shapes and orientations. The paper proposes an end-to-end direct-regression detector with one-step post-processing, achieving state-of-the-art performance, including 81% F1-measure on ICDAR2015 Incidental Scene Text.
Problem
Indirect regression and anchor mechanisms may be ineffective for multi-oriented text because suitable proposals are difficult to generate, especially for long and heavily inclined text.
Method
The method uses a fully convolutional network with pixel-wise text classification and direct regression of quadrilateral vertices, followed by one-step Recalled Non-Maximum Suppression.
Results
81% F1-measure on ICDAR2015 Incidental Scene Text is reported as new state-of-the-art, with state-of-the-art performance also reached on focused-text datasets.
Takeaways & Limitations
Direct regression is particularly beneficial for localizing incidental scene texts with irregular quadrilateral boundaries and large scale or perspective variation.
Takeaways & Limitations
The method still fails on Chinese text lines with wide character spaces or complex backgrounds, and failed focused-text cases include single-character text and letters at either end.
Abstract
from arXiv · showhide
In this paper, we first provide a new perspective to divide existing high performance object detection methods into direct and indirect regressions. Direct regression performs boundary regression by predicting the offsets from a given point, while indirect regression predicts the offsets from some bounding box proposals. Then we analyze the drawbacks of the indirect regression, which the recent state-of-the-art detection structures like Faster-RCNN and SSD follows, for multi-oriented scene text detection, and point out the potential superiority of direct regression. To verify this point of view, we propose a deep direct regression based method for multi-oriented scene text detection. Our detection framework is simple and effective with a fully convolutional network and one-step post processing. The fully convolutional network is optimized in an end-to-end way and has bi-task outputs where one is pixel-wise classification between text and non-text, and the other is direct regression to determine the vertex coordinates of quadrilateral text boundaries. The proposed method is particularly beneficial for localizing incidental scene texts. On the ICDAR2015 Incidental Scene Text benchmark, our method achieves the F1-measure of 81%, which is a new state-of-the-art and significantly outperforms previous approaches. On other standard datasets with focused scene texts, our method also reaches the state-of-the-art performance.
1. Introduction
The paper argues that indirect regression and anchor mechanisms are poorly suited to multi-oriented scene text, then proposes direct regression with a simple end-to-end detection pipeline. The method targets difficult incidental text and achieves strong benchmark performance.
- Regression perspective: Indirect regression predicts offsets from proposals, whereas direct regression predicts object boundary or size offsets from a given point.The distinction is illustrated through proposal-based versus point-based regression.
- Motivation: Multi-oriented texts challenge proposal-based detectors because robust word- or line-level proposals are scarce and long, heavily inclined text may lack a sufficiently overlapping anchor.The anchor deficiency is specifically illustrated for long and heavily inclined text words or lines.
- Results: 81% F1-measure on ICDAR2015 Incidental Scene Text establishes a new state-of-the-art result and surpasses the second-placed method by a large margin.The paper also reports state-of-the-art performance on other popular datasets of focused images.
- Method and advantages: Direct regression is particularly beneficial for incidental texts because it predicts irregular quadrilateral boundaries for words with difficult character identification and perspective variation.The method is designed to localize four vertices directly at word level.
- Method and advantages: The proposed pipeline combines a convolutional neural network with one-step Recalled Non-Maximum Suppression, removing line grouping and word partition modules.This simplified design reduces parameter-tuning effort and supports direct localization of irregular quadrilateral boundaries.
2. Related Work
Related work largely treats text as a composition of characters or converts text detection into generic object detection. The paper positions direct regression as a way to avoid proposal-generation difficulties caused by multi-oriented text variation.
- Scene text detection: Many scene text detectors first localize character or component candidates and then group them into words or text lines.Multi-oriented methods likewise use rule-based or graphical-model line grouping.
- Scene text detection: Some approaches detect text words or lines as whole objects, but rely on specialized features, text sequentiality, or generic object-detection frameworks.The cited examples include symmetric features, sequentiality-based post-processing, Faster-RCNN, and SSD.
- Generic object detection: Generic object detectors are divided into direct regression methods that predict object size and localization directly and indirect methods that regress offsets from proposals to ground truths.The paper applies this distinction to frameworks including Fast-RCNN, Faster-RCNN, Multi-Box, and SSD.
- Paper positioning: Direct regression might avoid proposal-generation difficulty for texts spanning wide ranges of scale, orientation, perspective distortion, and aspect ratio.The paper identifies this potential advantage as its main contribution.
3. Proposed Methodology
The proposed system combines a fully convolutional multi-task network with one-step Recalled NMS post-processing to detect multi-oriented text through pixel classification and direct quadrilateral regression.
- System Overview: The detection system combines convolutional feature extraction, multi-level feature fusion, multi-task learning, and recalled NMS.The first three modules form the network, while recalled NMS is the post-processing stage.
- Network Architecture: The network uses a receptive field larger than the input image to improve boundary prediction for long texts while reducing computation through fewer parameters.The feature fusion module captures texts at multiple scales and upsamples the fused feature to quarter input resolution.
- Multi-Task Outputs: The classification branch estimates text likelihood, while the regression branch outputs eight coordinates representing quadrilateral text vertices.Each output-map point is associated with a scored quadrilateral whose vertex offsets are predicted directly from that point.
- Ground Truth and Loss: The full loss combines classification and localization losses, with their balance controlled by λloc; smooth L1 is used for regression because it is less sensitive to outliers than L2.Class balancing and hard negative mining are also adopted for classification.
- Ground Truth and Loss: The classification ground truth labels center-line pixels as positive, surrounds them with NOT CARE regions, and excludes those regions from the training objective.This design reduces confusion between text and non-text and supports discriminative feature learning.
- Recalled Non-Maximum Suppression: Recalled NMS filters dense quadrilateral predictions by recalling high-score candidates and merging nearby results instead of only suppressing overlaps.It addresses retained quadrilaterals between close words while preserving relatively low-confidence text regions.
4. Experiments
The experiments evaluate the method on three benchmarks spanning incidental, multi-oriented, bilingual, and mostly horizontal scene text. Results report strong performance, including state-of-the-art outcomes, while examples expose specific failure cases.
- Benchmark setup: The method is evaluated on ICDAR2015 Incidental Scene Text, MSRA-TD500, and ICDAR2013, covering multi-oriented and mostly horizontal text.ICDAR2015 contains varied scales, resolution, blur, orientations, and viewpoints; MSRA-TD500 contains English and Chinese text; ICDAR2013 emphasizes horizontal text.
- ICDAR2015 Incidental Scene Text: Recalled NMS gives higher precision than traditional NMS on ICDAR2015 by filtering quadrilaterals between text lines.The method is evaluated with both recalled and traditional NMS in Table 1.
- MSRA-TD500: The method reaches state-of-the-art performance on MSRA-TD500 despite using English word-level training data rather than the provided training set or additional Chinese data.It shows compatibility with English and Chinese, but misses Chinese lines with wide character spacing or complex backgrounds.
- ICDAR2013 Focused Scene Text: The method achieves state-of-the-art performance on ICDAR2013 and is competitively fast, with failures mainly involving single characters and missing end characters.Examples also show word-level detection in cluttered scenes.
- Rationality of high performance: Direct regression avoids generating proposals for irregular multi-oriented text, while end-to-end localization replaces rule-based line grouping and word partitioning.The method uses a convolutional network with multi-task learning and one-step recalled NMS post-processing.
5. Conclusion
The paper frames object detection through direct and indirect regression, then proposes a direct-regression method for multi-oriented scene text detection. The method uses one-step post-processing, performs especially well on incidental text, and achieves state-of-the-art results across the reported benchmarks.
- Conclusion: The paper distinguishes direct and indirect regression and analyzes their advantages and disadvantages for irregularly shaped object detection.It then proposes a direct regression method for multi-oriented scene text detection.
- Conclusion: The detection framework is straightforward and effective with only one-step post-processing.The paper identifies a simpler detection pipeline as a central characteristic of the method.
- Conclusion: The method performs particularly well for incidental text detection and achieves new state-of-the-art performance on ICDAR2015 Incidental Scene Text.The paper reports that it outperformed previous methods by a large margin.
- Conclusion: The paper also analyzes the reasons for the high performance and compares the method with recent scene text detection systems.Future work concerns more robust and faster detection structures and further theoretical research on regression.