Source-linked AI summary
Look More Than Once: An Accurate Detector for Text of Arbitrary Shapes
Chengquan Zhang, Borong Liang, Zuming Huang, Mengyi En, Junyu Han, Errui Ding, Xinghao Ding
TL;DR
Existing detectors struggle with extremely long and arbitrarily shaped text because CNN receptive fields and simple geometric representations are limited. LOMO progressively refines direct-regression proposals and reconstructs irregular shapes through geometry-aware expression. It achieves state-of-the-art results across benchmarks covering long, curved, oriented, and multilingual text.
Problem
Limited CNN receptive fields and simple rectangle or quadrangle representations hinder detection of extremely long and arbitrarily shaped text.
Method
LOMO combines direct regression for preliminary quadrangle proposals, iterative refinement for long text, and a shape expression module using text-region, center-line, and border-offset geometry.
Results
LOMO achieves state-of-the-art performance across benchmarks containing long, curved, oriented, and multilingual text instances.
Takeaways & Limitations
The end-to-end LOMO architecture provides a detector for text instances with diverse lengths, shapes, orientations, and languages.
Abstract
from arXiv · showhide
Previous scene text detection methods have progressed substantially over the past years. However, limited by the receptive field of CNNs and the simple representations like rectangle bounding box or quadrangle adopted to describe text, previous methods may fall short when dealing with more challenging text instances, such as extremely long text and arbitrarily shaped text. To address these two problems, we present a novel text detector namely LOMO, which localizes the text progressively for multiple times (or in other word, LOok More than Once). LOMO consists of a direct regressor (DR), an iterative refinement module (IRM) and a shape expression module (SEM). At first, text proposals in the form of quadrangle are generated by DR branch. Next, IRM progressively perceives the entire long text by iterative refinement based on the extracted feature blocks of preliminary proposals. Finally, a SEM is introduced to reconstruct more precise representation of irregular text by considering the geometry properties of text instance, including text region, text center line and border offsets. The state-of-the-art results on several public benchmarks including ICDAR2017-RCTW, SCUT-CTW1500, Total-Text, ICDAR2015 and ICDAR17-MLT confirm the striking robustness and effectiveness of LOMO.
1. Introduction
LOMO addresses two scene-text detection challenges: incomplete localization of extremely long text and imprecise representations for curved or wavy text. It combines direct regression, iterative refinement, and shape expression modules in an end-to-end detector.
- Challenges: Extremely long text can exceed CNN receptive fields, preventing one-shot detectors from producing complete bounding-box proposals.The limitation arises because CNNs cannot encode sufficient information for long-distance dependencies.
- Challenges: Rectangles, rotated rectangles, and quadrangles may cover substantial non-text background when representing curved or wavy text.This imprecision can adversely affect subsequent text recognition in an OCR system.
- Approach: LOMO uses a direct regressor to generate preliminary quadrangle proposals, then iteratively refines them toward the ground truth for long text.The architecture is trained end to end with the direct regressor and iterative refinement module.
- Approach: Its shape expression module addresses arbitrary shapes by reconstructing text geometry through a more flexible representation.The proposed modules target long, arbitrary-shape, oriented, multilingual, and curved text instances.
2. Related Work
Scene-text detectors are commonly grouped into component-based, detection-based, and segmentation-based approaches. LOMO combines detection-based localization with segmentation-inspired handling of arbitrary shapes.
- Taxonomy: Scene-text detectors are broadly classified as component-based, detection-based, or segmentation-based according to the text elements they process.These categories reflect whether methods handle text parts, word-level regions, or pixels within text boxes.
- Component-based methods: Component-based methods detect text parts or characters and group them into words using post-processing.Examples include dense text components, text segments with links, and weakly supervised character detectors.
- Detection-based methods: Detection-based methods use object-detection frameworks or directly regress word quadrangles under word- or line-level supervision.Direct regression methods such as EAST and Deep Regression avoid anchors and proposals through end-to-end designs.
- Segmentation-based methods: Segmentation-based methods label text pixels and can extract arbitrary-shaped text, but proposal-free instance segmentation depends strongly on segmentation robustness.Text center-line maps are used by methods such as TextSnake and PSENet to separate instances.
- LOMO: LOMO integrates advantages of detection-based and segmentation-based methods through iterative refinement and a shape expression module.Its architecture uses IRM for long text and SEM, inspired by Mask R-CNN, for arbitrary-shape text.
3. Approach
LOMO combines direct regression, iterative refinement, and shape expression to detect long and irregular text through progressively refined quadrangle proposals and geometry-based polygon reconstruction.
- Framework: LOMO uses a shared backbone with separate DR, IRM, and SEM branches trained in an end-to-end architecture.The backbone is ResNet50 with FPN, producing shared feature maps at one-quarter of the input resolution.
- Direct Regressor: DR predicts pixel-wise text confidence and eight offsets from each positive pixel to the four corners of a text quadrangle.Its loss combines text/non-text classification with location regression, using smooth L1 loss for the latter.
- Iterative Refinement Module: IRM repeatedly refines preliminary quadrangles toward ground-truth boxes using RoI-transformed feature blocks and learned attention for each corner.Corner-specific attention maps weight contextual features before four regression headers predict corner offsets; refinement may run once or multiple times.
- Shape Expression Module: SEM reconstructs irregular text shapes by regressing text-region, text-center-line, and upper/lower border-offset maps from proposal features.These geometry properties provide a tighter representation than quadrangles for curved or wavy text.
- Shape Expression Module: SEM generates polygons by sampling center-line points, producing corresponding border points from offsets, and scoring the resulting polygon.It samples seven points for SCUT-CTW1500 curved-text experiments and two points for quadrangle-annotated benchmarks.
- Training and Inference: Training warms up DR on synthetic data before fine-tuning all three branches on real datasets, while inference runs DR, NMS, repeated IRM refinement, SEM reconstruction, and thresholding.The reported training schedule uses 10 warm-up epochs and approximately 10 fine-tuning epochs; inference uses a confidence threshold of 0.1.
4. Experiments
LOMO is evaluated across five public scene-text benchmarks covering long, curved, oriented, and multilingual text, with ablations isolating IRM, CAM, SEM, and center-line sampling. It achieves strong benchmark results, including state-of-the-art performance on several datasets and substantial gains for long and curved text.
- Ablation Study: IRM increases ICDAR2017-RCTW Hmean by 3.34%, 3.66%, and 3.77% with one, two, and three refinement times, respectively, over DR without IRM.The authors select two refinement times for the remaining experiments to preserve fast inference.
- Ablation Study: Removing the corner attention map reduces Hmean by 0.3% at IoU@0.5 and 1.5% at IoU@0.7 on ICDAR2017-RCTW.The ablation supports the role of corner-enhanced features in long-text detection.
- Ablation Study: SEM improves SCUT-CTW1500 Hmean by 7.17% without IRM and 6.34% with IRM.The authors report that SEM largely resolves the curved-text detection challenge.
- Ablation Study: On SCUT-CTW1500, Hmean rises from 62% to 78% as center-line sample points increase from 2 to 16, then converges; n=7 is used thereafter.With seven center-line samples, SEM generates polygons with 14 vertices.
- Benchmark Evaluation: LOMO achieves 62.3% Hmean on ICDAR2017-RCTW and 68.4% with multi-scale testing, while attaining state-of-the-art results on curved-text and multilingual benchmarks.It also reaches 87.2% Hmean on ICDAR 2015 and 73.1% on ICDAR2017-MLT with multi-scale testing.
5. Conclusion and Future Work
LOMO addresses extremely long and curved text detection with three modules and is trained end-to-end. Its robustness and effectiveness are reported across benchmarks covering long, curved or wavy, oriented, and multilingual text.
- LOMO uses DR for preliminary text proposals, IRM for iterative refinement, and SEM for flexible arbitrary-shape representation.The architecture is trained end-to-end.
- The method targets extremely long and curved text detection, with IRM refining long-text proposals iteratively.
- LOMO is evaluated on public benchmarks containing long, curved or wavy, oriented, and multilingual text cases.The paper reports that these evaluations demonstrate the approach’s robustness and effectiveness.