Source-linked AI summary
Adaptive Boundary Proposal Network for Arbitrary Shape Text Detection
Shi-Xue Zhang, Xiaobin Zhu, Chun Yang, Hongfa Wang, Xu-Cheng Yin
TL;DR
Arbitrary-shape text detection is challenging amid varied scene text, and existing segmentation approaches face difficulties with adjacent instances and noisy boundaries. This paper proposes an end-to-end network that iteratively deforms coarse boundary proposals using prior information. The authors report state-of-the-art performance on publicly available datasets.
Problem
Arbitrary-shape text detection remains challenging, while segmentation-based methods can have difficulty separating adjacent instances and producing clean contours.
Source: §1 · pp. 1–2Method
The network predicts prior maps and coarse boundaries, then uses an encoder-decoder with GCN and RNN components to iteratively deform boundaries.
Source: §1 · p. 2Results
The authors report state-of-the-art performance on several publicly available datasets.
Source: §1 · p. 2Takeaways & Limitations
The method directly generates accurate arbitrary-shape text boundaries without complex post-processing.
Source: §1 · p. 2
Abstract
from arXiv · showhide
Arbitrary shape text detection is a challenging task due to the high complexity and variety of scene texts. In this work, we propose a novel adaptive boundary proposal network for arbitrary shape text detection, which can learn to directly produce accurate boundary for arbitrary shape text without any post-processing. Our method mainly consists of a boundary proposal model and an innovative adaptive boundary deformation model. The boundary proposal model constructed by multi-layer dilated convolutions is adopted to produce prior information (including classification map, distance field, and direction field) and coarse boundary proposals. The adaptive boundary deformation model is an encoder-decoder network, in which the encoder mainly consists of a Graph Convolutional Network (GCN) and a Recurrent Neural Network (RNN). It aims to perform boundary deformation in an iterative way for obtaining text instance shape guided by prior information from the boundary proposal model. In this way, our method can directly and efficiently generate accurate text boundaries without complex post-processing. Extensive experiments on publicly available datasets demonstrate the state-of-the-art performance of our method.
1. Introduction
Arbitrary-shape text detection remains difficult because scene text varies in shape and segmentation-based methods struggle with adjacent instances and noisy contours. The paper proposes an end-to-end network that generates coarse boundaries and iteratively deforms them into accurate text boundaries without post-processing.
1. Introduction
- Segmentation-based methods can struggle to separate adjacent text instances and often require complex pixel-to-pixel merging to rebuild complete text regions.Source: §1 · p. 1
- Coarse-grained boundary annotations and open text contours can produce unpredictable boundary pixels, contributing to noisy and defective segmentation contours.Source: §1 · pp. 1–2
- The proposed network combines coarse boundary proposals with iterative adaptive deformation to generate accurate arbitrary-shape text boundaries without post-processing.Source: §1 · p. 2
- The boundary proposal model predicts classification, distance, and direction maps, while the deformation model uses a GCN and RNN to refine proposals iteratively.Source: §1 · p. 2
- Experiments on publicly available datasets are reported to demonstrate state-of-the-art performance.Source: §1 · p. 2
2. Related Work
Prior approaches to text detection use regression, connected components, segmentation, or contours, each with distinct limitations for arbitrary shapes, post-processing, or performance. The paper presents its method as an end-to-end framework with iterative optimization.
Regression-Based Methods
- Regression methods refine predefined anchors or regress boundary offsets, but often do not adapt well to arbitrary-shape text.Source: §2 · p. 2
CC-Based Methods
- Connected-component methods group detected text parts, but their grouping post-processing can be time-consuming and unsatisfactory.Source: §2 · p. 2
Segmentation-Based Methods
- Segmentation methods distinguish adjacent text with kernels or adaptive binarization, yet their performance is strongly affected by segmentation accuracy.Source: §2 · p. 2
Contour-based methods
- Contour-based methods predict text corners, boundary points, or contour points, but are reported to lag segmentation methods in detection performance and speed without recognition information.Source: §2 · p. 2
2. Related Work
- The paper's framework combines shared convolutions, a boundary proposal model, and adaptive boundary deformation in end-to-end iterative optimization.Source: Figure 2 · p. 3
3. Proposed Method
The method generates coarse text-boundary proposals from shared image features and prior maps, then iteratively deforms them using topology and sequence context. Its training objective combines proposal and deformation losses.
3.1. Overview
- Shared features produce text classification, distance, and direction maps that guide coarse boundary proposals and subsequent iterative refinement.Each proposal is represented by N points; the adaptive deformation model refines it under the guidance of the prior maps.Source: §3.1 · p. 3
3.2.1 Boundary Proposal Generation
- A distance threshold generates candidate boundaries, which are filtered by average classification confidence to remove low-confidence proposals.The direction field points each text pixel toward its nearest boundary pixel, while the distance field encodes normalized distance to that boundary.Source: §3.2.1 · pp. 3–4
3.2.2 Adaptive Boundary Deformation
- The deformation model combines GCN and RNN features to exploit boundary topology and sequence context while iteratively refining coarse text boundaries.Each control-point feature concatenates shared CNN features with classification, distance, and direction priors; the decoder predicts offsets toward the target boundary.Source: §3.2.2 · p. 5
3.3. Optimization
- Training combines boundary-proposal and adaptive-deformation losses, with the deformation-loss weight scheduled by training epoch.The proposal loss includes pixel classification, distance regression, and direction-field terms; the deformation loss matches predicted and ground-truth control points across text instances.Source: §3.3 · p. 5
4. Experiments
The experiments evaluate the network on datasets with varied text shapes and compare its design choices and benchmark performance. The reported results include strong comparisons on Total-Text, CTW-1500, and MSRA-TD500.
4.1. Datasets
- The described datasets span horizontal, multi-oriented, curved, multilingual, and long text, with polygon and word-level annotations in Total-Text, 14-vertex polygons in CTW-1500, and quadrangle annotations in ICDAR2017-MLT.The descriptions name Total-Text, CTW-1500, MSRA-TD500, SynthText, and ICDAR2017-MLT.Source: §4.1 · p. 5
4.2. Implementation Details
- Training uses SynthText pre-training followed by real-dataset fine-tuning, while inference resizes and pads images and runs on a single GPU.Pre-training uses 512 × 512 crops and fine-tuning uses 640×640 crops.Source: §4.2 · p. 6
4.3. Ablation Study
- The adaptive deformation model performs best among the compared variants on both datasets, with reported F-measure gains over RNN and GCN on Total-Text and no obvious increase in detection time.The reported gains are 0.94% over RNN and 0.85% over GCN.Source: §4.3 · p. 6, Table 1 · p. 6
- On Total-Text and CTW-1500, F-measures decline when control-point count is too high or too low; performance peaks around 20, while more iterations improve detection but reduce speed.The paper sets the control-point count to 20 and the default iteration count to 3, balancing speed and performance.Source: §4.3 · p. 6, Figure 6 · p. 6, Table 2 · p. 6
- On Total-Text, adding the distance field and direction field to the classification map improves F-measure by 5.45% and 1.55%, respectively.The ablation reports unsatisfactory performance when using only the classification map.Source: §4.3 · p. 7, Table 3 · p. 7
- Both FPN-P1 and FPN-P2 achieve state-of-the-art performance on Total-Text as shared features.FPN-P2 has one-quarter of the original image resolution.Source: §4.3 · p. 7, Table 4 · p. 7
4.4. Comparison with the state-of-the-arts
- 87.37% F-measure with SynthText pre-training and 87.85% F-measure with MLT on Total-Text; CTW-1500 reaches 87.81% precision and 85.0% F-measure, while MSRA-TD500 reaches 85.57% F-measure.The paper reports that the method outperforms prior state-of-the-art methods on these benchmarks.Source: Table 5 · p. 7, §4.4 · pp. 7–8, Table 6 · p. 8
5. Conclusion
The paper concludes that its proposal-and-deformation network can detect arbitrary-shape text boundaries accurately. The authors identify real-time arbitrary-shape text detection as future work.
5. Conclusion
- The network combines coarse boundary proposals with iterative deformation using GCN and RNN to obtain more accurate arbitrary-shape text boundaries.The authors report that experiments on challenging datasets show precise boundary detection and identify real-time development as future work.Source: §5 · p. 8