Source-linked AI summary
TextSnake: A Flexible Representation for Detecting Text of Arbitrary Shapes
Shangbang Long, Jiaqiang Ruan, Wenjie Zhang, Xin He, Wenhao Wu, Cong Yao
TL;DR
Existing scene-text detectors use simple geometric representations that can struggle with irregular shapes such as curved text. TextSnake represents text as ordered, overlapping disks with variable geometry estimated by an FCN, achieving state-of-the-art or comparable performance across benchmarks covering horizontal, oriented, and curved text.
Problem
Existing text-detection methods commonly represent instances with rectangles or quadrangles, which may provide imprecise geometry for irregular shapes such as curved text.
Method
TextSnake represents each text instance as ordered, overlapping disks centered along its axis, with variable radii and orientations estimated by a single Fully Convolutional Network.
Results
The detector achieves state-of-the-art or comparable performance on Total-Text, SCUT-CTW1500, ICDAR 2015, and MSRA-TD500, including 82.7% precision, 74.5% recall, and 78.4% F-measure on Total-Text.
Takeaways & Limitations
The flexible representation supports detection of horizontal, multi-oriented, and curved text instances while retaining precise information about their shape and course.
Takeaways & Limitations
The method assumes text instances are snake-shaped and do not fork into multiple branches.
Abstract
from arXiv · showhide
Driven by deep neural networks and large scale datasets, scene text detection methods have progressed substantially over the past years, continuously refreshing the performance records on various standard benchmarks. However, limited by the representations (axis-aligned rectangles, rotated rectangles or quadrangles) adopted to describe text, existing methods may fall short when dealing with much more free-form text instances, such as curved text, which are actually very common in real-world scenarios. To tackle this problem, we propose a more flexible representation for scene text, termed as TextSnake, which is able to effectively represent text instances in horizontal, oriented and curved forms. In TextSnake, a text instance is described as a sequence of ordered, overlapping disks centered at symmetric axes, each of which is associated with potentially variable radius and orientation. Such geometry attributes are estimated via a Fully Convolutional Network (FCN) model. In experiments, the text detector based on TextSnake achieves state-of-the-art or comparable performance on Total-Text and SCUT-CTW1500, the two newly published benchmarks with special emphasis on curved text in natural images, as well as the widely-used datasets ICDAR 2015 and MSRA-TD500. Specifically, TextSnake outperforms the baseline on Total-Text by more than 40% in F-measure.
1 Introduction
Scene text detection has advanced with deep learning, but conventional geometric representations remain inadequate for common curved and irregular text. TextSnake addresses this gap with a flexible representation for text of arbitrary shapes and strong benchmark performance across horizontal, oriented, and curved forms.
- Deep neural networks and large image datasets have substantially improved scene text detection on standard benchmarks.
- Most existing methods assume roughly linear text and use rectangles or quadrangles that struggle to describe curved text with perspective distortion precisely.
- TextSnake represents text as ordered, overlapping disks centered on the text axis, with variable radius and orientation.
- A single FCN estimates TextSnake’s central-axis points, radii, and orientations for horizontal, multi-oriented, and curved text.
- TextSnake achieves state-of-the-art performance on curved-text benchmarks while outperforming previous methods on horizontal and multi-oriented text.
2 Related Work
Prior scene text detectors largely use regression or segmentation frameworks, while relatively few methods specifically address curved text. TextSnake extends FCN-based geometric prediction with an arbitrary-shape representation and supports precise shape reconstruction for later recognition.
- Modern scene text detection methods are broadly classified as regression-based or segmentation-based deep neural network approaches.
- Regression methods adapt object-detection frameworks to predict text boxes, rotated boxes, or quadrangles per pixel.
- Segmentation methods formulate detection as semantic segmentation and vary in their predicted text properties or strategies for separating adjacent instances.
- Most reviewed methods do not specifically address curved text, whereas TextSnake targets horizontal, multi-oriented, and curved forms without requiring character-level annotations.
- TextSnake’s precise shape and regional-strike reconstruction can facilitate recognition by enabling canonical transformation with minimal distortion and background.
3 Methodology
TextSnake represents arbitrary-shaped text as ordered, overlapping disks along a center line, with geometry predicted by an FCN and reconstructed through post-processing. Its pipeline extracts text instances and their shape using center-line and region maps plus radius and orientation estimates.
- 3.1 Representation: TextSnake represents each text instance as ordered, overlapping disks centered on its symmetric axis, with variable radius and orientation.The disk union reconstructs the text region, while the representation adapts to rotation, scaling, bending, and varying lengths.
- 3.1 Representation: The representation does not assign disks to individual characters, but its geometry can rectify irregular text into straight regions for recognition.Rectification transforms irregular instances into rectangular image regions that are more suitable for text recognizers.
- 3.2 Pipeline: An FCN predicts text-region and text-center-line maps together with radius, cosθ, and sinθ geometry attributes.The network produces dense predictions at the input-image resolution, with separate classification and geometry channels.
- 3.3 Network Architecture: The network merges multilevel VGG-16 features through sequential merging units, upsampling and convolution to produce dense predictions.The architecture removes VGG-16 fully connected layers and combines feature maps from its convolution stages.
- 3.4 Inference: Inference thresholds the TR and TCL maps, intersects them, separates TCL pixels with disjoint sets, and reconstructs instances from ordered center-axis points.The striding procedure searches in opposite directions from a centralized starting point, while sliding draws circles using predicted radii.
- 3.5 Label Generation: Text-center-line labels for polygonal text are computed by matching sampled points on opposite sidelines, taking corresponding midpoints, and shrinking the ends.The label-generation procedure assumes snake-shaped instances without multiple branches and identifies head and tail edges geometrically.
4 Experiments
Experiments evaluate TextSnake across curved, incidental, and long straight text benchmarks, where it achieves strong performance and precisely describes text shape and course.
- Curved Text: 82.7% precision, 74.5% recall, and 78.4% F-measure are achieved on Total-Text, significantly outperforming previous methods.The F-measure is more than double the baseline reported in the original Total-Text paper.
- Curved Text: 75.6% F-measure is achieved on CTW1500, 2.2% higher than CTD+TLOC’s 73.4%.The corresponding precision and recall are 67.9% and 85.3%.
- Incidental Scene Text: On ICDAR 2015, single-scale testing outperforms most competitors, including methods evaluated with multi-scale testing.The authors use this result to characterize TextSnake as general and applicable to multi-oriented text in complex scenarios.
- Long Straight Text Lines: 78.3% F-measure on MSRA-TD500 is higher than the scores of the other methods.MSRA-TD500 evaluates multi-lingual, arbitrary-oriented, long text lines.
- Analyses and Discussions: TextSnake predicts precise text shape and course using text center lines, geometry attributes, and a striding algorithm for ordered central-axis points and reconstructed text areas.The TCL mechanism provides a slim representation of the text course, while local geometries support shape reconstruction.
- Analyses and Discussions: Without fine-tuning on curved text, TextSnake still performs well on curved benchmarks and significantly outperforms SegLink, EAST, and PixelLink.The authors attribute this generalization to integrating local representation elements while retaining their shape and course information.
5 Conclusion and Future Work
The paper introduces a flexible representation and detector for arbitrary-shaped scene text. It reports state-of-the-art or comparable performance across curved, horizontal, and multi-oriented benchmarks, with future work targeting end-to-end recognition.
- Conclusion: The paper presents a flexible representation for horizontal, multi-oriented, and curved scene text instances.The representation is designed to describe scene text with arbitrary shapes.
- Conclusion: The resulting detector achieves state-of-the-art or comparable performance on Total-Text, SCUT-CTW1500, ICDAR 2015, and MSRA-TD500.These benchmarks include curved-text and widely used scene-text datasets.
- Future Work: Future work will explore an end-to-end recognition system for text of arbitrary shapes.