Source-linked AI summary
Detecting Curve Text in the Wild: New Dataset and New Solution
Liu Yuliang, Jin Lianwen, Zhang Shuaitao, Zhang Sheng
TL;DR
Existing scene-text datasets and detectors provide limited support for common curve text and often localize it inadequately. The paper introduces CTW1500 and a direct polygon-based CTD with TLOC and post-processing methods. On CTW1500, the light-backbone method outperforms state-of-the-art methods by a large margin and remains best on curve and non-curve subsets, although it may be slightly slower than rigid rectangle-based detectors.
Problem
Current datasets contain very little curve text, while existing methods perform poorly or cannot directly detect strongly curved text that is common in real-world scenes.
Method
The paper constructs CTW1500 and proposes polygon-based CTD with TLOC, long-side interpolation, and NPS and PNMS post-processing.
Results
CTD with a light reduced ResNet-50 outperforms state-of-the-art methods by a large margin, and CTD+TLOC achieves the best results on curve and non-curve subsets.
Takeaways & Limitations
CTD provides a direct curve-text detector that can also be trained with rectangular or quadrilateral bounding boxes without additional manual efforts.
Takeaways & Limitations
The flexible detector may be slightly slower than a rigid rectangle-based detector.
Abstract
from arXiv · showhide
Scene text detection has been made great progress in recent years. The detection manners are evolving from axis-aligned rectangle to rotated rectangle and further to quadrangle. However, current datasets contain very little curve text, which can be widely observed in scene images such as signboard, product name and so on. To raise the concerns of reading curve text in the wild, in this paper, we construct a curve text dataset named CTW1500, which includes over 10k text annotations in 1,500 images (1000 for training and 500 for testing). Based on this dataset, we pioneering propose a polygon based curve text detector (CTD) which can directly detect curve text without empirical combination. Moreover, by seamlessly integrating the recurrent transverse and longitudinal offset connection (TLOC), the proposed method can be end-to-end trainable to learn the inherent connection among the position offsets. This allows the CTD to explore context information instead of predicting points independently, resulting in more smooth and accurate detection. We also propose two simple but effective post-processing methods named non-polygon suppress (NPS) and polygonal non-maximum suppression (PNMS) to further improve the detection accuracy. Furthermore, the proposed approach in this paper is designed in an universal manner, which can also be trained with rectangular or quadrilateral bounding boxes without extra efforts. Experimental results on CTW-1500 demonstrate our method with only a light backbone can outperform state-of-the-art methods with a large margin. By evaluating only in the curve or non-curve subset, the CTD + TLOC can still achieve the best results. Code is available at https://github.com/Yuliang-Liu/Curve-Text-Detector.
1. Introduction
Curve text is common and difficult to localize tightly with rectangles or quadrilaterals, motivating CTW1500 and a polygon-based detector designed for direct curve-text detection.
- Quadrilateral labels can create overlap, background noise, and interference between multiple text lines, whereas curve labels address these issues.
- Curve text appears in many real-world objects, but existing datasets contain very little of it and current methods cannot directly detect it.
- CTW1500 contains 1,500 images with over 10k text annotations, including 1,000 training and 500 testing images.
- CTD directly detects curve text with polygonal localization, separates width and height offset branches, and runs at 13 FPS below 4GB video memory.
- TLOC uses recurrent connections to learn relationships among locating points, producing smoother and more accurate detections.
- With a light reduced ResNet-50 backbone, CTD outperforms state-of-the-art methods by a large margin, while CTD+TLOC remains best on curve and non-curve subsets.
2. Related Work
Scene-text benchmarks and detectors evolved from rectangles to quadrilaterals, but curve text remains poorly addressed because datasets and four-point methods are inadequate.
- Scene-text datasets progressed from rectangular labels to rotated rectangles and quadrilaterals, alongside corresponding detector developments.
- Quadrilateral-based methods became prominent and could achieve strong performance on rotated or horizontal datasets.
- Current methods show disappointing performance on curve text, which commonly appears in real-world scenes.
- Limited curve-text data and unsatisfactory rectangular labels hinder evaluation and detection of strongly bending text.
- Four-point methods loosely localize curve text, causing severe mutual interference in cases such as multiple text lines.
- The paper addresses this gap by introducing CTW1500 and a method that directly detects curve text effectively.
3. CTW1500 Dataset and Annotation
CTW1500 combines diverse multilingual scene images with polygon annotations for curve text, using equidistant reference lines to reduce labeling effort and subjectivity.
- Data description: CTW1500 contains 1,500 images and 10,751 bounding boxes, including 3,530 curve boxes and at least one curve text per image.
- Data description: The dataset includes internet, image-library, and phone-camera images spanning indoor, outdoor, born-digital, blurred, and perspective-distorted scenes.
- Data description: CTW1500 is multilingual, with mainly Chinese and English text, and also contains horizontal and multi-oriented examples.
- Annotation: Horizontal and quadrilateral text require two or four clicks, while curve text uses ten equidistant reference lines to place ten additional points.
- Annotation: The reference-line procedure reduces labeling effort and subjective interference when annotating curve text.
- Annotation: Labeling one curve text takes approximately three times as long as labeling a quadrilateral, according to the reported timing comparison.
4. Methodology
The method detects curve text as polygons by extending a detector with 14-point localization, recurrent offset modeling, interpolation, and polygon-aware post-processing.
- Network Architecture: CTD combines a backbone, RPN, and regression module, modifying regression to add curve-locating points.The RPN roughly recalls text, while regression adjusts proposals more tightly.
- Network Architecture: The detector predicts 14 polygon points using separate width and height offsets, with 32 regression items including boundary terms.Relative positions use the circumscribed rectangle’s minimum x and y as the datum point.
- Recurrent Transverse and Longitudinal Offset Connection (TLOC): TLOC feeds width and height offset features into recurrent networks so sequential context connects neighboring localization points.The RNN receives fixed outputs from PSROIPooling, and the implementation uses a bidirectional LSTM with a 256D hidden layer.
- Recurrent Transverse and Longitudinal Offset Connection (TLOC): The training objective jointly optimizes classification and localization losses for scores, boundaries, and width/height offsets.The authors use balance factors to weight classification and localization losses.
- Long Side Interpolation: Equal-division interpolation converts two- or four-vertex bounding boxes into training targets for the detector’s 14-point output.The method is reported to train effectively with all text regions after interpolating points along the largest side and its opposite side.
- Polygonal Post Processing: NPS removes invalid self-intersecting polygons, while PNMS suppresses detections using polygon overlap rather than rectangular overlap.These post-processing methods address invalid shapes and dense, multi-oriented text.
5. Experiments
Experiments on CTW1500 show that CTD+TLOC substantially outperforms competing methods, while TLOC and PNMS improve detection and the detector remains relatively fast.
- Effectiveness of TLOC and PNMS: About 4 percent Hmean improvement results from adding TLOC to CTD.The comparison does not include NPS because it is prerequisite post-processing for polygonal overlap evaluation.
- Effectiveness of TLOC and PNMS: PNMS slightly outperforms classic NMS with both CTD and CTD + TLOC.NPS can slightly improve accuracy but is not compared separately in this evaluation.
- Comparison with state-of-the-arts methods: 13.3 or 15.2 FPS makes the method the second fastest among the compared detectors.The speed measurement covers the forward procedure without post-processing.
- Comparison with state-of-the-arts methods: Qualitative results demonstrate CTD’s curve-text detection and generalization ability on another dataset.Detection examples are visualized in Figure 7.
6. Conclusions and Future Work
The paper introduces CTW1500 and CTD with TLOC for directly detecting curve text, supplemented by flexible training and post-processing techniques.
- Conclusions and Future Work: CTW1500 provides a curve-text dataset whose polygon annotations require relatively little labeling effort.The dataset is mainly constructed from curve text, and referenced lines support annotation.
- Conclusions and Future Work: CTD directly detects curve text, while TLOC connects the detector with an RNN to improve detection performance.The approach is presented as a direct alternative to empirical grouping of text components.
- Conclusions and Future Work: Long-side interpolation makes CTD trainable with rectangular or quadrilateral bounding boxes without additional manual effort.The method is designed as a universal approach across these annotation types.
- Conclusions and Future Work: NPS and PNMS are proposed as effective post-processing methods for the detector.They are introduced as simple methods intended to improve detection accuracy.
- Conclusions and Future Work: The dataset could be enlarged into a curve-text recognition dataset in future work.The authors state that its labeling manner appears suitable for recognition.
- Conclusions and Future Work: Flexible detection may be slightly slower than rigid rectangle-based detection but can address more complicated problems such as curve-text detection.The authors identify this trade-off as worth further exploration.