Source-linked AI summary
ABCNet: Real-time Scene Text Spotting with Adaptive Bezier-Curve Network
Yuliang Liu, Hao Chen, Chunhua Shen, Tong He, Lianwen Jin, Liangwei Wang
TL;DR
Arbitrarily-shaped scene text spotting remains difficult to deploy in real time because prior methods require costly annotations, complex pipelines, or slow inference. ABCNet addresses this with parameterized Bezier-curve detection and BezierAlign in an end-to-end framework, achieving state-of-the-art benchmark performance while substantially improving speed.
Problem
Prior arbitrarily-shaped scene text spotters use expensive character-level annotations or complex segmentation pipelines, while nearly all are slow for real-time deployment.
Method
ABCNet is an end-to-end framework that represents arbitrarily-shaped text with parameterized Bezier curves and uses BezierAlign to extract aligned curved-text features.
Results
ABCNet achieves state-of-the-art performance on Total-Text and CTW1500 and is significantly faster than previous methods.
Takeaways & Limitations
The method provides real-time arbitrarily-shaped scene text spotting with negligible detection overhead and a lightweight recognition connection.
Takeaways & Limitations
The method relies on the empirical assumption that a cubic Bezier curve is sufficient to fit different arbitrarily-shaped scene text in practice.
Abstract
from arXiv · showhide
Scene text detection and recognition has received increasing research attention. Existing methods can be roughly categorized into two groups: character-based and segmentation-based. These methods either are costly for character annotation or need to maintain a complex pipeline, which is often not suitable for real-time applications. Here we address the problem by proposing the Adaptive Bezier-Curve Network (ABCNet). Our contributions are three-fold: 1) For the first time, we adaptively fit arbitrarily-shaped text by a parameterized Bezier curve. 2) We design a novel BezierAlign layer for extracting accurate convolution features of a text instance with arbitrary shapes, significantly improving the precision compared with previous methods. 3) Compared with standard bounding box detection, our Bezier curve detection introduces negligible computation overhead, resulting in superiority of our method in both efficiency and accuracy. Experiments on arbitrarily-shaped benchmark datasets, namely Total-Text and CTW1500, demonstrate that ABCNet achieves state-of-the-art accuracy, meanwhile significantly improving the speed. In particular, on Total-Text, our realtime version is over 10 times faster than recent state-of-the-art methods with a competitive recognition accuracy. Code is available at https://tinyurl.com/AdelaiDet
1. Introduction
ABCNet targets real-time arbitrarily-shaped scene text spotting with a simple end-to-end framework that represents text using Bezier curves and connects recognition through BezierAlign. It addresses the annotation, pipeline complexity, feature-alignment, and inference-speed limitations of prior approaches while reporting strong accuracy and efficiency.
- Motivation: Prior arbitrarily-shaped scene text spotters rely on complex segmentation pipelines or expensive character-level annotations, and are generally slow at inference.These limitations hinder deployment to real-time applications.
- ABCNet: ABCNet introduces an end-to-end framework that adaptively represents oriented and curved text with parameterized Bezier curves.The representation adds negligible computation overhead compared with standard bounding-box detection.
- ABCNet: BezierAlign samples convolutional features from structured Bezier detections, naturally connecting curved-text detection to a lightweight recognition branch.The recognition branch shares backbone features, avoiding a more complicated recognition design.
- Related work: Unlike earlier quadrilateral-oriented methods, ABCNet is designed to spot arbitrarily-shaped text while avoiding the limitations of methods that cannot handle such shapes.Prior methods such as TextAlign and FOTS are described as unable to spot arbitrarily-shaped scene text.
2. Adaptive Bezier Curve Network (ABCNet)
ABCNet represents arbitrarily shaped text with adaptive Bezier curves and aligns features directly along those curves. Its end-to-end design combines lightweight curve detection, BezierAlign, and a shared-feature recognition branch.
- Framework: ABCNet uses a single-shot, anchor-free convolutional framework for end-to-end arbitrarily-shaped scene text spotting.Removing anchor boxes simplifies detection, which is densely predicted on output feature maps.
- Bezier Curve Detection: Cubic Bezier curves parameterize curved text boundaries, reducing detection to bounding-box regression with eight control points.The method empirically treats cubic curves as sufficient for varied arbitrarily-shaped text and represents straight text as a special case.
- Bezier Curve Detection: Relative control-point offsets are predicted with one 16-channel convolution layer, adding nearly no detection cost.Offsets are measured relative to the minimum x and y coordinates of the four vertexes, allowing control points beyond image boundaries.
- Bezier Ground Truth Generation: Bezier ground truth is fitted from polygon annotations using least squares, with annotation-point spacing determining the curve parameters.The number of annotated boundary points is 5 for Total-Text and 7 for CTW1500.
- BezierAlign: BezierAlign samples features between upper and lower Bezier boundaries, computes positions by linear indexing, and applies bilinear interpolation.The resulting alignment warps curved text into a horizontal format without dramatic deformation.
- Recognition Branch: Shared backbone features and BezierAlign support a lightweight recognition branch with six convolutional layers, a bidirectional LSTM, and a fully connected layer.The branch uses CTC loss for text-string alignment and replaces training regions with detected Bezier curves during inference.
3. Experiments
Experiments evaluate ABCNet on two arbitrarily-shaped scene-text benchmarks, with ablations examining BezierAlign, sampling points, and Bezier curve detection. ABCNet achieves competitive or state-of-the-art accuracy while maintaining real-time efficiency.
- Datasets: ABCNet is evaluated on Total-Text and CTW1500, two benchmarks containing arbitrarily-shaped scene text and substantial straight text.Total-Text has 1,555 images, while CTW1500 contains 1,000 training and 500 testing images and includes English and Chinese text.
- Ablation studies: The number of sampling points significantly affects BezierAlign’s final performance and efficiency, with (7,32) providing the best F-measure–FPS trade-off.This setting is used in the following experiments.
- Ablation studies: Bezier curve detection introduces no extra computation compared with standard bounding box detection.The comparison is reported in the time-consumption ablation study.
- Comparison with state-of-the-art: With multi-scale inference, ABCNet achieves state-of-the-art performance and significantly outperforms previous methods in running time.Its faster version is more than 11 times faster than the previous best method with on-par accuracy.
- Qualitative results: Qualitative Total-Text results show accurate detection and recognition for most arbitrarily-shaped text, while some errors arise from misrecognizing individual characters.The method also handles straight text with nearly quadrilateral compact bounding boxes.
- CTW1500 results: On CTW1500, ABCNet significantly surpasses previous state-of-the-art methods, although long text lines make full-match word accuracy especially difficult.A single character error can produce a zero score for the whole text.
4. Conclusion
The conclusion presents ABCNet as a real-time end-to-end method for arbitrarily-shaped scene text spotting. It combines parameterized Bezier curves with BezierAlign and reports state-of-the-art performance with substantially higher speed than previous methods.
- Conclusion: ABCNet uses parameterized Bezier curves to detect arbitrarily-shaped scene text with negligible computation cost versus standard bounding box detection.The representation supports a lightweight recognition branch through regular Bezier curve bounding boxes.
- Conclusion: BezierAlign connects a lightweight recognition branch to the detection structure for curved text.The method naturally aligns recognition features using the regular Bezier curve representation.
- Conclusion: Experiments on Total-Text and CTW1500 demonstrate state-of-the-art performance with significantly faster inference than previous methods.The conclusion attributes this result to experiments using Bezier curve synthesized data and publicly available data.