Source-linked AI summary
Fourier Contour Embedding for Arbitrary-Shaped Text Detection
Yiqin Zhu, Jianyong Chen, Lingyu Liang, Zhanghui Kuang, Lianwen Jin, Wayne Zhang
TL;DR
Arbitrary-shaped text detection needs representations that capture diverse geometry, since masks can entail expensive post-processing and point sequences may struggle with highly-curved text. The paper proposes Fourier Contour Embedding and FCENet, which predict compact Fourier signatures and reconstruct contours with IFT. Experiments report effective generalization and superiority to SOTA methods on CTW1500 and Total-Text, especially for highly-curved text.
Problem
Existing masks may require expensive post-processing, while contour point sequences have limited capability for highly-curved text.
Method
FCE represents arbitrary-shaped contours as compact Fourier signatures, and FCENet predicts those signatures before reconstructing contours with Inverse Fourier Transformation.
Results
FCENet is reported effective and well-generalized, and superior to SOTA methods on CTW1500 and Total-Text, especially their highly-curved text subsets.
Takeaways & Limitations
Fourier-domain contour modeling provides a compact representation and enables end-to-end arbitrary-shaped text detection without complex post-processing.
Abstract
from arXiv · showhide
One of the main challenges for arbitrary-shaped text detection is to design a good text instance representation that allows networks to learn diverse text geometry variances. Most of existing methods model text instances in image spatial domain via masks or contour point sequences in the Cartesian or the polar coordinate system. However, the mask representation might lead to expensive post-processing, while the point sequence one may have limited capability to model texts with highly-curved shapes. To tackle these problems, we model text instances in the Fourier domain and propose one novel Fourier Contour Embedding (FCE) method to represent arbitrary shaped text contours as compact signatures. We further construct FCENet with a backbone, feature pyramid networks (FPN) and a simple post-processing with the Inverse Fourier Transformation (IFT) and Non-Maximum Suppression (NMS). Different from previous methods, FCENet first predicts compact Fourier signatures of text instances, and then reconstructs text contours via IFT and NMS during test. Extensive experiments demonstrate that FCE is accurate and robust to fit contours of scene texts even with highly-curved shapes, and also validate the effectiveness and the good generalization of FCENet for arbitrary-shaped text detection. Furthermore, experimental results show that our FCENet is superior to the state-of-the-art (SOTA) methods on CTW1500 and Total-Text, especially on challenging highly-curved text subset.
1. Introduction
Arbitrary-shaped text detection needs representations that capture diverse geometry without costly processing or limited curvature modeling. The paper addresses this with compact Fourier signatures and FCENet, achieving strong performance on curved-text benchmarks.
- Existing spatial-domain representations use masks or contour point sequences, but masks can require expensive post-processing and point sequences may poorly model highly-curved text.
- FCE converts contour point sequences to Fourier signatures using Fourier Transform and Inverse Fourier Transform, avoiding complex post-processing.
- FCENet combines a ResNet50-DCN backbone, FPN, classification and Fourier regression branches, reconstructing contours from predicted signatures with IFT.
- FCENet is reported superior to SOTA methods on CTW1500 and Total-Text, especially for highly-curved text subsets.
- Fourier Contour Embedding represents arbitrary-shaped text contours as compact Fourier signature vectors that can progressively approximate complex closed shapes.
2. Related Work
Prior arbitrary-shaped text detectors mainly use segmentation masks or explicit contour point sequences, each with practical or representational limitations. The paper motivates Fourier-domain contour modeling as a flexible alternative for complex text shapes.
- Segmentation-Based Methods: Segmentation-based methods detect pixels, fragments, characters, or kernels and group them into complete text instances or masks.
- Segmentation-Based Methods: Some segmentation methods predict transformed-space embeddings or direction fields before reconstructing or grouping text regions.
- Regression-Based Methods: Regression-based methods explicitly encode text contours with point sequences and are often simpler and easier to train than segmentation approaches.
- Regression-Based Methods: Point-sequence methods can have constrained representation capability for complex text instances, motivating refinement modules and other specialized designs.
- Contour Modeling: Effective contour modeling is important for irregular text detection and recognition, while Bezier control-point settings may limit representation in some cases.
- Fourier-Domain Modeling: FCE instead models text instances in the Fourier domain, which is described as fitting any closed continuous contour robustly and simply.
3. Approach
FCE represents arbitrary-shaped text contours with compact Fourier signatures, and FCENet predicts these signatures before reconstructing contours through IFT and NMS. Its architecture combines multi-scale features, classification masks, and Fourier regression for end-to-end detection.
- Fourier Contour Embedding: Low-frequency components describe rough contours while high-frequency components capture details; retaining K = 5 lowest frequencies gives satisfactory approximation.Fourier reconstruction progressively approximates the ground-truth contour as the degree increases.
- Fourier Contour Embedding: FCE resamples each contour at fixed intervals, transforms the points into Fourier coefficients, and represents the contour as a compact signature vector.The method uses N = 400 resampled points and a 2(2K +1)-dimensional vector containing real and imaginary coefficient components.
- Fourier Contour Embedding: The contour signature is made unique by fixing the starting point, clockwise sampling direction, and uniform sampling speed.The starting point is the rightmost contour intersection with the horizontal line through the contour center.
- FCENet: FCENet uses a ResNet50-DCN backbone, FPN, and separate classification and regression branches to predict text masks and Fourier signature vectors.FPN levels P3, P4, and P5 handle small, medium, and large text instances, respectively.
- FCENet: Predicted text-region and center-region heat maps are multiplied into classification scores, while predicted Fourier signatures are reconstructed with IFT and filtered by NMS.The final detections are obtained from reconstructed contours and their corresponding classification scores.
- FCENet: The regression loss compares reconstructed contours in the spatial domain, and using N′ = 50 sampling points produced absolute h-mean improvements of 6.9% on CTW1500 and 9.3% on Total-Text.The contour-based loss uses smooth-L1 distance after applying the IFT to predicted and ground-truth signatures.
4. Experiments
Experiments evaluate FCE’s contour-fitting ability and FCENet’s detection performance, ablations, generalization with reduced training data, and comparisons on benchmark datasets and highly-curved text.
- FCE evaluation: FCE obtains satisfactory contour fitting for most arbitrary-shaped texts using only small Fourier degrees K.Increasing K improves approximation theoretically, while experiments show small K is sufficient for most texts.
- FCE evaluation: TextRay fails to fit highly-curved ground-truth contours closely, whereas FCE produces accurate approximations with 22 parameters versus TextRay’s 44.FCE therefore uses half as many parameters as TextRay in this comparison.
- Ablation studies: The TCR loss and proposed regression loss substantially improve FCENet performance on both CTW1500 and Total-Text.These components belong to the classification and regression branches, respectively.
- Generalization ability: With training data reduced to 50% and 25%, FCENet maintains over 73% precision, recall, and F-measure, while other methods drop dramatically.Precision remains above 80% under the reduced-data settings.
- Benchmark comparisons: On CTW1500 and Total-Text, FCENet achieves the best precision and F-measure and competitive recall, without extra training data.It also achieves competitive results on ICDAR2015 without additional setup.
- Benchmark comparisons: FCENet uses a simple architecture and efficient IFT-and-NMS post-processing, supporting practical implementation across datasets.The highly-curved CTW1500 subset contains 106 selected samples, where comparisons also show FCENet’s effectiveness.
5. Conclusion
The paper introduces Fourier contour embedding for explicit shape modeling in arbitrary-shaped text detection and builds FCENet around it. FCE supports accurate contour approximation, while FCENet achieves strong benchmark performance and generalization.
- FCE approximates arbitrary closed shapes accurately using Fourier signatures.
- FCENet predicts Fourier signatures and reconstructs contour point sequences with the Inverse Fourier Transformation.
- FCENet is optimized end-to-end without complex post-processing.
- Experiments validate FCE’s representation capability, especially for highly-curved texts, and FCENet’s generalization with small training samples.
- FCENet achieves state-of-the-art performance on CTW1500 and Total-Text, with competitive results on ICDAR2015.