Source-linked AI summary
Deep Matching Prior Network: Toward Tighter Multi-oriented Text Detection
Yuliang Liu, Lianwen Jin
TL;DR
Incidental scene text is difficult to localize because it can be multi-oriented and perspective-distorted, while rectangular methods may add background or lose text information. DMPNet uses quadrilateral sliding windows, shared Monte-Carlo overlap computation, ordered relative regression, and smooth Ln loss for tighter detection, achieving state-of-the-art performance on ICDAR 2015 Challenge 4.
Problem
Multi-oriented, perspective-distorted scene text is difficult to localize tightly, and rectangular constraints may cause background noise, overlap, or information loss.
Method
DMPNet combines quadrilateral sliding-window recall, shared Monte-Carlo polygon-area computation, sequential point ordering with relative regression, and smooth Ln loss.
Results
DMPNet achieves state-of-the-art performance in detecting incidental scene text on the ICDAR 2015 robust reading challenge dataset.
Takeaways & Limitations
Quadrilateral proposals and regression provide the paper's approach to tighter scene-text localization with reduced background interference.
Takeaways & Limitations
DMPNet's high recall depends on numerous manually designed quadrilateral sliding windows, which may not be optimal.
Abstract
from arXiv · showhide
Detecting incidental scene text is a challenging task because of multi-orientation, perspective distortion, and variation of text size, color and scale. Retrospective research has only focused on using rectangular bounding box or horizontal sliding window to localize text, which may result in redundant background noise, unnecessary overlap or even information loss. To address these issues, we propose a new Convolutional Neural Networks (CNNs) based method, named Deep Matching Prior Network (DMPNet), to detect text with tighter quadrangle. First, we use quadrilateral sliding windows in several specific intermediate convolutional layers to roughly recall the text with higher overlapping area and then a shared Monte-Carlo method is proposed for fast and accurate computing of the polygonal areas. After that, we designed a sequential protocol for relative regression which can exactly predict text with compact quadrangle. Moreover, a auxiliary smooth Ln loss is also proposed for further regressing the position of text, which has better overall performance than L2 loss and smooth L1 loss in terms of robustness and stability. The effectiveness of our approach is evaluated on a public word-level, multi-oriented scene text database, ICDAR 2015 Robust Reading Competition Challenge 4 "Incidental scene text localization". The performance of our method is evaluated by using F-measure and found to be 70.64%, outperforming the existing state-of-the-art method with F-measure 63.76%.
1. Introduction
DMPNet addresses the difficulty of tightly localizing multi-oriented, distorted scene text by replacing rectangular localization with quadrilateral proposals and regression. Its components target recall, polygon-overlap computation, and stable tight localization.
- Scene text varies in orientation, perspective, size, color, and scale, making robust localization difficult.
- Rectangular constraints can introduce background noise, incomplete localization, redundant overlap, and information loss for distorted incidental text.
- DMPNet uses quadrilateral sliding windows to roughly recall text with higher overlap, then finely adjusts predictions using quadrilateral relative regression.
- A shared Monte-Carlo method computes polygonal overlapping areas quickly and accurately during proposal recall.
- The proposed smooth Ln loss outperforms L2 and smooth L1 losses in robustness and stability.
- DMPNet achieves state-of-the-art performance for incidental scene text detection on the ICDAR 2015 challenge.
2. Related work
Related work includes character-structure methods, sliding-window detectors, and CNN-based approaches. These methods improve text recall or suppress false positives, but horizontal windows can remain sensitive to background noise.
- Scene text detection methods commonly exploit individual character structure and relationships between characters.
- Sliding-window methods scan images across positions and scales to recall text, but background noise can produce false positives.
- CNN-based methods have been used to suppress false positives and generate efficient pixel-wise text/nontext maps.
3. Proposed methodology
DMPNet combines quadrilateral proposal windows, shared Monte-Carlo overlap computation, ordered point regression, and Smooth Ln loss to localize scene text with tighter quadrangles.
- Roughly recall text with quadrilateral sliding windows: Quadrilateral sliding windows improve rough text recall by achieving larger overlap with distorted text than rectangular windows.Their flexible shapes also reduce background noise and make confidence estimates more reliable for rejecting false positives.
- Shared Monte-Carlo method: A shared Monte-Carlo method estimates polygonal overlap using sampled points and reuses points across computations.Non-intersecting circumscribed rectangles are discarded early, and the procedure supports GPU parallelization for processing many windows.
- Finely localize text with quadrangle: The fine localization stage orders quadrangle vertices uniquely before regressing their coordinates relative to a central point.The protocol selects an initial point by minimum x and resolves the remaining points using line slopes and opposite sides.
- Finely localize text with quadrangle: DMPNet represents each quadrangle with ten parameters: a central coordinate plus four pairs of relative lengths.The central coordinate is based on the minimum circumscribed horizontal rectangle, while each pair describes a point’s relative position.
- Smooth Ln loss: Smooth Ln loss is continuous and differentiable, less sensitive to outliers than L2, and intended to produce tighter text bounding boxes.The paper reports better text localization and relatively tighter bounding boxes than the compared loss properties.
4. Experiments
DMPNet is evaluated on ICDAR 2015 Challenge 4 using word-level, multi-oriented incidental scene text images. It achieves higher F-measure than the previous best method while tightly localizing text with reduced background noise, though some text is missed.
- Dataset and evaluation: The official evaluation ranks submitted methods using recall rate, precision, and F-measure.Recall measures text-finding ability, precision measures bounding-box reliability, and F-measure is their harmonic mean.
- Experimental setup: DMPNet was trained on the official 1,000 training images without extra data augmentation, after modifying some rectangular labels into quadrilateral labels.The network uses a VGG-16-based structure and applies quadrilateral sliding windows to intermediate convolutional layers.
- Dataset and evaluation: The evaluation uses ICDAR 2015 Challenge 4, containing 1,000 training images and 500 testing images with word-level annotations.Texts may appear at any orientation and location, including small or low-resolution instances.
- Experimental results: 70.64% F-measure was achieved by DMPNet, compared with 63.76% for the previous best method on ICDAR 2015 Challenge 4.The two methods had comparable precision, while DMPNet’s recall rate greatly increased.
- Qualitative results: DMPNet tightly localizes diverse scene text with less background noise, but some false detections and missed inconspicuous text remain.The reported failures include missed labels in some test images.
5. Conclusion and future work
The paper concludes that DMPNet uses quadrilateral proposals, sequential point ordering, relative regression, Monte-Carlo area computation, and smooth Ln loss for tighter scene-text localization. It reports state-of-the-art performance, while identifying manually designed sliding-window shapes as a remaining limitation and motivating shape-adaptive designs.
- Conclusion: DMPNet combines quadrilateral sliding windows, sequential point ordering, relative regression, and shared Monte-Carlo computation for tighter text localization.The method is designed to reduce background interference while efficiently computing polygonal overlap.
- Conclusion: The smooth Ln loss shows better overall robustness and stability than L2 loss and smooth L1 loss for adjusting predictions.The loss is used for further prediction adjustment.
- Conclusion: Quadrilateral labeling may be more reasonable for scene text because rectangular labels can include background noise or omit marginal text under perspective distortion.The paper presents ICDAR 2015 Challenge 4 as the first dataset it knows to use quadrilateral labeling.
- Future work: The manually designed quadrilateral sliding windows may not be optimal, motivating future work on shape-adaptive sliding windows.The paper states that DMPNet’s high recall mainly depends on numerous prior-designed quadrilateral windows.