PaperlayerView exact revision on arXiv

Source-linked AI summary

MSR: Multi-Scale Shape Regression for Scene Text Detection

Chuhui Xue, Shijian Lu, Wei Zhang

arXiv:1901.02596v2cs.CV

TL;DR

Scene-text detection remains challenging for thin, curved lines of varied lengths and for text spanning large scale differences. MSR combines dense boundary-point regression with multi-scale feature fusion, and the paper reports superior detection across public datasets for varied text orientations, shapes, and lengths.

  • Problem

    Scene-text detectors can localize thin, curved lines of varied lengths inaccurately, and large text-scale variation can lead to missed small text or broken large-text detections.

    Source: §1 · p. 1

  • Method

    MSR regresses text pixels to nearby text boundaries, links the resulting dense points, and fuses features from multiple network channels and scales.

    Source: §1 · p. 1

  • Results

    Experiments on public datasets show superior detection performance for text with different orientations, shapes, and lengths.

    Source: §1 · p. 1

  • Takeaways & Limitations

    Dense boundary points support localization of arbitrary text orientations and curvatures, while multi-scale feature fusion demonstrates tolerance to text-scale variation.

    Source: §5 · p. 6

    MSR can fail on spatially overlapping text lines and produce broken detections when characters in a line are widely separated.

    Source: §4.5 · p. 6

Abstract

from arXiv · show

State-of-the-art scene text detection techniques predict quadrilateral boxes that are prone to localization errors while dealing with straight or curved text lines of different orientations and lengths in scenes. This paper presents a novel multi-scale shape regression network (MSR) that is capable of locating text lines of different lengths, shapes and curvatures in scenes. The proposed MSR detects scene texts by predicting dense text boundary points that inherently capture the location and shape of text lines accurately and are also more tolerant to the variation of text line length as compared with the state of the arts using proposals or segmentation. Additionally, the multi-scale network extracts and fuses features at different scales which demonstrates superb tolerance to the text scale variation. Extensive experiments over several public datasets show that the proposed MSR obtains superior detection performance for both curved and straight text lines of different lengths and orientations.

1 Introduction

Scene-text detectors face localization challenges for thin, curved lines of varied lengths and unreliable detection across large text-size variations. MSR combines dense boundary-point regression with multi-scale feature fusion to address these constraints and reports strong performance across public datasets.

1 Introduction

  • Thin, curved text lines of varied lengths challenge localization, while large scale variation can cause missed tiny text or broken detections of large text.The text-scale ratio in MSRA-TD500 images can reach 230 times.Source: §1 · p. 1
  • MSR regresses text pixels to nearby boundary points and links them to locate varied text shapes and lengths, rather than predicting quadrilateral vertices.The paper states that nearest-boundary regression introduces less error than vertex regression, especially for long lines.Source: §1 · p. 1
  • Multiple network channels extract and fuse features at different scales to improve tolerance to text-size variation.The framework takes an image and its down-scaled version as input and predicts central text regions, distances to boundaries, and dense boundary points.Source: §1 · p. 1, Figure 1 · p. 2
  • Experiments on public datasets show superior detection performance for text with different orientations, shapes, and lengths.The system is end-to-end trainable.Source: §1 · p. 1

2 Related Work

Prior scene-text methods adapt object detection or segmentation approaches, while quadrilateral prediction still has difficulty with varied text shapes and lengths. MSR instead regresses to nearby boundaries and combines multi-scale feature extraction with inputs at different image scales.

Scene Text Detection

  • Object-detection adaptations use text-specific proposals or default boxes, while segmentation approaches localize instances from pixel-level text maps.Other approaches regress each text pixel to localization-box vertices.Source: Scene Text Detection · p. 2
  • Quadrilateral prediction still struggles with varied text shapes and lengths; MSR links nearest-boundary predictions into dense points to locate them.This contrasts with box-vertex regression.Source: Scene Text Detection · p. 2

Large Scale Variation

  • Prior scale-handling approaches use features from multiple network layers or predictions from differently scaled images; MSR combines both by fusing features across stages and input scales.The paper describes feature-pyramid and U-Net approaches as examples of multi-layer feature use.Source: Large Scale Variation · p. 2

3 Methodology

MSR combines multi-scale, multi-stage features with shape regression: it classifies central text regions and predicts boundary distances to construct polygon detections.

3 Methodology

  • MSR fuses features from different image scales and network stages, then predicts central text regions and boundary-distance maps to form polygon detections.The network uses multiple channels for differently scaled images and feeds fused features to the shape-regression module.Source: §3 · p. 2, §3.2 · pp. 3–4

3.1 Shape Regression for Scene Text Localization

  • The regressor classifies central text regions, predicts horizontal and vertical distances to nearest boundaries, then encloses the resulting boundary points with normalized Alpha-Shape polygons.Training polygons are triangulated to derive central regions, whose pixels are assigned nearest annotation-boundary points as distance-map targets.Source: §3.1 · p. 3

3.2 Multi-Scale Multi-Stage Detection Network

  • Two image-scale channels fuse Conv2–Conv5 features across stages, using Concat UpConv and upsampling to align lower-scale features with preceding stages.The implemented system uses the original image and a half-scale image; Conv1 features are excluded as too low-level.Source: §3.2 · pp. 3–4, Figure 2 · p. 3

3.3 Network Training

  • Training minimizes classification and regression losses, using Dice loss for central-region pixels and Smooth L1 for boundary distances, with λ set to 1.0.The classification loss compares predicted and ground-truth central regions; regression uses horizontal and vertical distance targets.Source: §3.3 · p. 4

4 Experiments

Experiments across curved and straight-text datasets show strong detection performance, while ablations examine the contributions of multi-scale features and shape regression. The discussion identifies overlap and widely spaced characters as challenging cases.

4.2 Implementation Details

  • The network uses a ResNet-50 backbone, is pretrained on SynthText, and is fine-tuned on each evaluated dataset’s training images.Training uses Adam with a starting learning rate of 10^-4 and batch size 10.Source: §4.2 · p. 5

4.3 Experimental Results

  • The evaluation includes quantitative and qualitative results on four public datasets, with ablations reported separately.The experiments also include Tables 1–3 and Figures 4–5.Source: §4.3 · p. 5

4.1 Datasets

  • CTW1500 and Total-Text cover varied text appearances; ICDAR2015 and MSRA-TD500 provide straight-text evaluations with different orientations and annotation levels.CTW1500 and MSRA-TD500 use text-line annotations; Total-Text and ICDAR2015 use word-level annotations.Source: §4.1 · p. 4, Texts in Different Orientations and… · p. 5

Texts in Different Orientations and Shapes

  • 81.5% and 79.0% F-scores on CTW1500 and Total-Text; CTW1500 exceeds the prior best by 5.9, and Total-Text reaches state-of-the-art.The reported advantage is against methods that did not specifically address curved text lines.Source: Texts in Different Orientations and… · p. 5

Texts in Different Orientations and Lengths

4.4 Ablation Study

  • On CTW1500, both components together improve F-score by over 21 beyond baseline; shape regression alone improves precision and recall, while multi-scale alone trades some precision for recall.Figure 5 illustrates missing and broken detections recovered by the combined model.Source: §4.4 · p. 6, Table 3 · p. 6, Figure 5 · p. 6

4.5 Discussion

  • Overlapping text lines can defeat localization, and widely separated characters can yield broken detections because their grouping is ambiguous without text semantics.The authors describe these as specific scenarios in which the method can fail.Source: §4.5 · p. 6

5 Conclusion

MSR predicts dense text boundary points and fuses features across scales to locate varied scene text. Experiments on public datasets show superior performance.

5 Conclusion

  • Dense boundary points replace sparse quadrilateral vertices, which are prone to large regression errors on long text lines.The method also supports localization of varied orientations and curvatures where quadrilaterals may include background.Source: §5 · p. 6
  • The multi-scale network extracts and fuses features at different scales, demonstrating tolerance to text-scale variation.Source: §5 · p. 6
  • Experiments over several public datasets show superior performance for the proposed technique.Source: §5 · p. 6
Loading 1901.02596v2…