PaperlayerView exact revision on arXiv

Source-linked AI summary

Rotation-Sensitive Regression for Oriented Scene Text Detection

Minghui Liao, Zhen Zhu, Baoguang Shi, Gui-song Xia, Xiang Bai

arXiv:1803.05265v1cs.CV

TL;DR

Arbitrarily oriented scene text challenges detectors because classification and oriented bounding-box regression require incompatible feature characteristics. RRD separates these tasks, using rotation-invariant features for classification and rotation-sensitive features for regression, and reports strong performance across oriented detection benchmarks and tasks.

  • Problem

    Arbitrarily oriented scene text requires oriented bounding-box detection, but shared features create a conflict between rotation-invariant classification and orientation-sensitive regression.

  • Method

    RRD separates classification and regression into branches using rotation-invariant features for classification and actively rotated-filter features for oriented bounding-box regression.

  • Results

    RRD achieves comparable COCO-Text results at IoU 0.5 and outperforms previous methods by at least 6.1 percents at IoU 0.75.

  • Takeaways & Limitations

    The feature-separation strategy improves oriented detection across text and ship datasets and can be embedded into existing detection architectures without obvious speed loss.

  • Takeaways & Limitations

    RRD fails on text lines with large character spacing and can misclassify two vertical text lines as multiple horizontal ones.

Abstract

from arXiv · show

Text in natural images is of arbitrary orientations, requiring detection in terms of oriented bounding boxes. Normally, a multi-oriented text detector often involves two key tasks: 1) text presence detection, which is a classification problem disregarding text orientation; 2) oriented bounding box regression, which concerns about text orientation. Previous methods rely on shared features for both tasks, resulting in degraded performance due to the incompatibility of the two tasks. To address this issue, we propose to perform classification and regression on features of different characteristics, extracted by two network branches of different designs. Concretely, the regression branch extracts rotation-sensitive features by actively rotating the convolutional filters, while the classification branch extracts rotation-invariant features by pooling the rotation-sensitive features. The proposed method named Rotation-sensitive Regression Detector (RRD) achieves state-of-the-art performance on three oriented scene text benchmark datasets, including ICDAR 2015, MSRA-TD500, RCTW-17 and COCO-Text. Furthermore, RRD achieves a significant improvement on a ship collection dataset, demonstrating its generality on oriented object detection.

1. Introduction

Scene text detection requires oriented bounding boxes because natural text can be long, thin, and arbitrarily oriented. RRD separates classification and regression, using rotation-invariant features for text presence and rotation-sensitive features for box regression.

  • 1. Introduction: Arbitrary orientations, small sizes, and varied aspect ratios make scene text detection challenging for general object detectors.Horizontal boxes are inadequate for long, thin text at arbitrary orientations, which is better represented with oriented bounding boxes.
  • 1. Introduction: Shared features create a conflict because rotation invariance benefits classification but hinders regression of arbitrarily oriented bounding boxes.The conflict is especially relevant for long, thin non-Latin text lines that are often detected at line level.
  • 1. Introduction: RRD separates the tasks by using rotation-invariant features for classification and rotation-sensitive features for oriented bounding box regression.The method adopts oriented response convolution and oriented response pooling to produce the two feature types.
  • 1. Introduction: RRD adds an optional three-scale inception block to provide flexible receptive fields for covering long text.The method is presented as a general framework that can be embedded into existing detection architectures without obvious loss of speed.
  • 1. Introduction: RRD reports richer orientation information in its regression feature map and a more intensive classification feature map than the conventional shared feature map.The comparison is illustrated in Figure 1 through feature-map visualizations and detection results.

2. Related Work

CNN-based detectors commonly share features for classification and localization, but this design is poorly matched to oriented text. RRD instead assigns rotation-sensitive features to regression and rotation-invariant features to classification.

  • 2. Related Work: CNN-based object detectors commonly combine object classification with bounding box regression using shared translation- and rotation-invariant features.Pooling layers in classical CNN architectures contribute to these feature invariances.
  • 2. Related Work: Scene text detectors represent multi-oriented words or text lines with oriented rectangles or quadrilaterals rather than only horizontal boxes.Oriented representations require additional variables such as an angle or vertex coordinates.
  • 2. Related Work: Unlike modern multi-oriented text detectors with shared features, RRD explicitly uses rotation-sensitive features for oriented box regression and rotation-invariant features for classification.The paper presents this separation as improving regression accuracy for arbitrary oriented, especially long, text.
  • 2. Related Work: ORN produces rotation-sensitive feature maps with actively rotated filters and obtains rotation-invariant maps by pooling responses across orientations.RRD is inspired by ORN and applies this distinction to arbitrary oriented object detection.

3. Rotation-Sensitive Regression Detector

RRD separates classification and oriented bounding-box regression into branches using features with different rotation characteristics. Its regression branch preserves orientation information, while its classification branch pools those responses to obtain locally rotation-invariant features.

  • Overview: RRD is an end-to-end fully convolutional detector inspired by SSD, producing dense predictions from six backbone layers.It uses VGG16 as its backbone and adds extra layers in the same manner as SSD.
  • Overview: RRD extracts orientation-sensitive and orientation-insensitive feature maps separately for regression and classification, matching the differing requirements of the two tasks.The classification map is insensitive to text orientation, whereas the regression map remains sensitive.
  • Rotation-Sensitive Regression: Active rotating filters convolve each feature map with a canonical filter and rotated clones, producing orientation channels for rotation-sensitive regression.The method uses shared parameters across the rotated filters and sets the number of orientations N to 8 in practice.
  • Rotation-Invariant Classification: Orderless max pooling across all orientation responses produces a locally rotation-invariant feature map for text-presence classification.Pooling is applied along the depth axis of the rotation-sensitive feature maps.
  • Default Boxes and Training: For each horizontal default box, the regression branch predicts offsets to four quadrilateral vertices while the prediction layer also outputs classification scores.Positive quadrilaterals are filtered by non-maximum suppression; training matches default boxes to ground truth using box overlap and combines classification and regression losses.

4. Experiments

Experiments evaluate RRD across long-text, multi-oriented, horizontal-text, and oriented-object benchmarks, including ablations and qualitative comparisons. RRD improves over baselines and prior methods, while failures remain for text with large character spacing or vertically arranged lines.

  • Ablation Study: About 6% improvement from inception blocks indicates better handling of long, thin text through a larger effective receptive field.This comparison is against the baseline model.
  • Ablation Study: A further 6% gain from rotation-sensitive features supports their usefulness for long, thin, oriented text detection.The comparison is between Baseline+inc+rs and Baseline+inc.
  • Ablation Study: RRD outperforms all other RCTW-Long variants by a large margin and produces more convincing scores and more accurate boxes than Baseline+inc.RRD combines rotation-sensitive regression with rotation-invariant classification; low-score Baseline+inc boxes are discarded, whereas RRD boxes are retained.
  • Scene Text Benchmarks: On RCTW-17, RRD improves F-measure by about 3.3% over SegLink and 2.6% over EAST-ResNet, while multi-scale input adds 11.3% over single-scale RRD.On MSRA-TD500, RRD leads state-of-the-art precision, recall, and F-measure, and Baseline+inc trails it by 5% F-measure.
  • Scene Text and Oriented Object Benchmarks: RRD exceeds prior COCO-Text methods by at least 6.1% at IoU 0.75, gains about 1% over its baseline on IC15, remains competitive on IC13, and improves ship-benchmark mAP by around 8.6 points to 84.3.The IC15 gains are reported for both single-scale and multi-scale settings; IC13 contains horizontal text.
  • Limitations: RRD fails on text lines with large character spacing and can split two vertical text lines into multiple horizontal detections.The paper suggests higher-level semantic understanding and spatial analysis may be required.

5. Conclusion

RRD separates classification and regression by using rotation-insensitive and rotation-sensitive features, respectively. Its effectiveness and generality are demonstrated across multiple datasets and tasks, with potential applications to other oriented-object detection settings.

  • RRD performs classification and regression using rotation-insensitive and rotation-sensitive features, respectively.
  • RRD improves dense prediction of text presence and offsets used in many modern text detectors.
  • The strategy is demonstrated on multiple datasets and tasks and may benefit detectors beyond text and ship detection.
Loading 1803.05265v1…