PaperlayerView exact revision on arXiv

Source-linked AI summary

R2CNN: Rotational Region CNN for Orientation Robust Scene Text Detection

Yingying Jiang, Xiangyu Zhu, Xiaobing Wang, Shuli Yang, Wei Li, Hua Wang, Pei Fu, Zhenbo Luo

arXiv:1706.09579v2cs.CV

TL;DR

Arbitrary-oriented scene text detection must account for text orientation beyond axis-aligned localization. R2CNN extends Faster R-CNN by predicting inclined boxes from axis-aligned proposals, and it reports F-measures of 82.54% on ICDAR 2015 and 87.73% on ICDAR 2013.

  • Problem

    Scene text detection is difficult because texts vary in orientation, and orientation information is useful for scene text recognition and other tasks.

  • Method

    R2CNN uses an RPN for axis-aligned text proposals, concatenates features from differently sized ROIPoolings, predicts text scores and two box types, and applies inclined non-maximum suppression.

  • Results

    82.54% F-measure is reported on ICDAR 2015 incidental text detection and 87.73% on ICDAR 2013 focused text detection.

  • Takeaways & Limitations

    The proposed framework detects arbitrary-oriented scene texts while retaining a Faster R-CNN-based region-proposal strategy.

Abstract

from arXiv · show

In this paper, we propose a novel method called Rotational Region CNN (R2CNN) for detecting arbitrary-oriented texts in natural scene images. The framework is based on Faster R-CNN [1] architecture. First, we use the Region Proposal Network (RPN) to generate axis-aligned bounding boxes that enclose the texts with different orientations. Second, for each axis-aligned text box proposed by RPN, we extract its pooled features with different pooled sizes and the concatenated features are used to simultaneously predict the text/non-text score, axis-aligned box and inclined minimum area box. At last, we use an inclined non-maximum suppression to get the detection results. Our approach achieves competitive results on text detection benchmarks: ICDAR 2015 and ICDAR 2013.

1. Introduction

R2CNN addresses arbitrary-oriented scene text detection by extending Faster R-CNN to predict both axis-aligned and inclined text boxes. It uses multi-scale pooled features and inclined non-maximum suppression to produce final detections.

  • 1. Introduction: Scene text detection must predict orientation in addition to axis-aligned box information because text appears with varied orientations and visual properties.Challenges also include differences in size, aspect ratio, font style, lighting, and perspective distortion.
  • 1. Introduction: R2CNN uses an RPN to propose axis-aligned boxes enclosing arbitrary-oriented texts, then predicts refined boxes and inclined minimum area boxes.The framework is based on Faster R-CNN, with modified Fast R-CNN processing for classification, refinement, and inclined-box prediction.
  • 1. Introduction: 82.54% F-measure is reported on ICDAR 2015 incidental text detection, and 87.73% on ICDAR 2013 focused text detection.The results are evaluated using the competition benchmarks described in the introduction.
  • 1. Introduction: The approach formulates arbitrary-oriented text detection as a multi-task problem predicting text scores, axis-aligned boxes, and inclined minimum area boxes for each RPN proposal.These predictions are made jointly for each proposal.
  • 1. Introduction: Three ROIPoolings with sizes 7 × 7, 11 × 3, and 3 × 11 are concatenated to exploit text characteristics during detection.The resulting features are used for further detection predictions.
  • 1. Introduction: Inclined non-maximum suppression post-processes detection candidates to obtain final results, alongside a smaller anchor for detecting small scene texts.The suppression operates on inclined boxes rather than only axis-aligned boxes.

2. Related Work

Prior scene text detectors include traditional sliding-window and connected-components methods as well as deep-learning approaches. R2CNN instead uses Faster R-CNN proposals while predicting orientation from axis-aligned text candidates.

  • 2. Related Work: Traditional scene text detectors use densely scanned multi-scale windows or connected components to generate and classify character candidates.MSER-based connected-component methods achieved good performances in ICDAR 2015 and ICDAR 2013 competitions.
  • 2. Related Work: Region-proposal object detectors such as R-CNN, Fast R-CNN, and Faster R-CNN refine and classify proposals generated from convolutional feature maps.R2CNN is based on the Faster R-CNN architecture.
  • 2. Related Work: Many deep-learning scene text methods improve performance over traditional methods but generate axis-aligned boxes and therefore do not address text orientation.CTPN handles horizontal text well but is not fit for highly inclined text, while other methods target multi-oriented text through additional stages or representations.
  • 2. Related Work: Other arbitrary-oriented methods include FCN-based detection, segment-and-link approaches such as SegLink, and direct regression methods such as EAST, DMPNet, and Deep direct regression.These methods differ in whether they detect text blocks, segments and links, quadrangles, or directly regressed text boxes.
  • 2. Related Work: R2CNN differs from RRPN by using the RPN to generate axis-aligned text candidates and predicting orientation from those candidates instead of generating inclined proposals.The authors consider the RPN qualified to propose text candidates for arbitrary-oriented texts.

3. Proposed Approach

R2CNN adapts Faster R-CNN for arbitrary-oriented scene text by predicting axis-aligned and inclined boxes through a multi-task pipeline, then applying inclined NMS.

  • 3.1. Problem definition: R2CNN approximates arbitrary-oriented text detection with inclined minimum-area rectangles rather than angle-based representations.The rectangle is represented by the first two clockwise points and its height, avoiding unstable angle targets near vertical orientations.
  • 3.2. Rotational Region CNN (R2CNN): R2CNN concatenates ROIPooled features at 7 × 7, 11 × 3, and 3 × 11 sizes to capture varied text characteristics.The additional pooled sizes target texts with substantially different width-height proportions.
  • 3.2. Rotational Region CNN (R2CNN): The RPN generates axis-aligned proposals enclosing texts in horizontal, vertical, or diagonal directions.The proposal stage uses axis-aligned boxes even though the final detection targets are inclined boxes.
  • 3.2. Rotational Region CNN (R2CNN): Each proposal jointly predicts text/non-text scores, an axis-aligned box, and an inclined minimum-area box.The axis-aligned regression provides an additional constraint alongside the inclined-box prediction.
  • 3.2. Rotational Region CNN (R2CNN): Inclined NMS compares inclined-box IoU and avoids missing closely adjacent inclined texts when axis-aligned boxes overlap heavily.Figure 4 reports that normal NMS misses one such text, whereas inclined IoU remains low for the adjacent inclined boxes.
  • 3.3. Training objective (Multi-task loss): The proposal loss combines text/non-text classification with axis-aligned and inclined-box regression, balanced by λ1 and λ2.Box regression is conducted only on text proposals; the RPN loss remains the same as in Faster R-CNN.

4. Experiments

The method is trained on augmented ICDAR 2015 data together with collected focused-text images and evaluated on ICDAR 2015 and ICDAR 2013 using competition metrics.

  • Training Data: Training uses 1,000 ICDAR 2015 incidental-text images and 2,000 collected focused-text images.The collected images are described as clearer and different from ICDAR 2015's blurry texts, and are included for robustness to varied scene texts.
  • Training Data: Rotating training images through angles from −90° to 90° produces 39,000 augmented training images.The augmentation angles are spaced at 15° increments.
  • Training: The network is initialized with pretrained VGG16 weights and trained end-to-end for 20 × 10^4 iterations.Learning rates start at 10^−3 and are reduced by a factor of 10 after 5 × 10^4, 10 × 10^4, and 15 × 10^4 iterations.
  • Evaluation: Evaluation covers ICDAR 2015 and ICDAR 2013 using Precision, Recall, and F-measure from the ICDAR Robust Reading Competition.Results are obtained by submitting detections to the competition website for online evaluation.

A. ICDAR 2015

On ICDAR 2015, R2CNN evaluates design choices for arbitrary-oriented text detection and achieves competitive benchmark performance. Its gains come from inclined-box prediction, suitable anchors and NMS, while adding little single-scale detection time.

  • R2CNN achieves Recall 79.68%, Precision 85.62%, and F-measure 82.54% on ICDAR 2015.
  • Axis-aligned box and inclined box: Inclined-box regression improves F-measure from 56.63% with Faster R-CNN to 62.40%, while adding axis-aligned regression raises it to 68.49%.
  • Normal NMS on axis-aligned boxes vs. inclined NMS on inclined boxes: Inclined NMS consistently outperforms normal NMS across R2CNN-3, R2CNN-4 and R2CNN-5 under single-scale and multi-scale testing.
  • Anchor scales: Small anchors improve single-scale F-measure over R2CNN-2, reaching 72.94% with scales (4,8,16) and 72.93% with (4,8,16,32).
  • Anchor scales: Under multi-scale testing, adding the smaller anchor scale improves F-measure from 78.73% to 79.74%.
  • The method increases detection time only slightly relative to Faster R-CNN under single-scale testing and can be adapted to SSD and YOLO.

B. ICDAR 2013

On ICDAR 2013, R2CNN is evaluated for horizontal focused scene texts and reaches strong performance despite training data that excludes single characters. At a common 720-pixel test scale, it outperforms Faster R-CNN.

  • The ICDAR 2013 evaluation uses axis-aligned outputs because its texts are horizontal, although the model estimates both axis-aligned and inclined boxes.
  • R2CNN reaches an F-measure of 87.73% on the ICDAR 2013 test dataset.
  • The training data excludes single characters, which the ICDAR 2013 benchmark requires; the authors therefore expect better results with single-character training examples.
  • At a 720-pixel short-side test scale, R2CNN-720 achieves F-measure 83.16% versus 78.45% for Faster R-CNN.
  • Example results show that R2CNN detects horizontal focused scene texts, with a missed text identified as a single character.

5. Conclusion

R2CNN detects arbitrarily oriented scene text by extending Faster R-CNN with inclined-box prediction and inclined NMS. It achieves competitive results on ICDAR2015 and ICDAR2013 and can be adapted to other object-detection frameworks.

  • 5. Conclusion: R2CNN uses Faster R-CNN to generate axis-aligned proposals, estimate axis-aligned and inclined minimum-area boxes, and apply inclined NMS.The method performs multiple ROIPoolings with pooled sizes 7 × 7, 11 × 3, and 3 × 11 before prediction.
  • 5. Conclusion: Evaluation shows competitive results on the ICDAR2015 and ICDAR2013 text-detection benchmarks.
  • 5. Conclusion: The inclined-box prediction approach can be adapted to general object-detection frameworks such as SSD and YOLO.The paper describes the method as learning the inclined box from the axis-aligned box.
  • 5. Conclusion: Figure 6 distinguishes correct detections, false positives, and false negatives using green, red, and red dashed boxes, respectively.
Loading 1706.09579v2…