Light-Head R-CNN: In Defense of Two-Stage Object Detector

Zeming Li; Chao Peng; Gang Yu; Xiangyu Zhang; Yangdong Deng; Jian Sun

Source-linked AI summary

Light-Head R-CNN: In Defense of Two-Stage Object Detector

Zeming Li, Chao Peng, Gang Yu, Xiangyu Zhang, Yangdong Deng, Jian Sun

arXiv:1711.07264v2Nov 20, 2017cs.CV

TL;DR

The paper asks why typical two-stage detectors are slower than fast single-stage detectors and attributes the gap to computationally heavy heads. It proposes Light-Head R-CNN with thin feature maps and a cheap R-CNN subnet; with a small Xception-like backbone, it achieves 30.7 mmAP at 102 FPS on COCO while outperforming fast single-stage detectors.

Problem
Typical two-stage detectors use computationally heavy heads, limiting their speed advantage even when the backbone is reduced.
Source: §1 · p. 1
Method
Light-Head R-CNN uses thin feature maps and a cheap R-CNN subnet with pooling and a single fully connected layer.
Source: §1 · p. 2
Results
30.7 mmAP at 102 FPS on MS COCO is achieved with a small Xception-like base model, outperforming fast detectors such as YOLO and SSD.
Source: §5 · p. 7, §4.4 · p. 7
Takeaways & Limitations
The light-head design enables a two-stage detector to achieve superior performance with faster computational speed than fast single-stage detectors.
Source: §5 · p. 7
The ablation studies use ResNet-101 as the backbone to fairly compare with existing methods.
Source: §4.2 · p. 5

Abstract

from arXiv · show

In this paper, we first investigate why typical two-stage methods are not as fast as single-stage, fast detectors like YOLO and SSD. We find that Faster R-CNN and R-FCN perform an intensive computation after or before RoI warping. Faster R-CNN involves two fully connected layers for RoI recognition, while R-FCN produces a large score maps. Thus, the speed of these networks is slow due to the heavy-head design in the architecture. Even if we significantly reduce the base model, the computation cost cannot be largely decreased accordingly. We propose a new two-stage detector, Light-Head R-CNN, to address the shortcoming in current two-stage approaches. In our design, we make the head of network as light as possible, by using a thin feature map and a cheap R-CNN subnet (pooling and single fully-connected layer). Our ResNet-101 based light-head R-CNN outperforms state-of-art object detectors on COCO while keeping time efficiency. More importantly, simply replacing the backbone with a tiny network (e.g, Xception), our Light-Head R-CNN gets 30.7 mmAP at 102 FPS on COCO, significantly outperforming the single-stage, fast detectors like YOLO and SSD on both speed and accuracy. Code will be made publicly available.

1. Introduction

The paper argues that heavy heads make two-stage detectors slow, then proposes Light-Head R-CNN, which uses thin feature maps and a cheap R-CNN subnet to improve the speed–accuracy tradeoff.

Two-stage detectors divide detection into proposal generation and proposal recognition, typically favoring high accuracy over speed through heavy heads.Source: §1 · p. 1
Figure 1 compares Light-Head R-CNN using small Xception-like, ResNet-50, and ResNet-101 backbones under single-scale training.Source: Figure 1 · p. 1
Faster R-CNN uses large fully connected layers, while R-FCN generates large #classes × p × p score maps, making their heads costly.Source: §1 · pp. 1–2
Light-Head R-CNN produces thin feature maps with α × p × p channels, where α ≤10, and attaches a single cheap fully connected layer.Source: §1 · p. 2
Light-Head R-CNN significantly outperforms fast single-stage detectors such as SSD and YOLOv2 while using faster computational speed, and also outperforms state-of-the-art methods with ResNet-101.Source: §1 · p. 2

2. Related works

Related work frames object detection around accuracy and speed: both one-stage and two-stage methods achieve strong precision, but fast two-stage detectors remain underrepresented.

Recent object detectors are reviewed across accuracy and speed, following advances in deep convolutional networks.Source: §2 · p. 2

Accuracy perspective

R-CNN, Fast R-CNN, Faster R-CNN, and RetinaNet progressively develop proposal generation, multitask training, network-based proposals, and focal loss.Source: §2 · p. 2

Speed perspective

SPPnet, Fast/Faster R-CNN, and R-FCN improve speed by sharing computation across candidate boxes or RoI subnetworks.Source: §2 · p. 2

The paper identifies a lack of competitive fast two-stage detectors with accuracy comparable to single-stage approaches and introduces Light-Head R-CNN to address it.Source: §2 · p. 2

3. Our Approach

Light-Head R-CNN relocates efficiency into thin feature maps and a lightweight RoI subnet, addressing the computational costs of conventional two-stage heads.

3.1. Light-Head R-CNN

In this paper, the head is the structure attached to the backbone and consists of the R-CNN subnet and RoI warping.Source: §3.1 · p. 2

3.1.1 R-CNN subnet

Faster R-CNN concentrates computation in a powerful RoI classifier, whereas R-FCN shifts it to large shared score maps with #classes × p × p channels.Source: §3.1.1 · pp. 2–3
Light-Head R-CNN uses a simple fully connected layer and thin feature maps before RoI warping to reduce computation and memory costs.Source: §3.1.1 · p. 3, §3.1.2 · pp. 3–4

3.2. Light-Head R-CNN for Object Detection

Large separable convolution performs sequential k × 1 and 1 × k convolutions, with computational complexity controlled through Cmid and Cout.Source: Figure 3 · p. 4
The implementation employs a single 2048-channel fully connected layer in the R-CNN subnet, followed by sibling layers for classification and regression.Source: §3.2 · p. 4

4. Experiments

Experiments evaluate Light-Head R-CNN through baseline comparisons, ablations of thin feature maps and the R-CNN subnet, and COCO comparisons across large and small backbones. The results show that the design preserves or improves accuracy while reducing computational and memory costs.

Baselines

32.1% mmAP is achieved by the reproduced R-FCN baseline B1 on the COCO mini-validation set.B1 follows the publicly available R-FCN implementation details.Source: §4.2.1 · p. 5

Thin feature maps for RoI warping

3969 versus 490 feature-map channels yields comparable performance when reducing channels for PSRoI pooling.The thin-map configuration uses 490 channels, or 10 × 7 × 7, compared with the original R-FCN’s 3969 channels, or 81 × 7 × 7.Source: §4.2.2 · p. 5, Table 2 · p. 5
0.7 points of performance improvement comes from using a large-kernel separable convolution to enhance the thin feature map.The comparison is against the reproduced strong R-FCN baseline B2.Source: §4.2.2 · p. 6, Table 3 · p. 6

R-CNN subnet

37.7 mmAP is achieved by combining large-kernel feature maps with Light R-CNN, compared with 35.5/35.1 mmAP for Faster R-CNN and R-FCN.The method retains time efficiency even when thousands of proposals are used.Source: §4.2.3 · p. 6, Table 4 · p. 6

Light-Head R-CNN: High Accuracy

40.8% versus 39.1% mmAP is reported on COCO test-dev for the single-scale model against all listed competitors.The result is reported without additional bells and whistles and is presented as evidence for efficiency with large backbones.Source: §4.3 · p. 6

Light-Head R-CNN: High Speed

30.7 mmAP at 102 FPS is achieved on MS COCO with a tiny Xception-like backbone, outperforming fast detectors such as YOLO and SSD.The fast setting also uses reduced RPN channels, a thin feature map, and aligned PS pooling.Source: §4.4 · pp. 6–7

5. Conclusion

Light Head R-CNN presents a flexible light-head design for two-stage detection, achieving superior accuracy and speed with a tiny Xception-like backbone on COCO.

Light Head R-CNN introduces a better design principle for two-stage object detectors through a light head.Source: §5 · p. 7
Light R-CNN achieves superior performance on both accuracy and speed when paired with a tiny Xception* backbone.Source: Table 8 · p. 7
30.7 mmAP at 102 FPS demonstrates superior accuracy and speed with a small Xception-like base model on COCO.Source: §5 · p. 7