PaperlayerView exact revision on arXiv

Source-linked AI summary

ScratchDet: Training Single-Shot Object Detectors from Scratch

Rui Zhu, Shifeng Zhang, Xiaobo Wang, Longyin Wen, Hailin Shi, Liefeng Bo, Tao Mei

arXiv:1810.08425v4cs.CV

TL;DR

Pretrained detectors face translation-sensitivity and architecture-flexibility trade-offs, while scratch-trained detectors have faced convergence and performance challenges. ScratchDet uses BatchNorm and Root-ResNet to train detectors from scratch; it reports best-published results on three datasets and outperforms some one-stage pretrained methods. Training from scratch takes longer than fine-tuning in the reported COCO timing comparison.

  • Problem

    Fine-tuning ImageNet classifiers can constrain detector architecture and mismatch classification’s translation invariance with detection’s need for local texture, while scratch-trained detectors have faced convergence and performance challenges.

    Source: §1 · p. 1

  • Method

    ScratchDet integrates BatchNorm into SSD’s backbone and detection head, then redesigns the backbone as Root-ResNet to retain image information for detection.

    Source: §1 · p. 2, §3.1 · p. 2, §3.2 · p. 4

  • Results

    ScratchDet reports the best published performance on VOC 2007 and 2012 (86.3% mAP each) and COCO test-dev (39.1% AP), and outperforms some one-stage pretrained methods.

    Source: §5 · p. 8, §6 · p. 11

  • Takeaways & Limitations

    The authors conclude ScratchDet sets new state-of-the-art results among scratch-trained detectors on VOC 2007, VOC 2012, and MS COCO, outperforming some one-stage pretrained methods.

    Source: §5 · p. 8

    On the reported COCO setup, ScratchDet training takes 84.6 hours versus 29.7 hours for fine-tuning a pretrained classifier, though the comparison excludes weeks and millions of images used for pretraining.

    Source: §4.6 · p. 8

Abstract

from arXiv · show

Current state-of-the-art object objectors are fine-tuned from the off-the-shelf networks pretrained on large-scale classification dataset ImageNet, which incurs some additional problems: 1) The classification and detection have different degrees of sensitivity to translation, resulting in the learning objective bias; 2) The architecture is limited by the classification network, leading to the inconvenience of modification. To cope with these problems, training detectors from scratch is a feasible solution. However, the detectors trained from scratch generally perform worse than the pretrained ones, even suffer from the convergence issue in training. In this paper, we explore to train object detectors from scratch robustly. By analysing the previous work on optimization landscape, we find that one of the overlooked points in current trained-from-scratch detector is the BatchNorm. Resorting to the stable and predictable gradient brought by BatchNorm, detectors can be trained from scratch stably while keeping the favourable performance independent to the network architecture. Taking this advantage, we are able to explore various types of networks for object detection, without suffering from the poor convergence. By extensive experiments and analyses on downsampling factor, we propose the Root-ResNet backbone network, which makes full use of the information from original images. Our ScratchDet achieves the state-of-the-art accuracy on PASCAL VOC 2007, 2012 and MS COCO among all the train-from-scratch detectors and even performs better than several one-stage pretrained methods. Codes will be made publicly available at https://github.com/KimSoybean/ScratchDet.

1. Introduction

The paper argues that ImageNet-pretrained detectors carry translation-sensitivity and architecture-flexibility trade-offs, while scratch-trained detectors have struggled with convergence and performance. ScratchDet uses BatchNorm and a redesigned Root-ResNet to train across architectures and reports strong results on three detection datasets.

1. Introduction

  • Fine-tuning can mismatch classification’s preference for translation invariance with detection’s reliance on local texture, and makes architecture changes costly.A new architecture may require repeating large-scale pretraining.Source: §1 · p. 1
  • Scratch training removes those fine-tuning limitations, but DSOD’s DenseNet dependence and weaker performance leave architecture flexibility, convergence, and accuracy as open goals.The paper identifies matching pretrained performance while supporting other architectures as objectives.Source: §1 · p. 1
  • BatchNorm is proposed as a way to smooth the optimization landscape, while the paper treats poor convergence as a central scratch-training concern.The authors cite prior work on BatchNorm’s effect on optimization.Source: §1 · p. 1
  • ScratchDet integrates BatchNorm to support convergence across network types, then introduces Root-ResNet to improve accuracy, especially for small objects.The paper reports state-of-the-art results among scratch-trained detectors on VOC 2007, VOC 2012, and COCO.Source: §1 · p. 2

2. Related Work

Related work covers pretrained two-stage and one-stage detectors, scratch-trained detectors, and BatchNorm’s role in optimization. The paper positions its work against scratch methods that rely on DenseNet deep supervision and overlook BatchNorm and architecture flexibility.

2. Related Work

  • Two-stage detectors generate object proposals before predicting regions and labels, while one-stage methods predict locations, sizes, and labels simultaneously.The cited two-stage progression includes Faster R-CNN, R-FCN, FPN, and Mask R-CNN; one-stage examples include OverFeat, YOLO, and SSD.Source: §2 · p. 2
  • DSOD introduced scratch-trained one-stage detection, and GRP-DSOD added a Gated Recurrent Feature Pyramid.The paper says these methods focus on DenseNet deep supervision, with less attention to BatchNorm and architectural flexibility.Source: §2 · p. 2
  • BatchNorm is described as enabling larger learning rates, while cited work attributes its optimization benefit to a smoother landscape rather than reduced internal covariate shift..Source: §2 · p. 2

3. ScratchDet

ScratchDet studies BatchNorm as a means to stabilize SSD training from scratch, then analyzes backbone design and proposes Root-ResNet. Its design retains early image information for detection while using residual blocks for multiscale features.

3.1. BatchNorm for train-from-scratch

  • SSD combines a backbone with prediction blocks after detection layers; the paper adds BatchNorm to both subnetworks to support scratch training.The original SSD framework has no BatchNorm, and the baseline reaches 67.6% mAP on VOC 2007 test.Source: §3.1 · p. 2
  • BatchNorm in SSD’s detection head smooths the loss landscape and supports larger learning rates, with mAP rising from 67.6% to 71.0% and then to 75.6%.The paper associates the larger learning rate with faster convergence and more stable gradients.Source: §3.1 · p. 3, Figure 1 · p. 3
  • 78.7% mAP with BatchNorm in both backbone and head and a 0.05 base learning rate exceeds pretrained VGG-16 SSD’s 77.2%.The reported margin is 1.5% mAP.Source: §3.1 · p. 3

3.2. Backbone Network

  • Root-ResNet removes first-layer downsampling, replaces the 7 × 7 convolution with stacked 3 × 3 filters, and uses residual blocks for multiscale features.The authors connect retaining local image information with stronger small-object features and report parameter and computation efficiency without performance dropout.Source: §3.2 · pp. 3–4

4. Experiment

The experiments evaluate ScratchDet on PASCAL VOC and MS COCO, using SSD-like settings and fixed detection-layer sizes. Backbone analyses use Root-ResNet-18, while state-of-the-art comparisons use the deeper Root-ResNet-34.

4. Experiment

  • Experiments cover PASCAL VOC and MS COCO, with 20 and 80 object classes, respectively.The implementation uses Caffe; the authors say code and trained models will be made publicly available.Source: §4 · p. 4

4.1. Training details

  • ScratchDet uses 300 × 300 inputs and the same detection-layer spatial sizes as SSD300 and DSOD300, without larger detection feature maps.Training uses batch size 128 for scratch models to stabilize BatchNorm statistics; pretrained SSD uses batch size 32.Source: §4.1 · pp. 4–5
  • Root-ResNet-18 is used for model analysis, while Root-ResNet-34 is used for state-of-the-art comparisons.The paper states that the deeper backbone is selected for better performance and the shallower one to account for analysis compute cost.Source: §4.1 · pp. 4–5
  • For scratch SSD on VOC 2007 test using truncated VGG-16, the best performance was 78.7% mAP with BatchNorm in backbone and head, no pretraining, and a larger learning rate.“NAN” indicates that training is non-convergent.Source: Table 1 · p. 5

4.2. PASCAL VOC 2007

On VOC 2007, the section tests BatchNorm and backbone changes for SSD trained from scratch, then compares ScratchDet with other detectors. The results show gains from BatchNorm, reduced first-layer downsampling, and a redesigned root block.

4.2.1 Analysis of BatchNorm

  • 78.7% mAP: SSD trained from scratch with BatchNorm on every convolution layer exceeds pretrained SSD's 77.2% on VOC 2007.Backbone-only BatchNorm reaches 78.0% versus 67.6%; head-only BatchNorm reaches 71.0%, and removing BatchNorm from DSOD drops mAP from 77.7% to 71.5%.Source: §4.2.1 · p. 5

4.2.2 Analysis of the backbone subnetwork.

  • First-layer 3×3 and 7×7 kernels yield 73.2% and 73.1% mAP, respectively, with the smaller kernel slightly faster.The passage reports the same conclusion when the first-layer stride is set to 1 without downsampling.Source: §4.2.2 · p. 6
  • Removing first-layer downsampling raises ResNet-18 SSD from 73.1% to 77.6% mAP; removing only the second downsampling operation reaches 75.3%.The passage identifies first-layer downsampling as an obstacle, particularly for small-object detection.Source: §4.2.2 · p. 6, Table 2 · p. 6
  • Three 3×3 root-block layers raise mAP from 77.8% to 78.5%; Root-ResNet-34 then improves ScratchDet from 78.5% to 80.4%.The authors use three root-block layers for the 300×300 detection experiments and use Root-ResNet-34 in benchmark comparisons.Source: §4.2.2 · p. 6

4.2.3 Results

  • 80.4% mAP: ScratchDet300 beats RefineDet320 (80.0%), DES300 (79.7%), and GRP-DSOD (78.7%); ScratchDet300+ reaches 84.1% in multi-scale testing.The reported 300×300 comparison also exceeds SSD300-VGG16 (77.2%) and SSD321-ResNet101 (77.1%).Source: §4.2.3 · p. 6

4.3. PASCAL VOC 2012

ScratchDet reaches 78.5% mAP on PASCAL VOC 2012 at 300 × 300 input size, outperforming several methods with similar input sizes. Multi-scale testing raises its score to 83.6% mAP.

4.3. PASCAL VOC 2012

  • ScratchDet is trained on VOC 2012 trainval plus VOC 2007 trainval and test sets, then evaluated on VOC 2012 test.The training sets contain 21,503 images, and the test set contains 10,991 images.Source: §4.3 · p. 6
  • 78.5% mAP surpasses SSD321-ResNet101 (75.4%, 3.1% higher mAP) at similar input size.Table 3 reports detection results on PASCAL VOC 2012.Source: §4.3 · p. 7, Table 3 · p. 7
  • 83.6% mAP in multi-scale testing is reported as much better than the state-of-the-art one-stage and two-stage methods.ScratchDet also outperforms DSOD by 2.2% mAP with 60 fewer training epochs and GRP-DSOD by 1.5% mAP.Source: §4.3 · p. 7

4.4. MS COCO

On MS COCO, ScratchDet achieves strong results at 300 × 300 input size, including for small objects. The paper also reports strong VOC results after training with MS COCO and PASCAL VOC data.

4.4. MS COCO

  • On MS COCO test-dev, ScratchDet achieves 32.7% AP, 7.6% higher than SSD300’s 25.1% AP, and is reported better than other methods with similar input size.Source: §4.4 · p. 7
  • 39.1% AP in multi-scale testing is reported as the best result, surpassing prominent one-stage and two-stage approaches with larger input images.ScratchDet300's single-scale result is reported as better than other methods with similar input size.Source: §4.4 · p. 7, Table 4 · p. 8
  • 13.0% APS for small objects outperforms SSD321 by 6.8% on methods with similar input size.Source: §4.4 · p. 7

4.5. From MS COCO to PASCAL VOC

  • 84.0% and 82.1% mAP are reported on VOC 2007 and 2012 test sets after MS COCO pretraining and VOC fine-tuning.Multi-scale testing raises both reported VOC scores to 86.3% mAP; the paper reports top scores using MS COCO and PASCAL VOC training data.Source: §4.5 · pp. 7–8, Table 5 · p. 8

4.6. Comparison of the training time

On the stated four-GPU workstation and 300 × 300 MS COCO setting, training ScratchDet from scratch takes longer than fine-tuning a pretrained classifier. The authors note that the comparison uses the mmdetection framework.

4.6. Comparison of the training time

  • 84.6 vs. 29.7 hours: ScratchDet training from scratch takes longer than fine-tuning a pretrained classifier on four NVIDIA Tesla P40 GPUs with 300 × 300 MS COCO input.The authors state that the comparison is based on the mmdetection framework.Source: §4.6 · p. 8
  • The authors consider scratch training more attractive when accounting for the several weeks and millions of images involved in pretraining.Source: §4.6 · p. 8

5. Conclusion

The paper presents ScratchDet, a detector trained from scratch with BatchNorm, and Root-ResNet, a backbone intended to improve detection accuracy, especially for small objects. ScratchDet performs strongly on PASCAL VOC and MS COCO among scratch-trained detectors and exceeds some one-stage pretrained methods.

5. Conclusion

  • ScratchDet integrates BatchNorm and trains from scratch across network types, while Root-ResNet is proposed to improve accuracy, especially for small objects.Source: §5 · p. 8

6. Complete Object Detection Results

ScratchDet achieves the best reported results among published methods on PASCAL VOC 2007, PASCAL VOC 2012, and MS COCO, with qualitative examples across all three datasets.

6. Complete Object Detection Results

  • 86.3% mAP on PASCAL VOC 2007, 86.3% mAP on PASCAL VOC 2012, and 39.1% AP on MS COCO are reported as the best results among published methods.The corresponding results are presented in Tables 6, 7, and 8, respectively.Source: Table 6 · p. 11, Table 7 · p. 11, Table 8 · p. 11
  • ScratchDet300’s VOC 2007 and VOC 2012 qualitative-result figures correspond to 84.0% and 82.1% mAP, respectively.The figures use training data 07+12+COCO and 07++12+COCO, respectively.Source: Figure 4 · p. 11, Figure 5 · p. 12
  • ScratchDet300’s MS COCO test-dev qualitative-result figure corresponds to 32.7% mAP using COCO trainval35k.The paper also reports that its detection examples work well with occlusions, truncations, inter-class interference, and clustered backgrounds.Source: §6 · p. 11, Figure 6 · p. 12
Loading 1810.08425v4…