Source-linked AI summary
DSOD: Learning Deeply Supervised Object Detectors from Scratch
Zhiqiang Shen, Zhuang Liu, Jianguo Li, Yu-Gang Jiang, Yurong Chen, Xiangyang Xue
TL;DR
ImageNet pretraining can constrain detector design and introduce bias between classification and detection, especially across discrepant domains. DSOD trains a multi-scale detector from scratch using dense connections and deep supervision, achieving state-of-the-art or competitive accuracy with compact models across standard benchmarks. Its reported parameter counts are one-half of SSD, one-quarter of R-FCN, and one-tenth of Faster R-CNN.
Problem
ImageNet-pretrained detectors have limited structure design flexibility and learning bias from differences between classification and detection losses and category distributions, with larger mismatches across domains.
Method
DSOD combines dense layer-wise connections and deep supervision with a multi-scale proposal-free SSD-style detector trained from scratch.
Results
DSOD achieves competitive accuracy to SSD, Faster R-CNN, and R-FCN on PASCAL VOC 2007, VOC 2012, and MS COCO while using 1/2, 1/4, and 1/10 their parameters, respectively.
Takeaways & Limitations
DSOD provides a compact detector trained without ImageNet pretraining, with potential for domain-different inputs and resource-bounded devices.
Abstract
from arXiv · showhide
We present Deeply Supervised Object Detector (DSOD), a framework that can learn object detectors from scratch. State-of-the-art object objectors rely heavily on the off-the-shelf networks pre-trained on large-scale classification datasets like ImageNet, which incurs learning bias due to the difference on both the loss functions and the category distributions between classification and detection tasks. Model fine-tuning for the detection task could alleviate this bias to some extent but not fundamentally. Besides, transferring pre-trained models from classification to detection between discrepant domains is even more difficult (e.g. RGB to depth images). A better solution to tackle these two critical problems is to train object detectors from scratch, which motivates our proposed DSOD. Previous efforts in this direction mostly failed due to much more complicated loss functions and limited training data in object detection. In DSOD, we contribute a set of design principles for training object detectors from scratch. One of the key findings is that deep supervision, enabled by dense layer-wise connections, plays a critical role in learning a good detector. Combining with several other principles, we develop DSOD following the single-shot detection (SSD) framework. Experiments on PASCAL VOC 2007, 2012 and MS COCO datasets demonstrate that DSOD can achieve better results than the state-of-the-art solutions with much more compact models. For instance, DSOD outperforms SSD on all three benchmarks with real-time detection speed, while requires only 1/2 parameters to SSD and 1/10 parameters to Faster RCNN. Our code and models are available at: https://github.com/szq0214/DSOD .
1. Introduction
DSOD addresses limitations of ImageNet-pretrained detectors by training object detectors from scratch with design principles centered on deep supervision and dense connections. It achieves state-of-the-art performance across three benchmarks with real-time speed and compact models.
- Motivation: ImageNet-pretrained detectors have limited structure design flexibility and may suffer learning bias because classification and detection differ in losses and category distributions.Pretraining also creates difficulties when source and target domains differ substantially, such as RGB and depth images.
- Motivation: DSOD asks whether detectors can be trained from scratch while remaining accurate and resource efficient across computing platforms.The framework is intended to support server, desktop, mobile, and embedded devices.
- Design principles: Deep supervision through dense layer-wise connections is central to DSOD, extending dense structures to both the backbone and multi-scale prediction layers.Fusing and reusing multi-resolution prediction maps helps maintain or improve accuracy while reducing parameters.
- Contributions: DSOD introduces and validates design principles for efficient object detection networks trained from scratch through step-by-step ablation studies.The framework is presented as a first approach to achieve state-of-the-art performance without classification pretraining.
2. Related Work
Prior object detectors include region-proposal and proposal-free approaches, while specialized backbones generally still depend on ImageNet pretraining. DSOD instead targets object detection directly from scratch.
- Object Detection: CNN-based object detectors are divided into region-proposal methods and proposal-free methods.Region-proposal methods generate candidate regions before classification, whereas proposal-free methods directly predict object classes and locations.
- Object Detection: R-CNN, Fast R-CNN, Faster R-CNN, and R-FCN represent region-proposal approaches with progressively improved computation sharing and proposal generation.R-CNN processes each proposed region separately, while later methods share computation or improve speed.
- Object Detection: YOLO and SSD are proposal-free detectors designed for real-time detection, with SSD adding multi-scale prediction and default boxes to direct prediction.SSD uses small convolutional filters for categories and bounding-box offsets and pyramid features for different scales.
- Learning Deep Models from Scratch: Specialized detection backbones such as YOLO, YOLO9000, PVANet, and Inception-ResNet-v2 still require ImageNet classification pretraining.DSOD differs by removing that pretraining requirement.
3. DSOD
DSOD is a multi-scale, proposal-free detector derived from SSD, with a deeply supervised DenseNet-style backbone and dense prediction layers. Its design uses dense connections, resolution-preserving transitions, and a stem block to support scratch training.
- DSOD Architecture: DSOD consists of a feature-extraction backbone and a front-end subnet that predicts over multi-scale response maps.The backbone contains a stem block, four dense blocks, transition layers, and transition layers without pooling.
- Prediction Layers: DSOD uses dense prediction layers to concatenate multi-resolution feature maps, with down-sampling and channel reduction supporting these fusions.The down-sampling block uses 2×2 stride-2 pooling followed by a 1×1 convolution that reduces channels to 50%.
- Principle 1: Proposal-free: DSOD adopts a proposal-free framework derived from SSD because it combines high accuracy with fast processing speed.This is presented as the first design principle for training the detector from scratch.
- Principle 2: Deep Supervision: Dense layer-wise skip connections provide implicit deep supervision so earlier layers can share supervised signals from a single objective function.The paper validates the benefit of deep supervision experimentally.
- Transition w/o Pooling Layer: Transition layers without pooling allow additional dense blocks without reducing the final feature-map resolution.They remove the fixed relationship between dense-block count and output scale in the original DenseNet design.
- Training: All models are trained from scratch with SGD, while most training strategies follow SSD, including augmentation, default-box settings, and localization and classification losses.L2 normalization scales DSOD feature norms to 20 on all outputs.
4. Experiments
Experiments evaluate DSOD’s design principles and detection performance on PASCAL VOC and MS COCO. DSOD achieves strong accuracy from scratch while maintaining real-time inference and substantially smaller models.
- DSOD experiments cover PASCAL VOC 2007, VOC 2012, and MS COCO, using mAP as the detection metric.
- Design principles: 2.9% higher mAP is achieved with transition compression factor θ = 1 than with θ = 0.5 in the ablation study.The comparison evaluates feature-map reduction in DenseNet transition layers.
- Design principles: Deep supervision and the proposal-free SSD framework are central to successful training from scratch.Proposal-based Faster R-CNN and R-FCN training failed to converge, while scratch-trained SSD converged but achieved 69.6% mAP versus 75.8% with pre-training.
- Runtime and model size: 48.6ms per image is achieved with plain prediction and 57.5ms with dense prediction on one Titan X GPU.The corresponding speeds are 20.6 fps and 17.4 fps.
- Runtime and model size: 1/2 the parameters of SSD300 with VGGNet and 1/10 those of Faster R-CNN with VGGNet are used by DSOD.A 10.4M-parameter lite version runs at 25.8 fps with only a 1% mAP drop.
- PASCAL VOC 2007: 81.7% mAP is achieved on VOC 2007 after adding COCO training data, compared with 77.7% for dense prediction without COCO.Plain DSOD300 reaches 77.3%, slightly above SSD300* at 77.2%.
- PASCAL VOC 2012: 76.3% mAP is achieved on VOC 2012, exceeding SSD300* at 75.8%.
- MS COCO: 29.3%/47.3% is achieved on MS COCO test-dev, outperforming SSD300* and remaining comparable to single-scale R-FCN.The [0.5:0.95] result is better or comparable to R-FCN, while small-object precision is lower with the 300×300 input.
5. Discussion
The discussion frames DSOD as a response to the cost and transfer limitations of ImageNet-pretrained detectors. Its compact dense design targets efficient deployment while retaining detection performance.
- ImageNet-pretrained detectors constrain structure design because they inherit heavy classification networks and their computing requirements.
- Efficient models become increasingly important as growing datasets make training deep networks more expensive.
- Training from scratch is especially relevant when ImageNet and target domains differ substantially, including depth, multispectral, and medical images.
- 5.9M parameters and 73.6% mAP are achieved by the smallest dense DSOD model, indicating potential for low-end devices.
6. Conclusion
DSOD trains object detectors from scratch without ImageNet-pretrained models while achieving competitive accuracy on PASCAL VOC and MS COCO. It uses substantially fewer parameters than several established detectors.
- DSOD achieves competitive accuracy with SSD, Faster R-CNN, and R-FCN on PASCAL VOC 2007, 2012, and MS COCO.
- 1/2, 1/4 and 1/10 parameters compared to SSD, R-FCN and Faster R-CNN, respectively.
- DSOD has potential for domain-different scenarios including depth, medical, and multi-spectral images.