DSOD: Learning Deeply Supervised Object Detectors from Scratch

1. Introduction

• Pretrained classification networks limit detector design with heavy models, may bias learning because task losses and category distributions differ, and can mismatch target domains such as depth or medical images.Fine-tuning may mitigate category-distribution differences but does not resolve severe source-to-target domain mismatch.Source: §1 · pp. 1–2
• Dense layer-wise connections provide implicit deep supervision in both the backbone and multi-scale prediction layers, whose feature-map fusion and reuse help preserve or improve accuracy while reducing parameters.The prediction-layer comparison is shown for plain SSD and dense DSOD structures with 300×300 input.Source: Figure 1 · p. 2, §1 · p. 2
• DSOD aims to enable resource-efficient object detection from scratch while maintaining high accuracy, with structures adaptable to servers, desktops, mobile devices, and embedded platforms.The framework is motivated by whether detectors can be trained from scratch and how to design efficient structures without sacrificing accuracy.Source: §1 · p. 2
• The paper introduces and validates efficient detector-design principles through step-by-step ablation studies.Deep supervision is identified as a key principle, motivated by prior work [18] [35] and implemented through dense connections [10].Source: §1 · p. 2
• DSOD is presented as the first framework to train object detectors from scratch with state-of-the-art performance, achieving state-of-the-art results on three benchmarks with real-time speed and compact models.The cited benchmarks are PASCAL VOC 2007, PASCAL VOC 2012, and MS COCO.Source: §1 · p. 2

Object Detection

• Proposal-based detectors progress from R-CNN [5]’s costly selective-search regions to shared-computation Fast R-CNN and Faster R-CNN [27], then R-FCN ’s position-sensitive maps for speed and accuracy.R-CNN separately processes each proposed region with a CNN; Fast and Faster R-CNN share computation and use neural networks to generate proposals.Source: §2 · p. 2
• Proposal-free YOLO [25] and SSD [21] enable real-time detection; DSOD builds on SSD’s speed and accuracy advantages while producing smaller, more flexible models.SSD uses small filters, pyramid features, default boxes, and aspect ratios to predict categories and box locations at multiple scales.Source: §2 · p. 3

Network Architectures for Detection

• Detection methods commonly use ImageNet-pretrained classification backbones; YOLO [25], YOLO9000 [26], and PVANet [15] still require pretraining, and Faster R-CNN [27] with Inception-ResNet-v2 [31] achieved the highest performance [11].DSOD considers network structure for generic object detection but does not require ImageNet pretraining.Source: §2 · p. 3

Learning Deep Models from Scratch

• No prior object-detection networks were known to train from scratch; DSOD removes ImageNet pretraining, while segmentation work showed a designed DenseNet-based fully convolutional network could outperform state of the art without pretraining.Jégou et al. [13] added an upsampling path to recover full input resolution.Source: §2 · p. 3

3.1. DSOD Architecture

• Dense layer-wise connections provide deep supervision by carrying a single loss signal through skip connections to earlier layers, without auxiliary losses at each layer.The backbone is a variant of deeply supervised DenseNets, with a stem, four dense blocks, two transition layers, and two transitions without pooling.Source: §3.1 · pp. 3–4
• DSOD uses an SSD-like, multi-scale proposal-free detector; the authors report that only proposal-free methods converged successfully when trained from scratch.They conjecture that RoI pooling in proposal-based methods impedes gradient propagation from region-level outputs to convolutional feature maps.Source: §3.1 · p. 4
• Transitions without pooling let DSOD add dense blocks without reducing final feature-map resolution, removing DenseNet’s restriction on block count at a fixed output scale.The passage also notes that this transition can be used in standard DenseNet.Source: §3.1 · p. 4
• The stem block uses three 3×3 convolutions followed by 2×2 max pooling; the authors report improved detection performance and conjecture it reduces raw-image information loss.The first convolution has stride 2, while the other two have stride 1.Source: §3.1 · p. 4
• DSOD’s dense prediction structure fuses multi-scale features at each scale, learning half of its feature maps and reusing the other half from preceding scales.A pooling-plus-1×1-convolution block downsamples and reduces the reused maps; the authors state this structure can improve accuracy with fewer parameters than the plain structure.Source: §3.1 · pp. 4–5

3.2. Training Settings

• Models are trained from scratch with Caffe and SGD on an NVidia TitanX; DSOD applies L2 feature normalization to 20 on all output scales.Most training strategies follow SSD, including augmentation, default-box settings, and localization and classification losses; DSOD uses its own learning-rate schedule and mini-batch size.Source: §3.2 · p. 5

4.1. Ablation Study on PASCAL VOC2007

• The ablation study uses DSOD300 with 300×300 inputs, trains on VOC 2007+2012 trainval, and tests on VOC 2007 test.The experiments use a consistent setting except when examining components or structures.Source: §4.1 · p. 5

4.1.1 Configurations in Dense Blocks

• θ = 1 yields 2.9% higher mAP than θ = 0.5; wider bottlenecks improve mAP by 4.1%, more first-layer channels by 1.1%, and k = 48 versus 16 by 4.8%.The growth-rate comparison uses 4k bottleneck channels.Source: §4.1.1 · p. 5, Table 3 · p. 6

4.2. Results on PASCAL VOC2007

• 81.7% after adding COCO training data, versus 77.7% for DSOD300 with dense prediction and 77.2% for SSD300* on VOC 2007 test.DSOD300 with plain connections achieves 77.3%.Source: Table 4 · p. 6, §4.2 · p. 7

4.1.2 Effectiveness of Design Principles

• 77.7% mAP from DSOD300 trained from scratch exceeds SSD300S at 69.6% and fine-tuned SSD300 at 75.8%, supporting deep supervision; proposal-based Faster R-CNN and R-FCN failed to converge from scratch.The 69.6% SSD300S result uses VGG; the passage identifies the fine-tuned comparison as SSD300.Source: §4.1.2 · p. 5, Table 4 · p. 6
• The transition without pooling adds 1.7% performance gain, the stem block raises performance from 74.5% to 77.3%, and dense prediction raises mAP from 77.3% to 77.7% while reducing parameters from 18.2M to 14.8M.Dense prediction runs at 17.4 fps versus 20.6 fps for the plain structure; replacing SSD prediction layers also improves results from 75.8% to 76.1% with pre-training and 69.6% to 70.4% without it.Source: §4.1.2 · p. 6, Table 3 · p. 6, Table 4 · p. 6

4.1.3 Runtime Analysis

• 20.6 fps with plain prediction and 17.4 fps with dense prediction; DSOD uses half the parameters of SSD300 with VGGNet and one tenth those of Faster R-CNN with VGGNet.The lite DSOD has 10.4M parameters and runs at 25.8 fps with a 1% mAP drop; on Titan X, R-FCN ResNet-50/101 runs at 11/9 fps, while SSD300* runs at 12.1 fps with ResNet-101 and 46 fps with VGGNet.Source: §4.1.3 · pp. 6–7

4.3. Results on PASCAL VOC2012

• In the VOC 2007 experiments, Table 4 reports DSOD300 at 76.3% mAP, consistently better than SSD300* at 75.8%.Source: §4.3 · p. 7

4.4. Results on MS COCO

• 29.3%/47.3% on COCO test-dev outperforms SSD300* by a large margin and is comparable to single-scale R-FCN.DSOD is lower than R-FCN at 0.5 IoU but better or comparable at [0.5:0.95]; small-object precision is slightly lower, while large-object precision is much better, despite DSOD's 300×300 input versus R-FCN's ∼600×1000.Source: Table 6 · p. 7, §4.4 · p. 7

Better Model Structure vs. More Training Data

• DSOD trained on 16,551 VOC 2007 images achieves competitive or better performance than models trained on 1.2 million + 16,551 images, supporting better structure over more data.The discussion contrasts DSOD’s training data with models also trained using ImageNet data.Source: §5 · p. 8
• As datasets grow, training deep networks becomes increasingly expensive, making simple, efficient approaches more important; DSOD is presented as promising under this setting.Source: §5 · p. 8

Why Training from Scratch?

• Training from scratch matters when pretrained and target domains differ, including depth, multispectral, or medical images, and when fine-tuning restricts detector design for resource-limited IoT deployment.The discussion notes that most pretrained models use large-scale RGB ImageNet data and that fine-tuning restricts the structure-design space for detection networks.Source: §5 · p. 8

Model Compactness vs. Performance

• 73.6% mAP with 5.9M parameters: DS/64-64-16-1, the smallest dense model with dense prediction layers, shows potential for low-end devices.The paper identifies mobile phones and embedded electronics as examples of low-end devices for which large detection models may be unsuitable.Source: §5 · p. 8

• Without ImageNet pretraining, DSOD achieves competitive accuracy to SSD, Faster R-CNN, and R-FCN on PASCAL VOC 2007/2012 and MS COCO, using 1/2 SSD, 1/4 R-FCN, and 1/10 Faster R-CNN parameters.Source: §6 · pp. 8–9
• DSOD may suit depth, medical, and multispectral imagery; future work will consider these domains and ultra-efficient models for resource-bounded devices.Source: §6 · p. 9