Source-linked AI summary
Bag of Freebies for Training Object Detection Neural Networks
Zhi Zhang, Tong He, Hang Zhang, Zhongyue Zhang, Junyuan Xie, Mu Li
TL;DR
Object detection models use complex, divergent training pipelines, making it difficult to transfer effective heuristics across detector families. The paper evaluates architecture-preserving training tweaks, including visually coherent mixup and revised schedules, across YOLOv3 and Faster-RCNN. The resulting bag of freebies improves detection performance while adding no inference overhead, reaching up to 5% absolute precision improvement over baseline.
Problem
Object detection pipelines vary across models, limiting the ability of different detection methods to benefit from shared training advances.
Method
The paper systematically evaluates training tweaks, including visually coherent object-detection mixup, learning-rate scheduling, label smoothing, and synchronized BatchNorm, while keeping architectures unchanged.
Results
Up to 5% absolute precision improvement is achieved without modifying network architectures or increasing inference cost.
Takeaways & Limitations
Training-time bag-of-freebies modifications consistently improve YOLOv3 and Faster-RCNN on Pascal VOC and COCO without increasing inference time or changing network structures.
Abstract
from arXiv · showhide
Training heuristics greatly improve various image classification model accuracies~\cite{he2018bag}. Object detection models, however, have more complex neural network structures and optimization targets. The training strategies and pipelines dramatically vary among different models. In this works, we explore training tweaks that apply to various models including Faster R-CNN and YOLOv3. These tweaks do not change the model architectures, therefore, the inference costs remain the same. Our empirical results demonstrate that, however, these freebies can improve up to 5% absolute precision compared to state-of-the-art baselines.
1. Introduction
Object detection training pipelines vary substantially across detector families, limiting transfer of useful advances. The paper systematically evaluates architecture-preserving training tweaks, including visually coherent mixup and schedule changes, reporting improved performance without added inference cost.
- Different object detection methods use distinct preprocessing and training pipelines, making it difficult to benefit from advances across methods.
- The paper explores general training approaches for popular detectors without introducing extra inference computation.
- The approach includes visually coherent image mixup, learning-rate scheduling, label smoothing, and synchronized BatchNorm, stacked incrementally across single- and multiple-stage detectors.
- The work systematically evaluates training heuristics across object detection pipelines and presents practice guidelines for future research.
- Visually coherent image mixup is designed for object detection and empirically improves model generalization.
- Up to 5% absolute precision improvement, or 15 to 20% better than baseline, is achieved without modifying network architectures or increasing inference cost.
2. Related Work
Object detection research spans single- and multiple-stage pipelines whose architectural differences produce divergent data-processing and optimization practices. This work situates training heuristics from image classification and systematically examines tweaks that may benefit both detector families.
- 2.1. Scattering tricks from Image Classification: Image classification underpins major computer vision tasks and has motivated transferable training heuristics such as warmup, label smoothing, mixup, and cosine annealing.
- 2.2. Deep Object Detection Pipelines: Single-stage detectors generate predictions through one convolutional network, whereas multiple-stage detectors use sampled and pooled region-of-interest features.
- 2.2. Deep Object Detection Pipelines: These pipeline differences create divergent data processing and optimization practices, with spatial augmentation especially important in single-stage detection.
- 2.2. Deep Object Detection Pipelines: The paper systematically explores mutually beneficial training tweaks that may improve both single- and multiple-stage detection pipelines.
3. Bag of Freebies
The paper develops visually coherent mixup and training-pipeline adjustments for object detection, preserving spatial alignment and improving robustness and validation accuracy. It also adapts label smoothing, augmentation choices, and learning-rate scheduling to detector-specific pipelines.
- Visually Coherent Image Mixup for Object Detection: Geometry-preserved mixup blends image pixels while merging object labels, maintaining spatial coherence for object detection.The method avoids distortions during initial mixing steps and uses beta distributions with α and β at least 1.
- Visually Coherent Image Mixup for Object Detection: B(1.5, 1.5) sampling is marginally better than B(1, 1) and better than fixed even mixup in YOLOv3 on Pascal VOC.The comparison evaluates empirical mixup-ratio distributions for object detection.
- Visually Coherent Image Mixup for Object Detection: 94.12 vs. 42.95 recall demonstrates greater robustness to an elephant placed in an indoor scene for the visually coherent mixup model.The mixup model also preserves crowded furniture detections under heavy occlusion, although it produces lower average object scores.
- Classification Head Label Smoothing: Label smoothing replaces one-hot targets with softened distributions, reducing prediction confidence by narrowing the difference between largest and smallest logits.For YOLOv3 sigmoid outputs, smoothing corrects the upper and lower target limits.
- Data Preprocessing: Faster R-CNN omits random cropping because feature-map sampling already performs extensive cropping operations.This distinguishes its data-augmentation pipeline from one-stage detectors that preserve spatial alignments.
- Training Schedule Revamping: With warmup enabled, cosine learning-rate decay outperforms step decay in validation mAP throughout YOLOv3 training and suffers less plateauing.Step schedules make sharp transitions, whereas cosine adjustment changes the learning rate smoothly.
4. Experiments
Experiments evaluate bags of freebies on representative single-stage YOLOv3 and multi-stage Faster-RCNN pipelines using Pascal VOC and MS COCO. Incrementally stacking training tweaks improves both models, with gains varying by architecture and evaluation resolution.
- Experimental setup: YOLOv3 experiments use 416 × 416 evaluation on Pascal VOC, while Faster-RCNN uses input resolutions capped at 600 × 1000.The Pascal VOC setup uses specified training and validation preprocessing for each representative detector.
- Pascal VOC results: Data augmentation contributes nearly 16% to YOLOv3 baseline mAP, while stacked training tricks raise validation performance by 3.43% to 83.68% single-model single-scale mAP.The stack includes Synchronized BatchNorm, Random Training, cosine learning-rate scheduling, Sigmoid label smoothing, and detection mixup.
- Pascal VOC results: Faster-RCNN loses only 0.16% mAP without data augmentation, then gains 3.55% mAP from the proposed training tricks.The results are attributed to sampling-based proposals substituting for random cropping in this pipeline.
- MS COCO results: On MS COCO, BoF improves Faster-RCNN by 1.1% and 1.7% absolute mean AP with ResNet-50 and ResNet-101, respectively.The evaluation uses the COCO 2017 validation set and larger Faster-RCNN input resizing for smaller objects.
- MS COCO results: On MS COCO, YOLOv3 exceeds its baseline by 4.0% absolute mAP at 608 × 608 and 5.4% at lower resolutions.The same weights support multiple evaluation resolutions, with larger improvements at lower resolution levels.
- MS COCO results: Per-category AP analysis shows that most of the 80 categories benefit from BoF training tricks, despite mean AP not revealing category-level variation.Figures 7 and 8 encode gains in red and drops in blue for YOLOv3 and Faster-RCNN.
5. Conclusion
The paper concludes that stacked training enhancements improve YOLOv3 and Faster-RCNN on Pascal VOC and COCO without changing network structures or increasing inference time. These training-time modifications affect model weights while retaining inference compatibility.
- Conclusion: The proposed bag of training enhancements consistently improves YOLOv3 and Faster-RCNN on Pascal VOC and COCO.The conclusion reports no signs of degradation when the tweaks are stacked.
- Conclusion: The freebies modify training only, affecting model weights without changing network structures or increasing inference time.The paper characterizes this as zero overhead in the inference environment.
- Conclusion: The authors suggest wider adoption of these training tweaks in future object detection pipelines.Existing and future work is stated to be included in the open-source GluonCV repository.