PaperlayerView exact revision on arXiv

Source-linked AI summary

SNIPER: Efficient Multi-Scale Training

Bharat Singh, Mahyar Najibi, Larry S. Davis

arXiv:1805.09300v3cs.CV

TL;DR

Multi-scale detector training processes costly image pyramids, raising whether full high-resolution images are necessary. SNIPER instead samples scale-appropriate chips around instances and proposals, achieving comparable or better accuracy while training three times faster.

  • Problem

    Multi-scale recognition methods process entire image pyramids, motivating whether smaller scale-appropriate regions could replace full high-resolution processing.

  • Method

    SNIPER adaptively samples positive and negative chips from multiple image-pyramid scales using ground-truth instances and region proposals.

  • Results

    3× faster training is achieved while SNIPER significantly outperforms SNIP across all reported metrics when batch normalization is used.

  • Takeaways & Limitations

    SNIPER brings instance-level recognition training closer to image-classification protocols by enabling large-batch training on a single GPU.

  • Takeaways & Limitations

    Comparisons with other detectors are difficult because backbones, pretraining data, network structures, and test-time augmentations differ.

Abstract

from arXiv · show

We present SNIPER, an algorithm for performing efficient multi-scale training in instance level visual recognition tasks. Instead of processing every pixel in an image pyramid, SNIPER processes context regions around ground-truth instances (referred to as chips) at the appropriate scale. For background sampling, these context-regions are generated using proposals extracted from a region proposal network trained with a short learning schedule. Hence, the number of chips generated per image during training adaptively changes based on the scene complexity. SNIPER only processes 30% more pixels compared to the commonly used single scale training at 800x1333 pixels on the COCO dataset. But, it also observes samples from extreme resolutions of the image pyramid, like 1400x2000 pixels. As SNIPER operates on resampled low resolution chips (512x512 pixels), it can have a batch size as large as 20 on a single GPU even with a ResNet-101 backbone. Therefore it can benefit from batch-normalization during training without the need for synchronizing batch-normalization statistics across GPUs. SNIPER brings training of instance level recognition tasks like object detection closer to the protocol for image classification and suggests that the commonly accepted guideline that it is important to train on high resolution images for instance level visual recognition tasks might not be correct. Our implementation based on Faster-RCNN with a ResNet-101 backbone obtains an mAP of 47.6% on the COCO dataset for bounding box detection and can process 5 images per second during inference with a single GPU. Code is available at https://github.com/MahyarNajibi/SNIPER/.

1 Introduction

Multi-scale instance recognition must balance computation, context, and negative mining because processing entire high-resolution image pyramids is costly. SNIPER addresses this trade-off by adaptively sampling smaller chips around relevant objects at appropriate resolutions.

  • Motivation: Human vision illustrates the challenge: attention is limited to a small field of view, and objects at different scales are difficult to process simultaneously.Computer instance-recognition algorithms must handle visual content across scales despite this limitation.
  • Related work: Prior work shows that ignoring gradients from objects at extreme resolutions can benefit multi-scale training.For a three-scale image pyramid, gradients from large objects at large resolutions and small objects at small resolutions should be ignored.
  • Motivation: The introduction questions whether full images must be processed at every scale, proposing smaller regions of interest around objects instead.It also asks whether upsampling already high-resolution images containing large objects provides useful benefit.
  • Motivation: SNIPER is motivated by the trade-off between computation, contextual information, and negative mining when accelerating multi-scale training.Smaller chips reduce computation but may omit recognition-relevant context and background.
  • Contribution: SNIPER is introduced as a training algorithm that adaptively samples chips for efficient multi-scale instance recognition.The approach is designed to preserve needed context while reducing unnecessary image processing.

2 Background

Object detection evolved from R-CNN’s proposal-wise canonical resizing toward methods that process image pyramids, but multi-scale training can waste computation by resampling every proposal or nearly all pixels. SNIPER instead samples scale-appropriate context chips and uses proposal-guided background chips to focus training on relevant regions.

  • Prior detectors: R-CNN [11] resized object proposals to canonical 224x224 images for CNN classification, but its training and inference cost scaled linearly with the number of proposals.The original proposals were generated by an unsupervised algorithm [34].
  • Prior detectors: Fast-RCNN’s multi-scale training resampled every proposal, whereas R-CNN resized only regions likely to contain objects to an appropriate resolution.This distinction motivates region-focused processing rather than resampling every image pixel.
  • Multi-scale training: SNIP still trained on almost all image-pyramid pixels while ignoring gradients from extreme-resolution objects, whereas SNIPER sampled 512-pixel crops from three pyramid scales.SNIP used 480- and 800-pixel resolutions plus multiple 1000-pixel crops from the 1400-pixel resolution.
  • SNIPER: SNIPER adaptively selects context chips around objects and adjusts upsampling or downsampling according to object size.The figure distinguishes covered objects from invalid objects at each scale.
  • SNIPER: For background regions, SNIPER samples up to two chips from likely false-positive areas identified by a lightweight RPN trained with a short schedule.The proposal mechanism could potentially be replaced by unsupervised proposals such as EdgeBoxes.

3 SNIPER

SNIPER trains detection networks on multi-scale 512x512 chips selected to cover ground-truth instances and likely false-positive regions rather than processing entire image pyramids. This adaptive sampling reduces high-resolution computation while retaining end-to-end Faster-RCNN-style training and improving batch packing.

  • Positive chip mining: SNIPER generates multi-scale chips on resized image canvases at 32-pixel intervals, assigning each ground-truth box to scales whose area ranges contain it and greedily covering valid boxes.Every ground-truth box is covered at an appropriate scale, while overlapping scale ranges can assign an instance to multiple scales or chips.
  • Negative chip mining: Negative chips are mined from RPN proposals not covered by positive chips, selecting regions that cover at least M proposals and randomly sampling a fixed number per image and epoch.The preliminary RPN is trained briefly without negative chips and only roughly guides selection toward likely false positives.
  • Label assignment: SNIPER trains end to end like Faster-RCNN, assigning labels and regression targets from all ground-truth boxes inside each chip while ignoring proposals outside the scale’s valid area range.Cropped ground-truth boxes remain in the chip, and proposals are trained against the ground truth present there.
  • Benefits of SNIPER: 30% more pixels are processed than 800x1333 single-scale training, while training uses approximately 5 512x512 chips per COCO image across three scales.Because chips share a fixed size, data is better packed for GPU utilization; high-resolution backgrounds are largely avoided because chips are more than 10x smaller than the image.
  • Benefits of SNIPER: On OpenImagesV4, SNIPER generates 3.5 million 512x512 chips from scales (512/ms, 1), with adaptive downsampling reducing the need for upsampling large objects.At smaller scales, chips cover more background, diminishing the impact of negative sampling.

4 Experimental Details

SNIPER is evaluated on COCO with a three-scale, chip-based training setup and Faster R-CNN detectors. Negative chip mining improves detection mAP, while the method retains efficient training and inference at high-resolution image-pyramid scales.

  • Training Setup: SNIPER is trained on COCO using 512×512 chips, batch size 128, learning rate 0.015, three scales, six epochs, and image flipping.The training scales are (512/ms, 1.667, 3), with desired area ranges (0,802), (322,1502), and (1202, inf).
  • Chip Generation: SNIPER generates 1.2 million chips after flipping, averaging about 5 chips per image and adapting from 1 to 10 chips with scene complexity.This adaptive sampling reduces imbalance in instance-level gradients relative to full-resolution training.
  • Proposal Generation: RPN recall is unchanged without negative sampling because recall ignores false positives and positive chips cover every ground-truth instance.Thus, positive-only training is sufficient for proposal generation, whereas negative mining remains important for detection mAP.
  • Negative Chip Mining: Negative chip mining improves average precision from 43.4 to 46.1 across localization thresholds and object sizes.The comparison is against SNIPER without negative chips, with all other parameters unchanged; negative mining reduces false positives.
  • Efficiency: End-to-end training takes 14 hours on 8 V100 GPUs, RPN pre-training takes 2 hours, and inference reaches about 5 images per second on one V100.Training uses three image-pyramid scales: maximum size 512, 1.667, and 3.
  • Results: 47.6% mAP is achieved with SNIPER while processing 5 images per second on a single V100 GPU.The result uses efficient batch inference and demonstrates practical high-resolution image-pyramid inference.

5 Related Work

SNIPER builds on prior findings about batch normalization, large-batch training, scale-aware sampling, and multi-scale context modeling. Its results additionally suggest that context may be less important for high-performance object-detector training than previously emphasized.

  • Related techniques: SNIPER benefits from batch-normalization statistics [22] and large batch sizes previously shown effective for object detection.These techniques are identified as important contributors to SNIPER’s improved performance.
  • Related techniques: Prior work studied context [38] [1], scale [5] [20], sampling [21] [2] [3], and architectures encoding multi-scale information [23].The passage frames these as established directions across visual recognition research.
  • Implications: SNIPER’s results suggest that context may not be very important for training high-performance object detectors.This conclusion contrasts with the considerable emphasis previously placed on context and multi-scale feature encoding for classification.

6 Conclusion and Future Work

SNIPER accelerates multi-scale training by 3× through low-resolution chip sampling without compromising detector performance, while enabling large single-GPU batches. Future work targets faster multi-scale inference and determining when chip context harms detection.

  • Conclusion: 3× acceleration is achieved by sampling low-resolution chips from a multi-scale image pyramid without compromising detector performance.Effective positive and negative chip sampling supports the efficiency gain.
  • Conclusion: Large batch sizes on a single GPU bring SNIPER closer to image-classification training protocols than conventional high-resolution instance-recognition training.SNIPER operates on re-sampled low-resolution chips, enabling this training configuration.
  • Future Work: Future work will accelerate multi-scale inference by eliminating background computation and evaluate when chip context begins to hurt detector performance.The authors note that a significant portion of the background may be removed without expensive computation.
Loading 1805.09300v3…