Source-linked AI summary
Scalable Object Detection using Deep Neural Networks
Dumitru Erhan, Christian Szegedy, Alexander Toshev, Dragomir Anguelov
TL;DR
Object detection must handle exhaustive search and multiple instances without scaling poorly across classes. DeepMultiBox predicts class-agnostic boxes with confidence scores, trains them using assignment-based losses, and achieves competitive benchmark performance using few locations and evaluations.
Problem
Exhaustive location-and-scale search and class-specific detectors create computational and scaling challenges, while single-box outputs cannot naturally handle multiple instances of one class.
Method
DeepMultiBox uses a single DNN to predict class-agnostic bounding boxes and confidences, training predictions through ground-truth assignment, prior matching, and backpropagation.
Results
Competitive results were obtained on VOC2007 and ILSVRC-2012 using only a few predicted locations; VOC2007 mean AP was 0.29 using the top 10 boxes.
Takeaways & Limitations
The approach scales across object classes, generalizes to unseen categories, and captures multiple instances of objects of the same class.
Takeaways & Limitations
The current system requires two passes, totaling 5-10 network evaluations at roughly 1 CPU-sec each, although this cost does not scale linearly with class count.
Abstract
from arXiv · showhide
Deep convolutional neural networks have recently achieved state-of-the-art performance on a number of image recognition benchmarks, including the ImageNet Large-Scale Visual Recognition Challenge (ILSVRC-2012). The winning model on the localization sub-task was a network that predicts a single bounding box and a confidence score for each object category in the image. Such a model captures the whole-image context around the objects but cannot handle multiple instances of the same object in the image without naively replicating the number of outputs for each instance. In this work, we propose a saliency-inspired neural network model for detection, which predicts a set of class-agnostic bounding boxes along with a single score for each box, corresponding to its likelihood of containing any object of interest. The model naturally handles a variable number of instances for each class and allows for cross-class generalization at the highest levels of the network. We are able to obtain competitive recognition performance on VOC2007 and ILSVRC2012, while using only the top few predicted locations in each image and a small number of neural network evaluations.
1. Introduction
DeepMultiBox addresses the computational and scaling limits of exhaustive, class-specific detection by predicting a small set of class-agnostic object boxes. Its jointly trained localization and confidence outputs support efficient detection across multiple instances and classes.
- Exhaustive search across locations and scales becomes computationally difficult, especially when separate detectors are trained for many classes.
- DeepMultiBox uses a single DNN to regress several class-agnostic bounding boxes and assign each a confidence score for containing an object.
- The loss jointly trains box coordinates, confidences, and underlying features through assignment-based backpropagation.
- A class-agnostic box predictor enables efficient detection of many classes and can generalize to unseen classes.
- Using less than ten boxes from one network application, the method achieves state-of-art detection results.
2. Previous work
Prior detection methods face high costs from exhaustive multi-scale search and class-specific scaling. Earlier work reduces these costs through search strategies, shared representations, classless proposals, or deep regression, but multiple-class scalability remains challenging.
- Part-based detectors rely on exhaustive multi-scale template application, making them expensive and linear in the number of classes.
- Branch-and-bound methods reduce the need to evaluate every potential object location.
- Low-dimensional part bases and hashing share or accelerate part detection across object classes.
- Classless segmentation and inexpensive objectness classifiers generate or score object hypotheses before subsequent detection steps.
- Deep bounding-box regression has achieved strong detection performance, but mask regression does not scale to multiple classes because of its cost.
3. Proposed approach
The proposed approach predicts a fixed set of box locations and objectness confidences, then trains them with assignment-based losses and backpropagation. Priors and prior matching encourage efficient, diversified predictions that can be classified afterward.
- The DNN outputs a fixed number of bounding boxes and a confidence score for each box indicating its likelihood of containing an object.
- Each box is represented by four normalized coordinates, while its confidence is produced through a sigmoid output.
- At inference, the model produces K boxes that can be reduced with confidence scores and non-maximum suppression before classification by a subsequent DNN.
- Training formulates an assignment between predictions and ground-truth objects, using normalized-coordinate L2 distance to measure box dissimilarity.
- The objective combines matching and confidence losses, with matched predictions encouraged toward ground-truth boxes and high confidence.
- The assignment is a polynomial-time bipartite-matching variant, after which network parameters are optimized through backpropagation.
- Ground-truth clustering supplies K location priors, and matching priors to ground truth is hypothesized to diversify predictions.
- A class-specific variant is possible, but its parameters grow linearly with classes; the two-step localization-then-recognition process shares data across object types.
4.1. Network Architecture and Experiment Details
The experiments use DNN-based localization and classification models with specified training procedures, data generation, and validation choices.
- Adagrad, mini-batches of 128, distributed replicas, k-means priors, and α = 0.3 define the main training configuration.The priors balance localization and confidence losses through the stated α value.
- Approximately 30 million images were generated for localizer training from the training set.Samples were balanced across four bounding-box coverage ranges: 0−5%, 5−15%, 15−50%, and 50−100%.
- The training-set selection and most hyperparameters were based on prior experience with non-public datasets.The experiments did not explore non-standard data generation or regularization options.
- Hyperparameters were selected using a held-out portion comprising 10% of the training set.The supplied passage continues with a random choice of examples.
4.2. VOC 2007
VOC 2007 evaluation uses models trained on VOC 2012 and classifies a small set of localized candidate boxes to produce detections.
- VOC contains complex scene images with labeled bounding boxes for 20 diverse object classes.The evaluation focuses on the 2007 edition, whose test set was released.
- Training on VOC 2012 used approximately 11,000 images and produced a 100-box localizer with a deep-network classifier.The classifier training data included positive crops overlapping objects and negative background crops.
- The classifier used 10 million positive crops with at least 0.5 Jaccard overlap and 20 million negative crops with at most 0.2 overlap.Positive crops received VOC class labels, while negative crops received the background label.
- A single localization pass produced up to 100 candidate boxes, after which non-maximum suppression retained the top 10 for 21-way classification.Final detection scores multiplied the localizer and classifier scores.
4.3. Discussion
The localizer detects objects with a small box budget, and higher-resolution crops improve localization; classification achieves competitive VOC performance with low runtime complexity.
- 45.3% of objects were localized with 10 boxes using the first model, versus 48% using the second model.The results were obtained with models trained on VOC 2012.
- Using higher-resolution image crops provides an additional localization boost over the maximum center crop alone.The experiment compares the maximum center crop with a second scale using 3 × 3 windows covering 60% of the image.
- The classified detections achieve a mean AP of 0.29 on VOC 2007, described as on par with state-of-the-art.Runtime complexity is low because only the top 10 boxes are used.
- Using only the maximum-centered square crop still detects relatively small objects, including boats and sheep in the example detections.The examples and full precision-recall curves are shown in Figures 2 and 3.
4.4. ILSVRC 2012 Detection Challenge
DeepMultiBox is evaluated on ILSVRC-2012 using a localization model, a recognition model, and post-classification of a small number of predicted windows. With 5–10 windows, it performs about as well as a one-box-per-class approach while offering class-agnostic and cross-class localization.
- 544,545 training images cover 1,000 object categories, while evaluation uses 48,238 validation images.
- DeepMultiBox predicts 100 locations, reduces them with non-maximum suppression, and ranks classifier-scored windows for final detection.
- The ILSVRC-2012 metric detection@5 permits one box for each of five labels and requires 0.5 Jaccard overlap with a ground-truth box.
- With 5–10 windows, DeepMultiBox performs about as well as the competing one-box-per-class approach.
- The one-box-per-class approach scales linearly with classes, whereas DeepMultiBox can transfer localization to unseen categories and handle multiple instances of one class.
- An ImageNet-trained localization model captures more VOC windows than a VOC-trained model captures ImageNet windows, hypothesized to reflect ImageNet’s richer class set.
5. Discussion and Conclusion
The paper concludes that DeepMultiBox predicts multiple object locations with a class-agnostic deep network and remains competitive while using few classifier probes. It also generalizes across datasets and captures repeated instances, although localization and recognition still require separate network passes.
- DeepMultiBox predicts multiple bounding boxes using a deep convolutional neural network and a cost that accommodates variable numbers of ground-truth locations.
- The method learns to predict object locations in unseen images, including locations from categories on which it was not trained.
- DeepMultiBox is competitive on VOC2007 and ILSVRC-2012 while requiring only a few locations for subsequent classification.
- The method captures multiple instances of objects from the same class, supporting image-understanding systems that must represent repeated objects.
- The current two-pass procedure uses 5–10 network evaluations, while a future version would combine localization and recognition in one feed-forward pass.