PaperlayerView exact revision on arXiv

Source-linked AI summary

Scalable, High-Quality Object Detection

Christian Szegedy, Scott Reed, Dumitru Erhan, Dragomir Anguelov, Sergey Ioffe

arXiv:1412.1441v3cs.CV

TL;DR

High-quality detection depends on proposal generation whose weak ranking can limit runtime-quality trade-offs, motivating whether data-driven methods can replace careful engineering. The paper develops MSC-MultiBox, a multi-scale learned proposal system, and reports strong detection and proposal results across ImageNet and COCO.

  • Problem

    Domain-agnostic proposal methods are often unranked or weakly ranked, making runtime-quality trade-offs difficult and raising whether high-quality proposals can be learned from data.

  • Method

    MSC-MultiBox uses an improved Inception-style convolutional network with multi-scale predictors that output bounding-box coordinates and confidence scores.

  • Results

    0.50 mAP is achieved with a single model and 0.52 mAP with an ensemble, while the system exceeds previously reported ILSVRC2014 detection performance using several dozen proposals per image.

  • Takeaways & Limitations

    Learning-based proposal generation closes the performance gap with engineered methods while reducing detection computational cost and supporting flexible runtime-quality operating points.

  • Takeaways & Limitations

    YOLO's transferability to datasets with significantly more objects, such as the ILSVRC detection challenge, is unclear.

Abstract

from arXiv · show

Current high-quality object detection approaches use the scheme of salience-based object proposal methods followed by post-classification using deep convolutional features. This spurred recent research in improving object proposal methods. However, domain agnostic proposal generation has the principal drawback that the proposals come unranked or with very weak ranking, making it hard to trade-off quality for running time. This raises the more fundamental question of whether high-quality proposal generation requires careful engineering or can be derived just from data alone. We demonstrate that learning-based proposal methods can effectively match the performance of hand-engineered methods while allowing for very efficient runtime-quality trade-offs. Using the multi-scale convolutional MultiBox (MSC-MultiBox) approach, we substantially advance the state-of-the-art on the ILSVRC 2014 detection challenge data set, with $0.5$ mAP for a single model and $0.52$ mAP for an ensemble of two models. MSC-Multibox significantly improves the proposal quality over its predecessor MultiBox~method: AP increases from $0.42$ to $0.53$ for the ILSVRC detection challenge. Finally, we demonstrate improved bounding-box recall compared to Multiscale Combinatorial Grouping with less proposals on the Microsoft-COCO data set.

1. Introduction

The paper argues that learned, multi-scale proposal generation can match engineered methods while offering efficient runtime-quality trade-offs. MSC-MultiBox improves detection and proposal quality through architectural, contextual, and training advances.

  • Current high-performing detectors commonly use salience-based localization followed by deep-convolutional post-classification.
  • Strong proposal ranking can reduce proposal counts while balancing recall, quality, and running time through threshold selection.
  • Learned proposal generation closely rivals salience-based methods at significantly lower computational cost and adapts readily to new domains and object-specific uses.
  • MSC-MultiBox combines an improved architecture, multi-scale convolutional predictors, Inception-based post-classification, and contextual modeling.
  • 0.45 mAP is achieved for 200-class detection using 15 proposals per image.
  • 0.50 mAP is achieved with a single model, while 0.52 mAP is achieved with an ensemble of three post-classifiers and two proposal generators.

2. Related Work

Related work spans part-based and deep-network detectors, learned proposal systems, and increasingly unified detection architectures. The paper positions MultiBox among approaches that trade detection quality against scalability and proposal-generation design.

  • Part-based DPM systems established configurable runtime-quality trade-offs, including real-time detection on PASCAL 2007.
  • Deep convolutional architectures became dominant for large-scale recognition and detection, including regression and multi-scale sliding-window formulations.
  • MultiBox predicts class-agnostic box coordinates and confidences, avoiding linear scaling with the number of classes while improving location recall in this work.
  • SPP improves R-CNN efficiency by reusing midlevel CNN features and reports roughly 100x speedup over R-CNN.
  • Unified detectors such as the two-branch model and YOLO reduce reliance on data-independent proposals by predicting localization-related outputs within one network.
  • Faster R-CNN uses a Region Proposal Network to predict boxes and objectness scores, sharing translation-invariant priors with MultiBox but differing in multi-scale design.

3. Model

MultiBox trains a convolutional network to output bounding-box coordinates and rank proposals by object-likelihood using a weighted confidence-and-location objective.

  • MultiBox directly predicts bounding-box coordinates and confidence scores for proposals.
  • Confidence uses a logistic loss for whether a proposal corresponds to an object of interest.
  • Location uses a similarity loss between predicted boxes and their closest matching object boxes, with L2 distance as the default.
  • The network uses an improved Inception-style convolutional architecture with a structured output module for coordinates and confidences.
  • Training matches predictions to ground-truth boxes through bipartite matching before computing the losses.
  • The overall training objective is a weighted sum of the confidence and location losses, optimized with stochastic gradient descent.

3.2. Convolutional Priors

Convolutional priors impose structured, multi-scale proposal locations and shapes, aiming to maximize object coverage at IOU threshold 0.5 while improving proposal quality.

  • MultiBox predicts a constant set of proposal locations and confidences, seeking high coverage among high-confidence predictions.
  • Each output slot receives a prior rectangle computed from the training-set distribution of object boxes to impose proposal diversity.
  • 0.417 to 0.529: class-agnostic average precision rises for convolutional versus non-convolutional priors at IOU threshold 0.5.
  • At IOU threshold 0.5, the prior objective maximizes expected coverage of matching ground-truth objects.
  • Predicted box coordinates are interpreted as offsets from their associated priors and converted back by adding the prior at inference.
  • The prior set uses regularly placed grid boxes, exploiting expected translation invariance in object locations.
  • Prediction layers operate on 8×8, 6×6, 4×4, 3×3, 2×2, and 1×1 grids, with multiple aspect-ratio priors per tile.

3.3. Training with missing positive labels

The method addresses missing positive labels, which can penalize confident predictions of unlabeled objects and potentially reduce proposal recall, using confidence bootstrapping.

  • Missing true-positive labels can impose a large confidence loss on predictions of real but unlabeled objects.
  • The authors hypothesize that missing or noisy labels may make MultiBox overly conservative and reduce proposal recall.
  • Training reformulates the confidence objective around the top-L most confident predictions for each image.
  • The model is initialized without bootstrapping and then fine-tuned using the bootstrapped objective.
  • Figure 2 illustrates multi-scale convolutional prediction of proposal locations and confidences.

3.4. MultiBox network architecture

The MultiBox localizer and post-classifier use new Inception variants, with a deep convolutional network operating over a 299×299 receptive field.

  • Both the MultiBox localizer and post-classifier use new variants of the Inception architecture.
  • The convolutional network is 42 layers deep over a 299×299 receptive field and contains over 130 layers.
  • The architecture uses the top 8 × 8 × 2048 convolutional layer and removes extra side heads for simplicity.

3.5. Post-classification

MSC-MultiBox supports one-shot detection or class-agnostic proposals followed by post-classification. The post-classifier uses an Inception-based architecture designed to preserve spatial information during grid reduction.

  • 3.5. Post-classification: MSC-MultiBox can operate as a one-shot detector or as a class-agnostic localizer feeding a post-classifier.In the high-quality regime, the paper uses an additional post-classification step.
  • 3.5. Post-classification: The post-classifier must classify objects while determining whether each object overlaps the crop’s central receptive-field region.This spatial sensitivity motivates architectural changes beyond ordinary image classification.
  • 3.5. Post-classification: Stride-2 convolutions are added alongside large pooling layers when reducing grid size to preserve spatial information.The paper hypothesizes that large pooling layers can harm accurate spatial prediction.

3.7. Context Modeling

The context model combines whole-image features with object-region features for post-classification. Separate context and object models are used, with distinct training objectives for context classification and box classification.

  • 3.7. Context Modeling: Whole-image features are concatenated with object features before the combiner classifier scores each region.The whole-image feature comes from the topmost layer before the classifier.
  • 3.7. Context Modeling: The context and object-feature models are separate networks and do not share weights.The context network supplies whole-image features rather than its classifier output at proposal evaluation time.
  • 3.7. Context Modeling: The context network uses separate logistic classifiers per class, while the combiner uses a softmax classifier with a background class.The combiner is trained after whole-image features have been extracted.

3.8. In-Model Context Ensembling

The method ensembles context predictions across multiple image crops by averaging combiner scores for each proposal. This adds a small, consistent mAP gain at relatively low cost.

  • 3.8. In-Model Context Ensembling: Context features are extracted from six large crops: the whole image, four corner squares, and one center square.The final score averages combiner outputs across the context-feature and object-feature pairs.
  • 3.8. In-Model Context Ensembling: The crop-level combiner scores are averaged to form the final proposal score.This provides computationally efficient ensembling during evaluation.
  • 3.8. In-Model Context Ensembling: 0.005-0.01 mAP is the modest but consistent improvement from context ensembling.The additional cost is relatively small when many proposals are evaluated and the combiner is cheaper than feature extraction.

4. Results

Inception-based multi-scale MultiBox improves proposal quality, and the ranked proposals enable graceful runtime-quality trade-offs. Context, multi-crop processing, and cross-model ensembling further improve detection results, while MultiBox compares favorably with MCG under proposal budgets.

  • 4.1. Network architecture improvements: For a fixed proposal budget K, switching from Zeiler-Fergus to Inception substantially increases class-agnostic AP and maximum recall.The improvement is attributed to the underlying network architecture.
  • 4.1. Network architecture improvements: Increasing priors from around 150 to 800 provides a large benefit, with no significant improvement beyond 800.This result is reported for Inception-style convolutional networks.
  • 4.2. Runtime-quality trade-off: More than 90% of the highest-quality operating point’s mAP can be maintained with an order of magnitude fewer network evaluations.Quality eventually decreases when many low-quality proposals swamp the post-classifier.
  • 4.3. Contextual features: Adding contextual features greatly improves post-classification results compared with non-contextual models.The comparison uses multi-scale MultiBox proposals and the multi-crop methodology.
  • 4.4. Multibox on many image crops: Almost 5% mAP improvement results from using multiple image crops, increasing proposals from 13 to 51 per image on average.The resulting proposal count remains significantly below that of Selective Search.
  • 4.5. ILSVRC2014 detection challenge: 0.499 mAP is obtained with one MultiBox region proposal network and one post-classifier model.A three-post-classifier ensemble reaches 0.506 mAP, while the proposed two-MultiBox-model ensembling reaches 0.52 mAP.
  • 4.6. Microsoft-COCO: MultiBox significantly outperforms MCG below 2000 proposals, especially at lower overlap thresholds.MCG surpasses MultiBox at 0.8 or higher thresholds with over 300 proposals, while the paper notes a possible NMS adjustment for tighter-threshold recall.

5. Conclusions

The framework provides scalable high-quality object detection with a flexible runtime-quality operating point. Learning-based multi-scale proposal generation closes the gap with engineered methods while reducing computational cost and improving overall detection performance.

  • The framework is simple, efficient, and practical for high-quality object detection at scale.
  • Single-crop multi-scale convolutional MultiBox exceeds previously reported ILSVRC2014 detection performance with only several dozen proposals per image on average.
  • Over 10% relative mAP improvement is achieved at the high-quality end of the runtime-quality curve.
  • Learning-based proposal generation closes the performance gap with engineered methods while reducing detection computational cost.
  • Higher-quality proposals in smaller numbers improve overall object detection performance, beyond serving as a more efficient replacement for static proposal algorithms.
Loading 1412.1441v3…