PaperlayerView exact revision on arXiv

Source-linked AI summary

Active Learning for Deep Object Detection via Probabilistic Modeling

Jiwoong Choi, Ismail Elezi, Hyuk-Jae Lee, Clement Farabet, Jose M. Alvarez

arXiv:2103.16130v2cs.CV

TL;DR

Active learning for object detection needs informative sample selection, but prior approaches often rely on multiple models or omit localization uncertainty. This paper uses mixture density networks to estimate both aleatoric and epistemic uncertainty for localization and classification in one forward pass, then aggregates them into image scores. Across PASCAL VOC and MS-COCO, it outperforms single-model methods and matches multi-model accuracy at substantially lower computing cost.

  • Problem

    Active learning for object detection is limited by methods that use multiple models or estimate informativeness mainly from classification, while labeling costs motivate efficient sample selection.

  • Method

    Mixture density networks estimate aleatoric and epistemic uncertainty for localization and classification in a single model and forward pass, with a scoring function aggregating these uncertainties.

  • Results

    The approach outperforms single-model methods and achieves accuracy comparable to multi-model methods while significantly reducing computing cost across PASCAL VOC and MS-COCO.

  • Takeaways & Limitations

    Combining localization and classification uncertainties provides an efficient active-learning strategy that scales across detector architectures and datasets.

Abstract

from arXiv · show

Active learning aims to reduce labeling costs by selecting only the most informative samples on a dataset. Few existing works have addressed active learning for object detection. Most of these methods are based on multiple models or are straightforward extensions of classification methods, hence estimate an image's informativeness using only the classification head. In this paper, we propose a novel deep active learning approach for object detection. Our approach relies on mixture density networks that estimate a probabilistic distribution for each localization and classification head's output. We explicitly estimate the aleatoric and epistemic uncertainty in a single forward pass of a single model. Our method uses a scoring function that aggregates these two types of uncertainties for both heads to obtain every image's informativeness score. We demonstrate the efficacy of our approach in PASCAL VOC and MS-COCO datasets. Our approach outperforms single-model based methods and performs on par with multi-model based methods at a fraction of the computing cost.

1. Introduction

The paper proposes a single-model active-learning method for object detection that estimates aleatoric and epistemic uncertainty from both localization and classification outputs. It uses mixture density networks and a scoring function to select informative images while reducing computing cost.

  • 1. Introduction: Its scoring function aggregates uncertainty across both heads and selects the top-K images for labeling.
  • 1. Introduction: The method addresses the high cost of multi-model uncertainty estimation while retaining both uncertainty types for active object detection.
  • 1. Introduction: The method estimates aleatoric and epistemic uncertainty for localization and classification in a single forward pass of one model.Mixture density networks learn Gaussian mixture models for both output heads.
  • 1. Introduction: A novel loss regularizes inconsistent data and is proposed to improve the robustness and overall performance of the GMM-based detector.
  • 1. Introduction: Experiments evaluate the approach across different models and datasets, including PASCAL VOC and MS-COCO.

2. Related Work

Prior active-learning methods for object detection use ensembles, multiple passes, heuristics, or omit localization uncertainty, creating accuracy and computational-cost limitations. The paper instead combines localization and classification uncertainties in one forward pass of a single model.

  • 2. Related Work: Existing methods often require multiple models or forward passes, resulting in high computational cost.
  • 2. Related Work: Many prior approaches estimate localization uncertainty heuristically or cannot estimate it at all.
  • 2. Related Work: The proposed approach combines localization and classification uncertainties while using a single forward pass to target accuracy and cost limitations.
  • 2. Related Work: Earlier mixture density network studies did not jointly address classification, both uncertainty types, and active learning for object detection.

3. Active Learning for Object Detection

The method replaces deterministic localization and classification outputs with Gaussian-mixture distributions, enabling uncertainty-aware active learning for object detection. It aggregates localization and classification uncertainties across detected objects to score image informativeness while improving parameter efficiency in the classification head.

  • 3.1. Object detection with probabilistic modeling: The network predicts K-component GMM parameters for localization coordinates and class outputs instead of deterministic values.Each mixture component uses means, variances, and weights to represent output distributions.
  • 3.1. Object detection with probabilistic modeling: Localization training uses negative log-likelihood to model bounding-box ambiguity, while classification training combines positive and hard-negative terms.Negative matches are ranked by the proposed mixture classification loss, with the top M × N selected for training.
  • 3.2. Improving parameter efficiency: The classification head improves parameter efficiency by removing explicit variance estimation and using class probabilities to estimate aleatoric uncertainty.The revised head reduces its added parameters to F × F × D × (C × K + K).
  • 3.3. Scoring function: The scoring function normalizes uncertainty values, takes the maximum across coordinates and detected objects, and aggregates four localization- and classification-based uncertainties per image.The four values are aleatoric and epistemic uncertainties for classification and localization; taking the maximum over them achieves the highest results among explored combinations.
  • 3. Active Learning for Object Detection: Table 1 compares single- and multiple-Gaussian variants applied to localization, classification, or both against the original SSD network.The table reports mAP for SGM and MDN configurations across Loc, Cl, and Loc+Cl settings.

4. Experiments

Experiments evaluate probabilistic object detection and active learning on PASCAL VOC and MS-COCO, including uncertainty aggregation, comparisons with single- and multiple-model methods, computing cost, and transferability.

  • 4.1. Object detection with probabilistic modeling: On PASCAL VOC and MS-COCO, probabilistic models outperform SSD, with Oursgmm leading the tested variations and baseline across reported metrics.The authors attribute the improvement to the proposed loss's aleatoric-uncertainty-based regularization and robustness to noisy data.
  • 4.2. Active learning evaluation: The maximum of all aleatoric and epistemic uncertainties across localization and classification consistently gives the best active-learning data selection across iterations.This aggregation is selected as the scoring function for comparisons with other active-learning studies.
  • 4.2. Active learning evaluation: 14% overlap between selections using localization and classification uncertainties together indicates diversified uncertainty measures whose combination improves image selection.The corresponding overlap is 48% for localization and 33% for classification individually.
  • 4.2. Active learning evaluation: On VOC07+12, the proposed method performs on par with ensembles and MC-dropout while requiring significantly less computing cost.The comparison uses Fig. 4 for accuracy and Fig. 5 for computational cost.
  • 4.2. Active learning evaluation: On MS-COCO, both proposed instances consistently outperform the other single-model methods in each active-learning cycle.The comparison applies all sampling methods to the same proposed GMM architecture.
  • 4.3. Scalability and dataset transferability: The approach transfers to Faster-RCNN with FPN, improving mAP by up to 1.13 points over the original model with negligible added computing cost.The uncertainty computation is applied after region proposal, excluding the number of anchor boxes from the added computation.

5. Conclusions

The paper concludes that mixture density networks enable single-pass estimation of localization and classification uncertainties for active object detection, with strong accuracy and computing-cost results across datasets and architectures.

  • 5. Conclusions: The method estimates aleatoric and epistemic uncertainty for localization and classification in one forward pass of one model and aggregates them for scoring.Experiments cover PASCAL VOC and MS-COCO, and the authors report scalability to architectures with different designs.
  • 5. Conclusions: Across PASCAL VOC and MS-COCO, the proposed probabilistic modeling and scoring function achieve gains in accuracy and computing cost.The conclusion characterizes the experiments as spanning two publicly available datasets.

A.1. Accuracy as a function of K

The supplementary experiment studies how the number of Gaussian mixture components affects accuracy, model size, and forward time.

  • A.1. Accuracy as a function of K: K=1, 2, 4, and 8 are compared using average mAP, standard deviation, parameter count, and forward time.The experiment reports normal IoU>0.5 and strict IoU>0.75 metrics over three repetitions.

A.2. Accuracy as a function of input image resolution

The supplementary experiment evaluates robustness to input resolution by comparing SSD with the proposed method using 512×512 images.

  • A.2. Accuracy as a function of input image resolution: Using 512×512 inputs significantly improves mAP for all compared methods.The experiment is analogous to the VOC07 experiment in Table 1a and compares SSD with the proposed method.

A.3. Accuracy as a function of budget number in active learning

The experiment evaluates active-learning mAP across budgets of 1k, 3k, and 9k labels on VOC07+12.

  • mAP is compared as a function of active-learning budget on VOC07+12.The reported experiment uses budgets of 1k, 3k, and 9k and averages mAP with standard deviation over three independent trials.

B. More visual examples selected by our approach

Additional selected examples illustrate how aleatoric and epistemic uncertainties expose different detection failures. The accompanying materials document comparisons with single-model and multi-model scoring methods and their computational settings.

  • Uncertainty values distinguish false positives and class misclassifications among representative selected detections.Examples include false-positive person and cow boxes, plus horses and motorbikes assigned incorrect classes.
  • The computational comparison reports model parameters and forward time at 300 × 300 resolution with K = 4.
  • The reported VOC07+12 comparisons cover published single-model methods, MC-dropout, and ensembles.MC-dropout is evaluated with 25 and 50 forward passes.

D. Discussion of the classification loss

The paper compares alternative classification losses and finds that their relative accuracy depends on the dataset, while the preferred loss does not fully resolve mixture-weight bias on larger datasets.

  • Classification-loss accuracy differs little on VOC07 but substantially on MS-COCO.The comparison concerns the Type-1 and Type-2 losses.
  • Type-2 loss alleviates mixture-weight bias relative to Type-1 loss but does not provide sufficient accuracy improvement on larger datasets with more classes.
  • The main paper reports results using Type-1 loss, leaving improved classification-loss design as future work.
Loading 2103.16130v2…