Source-linked AI summary
Multiple instance active learning for object detection
Tianning Yuan, Fang Wan, Mengying Fu, Jianzhuang Liu, Songcen Xu, Xiangyang Ji, Qixiang Ye
TL;DR
Active object detection lacks an instance-level active learning method for selecting informative images amid noisy background instances. MI-AOD learns instance uncertainty with two adversarial classifiers and re-weights it through multiple instance learning, improving state-of-the-art methods on common datasets, especially when labeled sets are small.
Problem
Active object detection lacks an instance-level active learning method, while noisy background instances interfere with estimating image uncertainty.
Method
MI-AOD learns unlabeled-instance uncertainty from the prediction discrepancy of two adversarial classifiers and re-weights instances with multiple instance learning.
Results
MI-AOD improves state-of-the-art methods with significant margins on commonly used object detection datasets, particularly with small labeled sets.
Takeaways & Limitations
MI-AOD establishes a solid baseline for active object detection by bridging instance-level uncertainty and image-level evaluation while suppressing noisy instances.
Abstract
from arXiv · showhide
Despite the substantial progress of active learning for image recognition, there still lacks an instance-level active learning method specified for object detection. In this paper, we propose Multiple Instance Active Object Detection (MI-AOD), to select the most informative images for detector training by observing instance-level uncertainty. MI-AOD defines an instance uncertainty learning module, which leverages the discrepancy of two adversarial instance classifiers trained on the labeled set to predict instance uncertainty of the unlabeled set. MI-AOD treats unlabeled images as instance bags and feature anchors in images as instances, and estimates the image uncertainty by re-weighting instances in a multiple instance learning (MIL) fashion. Iterative instance uncertainty learning and re-weighting facilitate suppressing noisy instances, toward bridging the gap between instance uncertainty and image-level uncertainty. Experiments validate that MI-AOD sets a solid baseline for instance-level active learning. On commonly used object detection datasets, MI-AOD outperforms state-of-the-art methods with significant margins, particularly when the labeled sets are small. Code is available at https://github.com/yuantn/MI-AOD.
1. Introduction
Active learning for object detection lacks an instance-level method that can select informative images despite many noisy background instances. MI-AOD addresses this by learning and re-weighting instance uncertainty with adversarial classifiers and multiple instance learning, and improves state-of-the-art methods on common datasets.
- Motivation: Object detection lacks instance-level active learning methods that account for noisy background instances when estimating image uncertainty.Existing approaches average instance or pixel uncertainty, which can obscure the relationship between instance and image uncertainty.
- Method: MI-AOD uses two adversarial instance classifiers to learn uncertainty for unlabeled instances through prediction discrepancy.The method targets informative image selection by learning instance uncertainty from the unlabeled set.
- Method: Multiple instance learning treats each unlabeled image as an instance bag and re-weights instance uncertainty to connect instance-level observations with image-level evaluation.The image-level classification loss suppresses noisy instances during re-weighting.
- Contribution: IUL and IUR highlight informative instances while filtering noisy ones in object detection.These modules provide the paper’s proposed approach for handling instance uncertainty and image uncertainty.
- Results: MI-AOD improves state-of-the-art methods with significant margins on commonly used object detection datasets.The paper presents this result as validation of MI-AOD’s effectiveness for active object detection.
2. Related Work
Prior active learning work mainly uses uncertainty or sample-distribution estimates, while active object detection must handle complex and noisy instance distributions within images. MI-AOD is positioned against these approaches as an instance-aware method for object detection.
- Uncertainty-based Methods: Uncertainty-based methods select samples using posterior probabilities, class margins, entropy, or expected model change.MIL-based methods also discover representative instances, but the cited approaches target image classification rather than object detection.
- Distribution-based Methods: Distribution-based methods select diverse samples by estimating unlabeled-sample distributions with clustering, discrete optimization, context, or core-set selection.These approaches emphasize sample diversity or representativeness.
- Active Learning: Active learning methods commonly follow uncertainty-based or distribution-based routines, with extensions to open sets and self-paced learning.The effectiveness of intermediate feature representations for sample selection remains questionable in the cited discussion.
- Active Object Detection: Active object detection faces complex instance distributions within images and is more challenging than active image classification.Existing adaptations include sorting instance loss predictions, using background-pixel uncertainty, and adding spatial context.
- MI-AOD: MI-AOD is illustrated with instance uncertainty learning and instance uncertainty re-weighting modules for active object detection.The figure distinguishes adversarial-classifier uncertainty learning from MIL-based re-weighting.
3. The Proposed Approach
MI-AOD selects informative images by learning instance-level uncertainty with adversarial classifier discrepancy, then re-weighting instances through multiple instance learning to align instance and image uncertainty. Its iterative pipeline suppresses noisy instances before selecting images for the next active-learning cycle.
- Overview: MI-AOD initializes a detector on labeled images, selects informative unlabeled images, merges them into the labeled set, and repeats training and selection until reaching the annotation budget.The labeled data include bounding-box and category labels, while selected unlabeled images are manually labeled before merging.
- Overview: Active object detection selects unlabeled images for annotation, but conventional methods can obscure informative instances by averaging instance or pixel uncertainty and ignoring abundant negative instances.A RetinaNet image may contain approximately 100k anchor instances, making noisy background instances a central challenge.
- Instance Uncertainty Learning: Instance uncertainty learning uses two independently initialized adversarial classifiers whose prediction discrepancy highlights uncertain instances, especially those near the labeled–unlabeled distribution boundary.The feature extractor is fixed while classifier parameters are optimized to maximize discrepancy on unlabeled instances while preserving labeled-set detection performance.
- Instance Uncertainty Learning: The feature extractor is then optimized to minimize classifier discrepancy, reducing distribution bias and progressively aligning labeled and unlabeled instance features.Maximization and minimization alternate within active-learning cycles as an unsupervised procedure using unlabeled-set information.
- Instance Uncertainty Re-weighting: Instance uncertainty re-weighting treats each image as an instance bag and uses an MIL classifier plus image classification loss to suppress noisy instances and connect instance uncertainty to image uncertainty.The MIL score is large when an instance belongs to an image class and its instance-classification scores exceed those of other classes.
- Instance Uncertainty Re-weighting: After IUL and IUR, MI-AOD ranks each image by its top-k instance uncertainties and selects the highest-ranking images for the next learning cycle.The method assumes re-weighting has suppressed noisy instances so instance uncertainty is consistent with image uncertainty.
4. Experiments
Experiments evaluate MI-AOD across object detection datasets, detector architectures, module ablations, visualizations, and computational cost. MI-AOD consistently improves selection and detection performance, especially with limited labels.
- Experimental setup: PASCAL VOC experiments use RetinaNet and SSD, while MS COCO experiments evaluate active learning under larger scale, denser objects, and more categories.VOC uses 5.0% initialization and expands to 20.0%; COCO uses 2.0% initialization and expands to 10.0%.
- PASCAL VOC: 18.08%, 7.78%, and 5.19% are MI-AOD's margins over state-of-the-art methods at 5.0%, 7.5%, and 10.0% labeled samples on PASCAL VOC.At 20.0% samples, MI-AOD reaches 72.27% mAP and exceeds CDAL by 3.20%.
- MS COCO: 0.6%, 0.5%, and 2.0% are MI-AOD's margins over Core-set at 2.0%, 4.0%, and 10.0% labeled MS COCO images.Against CDAL, the corresponding margins are 0.6%, 1.3%, and 2.6%.
- Ablation study: 7.06% is the IUL improvement in CIFAR-10 classification at 2.0% samples, from 51.01% to 58.07%.The result supports discrepancy learning for instance uncertainty estimation.
- Ablation study: 5.04%∼17.09% are IUR's improvements in the first three cycles, while the last-cycle gains are 1.28% over IUL and 1.39% over Random.With all images for training, IUR improves performance by 1.09% over no IUR, 78.37% versus 77.28%.
- Analysis: MI-AOD selects more true-positive instances in every active learning cycle and costs less time than CDAL at early cycles.The visualization analysis also reports that IUR suppresses background and surrounding interference while recovering representative instances.
5. Conclusion
MI-AOD selects informative training images by learning instance uncertainty and re-weighting it at the image level. Experiments validate its superiority over state-of-the-art methods and establish a baseline for active object detection.
- Method: MI-AOD uses adversarial instance classifiers to learn uncertainty for unlabeled instances.It treats unlabeled images as instance bags and re-weights instances using multiple instance learning.
- Method: Iterative instance uncertainty learning and re-weighting suppress noisy instances while selecting informative images for detector training.The method is designed to bridge instance uncertainty and image-level uncertainty.
- Conclusion: Experiments on large-scale datasets validate MI-AOD's superiority over state-of-the-art methods.The paper presents MI-AOD as a solid baseline for active object detection.