Source-linked AI summary
Active Learning for Deep Detection Neural Networks
Hamed H. Aghdam, Abel Gonzalez-Garcia, Joost van de Weijer, Antonio M. López
TL;DR
Large-scale bounding-box annotation is prohibitively expensive, motivating active selection of informative images for detector training. The paper proposes an iterative scoring method for deep object detectors and reports better performance than random selection, while supporting still images and videos with temporal complements.
Problem
Bounding-box labeling for large image collections is costly, while random selection may miss diverse visual patterns needed for accurate detection.
Method
The method computes detection-based pixel scores, aggregates them into image-level scores, ranks unlabeled images, and iteratively finetunes a convolutional detector.
Results
The method outperforms random selection overall and performs significantly better than guided random on the Caltech Pedestrian dataset by the 14th cycle.
Takeaways & Limitations
Active learning can reduce detector-labeling requirements by selecting informative images for still-image datasets or videos, with temporal rules complementing video selection.
Takeaways & Limitations
The method's advantage depends on the detector having sufficient capacity for the targeted domain; a lightweight network may select redundant samples and remain less accurate on complex data.
Abstract
from arXiv · showhide
The cost of drawing object bounding boxes (i.e. labeling) for millions of images is prohibitively high. For instance, labeling pedestrians in a regular urban image could take 35 seconds on average. Active learning aims to reduce the cost of labeling by selecting only those images that are informative to improve the detection network accuracy. In this paper, we propose a method to perform active learning of object detectors based on convolutional neural networks. We propose a new image-level scoring process to rank unlabeled images for their automatic selection, which clearly outperforms classical scores. The proposed method can be applied to videos and sets of still images. In the former case, temporal selection rules can complement our scoring process. As a relevant use case, we extensively study the performance of our method on the task of pedestrian detection. Overall, the experiments show that the proposed method performs better than random selection. Our codes are publicly available at www.gitlab.com/haghdam/deep_active_learning.
1. Introduction
Large-scale object-detection annotation is costly, while random sampling may miss diverse visual patterns and reduce accuracy. The paper proposes active selection for deep detectors using pixel-level importance aggregated into image-level scores, with iterative labeling for still images or videos.
- 1. Introduction: 35 seconds is the minimum average time to label pedestrians in a typical urban scene, making annotation prohibitively costly at hundred-thousands-of-image scale.The reported timing was measured across six labeling tools and can increase with tool and annotator experience.
- 1. Introduction: Randomly selecting images may fail to capture diverse visual patterns, producing lower accuracy than training on the full dataset.
- 1. Introduction: The paper proposes active learning for convolutional-neural-network detectors, targeting informative unlabeled images rather than random subsets.The goal is to reduce labeling while improving detector accuracy through selected samples.
- 1. Introduction: Pixel-level importance scores are aggregated into image-level scores to rank unlabeled images for labeling and repeated detector finetuning.The procedure supports multiple iterations and applies to still-image datasets and videos.
- 1. Introduction: The method is evaluated on pedestrian detection against random selection and classical image-level scoring methods, with temporal rules added for video data.
2. Related Work
Prior active-learning research has largely addressed image classification, while deep object-detection methods remain limited. The paper distinguishes its approach through a detection-specific pixel score, a new aggregation method, and temporal redundancy handling for videos.
- 2. Related Work: Most active-learning research has focused on image classification, while only a few methods address object detection with convolutional neural networks.
- 2. Related Work: Existing classification methods may select redundant samples because they do not explicitly consider similarity among selected images.
- 2. Related Work: Compared with prior deep-detector methods, this work introduces a pixel-level score designed for object detection instead of relying on marginal or entropy scores.
- 2. Related Work: The method also replaces simple average or maximum pixel-score merging with another aggregation strategy and adds temporal rules to avoid redundant video frames.
3. Proposed Method
The method scores unlabeled images by measuring local disagreement among detection predictions, aggregates pixel scores into image scores, and selects the highest-scoring images for labeling. It supports iterative selection for still images and adds temporal reasoning for video sequences.
- Pixel-level scores: Local divergence between predictions for translated image patches is assumed to be low for correct predictions and high for false positives or false negatives.This motivates using neighborhood prediction disagreement to identify pixels that may improve the detector.
- Active learning cycle: The active-learning cycle predicts per-pixel probabilities, computes informative pixel scores, aggregates them into image scores, selects b images, and retrains on the accumulated labeled set.The cycle repeats as selected images move from Xu to Xal.
- Pixel-level scores: Pixel-level scores combine spatially averaged predictions, entropy-based disagreement, and scores from multiple probability matrices to measure local inconsistency.The final pixel score sums corresponding scores across prediction matrices.
- Aggregating scores: Image-level scores average the maximum pixel score from each non-overlapping region after partitioning the score matrix into regions.This max-pooling aggregation converts a score matrix into a scalar used to rank unlabeled images.
- Selecting images: Still-image selection chooses the top b image scores, whereas video selection uses temporal smoothing and selection rules to reduce redundant neighboring frames.The temporal score weights image-level scores within a window around each frame.
4. Experiments
Experiments evaluate active learning for pedestrian detection across datasets, budgets, cycles, score functions, and selection strategies. The proposed method generally improves over guided random selection, though gains depend on dataset and network capacity.
- Time-to-completion vs. budget: Budget size creates a time–selection trade-off: with B=7500, b=2500 takes about 46 hours, whereas b=50 takes about 1800 hours.The authors consider b=500 more practical, with an estimated completion time of about 190 hours.
- Our method vs. random: The proposed method selects more accurate training subsets than guided random on Caltech Pedestrian, with significantly better performance by the 14th cycle.At the fourth cycle, 2K frames had been selected; by the 14th, the method was significantly better on Caltech Pedestrian.
- Other pixel-level score functions: MC-Dropout outperforms guided random but remains less accurate than the proposed score, while binary entropy performs poorly even against guided random.The comparison replaces the proposed pixel-level score with binary entropy or MC-Dropout at dropout ratios of 50% and 10%.
- Statistics of Xal: The selected 7K frames contain 5706 pedestrian instances on average, compared with 2741 for guided random, indicating denser pedestrian content in the proposed selection.MC-Dropout and entropy selections contain 3700 and 5243 pedestrian instances, respectively.
- Per cycle comparison: On BDD100K, the proposed method performs only slightly better than guided random, and the authors attribute the limited improvement to the need for a more complex network architecture.The same dataset also shows more pedestrian instances and more frames containing pedestrians under the proposed selection.
5. Conclusion
The paper proposes convolutional-neural-network active learning for object detectors and finds that it outperforms random selection when the detector has sufficient capacity. The method supports still images and videos, with temporal reasoning for videos.
- The proposed active learning method outperforms random selection when the detector has sufficient capacity for the targeted domain.
- The method supports unlabeled sets of still images or videos, with temporal reasoning as a complementary selection mechanism for videos.
- An ablation study identifies max-pooling-based aggregation as especially relevant because it outperforms other proposed aggregation methods.
(Supplementary Materials)
The supplementary materials identify the paper, its authors and affiliations, and show a figure concerning pedestrian-detection performance on Caltech using different negative-to-positive ratios.
- The authors are affiliated with the Computer Vision Center and Computer Science Department at Universitat Autònoma de Barcelona.
- Figure 1 reports Caltech pedestrian-detection performance for different negative-to-positive ratios.
1. Caltech Pedestrian dataset
On Caltech, the experiments examine detector behavior under different negative-to-positive ratios and compare active selection with guided random selection across cycles. The proposed method becomes more accurate after the first cycle and selects substantially more pedestrian-containing frames.
- Lower-bound error: N2P affects overall detector performance, and N2P=15 produces the best results with lower miss rate.
- Per cycle comparison: The Caltech experiments compare the proposed method and its MC-Dropout and binary-entropy variants with guided random selection across cycles.
- Per cycle comparison: Starting from the second cycle, the proposed method selected images that produced a more accurate detector than guided random selection.
- Statistics of Xal: At cycle 14, the proposed method selected pedestrian-containing instances in 2,895 of 7K frames (41%), versus 1,460 frames (21%) for guided random selection.
2. BDD100K dataset
On BDD100K, the proposed method performs slightly better than random selection, but the improvement is less significant than on Caltech. The authors attribute this limitation to the need for a more capable detector architecture.
- Lower-bound error: The authors report that their network is less accurate on BDD100K than on Caltech because it is too lightweight for BDD100K’s complexity.
- Per cycle comparison: On BDD100K, the proposed method performs slightly better than random selection, but the improvement is less significant than on Caltech.
- Statistics of Xal: On BDD100K, the proposed method selects frames containing more pedestrian instances and more frames containing at least one pedestrian than random selection.
- Per cycle comparison: The BDD100K comparisons include the proposed method, random selection, and related variants shown in the supplied figures.