Source-linked AI summary
Weakly Supervised Deep Detection Networks
Hakan Bilen, Andrea Vedaldi
TL;DR
Weakly supervised object detection seeks to learn detectors from image-level labels, addressing the annotation demands of complex, data-hungry visual learning. The paper modifies a pre-trained classification CNN to operate on regions with separate recognition and detection streams, and reports stronger weakly supervised detection and image classification than established alternatives and standard fine-tuning techniques.
Problem
Weakly supervised detection aims to learn object detectors using only image-level labels, an important challenge because image understanding involves many complex visual concepts and CNN training is data-hungry.
Method
WSDDN extends a pre-trained image-classification CNN with region-level descriptors and parallel recognition and detection streams whose aggregated scores provide image-level supervision.
Results
WSDDN achieves significantly better weakly supervised detection performance than existing methods and outperforms traditional fine-tuning techniques for image classification.
Takeaways & Limitations
A simple end-to-end modification of a pre-trained CNN can perform weakly supervised detection using fine-tuning, region proposals, back-propagation, and image-level labels.
Takeaways & Limitations
Detecting highly distinctive object parts instead of complete objects damages performance in selected categories, motivating complementary cues favoring whole-object detection.
Abstract
from arXiv · showhide
Weakly supervised learning of object detection is an important problem in image understanding that still does not have a satisfactory solution. In this paper, we address this problem by exploiting the power of deep convolutional neural networks pre-trained on large-scale image-level classification tasks. We propose a weakly supervised deep detection architecture that modifies one such network to operate at the level of image regions, performing simultaneously region selection and classification. Trained as an image classifier, the architecture implicitly learns object detectors that are better than alternative weakly supervised detection systems on the PASCAL VOC data. The model, which is a simple and elegant end-to-end architecture, outperforms standard data augmentation and fine-tuning techniques for the task of image-level classification as well.
1. Introduction
The paper leverages pre-trained CNN representations to learn weakly supervised object detectors from image-level labels. WSDDN extends a classification CNN to reason over regions through separate recognition and detection streams, enabling end-to-end training and strong detection and classification performance.
- CNNs pre-trained on large labelled datasets learn generic visual features that transfer across recognition, detection, segmentation, and pose estimation tasks.
- Weakly supervised detection learns object detectors using only image-level labels, reducing annotation demands for complex visual concepts and data-hungry CNN training.
- WSDDN modifies a pre-trained CNN to extract region descriptors, then uses parallel recognition and detection streams whose scores are aggregated into image-level predictions.
- The detection stream selects salient regions independently of recognition, avoiding a multiple-instance-learning tendency to become stuck in local optima.
- The network can be fine-tuned on target data using image-level labels, region proposals, and back-propagation, while remaining trainable end-to-end with standard CNN packages.
- On PASCAL VOC, WSDDN achieves state-of-the-art weakly supervised detection and also improves image classification over standard fine-tuning techniques.
2. Related Work
Prior weakly supervised detection work largely formulates detection as multiple instance learning or selects regions through similarity-based methods. Other CNN approaches address localization or transformation, but use different mechanisms from WSDDN.
- Multiple instance learning treats each image as a bag of regions and alternates between estimating object appearance and selecting positive regions.
- MIL optimization is non-convex, so solution quality can depend strongly on initialization because solvers may become trapped in local optima.
- Similarity-based approaches select or jointly learn object regions using graph connections, latent semantic clustering, or discriminative convex clustering.
- Weakly supervised localization methods modify or reuse CNNs to obtain coarse object localization while predicting image labels without location annotations.
- Transformer networks learn an image pre-transformation for classification, whereas WSDDN learns to pre-select likely object regions through a different selection process.
3. Method
WSDDN modifies a CNN pretrained with image-level supervision to process candidate regions through separate classification and detection streams. Their scores are combined for region detections and summed into image-level predictions, while training uses image-level labels plus a spatial regularizer.
- Architecture: WSDDN starts from an ImageNet-pretrained CNN and modifies it with spatial pyramid pooling to produce region-level representations.Candidate regions are processed efficiently, with shared convolutional features and an SPP layer compatible with fully connected layers.
- Architecture: The architecture branches region features into classification and detection streams that produce class-region score matrices.The classification stream scores classes for each region, whereas the detection stream scores regions relative to one another for each class.
- Architecture: Different softmax normalizations make the classification stream rank classes within regions and the detection stream rank regions within classes.This asymmetry gives the two branches distinct roles despite their similar structure.
- Detection: Region scores are formed by the element-wise product of the two stream outputs, then ranked and filtered with non-maxima suppression using a 40% IoU threshold.The resulting detections are class-specific.
- Image-level prediction: Image-level class scores are obtained by summing region scores, producing values in the range (0, 1) while allowing multiple object classes per image.The scores combine softmax-normalized stream outputs over candidate regions.
- Training: Training uses stochastic gradient descent with momentum on binary log-loss terms from image-level class labels.A soft regularizer additionally penalizes fc7 feature discrepancies between the highest-scoring region and regions with at least 60% IoU.
4. Experiments
Experiments evaluate WSDDN on PASCAL VOC detection and classification, examining architecture, proposals, regularization, and comparisons with prior methods. The results show strong weakly supervised detection and improved image classification, with object-part localization remaining a key failure mode.
- Experimental setup: WSDDN is evaluated on PASCAL VOC 2007 and 2010 for weakly supervised detection and image classification.Detection uses AP at 50% IoU and CorLoc as evaluation measures.
- Architecture and training: The architecture replaces pool5 with spatial pyramid pooling, adds a detection branch, and combines classification and detection streams before binary log-loss.The network is fine-tuned across all layers using only weak image-level supervision.
- Baselines: 21.6% mAP is achieved by the single-stream baseline, below the 31.6% state-of-the-art result reported for Wang et al.The baseline uses a soft approximation of the maximum region score rather than selecting only the highest-scoring region.
- Object proposals: 2% improvement is obtained for models S and M using EdgeBoxes objectness scores, while model L gains 5%.The scores are incorporated by scaling region feature maps according to EdgeBoxes objectness.
- Spatial regularization: 1, 2, and 4 mAP points are gained for models S, M, and L respectively from spatial regularization.The authors report that larger networks benefit more from spatial invariance around high-confidence regions.
- Detection results: WSDDN generally outperforms alternative weakly supervised detectors, while model L often detects object parts instead of whole instances and performance is weaker for chair, person, and pottedplant.The method uses a simple CNN modification fine-tuned end-to-end, and ensembles further improve performance.
- Classification results: 8 and 7 points are gained over VGG-F and VGG-M-1024 respectively on image classification.WSDDN S and M are compared with corresponding pre-trained CNN baselines and standard fine-tuning techniques.
5. Conclusions
WSDDN modifies a pre-trained CNN into an end-to-end weakly supervised detector using region proposals and image-level labels. It outperforms existing weakly supervised detection methods, while object-part detections remain a documented failure mode.
- Contributions: WSDDN is a simple modification of a pre-trained image-classification CNN that performs weakly supervised object detection.The method requires fine-tuning on a target dataset using back-propagation, region proposals, and image-level labels.
- Contributions: The model achieves significantly better performance than existing weakly supervised detection methods.It is also efficient at training and test time because it operates on a spatial pyramid pooling layer.
- Limitations: Object-part detections damage performance in selected categories because the method favors highly distinctive regions over complete objects.The authors are exploring complementary cues that would favor detecting whole objects.