Source-linked AI summary
Weakly-supervised Discovery of Visual Pattern Configurations
Hyun Oh Song, Yong Jae Lee, Stefanie Jegelka, Trevor Darrell
TL;DR
Weakly supervised detection seeks to learn from image-level labels when detailed object locations are scarce or costly. The paper discovers discriminative multi-patch configurations using constrained submodular optimization, then uses them for localization and training. The resulting approach achieves state-of-the-art weakly supervised detection results on PASCAL VOC.
Problem
Weakly supervised detection addresses the scarcity and cost of detailed object-location annotations by learning from image-level presence labels.
Method
The method discovers frequent configurations of discriminative patches using an independence-constrained submodular selection problem.
Results
State-of-the-art weakly supervised detection results on PASCAL VOC follow from improved object coverage and informative hard negatives generated by discovered configurations.
Takeaways & Limitations
The configurations provide a practical way to improve foreground localization and construct training examples that address mislocalization.
Abstract
from arXiv · showhide
The increasing prominence of weakly labeled data nurtures a growing demand for object detection methods that can cope with minimal supervision. We propose an approach that automatically identifies discriminative configurations of visual patterns that are characteristic of a given object class. We formulate the problem as a constrained submodular optimization problem and demonstrate the benefits of the discovered configurations in remedying mislocalizations and finding informative positive and negative training examples. Together, these lead to state-of-the-art weakly-supervised detection results on the challenging PASCAL VOC dataset.
1 Introduction
The paper targets weakly supervised detection from image-level labels by discovering discriminative configurations of multiple visual patterns. These configurations improve localization and training data, yielding stronger detection on challenging datasets.
- Motivation: Weakly supervised detection replaces costly object-location annotations with binary image-level labels, enabling learning from abundant but sparsely labeled visual data.The setting is also relevant when detailed annotations are noisy or ill-defined.
- Motivation: Mislocalized or poorly sized boxes harm detection directly and can weaken subsequent training through inaccurate positive and hard-negative examples.Alternating optimization is especially influenced by the quality of initial localizations.
- Approach: The method discovers characteristic configurations of multiple discriminative patches rather than relying on a single patch, which often represents only an object part.The input consists of images labeled by object presence or absence.
- Approach: Patch selection requires both discriminative occurrence across positive and negative images and an independence constraint preventing redundant or fully overlapping regions.The resulting constrained selection problem is formulated as submodular optimization subject to a matroid intersection constraint.
- Benefits: Configuration combinations can cover more of the object and generate informative hard negatives from overlapping regions, reducing localization errors during testing.These benefits address both foreground estimation and detector training.
- Results: The method discovers frequent discriminative pattern configurations and outperforms state-of-the-art methods on PASCAL VOC under image-level supervision.The paper positions this result against prior weakly supervised methods developed for increasingly realistic datasets.
2 Approach
The approach discovers discriminative, diverse patch configurations from image-level labels, then uses their spatial relationships for localization and hard-negative generation. It formulates patch selection as constrained submodular optimization and finds configurations efficiently with greedy procedures.
- Discriminative candidate patches: The method first identifies discriminative patches whose neighborhoods occur mainly in positively labeled images, using a bipartite covering formulation.Representative patches maximize a monotone submodular covering function.
- Diverse patch selection: To preserve multiple object parts, selection forbids patches whose neighborhoods overlap beyond a threshold rather than treating overlapping candidates as identical.Treating dense overlapping candidates as identical would prevent discovering co-occurring configurations.
- Constrained optimization: The feasible sets are independent sets in a graph encoding prohibited overlaps, and the resulting constraint family is represented as an intersection of partition matroids.The graph-based constraint can be checked without explicitly computing the individual matroids.
- Constrained optimization: A greedy algorithm approximately maximizes coverage under the diversity constraints, achieving F(Sg) ≥ 1/(∆ + 2)F(S∗), or 1/(∆ + 1) when neighborhoods are disjoint.The optimization problem is NP-hard, so the algorithm adds the patch with greatest marginal neighborhood coverage and deletes overlapping neighbors.
- Configuration discovery: Selected patches form representative clusters, and frequent configurations combine co-occurring clusters with relative location, viewpoint, and scale.If configuration evidence is insufficient, the method falls back to the most frequent single cluster; the final localization is the smallest box containing the configuration.
- Hard-negative generation: For each discovered configuration, the smallest box containing its patches estimates the foreground, while regions capturing only fragments become up to four hard negative examples.These hard negatives are intended for detector training and are generated from regions that do not overlap both configuration patches simultaneously.
3 Experiments
Experiments show that the method discovers meaningful visual-part configurations across PASCAL classes, while using them to improve localization and weakly supervised detection. The evaluation includes qualitative analyses, detection comparisons, and hard-negative studies on PASCAL VOC 2007.
- Discovered configurations: The method consistently discovers meaningful part configurations, including bicycle wheels with bodies, people’s faces with torsos, and car windows with body frames.Some failures involve consistent configurations spanning different co-occurring objects, such as lamps and sofas.
- Weakly-supervised object detection: Detection average precision is evaluated on the full PASCAL VOC 2007 test set using image-level object-presence supervision.The comparison excludes extra instance-level annotations such as pose, difficult, and truncated labels.
- Weakly-supervised object detection: The method improves detection performance on most classes relative to the two reported baselines, with consistent gains for rigid man-made objects and significant improvement on person.Figure 4 presents example test-set detections comparing the proposed method with [27].
- Impact of discovered hard negatives: The hard-negative experiment compares using no positive-image negatives against adding neighboring regions around foreground estimates.Table 2 reports the effect of these hard-negative choices on the full PASCAL VOC 2007 test set.
- Experimental conclusions: Discovered configurations provide tighter full-object spatial coverage and generate useful hard negatives, supporting state-of-the-art weakly supervised detection on PASCAL VOC.Figure 5 visualizes discriminative parts in green and yellow and their tightly fitting configuration boxes in magenta.