Source-linked AI summary
C-MIL: Continuation Multiple Instance Learning for Weakly Supervised Object Detection
Fang Wan, Chang Liu, Wei Ke, Xiangyang Ji, Jianbin Jiao, Qixiang Ye
TL;DR
Weakly supervised object detection remains an open problem, with WSOD trailing fully supervised detection by about 20% on PASCAL VOC. C-MIL introduces continuation optimization into multiple instance learning, using progressively smoothed losses over spatially and class-related instance subsets, and achieves state-of-the-art detection and localization performance.
Problem
WSOD remains open, with about a 20% performance gap from fully supervised detection on the PASCAL VOC detection benchmark.
Method
C-MIL applies continuation optimization to MIL by progressively smoothing losses over spatially and class-related instance subsets.
Results
C-MIL achieves new state-of-the-art performance for weakly supervised detection and localization on commonly used object detection benchmarks.
Takeaways & Limitations
C-MIL's parametric subset partition strategy combined with deep learning activates full object extent while alleviating MIL's non-convexity problem.
Abstract
from arXiv · showhide
Weakly supervised object detection (WSOD) is a challenging task when provided with image category supervision but required to simultaneously learn object locations and object detectors. Many WSOD approaches adopt multiple instance learning (MIL) and have non-convex loss functions which are prone to get stuck into local minima (falsely localize object parts) while missing full object extent during training. In this paper, we introduce a continuation optimization method into MIL and thereby creating continuation multiple instance learning (C-MIL), with the intention of alleviating the non-convexity problem in a systematic way. We partition instances into spatially related and class related subsets, and approximate the original loss function with a series of smoothed loss functions defined within the subsets. Optimizing smoothed loss functions prevents the training procedure falling prematurely into local minima and facilitates the discovery of Stable Semantic Extremal Regions (SSERs) which indicate full object extent. On the PASCAL VOC 2007 and 2012 datasets, C-MIL improves the state-of-the-art of weakly supervised object detection and weakly supervised object localization with large margins.
1. Introduction
WSOD reduces annotation effort but remains difficult because MIL’s non-convex optimization can localize object parts instead of full objects. C-MIL applies continuation optimization through smoothed losses and instance subsets to address this problem, achieving state-of-the-art detection and localization performance.
- WSOD uses image-level category annotations instead of labor-intensive bounding-box annotations, reducing human involvement.
- A roughly 20% performance gap remains between WSOD and fully supervised detection on PASCAL VOC.
- MIL’s non-convex loss can trap training in local minima that select discriminative object parts while missing full object extent.
- C-MIL smooths the loss into easier sub-problems and partitions instances by spatial overlap and class-related scores.
- Continuation parameters move training from a single subset containing all instances toward subsets containing individual instances, approximating the original MIL loss.
- C-MIL reports new state-of-the-art performance for weakly supervised detection and localization on standard object detection benchmarks.
2. Related Work
Related work addresses MIL’s local-minimum problem with clustering, bag splitting, multi-start optimization, regularization, context, and progressive refinement. The paper positions continuation optimization as a principled way to smooth non-convex MIL objectives through easier sub-problems.
- MIL-based methods: MIL treats each image as a bag and iteratively selects high-scoring instances while jointly estimating instances and detectors.
- MIL-based methods: MIL frequently encounters local minima because its loss is non-convex, especially when the solution space is large.
- Optimization strategies: Prior strategies reduce this difficulty through clustering, bag splitting, or multi-fold training with cross-validation and multiple starting points.
- MIL networks: MIL networks have incorporated spatial regularization, context information, and progressive optimization to mitigate localization and optimization problems.
- Open problem: Despite these improvements, existing methods lack a principled and systematic optimization-based solution to the local-minimum problem.
- Continuation optimization: Continuation methods smooth complex objectives into easier sub-problems that approach the optimization problem of interest as continuation parameters are tuned.
- Continuation optimization: This study applies continuation optimization to spatially and class-related instance subsets to learn full object extent.
3. Methodology
C-MIL replaces MIL’s single-instance selection with continuation-based subset selection and gradually transitions from a smooth, convex objective toward the original MIL objective. Spatially and class-related subsets are used to activate object extent and support detector estimation despite inaccurate early instance selection.
- MIL Revisit: MIL alternates between selecting the highest-scoring instance and estimating detectors, but its loss is non-convex for positive bags and can have many local minima.False positives selected early can mislead detector learning.
- Continuation Detector Estimation: C-MIL equally uses instances in the selected subset during back-propagation, allowing spatially overlapping, class-related proposals to contribute to object-extent activation.The most discriminative subset, rather than only its single highest-scoring instance, supplies pseudo-objects for detector estimation.
- Continuation Instance Selection: C-MIL partitions each image’s region proposals into disjoint, spatially overlapping and class-related subsets, constructed around high-scoring instances.Subsets form a minimum sufficient cover of each bag, and overlapping proposals are merged according to λ.
- Continuation MIL: As λ increases from 0 to 1, C-MIL changes each bag from one subset containing all instances to singleton subsets, connecting a convex classification loss to the original MIL loss.The continuation traces a sequence of smoothed losses from a start solution at λ=0 to the target solution at λ=1.
- Continuation Instance Selection: For intermediate λ, subset scores average member scores, producing smoother loss functions than MIL and reducing the non-convexity of instance selection.At λ=0, the relevant max operation is replaced by an average over all instances, making the loss convex.
- Continuation Detector Estimation: Detector estimation selects the highest-average-score subset, while instances are labeled positive, negative, or ignored according to IoU with its highest-scoring instance.The IoU thresholds are 1−λ/2 for positives and λ/2 for negatives; intermediate overlaps are ignored.
4. Experiments
C-MIL was evaluated on PASCAL VOC benchmarks for detection and localization, including continuation schedules, ablations, training dynamics, SSERs, and state-of-the-art comparisons. Continuation improved optimization behavior and produced strong detection and localization results.
- Experimental setup: C-MIL was evaluated on PASCAL VOC 2007 and 2012 using mAP and CorLoc metrics.CorLoc measures images whose highest-scoring region reaches at least 0.5 IoU with the ground-truth object region.
- Continuation schedule: Continuation optimization improved detection by 1.1%–4.7% and localization by 1.4%–4.5% across five continuation-parameter functions.The functions controlled how λ changed during training on VOC 2007 with VGGF.
- Continuation schedule: The “Log” function performed best by increasing λ quickly early and changing slowly later, matching the transition from broad subsets to detector estimation.Early subsets collected object parts broadly; later subsets stabilized while training focused on detector estimation.
- Ablation: Combining continuation instance selection and detector estimation improved detection by 4.7% (40.7% vs. 36.0%) over the baseline.Instance selection alone improved performance by 3.0%, while continuation object estimation contributed a further 1.4%.
- SSER analysis: C-MIL’s activated subsets dwindled as λ increased from 0 to 1, forming SSERs around object boundaries that often captured full objects.The training process initially collected object regions broadly and later suppressed backgrounds while stabilizing object activations.
- Detection performance: On VOC 2007, C-MIL reached 40.7% with VGGF and 50.5% with VGG16, outperforming several state-of-the-art detectors.With VGG16, reported gains were 6.2% over WeakRPN, 5.2% over TS2C, and 3.2% over MELM.
- Detection and localization performance: On VOC 2012 with VGG16, C-MIL achieved 46.7% detection performance and 67.4% localization performance, exceeding the cited comparison methods.Detection gains ranged from 4.3% to 6.7%, while localization gains ranged from 2.5% to 3.0%.
5. Conclusion
The paper presents C-MIL as a continuation-based approach for weakly supervised object detection that addresses MIL’s non-convexity through smoothed loss functions. It reports improved detection and localization and links the method’s behavior to the emergence of SSERs.
- 5. Conclusion: C-MIL alleviates MIL’s non-convexity using a parametric sequence of smoothed loss functions over instance subsets.The strategy combines deep feature learning with continuation optimization to first collect object regions and then discover SSERs.
- 5. Conclusion: C-MIL significantly improved weakly supervised object detection and localization compared with state-of-the-art approaches.The conclusion attributes this behavior to continuation optimization combined with deep feature learning.
- 5. Conclusion: SSERs provide a learned semantic indication of full object extent during weakly supervised localization.The paper presents their emergence as a fresh insight into the weakly supervised object detection problem.