Source-linked AI summary
We don't need no bounding-boxes: Training object class detectors using only human verification
Dim P. Papadopoulos, Jasper R. R. Uijlings, Frank Keller, Vittorio Ferrari
TL;DR
Bounding-box annotation is time-consuming, while weakly supervised training can produce lower-quality detectors. The paper trains detectors by iterating between human verification, retraining, and re-localization. On PASCAL VOC 2007, it reports near-fully-supervised performance without drawing boxes and a 6×–9× reduction in annotation time.
Problem
Detector training typically needs many manually boxed images, and bounding-box annotation is tedious, time-consuming, and expensive.
Source: §1 · p. 1Method
The scheme iteratively verifies automatically generated boxes, retrains on positively verified boxes, and uses verification signals to reduce the re-localization search space.
Source: §1 · p. 1Results
On PASCAL VOC 2007, detectors perform almost as well as fully supervised models without drawing boxes, and total annotation time falls by 6×–9×.
Source: §1 · p. 2, §5.5 · p. 8Takeaways & Limitations
The authors report that verification-based training produces high-quality bounding-box annotations while minimizing human annotation effort.
Source: §1 · p. 1, §5.5 · p. 8The scheme returns exactly one box per class per training image, although test-time detections can localize multiple same-class objects.
Source: §5.1 · p. 5
Abstract
from arXiv · showhide
Training object class detectors typically requires a large set of images in which objects are annotated by bounding-boxes. However, manually drawing bounding-boxes is very time consuming. We propose a new scheme for training object detectors which only requires annotators to verify bounding-boxes produced automatically by the learning algorithm. Our scheme iterates between re-training the detector, re-localizing objects in the training images, and human verification. We use the verification signal both to improve re-training and to reduce the search space for re-localisation, which makes these steps different to what is normally done in a weakly supervised setting. Extensive experiments on PASCAL VOC 2007 show that (1) using human verification to update detectors and reduce the search space leads to the rapid production of high-quality bounding-box annotations; (2) our scheme delivers detectors performing almost as good as those trained in a fully supervised setting, without ever drawing any bounding-box; (3) as the verification task is very quick, our scheme substantially reduces total annotation time by a factor 6x-9x.
1. Introduction
Bounding-box labels make detector training expensive, while existing lower-cost approaches either produce weaker detectors or still require substantial box drawing. The paper replaces drawing with verification and reports near-fully-supervised performance with substantially less annotation time.
1. Introduction
- Training detectors typically requires many manually boxed images, and bounding-box annotation is tedious, time-consuming, and expensive.The passage gives 42 seconds per box for ILSVRC crowd-sourced annotation using a technique developed for efficiency.Source: §1 · p. 1
- Weak supervision is cheaper but typically performs about half as well as box-supervised training, while active learning still requires many human-drawn boxes.The passage describes active learning as producing high-quality detectors but yielding limited total-time gains because box drawing remains costly.Source: §1 · p. 1
- The scheme alternates detector retraining, object re-localization, and human verification, using verification signals in both retraining and search.Figure 1 depicts the iterative framework and the signal's use in both stages.Source: §1 · p. 1, Figure 1 · p. 2
- 6×–9× less total annotation time, with detectors almost as good as fully supervised models and no boxes drawn.Experiments on PASCAL VOC 2007 also report rapid production of high-quality bounding-box annotations.Source: §1 · pp. 1–2
2. Related Work
Prior work reduces annotation needs through weak supervision, human-machine collaboration, active learning, and other data sources. This paper's approach differs by using verification to alter both detector retraining and object re-localization.
2. Related Work
- WSOL learns from image-level labels without object locations, jointly localizing training objects and learning detectors for new images; performance has remained about half that of fully supervised counterparts.The paper describes many WSOL methods as alternating detector retraining and re-localization, and introduces verification signals into both steps.Source: §2 · p. 2
- Human-machine collaboration has addressed difficult vision tasks, while Russakovsky et al. detect objects in complex scenes using detectors pretrained on boxed ILSVRC images.Their goal was not an efficient training scheme.Source: §2 · p. 2
- Object-detection active learning can produce high-quality detectors, but prior schemes still require drawing or correcting many boxes, limiting annotation-time gains.One approach asks annotators to draw boxes on about one third of web-crawled training images.Source: §2 · pp. 2–3
- Other annotation-reduction approaches exploit video, transfer learning, domain adaptation, web text, or eye-tracking signals.The passage cites video methods, transfer learning, domain adaptation [24], text, and eye tracking.Source: §2 · p. 3
3. Method
The method iteratively verifies automatically generated detections, retrains on positively verified boxes, and uses verification outcomes to narrow later localization searches. Its richer error labels support more targeted pruning than a simple yes/no response.
3. Method
- The framework alternates detector retraining, re-localization, and annotator queries, using verification to update training and narrow the later search space.Images without positively verified boxes and their possible object locations are tracked across iterations.Source: §3 · p. 3
3.1. Verification by annotators
- Annotators either mark detections Yes or No, or label them Yes, Part, Container, Mixed, or Missed; Yes requires IoU ≥0.5 with the true box.The richer labels provide more information but are expected to take longer to assess.Source: §3.1 · p. 3, Figure 2 · p. 3
3.2. Re-training object detectors
- Retraining uses only positively verified boxes, rather than all detections, which the paper says yields stronger, more reliable detectors.Background samples are proposals with IoU in [0, 0.5) relative to positively verified boxes.Source: §3.2 · pp. 3–4
3.3. Re-localizing objects by search space reduction
- Re-localization scores remaining proposals; negative verifications prune the search, with YPCMM error categories removing different proposal sets.Yes/No removes the rejected proposal; YPCMM uses category-specific rules, including IoA thresholds of 0.9 for Part and Container cases. Figure 3 illustrates the resulting steering toward the small cat and away from the dog.Source: §3.3 · p. 4, Figure 3 · p. 4
4. Implementation details
The implementation combines a Fast R-CNN detector with EdgeBoxes proposals and initializes detections through multiple instance learning. Multi-fold training and objectness scoring are used to improve that initialization.
4.1. Object class detector
- The detector is Fast R-CNN with EdgeBoxes proposals; retraining omits box regression to keep proposals fixed, while test detectors use box regression.Most experiments use AlexNet as the CNN architecture.Source: §4.1 · p. 4, §4 · p. 4
4. Implementation details
- Initial detections come from MIL, which alternates retraining and re-localization until two successive steps produce the same detections, typically within 10 iterations.The first classifier iteration uses complete images as positive examples.Source: §4 · p. 4, §4.2 · p. 5
4.2. Initialization by Multiple Instance Learning
- MIL initialization uses 10-fold multi-fold training to reduce premature lock-in and combines detector scores with an objectness measure; YPCMM's extra error signal makes the search more targeted.The figure's bird example reports fewer verification steps with YPCMM; the objectness measure used is EdgeBoxes.Source: §4.2 · p. 5, Figure 4 · p. 5
5. Experimental Results
On PASCAL VOC 2007, experiments evaluate localization and detection quality against annotation effort; verification-guided training approaches fully supervised detection performance with 6×–9× less annotation time.
5.1. Dataset and evaluation protocol
- The evaluation uses image-level-labeled training images, measures localization by CorLoc and test detection by mAP, and tracks effort through verification counts and human time.The protocol excludes images containing only difficult and truncated instances in its reduced training set and compares verification with manual-box supervision and MIL WSOL.Source: §5.1 · p. 5
5.2. Simulated verification
- 96% CorLoc was reached with YPCMM after an average of 2.5 checks per image; using verification to reduce the search space reached high CorLoc faster than retraining alone.Retraining alone raised CorLoc from MIL WSOL’s 43% to 82% after about four checks per image; search-space reduction variants reached 92%, 95%, and 96%.Source: §5.2 · p. 6, Figure 5 · p. 6
5.3. Human verification
- Human verification lifts the MIL baseline from 43% CorLoc and 27% mAP to 83% CorLoc and 45% mAP at 5.8 hours, exceeding the cited WSOL methods’ reported results.The compared WSOL results use AlexNet, as does this method’s comparison.Source: §5.3 · p. 7
- 45% mAP nearly matches fully supervised 51% mAP, while requiring 6×–9× less annotation time.The verification scheme takes 5.8 hours, versus 33 or 53 hours for full supervision under the stated annotation-time assumptions.Source: Figure 8 · p. 8, §5.3 · p. 8
5.4. Complete training set and VGG16
- On the complete 5011-image training set, verification reaches 50% mAP with AlexNet and 58% with VGG16, about 90% of the corresponding fully supervised results.The reported fully supervised results are 55% mAP with AlexNet and 66% with VGG16; Table 1 compares mAP across training sets and architectures.Source: Table 1 · p. 8, §5.4 · p. 8
5.5. Conclusions
- The authors conclude that verification-guided retraining and relocalization produce near-fully-supervised detectors without drawing boxes and reduce annotation time by 6×–9×.The conclusion is based on experiments on PASCAL VOC 2007.Source: §5.5 · p. 8