Source-linked AI summary
YOLO9000: Better, Faster, Stronger
Joseph Redmon, Ali Farhadi
TL;DR
Object detection aims to be fast, accurate, and broad, but available detection methods and labeled data remain limited in scope. YOLO9000 combines classification and detection data through hierarchical dataset integration and joint training, achieving real-time detection across more than 9,000 categories, including 19.7 mAP on ImageNet detection.
Problem
Object detection needs to be fast, accurate, and broad, but existing methods recognize relatively few objects and detection labeling is expensive.
Method
YOLO9000 combines classification and detection datasets through a hierarchical classification structure and jointly trains a real-time detector on ImageNet and COCO.
Results
YOLO9000 achieves 19.7 mAP overall on ImageNet detection and 16.0 mAP on 156 classes without labeled detection data while detecting more than 9,000 categories in real time.
Takeaways & Limitations
YOLO9000 demonstrates real-time detection at a scale closer to image classification by leveraging classification data alongside detection data.
Takeaways & Limitations
YOLO9000 struggles with clothing categories because COCO provides bounding-box labels for people but not clothing types.
Abstract
from arXiv · showhide
We introduce YOLO9000, a state-of-the-art, real-time object detection system that can detect over 9000 object categories. First we propose various improvements to the YOLO detection method, both novel and drawn from prior work. The improved model, YOLOv2, is state-of-the-art on standard detection tasks like PASCAL VOC and COCO. At 67 FPS, YOLOv2 gets 76.8 mAP on VOC 2007. At 40 FPS, YOLOv2 gets 78.6 mAP, outperforming state-of-the-art methods like Faster RCNN with ResNet and SSD while still running significantly faster. Finally we propose a method to jointly train on object detection and classification. Using this method we train YOLO9000 simultaneously on the COCO detection dataset and the ImageNet classification dataset. Our joint training allows YOLO9000 to predict detections for object classes that don't have labelled detection data. We validate our approach on the ImageNet detection task. YOLO9000 gets 19.7 mAP on the ImageNet detection validation set despite only having detection data for 44 of the 200 classes. On the 156 classes not in COCO, YOLO9000 gets 16.0 mAP. But YOLO can detect more than just 200 classes; it predicts detections for more than 9000 different object categories. And it still runs in real-time.
1. Introduction
Object detection remains limited by smaller datasets and more expensive annotations than classification. YOLO9000 addresses this gap by combining hierarchical classification data with detection data and joint training to expand vocabulary while preserving localization and real-time performance.
- Motivation: Detection methods are increasingly fast and accurate, but most remain constrained to a small set of objects.General-purpose detection is intended to be fast, accurate, and broad in object coverage.
- Motivation: Detection datasets contain thousands to hundreds of thousands of images and dozens to hundreds of tags, versus millions of classification images and tens or hundreds of thousands of categories.The dataset-scale gap limits detection’s ability to match classification’s breadth.
- Motivation: Detection annotation is far more expensive than classification or tagging, making similarly large detection datasets unlikely in the near future.Tags are often user-supplied for free, unlike detection labels.
- Approach: The proposed method combines distinct datasets through a hierarchical view of object classification to expand current detection systems.This approach harnesses existing classification data rather than requiring detection labels at the same scale.
- Approach: Joint training uses detection images for precise localization and classification images to increase detector vocabulary and robustness.The algorithm trains object detectors on both detection and classification data.
- Result: YOLO9000 detects over 9000 object categories in real time by improving YOLO into YOLOv2 and jointly training on ImageNet classification and COCO detection data.The system is described as a state-of-the-art, real-time detector trained on more than 9000 ImageNet classes alongside COCO detection data.
2. Better
YOLOv2 improves localization and recall while preserving real-time detection by combining architectural changes, anchor-box refinements, and training strategies. These improvements yield strong speed–accuracy tradeoffs across resolutions, including 78.6 mAP on VOC 2007 above real-time speeds.
- Motivation: YOLO’s main weaknesses are significant localization errors and relatively low recall, motivating improvements focused on both while maintaining classification accuracy.
- Training improvements: More than 2% mAP improvement comes from adding batch normalization to every convolutional layer, allowing dropout removal without overfitting.Batch normalization also improves convergence and regularizes the model.
- Training improvements: Almost 4% mAP improvement comes from fine-tuning the classifier at 448 × 448 resolution for 10 ImageNet epochs before detection training.This gives filters time to adapt to higher-resolution inputs.
- Anchor boxes: 69.2 mAP with 88% recall using anchor boxes trades a small mAP decrease for higher recall than 69.5 mAP with 81% recall without anchors.YOLOv2 removes fully connected layers, predicts class and objectness for each anchor, and produces more than a thousand boxes per image.
- Dimension clusters: 61.0 average IOU with 5 clustered priors matches 60.9 from 9 hand-picked anchors, while 9 centroids achieve much higher average IOU.The method uses k-means with an IOU-based distance and selects k = 5 as a tradeoff between model complexity and recall.
- Stability and multi-scale detection: Almost 5% improvement over the anchor-box version comes from combining dimension clusters with direct center prediction, while multi-resolution operation enables speed–accuracy tradeoffs.At 288 × 288, YOLOv2 exceeds 90 FPS with mAP almost as good as Fast R-CNN; at high resolution it reaches 78.6 mAP on VOC 2007 above real-time speeds.
3. Faster
YOLOv2 is designed for low-latency detection by using a faster base network and a streamlined detection architecture. Its Darknet-19 backbone requires 5.58 billion operations while achieving strong ImageNet classification accuracy.
- Motivation: YOLOv2 is designed for fast, low-latency predictions needed by applications such as robotics and self-driving cars.The authors optimize the system for speed from the ground up.
- Architecture changes: 33% less computation results from using YOLOv2’s new network, while anchor boxes increase recall without changing mAP.These are the two exceptions among the listed design decisions that do not significantly increase mAP.
- Detection speed: 2−10× faster is YOLOv2 than state-of-the-art detectors such as Faster R-CNN with ResNet and SSD512 on PASCAL VOC2012, with comparable detection performance.The table reports that YOLOv2 performs on par with these detectors while running substantially faster.
- Base network: 30.69 billion operations are required by VGG-16 per 224 × 224 image, compared with 8.52 billion for YOLO’s GoogLeNet-based network.The YOLO framework uses a custom feature extractor to reduce computation while maintaining detection capability.
- Darknet-19: 5.58 billion operations are required by Darknet-19 to process an image, while it achieves 72.9% top-1 and 91.2% top-5 ImageNet accuracy.Darknet-19 contains 19 convolutional layers and 5 maxpooling layers.
4. Stronger
YOLO9000 jointly trains detection and classification data using WordTree, enabling a detector to learn bounding boxes and objectness from COCO while expanding its recognizable categories with ImageNet labels. On ImageNet detection, it achieves 19.7 mAP overall and 16.0 mAP on 156 classes without labeled detection data, while learning animal species better than clothing and equipment categories.
- Joint classification and detection: Joint training mixes detection and classification images, applying the full YOLOv2 loss to detection data and only classification-specific losses to classification data.This lets detection images teach bounding-box prediction and objectness while class-only images expand the detectable category set.
- Hierarchical classification: WordTree resolves incompatible dataset labels by organizing visual concepts hierarchically and predicting conditional probabilities over each node’s hyponyms.Absolute class probabilities are computed by multiplying conditional probabilities along the path to the root, allowing labels such as “Norfolk terrier” and “dog” to coexist.
- Joint classification and detection: YOLO9000 combines COCO detection data with the top 9000 ImageNet classes, mapping categories from both datasets into a shared WordTree.The resulting joint model uses COCO to learn object localization and ImageNet to classify a wider variety of objects.
- Results: 19.7 mAP overall and 16.0 mAP on 156 disjoint classes were achieved on ImageNet detection, despite labeled detection data overlapping with COCO for only 44 categories.The 156 classes had no labeled detection data during training, so their performance demonstrates transfer from classification supervision.
- Results: YOLO9000 learns new animal species well but struggles with clothing and equipment categories because COCO’s objectness and bounding-box supervision generalize unevenly.Animal objectness transfers from COCO, whereas COCO lacks bounding-box labels for clothing categories beyond person.
5. Conclusion
The paper introduces YOLOv2 and YOLO9000 as real-time detection systems, with YOLO9000 jointly optimizing detection and classification across data sources to detect more than 9000 categories. The authors also identify broader applications for their techniques and future work in weakly supervised segmentation and improved label matching.
- 5. Conclusion: YOLOv2 is state-of-the-art and faster across varied detection datasets, while supporting a smooth speed–accuracy tradeoff through multiple image sizes.It can be run at different image sizes to adjust speed and accuracy.
- 5. Conclusion: YOLO9000 detects more than 9000 object categories by jointly optimizing detection and classification with WordTree across ImageNet and COCO.The framework combines data from various sources and addresses the dataset size gap between detection and classification.
- 5. Conclusion: WordTree, hierarchical dataset combination, and multi-scale training may benefit image classification, segmentation, and other visual tasks beyond object detection.The authors describe WordTree as providing a richer output space for image classification.
- 5. Conclusion: Future work targets weakly supervised image segmentation, stronger matching strategies for assigning weak labels, and combining diverse data sources and structures.The authors plan to continue using labelled data to build stronger models of the visual world.