Source-linked AI summary
YOLOv3: An Incremental Improvement
Joseph Redmon, Ali Farhadi
TL;DR
YOLO needed incremental improvements without sacrificing speed. YOLOv3 combines design updates, a stronger classifier, and multiscale predictions, achieving fast detection with strong performance at the older AP50 metric.
Problem
YOLO required further improvement, motivating a set of incremental design changes and a stronger network.
Method
YOLOv3 combines ideas from prior work, a new classifier network, and three-scale predictions for bounding boxes, objectness, and classes.
Results
YOLOv3 is on par with SSD variants while 3× faster on COCO average AP, and nearly matches RetinaNet while excelling at AP50.
Takeaways & Limitations
YOLOv3 is a fast, strong detector for producing decent object boxes, though its localization weakens at higher IoU thresholds.
Takeaways & Limitations
The paper acknowledges that its criticism of COCO metrics lacked sufficient justification.
Abstract
from arXiv · showhide
We present some updates to YOLO! We made a bunch of little design changes to make it better. We also trained this new network that's pretty swell. It's a little bigger than last time but more accurate. It's still fast though, don't worry. At 320x320 YOLOv3 runs in 22 ms at 28.2 mAP, as accurate as SSD but three times faster. When we look at the old .5 IOU mAP detection metric YOLOv3 is quite good. It achieves 57.9 mAP@50 in 51 ms on a Titan X, compared to 57.5 mAP@50 in 198 ms by RetinaNet, similar performance but 3.8x faster. As always, all the code is online at https://pjreddie.com/yolo/
1. Introduction
The introduction presents YOLOv3 as the result of incremental improvements consisting of many small changes. It also previews the paper’s organization: the method, unsuccessful experiments, and broader implications.
- 1. Introduction: YOLO was improved through a collection of small changes rather than a major research advance.The author describes the work as building on leftover momentum from the previous year.
- 1. Introduction: The paper is presented as a technical report documenting random YOLO updates for a camera-ready deadline.The author notes that they lacked another source for citing these updates.
- 1. Introduction: The paper proceeds by explaining YOLOv3, describing the approach, discussing failed experiments, and considering the implications.The introduction explicitly signposts these four parts of the paper.
2. The Deal
YOLOv3 combines established ideas with a new classifier backbone, multi-scale box prediction, and multilabel classification. Its Darknet-53 backbone matches strong classifiers while using fewer operations and running faster.
- Detection system: YOLOv3 predicts bounding boxes at 3 different scales using feature-pyramid-like feature extraction.For COCO, it predicts 3 boxes per scale, encoding 4 box offsets, 1 objectness score, and 80 class predictions.
- Classifier backbone: 1.5× faster: Darknet-53 outperforms ResNet-101 under the stated comparison and has similar performance to ResNet-152.The networks use identical training settings and are tested with single-crop accuracy at 256×256 on a Titan X.
- Classifier backbone: Darknet-53 achieves the highest measured floating point operations per second, improving GPU utilization and evaluation speed.The paper attributes this efficiency partly to ResNets having too many layers.
3. How We Do
YOLOv3 matches SSD variants on COCO’s average mean AP while running 3× faster, and is especially strong at AP50, where it nearly matches RetinaNet and outperforms SSD variants. Its multi-scale predictions improve small-object performance, though localization worsens at higher IoU thresholds and performance is comparatively weaker for medium and large objects.
- 3× faster than SSD variants while remaining on par with them on COCO’s average mean AP.
- At mAP at IOU= .5, YOLOv3 is almost on par with RetinaNet and far above SSD variants.This indicates strong detection of reasonably good object boxes.
- Performance drops significantly as the IOU threshold increases, indicating difficulty aligning boxes perfectly with objects.
- Multi-scale predictions reverse YOLO’s previous small-object weakness, yielding relatively high APS performance but comparatively worse medium- and large-object performance.The paper notes that more investigation is needed to explain this pattern.
- On the AP50 accuracy–speed plot, YOLOv3 is faster and better than other detection systems.
4. Things We Tried That Didn’t Work
The authors tested several alternative training and prediction strategies for YOLOv3, but these reduced stability or mAP or failed to produce good results. They therefore retained their current formulation, while noting that further tuning might improve the unsuccessful techniques.
- Prediction and training strategies: The normal anchor-box mechanism for x, y offsets decreased model stability and did not work well.This mechanism predicted each offset as a multiple of the box width or height using linear activation.
- Prediction and training strategies: Linear x, y predictions instead of logistic activation caused a couple-point drop in mAP.The linear activation directly predicted the x, y offset rather than using logistic activation.
- Prediction and training strategies: Focal loss dropped mAP about 2 points, possibly because YOLOv3 separates objectness and conditional class predictions.The authors were uncertain whether this existing structure makes focal loss unnecessary for most examples.
- Prediction and training strategies: A dual-IOU-threshold truth-assignment strategy similar to Faster R-CNN failed to produce good results.The tested strategy used positive, ignored, and negative examples based on overlap thresholds.
- Conclusion: The authors considered their current formulation near a local optimum, while suggesting unsuccessful techniques might need tuning to stabilize training.They expressed preference for the current formulation but did not rule out future improvements.
5. What This All Means
YOLOv3 is fast and accurate, especially under the older .5 IOU metric, though weaker on COCO’s average AP across .5–.95 IOU. The paper also argues that computer-vision researchers should consider and mitigate potential harms from the technology.
- Performance: YOLOv3 is fast and accurate, but performs less strongly on COCO’s average AP between .5 and .95 IOU.It performs very well under the older .5 IOU detection metric.
- Evaluation metrics: The paper questions how much fine-grained IOU distinctions matter because humans struggle to distinguish bounding boxes at IOU .3 versus .5.This concern motivates skepticism about switching evaluation metrics.
- Broader implications: The paper notes that major technology companies and the military heavily fund vision research, raising concerns about potential misuse.It specifically mentions personal-information harvesting and harmful military applications.
- Broader implications: Computer vision is already used in questionable ways, so researchers should consider the harms their work may cause and seek ways to mitigate them.The paper contrasts these risks with beneficial applications such as counting zebras and tracking pets.
Rebuttal
The rebuttal concedes that its criticism of COCO metrics was partly defensive, but argues that updated metrics need stronger justification and should better reflect real-world detection priorities. It proposes evaluating how well detectors find and classify objects globally or per image rather than relying solely on per-class AP.
- Metric criticism: The authors concede that their arguments against COCO metrics were weak and that they criticized the metrics partly to justify their project.They nevertheless maintain their criticism of the metrics.
- Metric criticism: The rebuttal argues that updating mAP should address known issues or justify why the updated metric is better.It contrasts COCO with PASCAL VOC’s explicitly justified low IoU threshold and notes that COCO may have better labels because it uses segmentation masks.
- Metric criticism: The COCO metric emphasizes more precise bounding boxes while de-emphasizing classification accuracy, whose errors may be more obvious to people.The rebuttal questions whether tighter boxes are more important than better classification.
- Metric criticism: mAP can rank two detectors as equally good despite obvious discrepancies in their hypothetical outputs, because it depends only on per-class rank ordering.The rebuttal calls this an exaggeration but uses it to motivate metrics aligned with real-world concerns.
- Metric proposal: The proposed alternatives measure global average precision or average per-image precision to capture how well a detector finds and classifies objects in an image.These proposals remove or reduce reliance on per-class AP.