Source-linked AI summary
Multiscale Domain Adaptive YOLO for Cross-Domain Object Detection
Mazin Hnewa, Hayder Radha
TL;DR
Object detection can degrade when source training and target testing distributions differ, while target-domain annotations may be unavailable. MS-DAYOLO adds multiscale domain-adaptation paths to YOLOv4 during training. Experiments report improved detection in foggy and rainy target settings.
Problem
Object detection performance drops when testing data differs from training data, and annotated target-domain data may be unavailable.
Source: §1 · p. 1Method
MS-DAYOLO adapts YOLOv4 using domain-adaptation paths at multiple backbone feature scales and removes the DAN for inference.
Source: §1 · p. 2, §2 · p. 2, §2.1 · p. 2Results
MS-DAYOLO improves target-domain detection over original YOLOv4 in clear-to-foggy and sunny-to-rainy experiments.
Source: §3.2 · p. 4Takeaways & Limitations
The authors report adapting YOLO to target domains without annotation and improved performance across diverse autonomous-driving testing scenarios.
Source: §4 · p. 5
Abstract
from arXiv · showhide
The area of domain adaptation has been instrumental in addressing the domain shift problem encountered by many applications. This problem arises due to the difference between the distributions of source data used for training in comparison with target data used during realistic testing scenarios. In this paper, we introduce a novel MultiScale Domain Adaptive YOLO (MS-DAYOLO) framework that employs multiple domain adaptation paths and corresponding domain classifiers at different scales of the recently introduced YOLOv4 object detector to generate domain-invariant features. We train and test our proposed method using popular datasets. Our experiments show significant improvements in object detection performance when training YOLOv4 using the proposed MS-DAYOLO and when tested on target data representing challenging weather conditions for autonomous driving applications.
1. INTRODUCTION
Object detection degrades when testing data differs from training data, including shifts caused by weather or viewpoint. The paper motivates adapting YOLOv4 because target labels are scarce and YOLO is suited to time-critical applications, then introduces MS-DAYOLO for cross-domain detection.
1. INTRODUCTION
- Under domain shift, state-of-the-art object detectors can lose performance when testing data differs from training data, for example across weather or viewpoints.The paper identifies clear-weather training and foggy-weather testing as an autonomous-driving example.Source: §1 · p. 1
- Domain adaptation addresses scarce target annotations by training with labeled source data and unlabeled target data while learning domain-invariant features.A domain classifier distinguishes source from target as the detector’s feature extractor learns to confuse it.Source: §1 · p. 1
- Prior domain-adaptive detection focused on Faster R-CNN, while YOLO’s speed suits time-critical driving despite receiving little adaptation research.The paper identifies this gap as motivation for adapting YOLOv4.Source: §1 · p. 2
- MS-DAYOLO applies domain adaptation at multiple feature-extraction layers of YOLOv4, using multiscale inputs and multiple domain classifiers.The authors report significant improvement in YOLOv4 detection performance on target domains.Source: §1 · p. 2
2. PROPOSED MS-DAYOLO
MS-DAYOLO adapts YOLOv4’s backbone features at three scales through a Domain Adaptive Network (DAN). The DAN uses adversarial domain classification during training and is omitted at inference; separate adaptation paths address the limited influence of adapting only the final scale.
2. PROPOSED MS-DAYOLO
- YOLOv4 extracts multiscale backbone features, combines them in the neck, and uses the head to predict object boxes and class probabilities.The proposed adaptation targets backbone features F1, F2, and F3 that feed the neck.Source: §2 · p. 2
2.1. Domain Adaptive Network for YOLO
- During training, DAN classifies source versus target features while the backbone learns domain-invariant representations; DAN is removed for inference, leaving detector complexity unchanged.Detection loss uses labeled source images, while domain-classification loss uses labeled source and unlabeled target images; a Gradient Reversal Layer supports the opposing objectives.Source: §2.1 · pp. 2–3
2.2. DAN Architecture
- Three separate adaptation paths connect features F1, F2, and F3 to DAN because adaptation at only the final scale may have limited effect on earlier scales due to gradient vanishing.Each path uses a GRL followed by convolutional layers and a domain classifier.Source: §2.2 · p. 3
3. EXPERIMENTS
Experiments evaluate MS-DAYOLO against original YOLOv4 on target-domain weather shifts in driving datasets. Three-scale adaptation achieves the best clear-to-foggy result, while MS-DAYOLO improves performance in sunny-to-rainy tests on both BDD100K and INIT.
3.1. Setup
- Training uses YOLOv4 defaults with 32 labeled-source and 32 unlabeled-target images per batch; evaluation reports AP and mAP at threshold 0.5.Original YOLOv4 and MS-DAYOLO are compared on the same target-domain validation set, rather than against Faster R-CNN-based methods.Source: §3.1 · pp. 3–4
3.2. Results and Discussion
- Clear-to-foggy tests use Cityscapes as labeled source and Foggy Cityscapes as unlabeled target, excluding truck, bus, and train from mAP due to low counts.Labeled Foggy Cityscapes training data is also used to train an oracle model.Source: §3.2 · p. 4
- The ablation tests each of three feature scales separately, then every two-scale combination, against adaptation at all three scales.The datasets cover clear-to-foggy Cityscapes and sunny-to-rainy BDD100K and INIT weather shifts.Source: §3.2 · p. 4, §3.1 · p. 4
- Adapting all three feature scales achieves the best clear-to-foggy detection result and nearly reaches the oracle, especially for some classes and overall mAP.Table 1 reports clear-to-foggy Cityscapes results; Figure 1 shows example detections compared with original YOLOv4.Source: Table 1 · p. 4, §3.2 · p. 4
- MS-DAYOLO improves performance over original YOLO on sunny-to-rainy adaptation in both BDD100K and INIT.The method trains with labeled sunny images and unlabeled rainy images, then evaluates on labeled rainy data.Source: Table 2 · p. 4, §3.2 · p. 4
4. CONCLUSION
The conclusion presents MS-DAYOLO as multiscale adaptation for real-time YOLO, applying adaptation to three feature scales. It reports improved target-domain detection without target annotation and better performance than YOLOv4 in diverse autonomous-driving tests.
4. CONCLUSION
- MS-DAYOLO applies domain adaptation to three feature scales in the YOLO feature extractor.Source: §4 · p. 5
- The framework improves overall target-domain detection without requiring target-domain annotation.Source: §4 · p. 5
- MS-DAYOLO outperforms state-of-the-art YOLOv4 across diverse autonomous-driving testing scenarios.Source: §4 · p. 5