Source-linked AI summary
SSDA-YOLO: Semi-supervised Domain Adaptive YOLO for Cross-Domain Object Detection
Huayi Zhou, Fei Jiang, Hongtao Lu
TL;DR
Source-trained detectors can degrade under domain shifts, and many DAOD methods rely on two-stage Faster R-CNN. SSDA-YOLO combines YOLOv5, Mean Teacher distillation, translated images, and consistency loss; the authors report improvements across public and classroom detection tasks.
Problem
Domain shifts can degrade object detection, while many DAOD methods rely on computationally intensive two-stage Faster R-CNN.
Source: §1 · p. 1Method
Results
SSDA-YOLO reports comparable or superior improvements across two public domain transfers and generalization in classroom yawning detection; on Foggy Cityscapes, BaseDC reaches 55.9 mAP versus TDD's 49.2.
Source: §1 · p. 2, §5.2.2 · p. 8, §6 · p. 9Takeaways & Limitations
Abstract
from arXiv · showhide
Domain adaptive object detection (DAOD) aims to alleviate transfer performance degradation caused by the cross-domain discrepancy. However, most existing DAOD methods are dominated by outdated and computationally intensive two-stage Faster R-CNN, which is not the first choice for industrial applications. In this paper, we propose a novel semi-supervised domain adaptive YOLO (SSDA-YOLO) based method to improve cross-domain detection performance by integrating the compact one-stage stronger detector YOLOv5 with domain adaptation. Specifically, we adapt the knowledge distillation framework with the Mean Teacher model to assist the student model in obtaining instance-level features of the unlabeled target domain. We also utilize the scene style transfer to cross-generate pseudo images in different domains for remedying image-level differences. In addition, an intuitive consistency loss is proposed to further align cross-domain predictions. We evaluate SSDA-YOLO on public benchmarks including PascalVOC, Clipart1k, Cityscapes, and Foggy Cityscapes. Moreover, to verify its generalization, we conduct experiments on yawning detection datasets collected from various real classrooms. The results show considerable improvements of our method in these DAOD tasks, which reveals both the effectiveness of proposed adaptive modules and the urgency of applying more advanced detectors in DAOD. Our code is available on \url{https://github.com/hnuzhy/SSDA-YOLO}.
1. INTRODUCTION
Object detectors trained on a source domain can lose performance on target scenes with different styles, lighting, image quality, or camera perspectives, while obtaining more labeled data can be impractical. SSDA-YOLO combines YOLOv5 with semi-supervised adaptation and reports improvements across public and classroom detection tasks.
1. INTRODUCTION
- Cross-domain shifts can degrade detection, while collecting additional labeled data may be impractical or impossible in some scenarios.Target scenes may differ in image style, lighting, image quality, or camera perspective.Source: §1 · p. 1
- Most DAOD approaches follow two-stage Faster R-CNN, despite one-stage detectors offering computational advantages and YOLO-based methods being lighter-weight.The paper frames the continued dominance of Faster R-CNN as a concern for resource-limited and time-critical applications.Source: §1 · p. 1
- SSDA-YOLO uses YOLOv5 with a Mean Teacher-guided teacher-student framework to learn from unlabeled target images and iteratively generate filtered pseudo-labels.It also uses offline CUT-generated pseudo-images as additional inputs to address image-level differences.Source: §1 · p. 2
- The authors report comparable or superior accuracy improvements on PascalVOC→Clipart1k and Cityscapes→Foggy Cityscapes, plus generalization in classroom yawning detection.They identify distillation and consistency losses as effective components and argue for using more advanced detectors in DAOD.Source: §1 · p. 2
2. RELATED WORKS
The related work contrasts detector architectures and traces domain adaptation from Faster R-CNN-based methods toward one-stage and semi-supervised approaches. It motivates SSDA-YOLO as a combination of YOLOv5, teacher-student learning, and translated training images.
2.1. Object Detection
- Two-stage detectors use region proposals, while one-stage detectors directly predict boxes and classes; the latter are faster and more compact for time-sensitive or computation-limited settings.The paper selects YOLOv5, citing its efficiency, training optimizations, and data augmentation versatility.Source: §2.1 · p. 2
2.2. Cross-Domain Object Detection
- DAOD has largely used Faster R-CNN for local adaptation, while newer one-stage methods adapt FCOS, SSD, and YOLO architectures.Prior work includes feature alignment, noisy-label learning, image generation, graph matching, and other adaptation designs.Source: §2.2 · pp. 2–3
- The authors question whether DAOD's reliance on Faster R-CNN limits performance, and argue that detector architecture and training strategy both matter.They position SSDA-YOLO as an effort to replace this status quo with YOLOv5 and adaptive modules.Source: §2.2 · p. 3
2.3. Semi-supervised Domain Adaptation
- In the stated UDA setting, target labels are unavailable during training; semi-supervised work also considers limited target labels and Mean Teacher-based adaptation.Prior approaches include target image-level annotations and teacher-student or knowledge-distillation strategies.Source: §2.3 · p. 3
- SSDA-YOLO uses source supervision and target-domain unsupervised learning, with target images translated to have source-like global scenes before entering the teacher model.The authors describe this combination as their semi-supervised domain adaptation approach.Source: §2.3 · p. 3
3. PRELIMINARIES AND MOTIVATION
The paper identifies challenges in adapting one-stage detectors and in optimizing shared-network adversarial training. It motivates a teacher-student approach that distills final responses and combines translated images with Mean Teacher target-feature learning.
3. PRELIMINARIES AND MOTIVATION
- Shared-network adversarial adaptation can be difficult to optimize and converge, while DAOD methods commonly align features at instance and image levels.The paper notes that Faster R-CNN offers clear classification and localization branches that facilitate such alignment.Source: §3 · p. 3
- Earlier DAOD methods use a shared network and adversarial objectives, whereas teacher-student methods distill target-relevant knowledge into a detector.The paper distinguishes its approach by distilling final responses from identical-architecture YOLOv5 teacher and student models rather than intermediate features.Source: §3 · pp. 3–4
- Prior one-stage adaptation methods design discriminators or adaptive modules for cross-domain features, while SSDA-YOLO combines cross-generated images with Mean Teacher target-feature learning.The paper assigns translated images to image-level shifts and the Mean Teacher model to instance-level target features.Source: §3 · p. 4
4. PROPOSED METHOD
SSDA-YOLO combines YOLOv5 with teacher–student distillation, cross-domain pseudo-images, and a consistency loss to adapt detection using labeled source and unlabeled target data. Its training uses jointly optimized losses, while inference uses only the trained student on target images.
4. PROPOSED METHOD
- SSDA-YOLO adapts YOLOv5 with a Mean Teacher distillation framework, cross-generated pseudo-images, an updated distillation loss, and a consistency loss.These components target instance-level and image-level domain differences and cross-domain objectness bias.Source: §4 · p. 4
4.1. Definition of Terms
- The task uses annotated source images and unlabeled target images, with the goal of maximizing detection performance on the target domain.The supervised source loss combines GIoU box loss with Focal loss for classification and objectness.Source: §4.1 · p. 4
4.2. Mean Teacher Model
- The Mean Teacher framework updates the teacher from student weights by exponential moving average and uses filtered teacher predictions as target-domain pseudo-labels for student training.High-confidence boxes and category scores are retained to provide target-domain instance-level features.Source: §4.2 · pp. 4–5
4.3. Pseudo Training Images Generation
- CUT generates target-like source images and source-like target images offline, with paired source images and their translated versions appearing together during training.The authors select CUT for faster, more robust unpaired scene transfer.Source: §4.3 · p. 5
4.4. Remedying Cross-Domain Discrepancy
- Target-like source images train a supervised student branch like source images, while source-like target images replace teacher inputs and the student continues using unlabeled target images.The revised distillation setup is linked by the Mean Teacher EMA update and is intended to avoid a teacher biased only toward target-domain objects.Source: §4.4 · p. 5
4.5. Consistency Loss Function
- Because source and target-like images share a label space despite different scene distributions, the method constrains their final predictions rather than intermediate features.The consistency loss uses the L2 distance between outputs; L1 is also considered for ablation.Source: §4.5 · pp. 5–6
4.6. Overall Optimization
- Translated images are generated offline and all related losses are optimized jointly, while inference uses only the trained student with a target image as input.The overall loss uses trade-off hyperparameters α and β.Source: §4.6 · p. 6
5.1. Training Configuration
Training uses paired labeled source and unlabeled target images with YOLOv5-L, fixed image sizing, and specified teacher, filtering, and loss hyperparameters.
5.1. Training Configuration
- 960 × 960 × 3 images are used, and each batch contains two labeled source-image pairs and two unlabeled target-image pairs.The batch size is 10 on one 24GB GTX3090 GPU, with 200 training epochs.Source: §5.1 · p. 6
- YOLOv5-L is the detector, with teacher EMA γ = 0.99 and pseudo-label thresholds τbox = 0.3 and τcls = 0.8.The authors select YOLOv5-L for comparable detection accuracy to state-of-the-art methods at about real-time speed.Source: §5.1 · p. 6
- The overall-loss weights are α = 0.005 and β = 2.0, and the consistency loss uses L2 distance.The authors state that these settings are discussed in ablation studies.Source: §5.1 · p. 6
5.2. Transfer Experiment Design
The transfer experiments test SSDA-YOLO on real-to-virtual, clear-to-foggy, and cross-classroom yawning detection. They compare adaptation results across public benchmarks and collected classroom data.
5.2. Transfer Experiment Design
- The suite covers PascalVOC-to-Clipart1k, Cityscapes-to-Foggy Cityscapes, and yawning detection across classroom domains.The VOC setup combines 2007 and 2012 data (16,551 images) with 1,000 Clipart1k images; the classroom sets contain 12,924 source and 1,990 target images.Source: §5.2 · p. 6, §5.2.1 · p. 6, §5.2.2 · p. 7
5.2.1. Real to Virtual Adaptation
- 44.3 mAP and 16.8 Gain for BaseDC on Clipart1k are reported as comparable to UMT's 44.1 mAP.BaseDC's mAP exceeds BaseD's 41.2 and BaseC's 42.2; the Base configuration using cross-generated fake images alone scores 40.4.Source: §5.2.1 · p. 6, Table 1 · p. 7
5.2.2. Normal to Adverse Weather Adaptation
- 55.9 mAP for BaseDC on Foggy Cityscapes exceeds TDD's 49.2 mAP; YOLOv5 Source Only reaches 35.9 mAP.The authors report that BaseDC is below PT and TDD on relative improvement, while outperforming several methods proposed the same year.Source: Table 2 · p. 7, §5.2.2 · p. 8
5.2.3. Cross Classroom Yawning Detection Adaptation
- 52.5 mAP.5:.95 for classroom BaseDC compares with 38.7 for Source Only and 65.1 for Oracle, leaving a gap to Oracle.Base, BaseD, and BaseC score 43.7, 46.2, and 47.6 mAP.5:.95, respectively; the authors also note lower relative results than in earlier experiments.Source: §5.2.3 · p. 8, Table 3 · p. 8
5.3. Ablation Studies
This ablation examines loss-weight choices and the consistency loss using YOLOv5-S on PascalVOC7→Clipart1k, with training shortened to 100 epochs.
5.3.1. Weight α for Distillation Loss
- α = 0.005 gives the highest mAP.50; smaller or larger weights reduce distillation-loss performance.The tested α values ranged from 0.001 to 0.1, increasing by about 2× at each step.Source: §5.3.1 · p. 8, Figure 5 · p. 9
5.3.2. Weight β for Consistency Loss
- 34.7 mAP with L2 loss at β = 2.0 exceeds 33.8 with L1 loss at β = 0.5; β = 2.0 is the reported optimum.The weight β is larger than α because of the absolute values of the corresponding losses.Source: §5.3.2 · p. 9, Figure 6 · p. 9
5.3.3. Validity of Consistency Loss
- Adding consistency loss promotes convergence of Lcls, Lbox, and Lobj, especially the objectness loss Lobj.The comparison evaluates the target-domain test set with consistency loss added versus the base setup.Source: §5.3.3 · p. 9
6. CONCLUSION
SSDA-YOLO combines YOLOv5 with teacher–student distillation, style-transfer-generated pseudo-images, and consistency loss for cross-domain detection. The paper reports results on public benchmarks and self-made classroom yawning datasets.
6. CONCLUSION
- SSDA-YOLO uses YOLOv5 with a Mean Teacher-based teacher and student, style transfer to cross-generate pseudo-training images, and consistency loss to correct prediction shifts.Source: §6 · p. 9
- The target-domain loss-trend comparison uses a base setup, a distillation-only setup, and a consistency-only setup.Source: Figure 7 · p. 9
- The authors report that final results demonstrate SSDA-YOLO’s effectiveness and superiority in real cross-domain detection applications, based on public benchmarks and self-made yawning datasets.Source: §6 · p. 9