Source-linked AI summary
Cross-Domain Adaptive Teacher for Object Detection
Yu-Jhe Li, Xiaoliang Dai, Chih-Yao Ma, Yen-Cheng Liu, Kan Chen, Bichen Wu, Zijian He, Kris Kitani, Peter Vajda
TL;DR
Cross-domain object detection must transfer knowledge from labeled source data to unlabeled target data despite domain shift and noisy teacher-generated pseudo labels. Adaptive Teacher addresses this with a target-specific Teacher, cross-domain Student, weak-strong augmentation, mutual learning, and feature-level adversarial alignment. It reports 50.9% mAP on Foggy Cityscape and 49.3% on Clipart1K, exceeding prior state-of-the-art and Oracle models in the reported comparisons.
Problem
Cross-domain teacher-student detection suffers from source-domain bias and low-quality target pseudo labels, including false positives, under domain shift.
Method
Adaptive Teacher combines a target-specific Teacher using weak target images with a cross-domain Student using strongly augmented source and target images, mutual learning, and feature-level adversarial alignment.
Results
50.9% mAP on Foggy Cityscape and 49.3% mAP on Clipart1K exceed previous state-of-the-art and Oracle models in the reported comparisons.
Takeaways & Limitations
Experiments on two benchmarks support Adaptive Teacher's effectiveness and superiority for cross-domain object detection, including performance above fully supervised Oracle models.
Takeaways & Limitations
Adversarial learning is applied to the Student because it processes both domains and is updated through objective losses, while unsupervised bounding-box regression losses are not applied.
Abstract
from arXiv · showhide
We address the task of domain adaptation in object detection, where there is a domain gap between a domain with annotations (source) and a domain of interest without annotations (target). As an effective semi-supervised learning method, the teacher-student framework (a student model is supervised by the pseudo labels from a teacher model) has also yielded a large accuracy gain in cross-domain object detection. However, it suffers from the domain shift and generates many low-quality pseudo labels (\textit{e.g.,} false positives), which leads to sub-optimal performance. To mitigate this problem, we propose a teacher-student framework named Adaptive Teacher (AT) which leverages domain adversarial learning and weak-strong data augmentation to address the domain gap. Specifically, we employ feature-level adversarial training in the student model, allowing features derived from the source and target domains to share similar distributions. This process ensures the student model produces domain-invariant features. Furthermore, we apply weak-strong augmentation and mutual learning between the teacher model (taking data from the target domain) and the student model (taking data from both domains). This enables the teacher model to learn the knowledge from the student model without being biased to the source domain. We show that AT demonstrates superiority over existing approaches and even Oracle (fully-supervised) models by a large margin. For example, we achieve 50.9% (49.3%) mAP on Foggy Cityscape (Clipart1K), which is 9.2% (5.2%) and 8.2% (11.0%) higher than previous state-of-the-art and Oracle, respectively.
1. Introduction
Cross-domain object detection transfers knowledge from labeled source data to unlabeled target data, but domain shift limits adversarial and teacher-student methods. Adaptive Teacher combines adversarial learning, mutual learning, and weak-strong augmentation to improve target pseudo-label quality and detection performance.
- Cross-domain object detection transfers knowledge from labeled source data to unlabeled target data, but existing methods often retain a substantial gap from fully supervised Oracle models.
- Teacher-student adaptation gains accuracy without target annotations, yet source-biased teachers generate erroneous and false-positive pseudo labels under large domain gaps.
- Adaptive Teacher combines a target-specific Teacher, a cross-domain Student, adversarial learning, mutual learning, and weak-strong augmentation.
- Weak augmentation on target images helps the Teacher generate reliable pseudo labels, while adversarial alignment mitigates source-domain bias in Student features.
- 35%: Adaptive Teacher suppresses the false positive ratio by up to 35%, producing substantial accuracy gains across domain adaptation experiments.
- 50.9% mAP: Adaptive Teacher exceeds prior state-of-the-art by 9.2% and Oracle by 8.2% on Foggy Cityscape.
2. Related Works
Related work spans general object detection, domain adaptation, adversarial feature alignment, and teacher-student self-training. Adaptive Teacher combines these directions for cross-domain object detection.
- Object Detection: Object detection localizes objects and their bounding boxes, using anchor-based or anchor-free detectors; this work employs Faster R-CNN as its backbone.
- Domain Adaptation: Domain adaptation learns from labeled source data and unlabeled target data, using discrepancy minimization, adversarial domain classifiers, or image-level translation.
- Cross-Domain Object Detection: Cross-domain object detection is challenging because models must predict both bounding boxes and class labels while handling domain differences.
- Cross-Domain Object Detection: Prior approaches use gradient-reversal adversarial learning, annotation-level adaptation, curriculum learning, or Mean Teacher-based consistency methods.
- Cross-Domain Object Detection: Adaptive Teacher uses a target-specific Teacher and cross-domain Student trained through Teacher-Student mutual learning and adversarial learning.
3. Adaptive Teacher
Adaptive Teacher combines a target-specific Teacher with a cross-domain Student, using weak-strong augmentation, pseudo-label filtering, mutual learning, and adversarial feature alignment. The framework is designed to reduce source-domain bias and improve target-domain pseudo-label quality.
- Problem Formulation: The problem formulation provides labeled source images with bounding boxes and class labels but unlabeled target images, aiming to learn domain-invariant detectors.
- Framework Overview: Adaptive Teacher uses a target-specific Teacher and a cross-domain Student trained through Teacher-Student mutual learning and adversarial learning.
- Weak-Strong Augmentation: The Teacher receives weakly augmented target images, while the Student receives strongly augmented images from both source and target domains.
- Pseudo-Label Optimization: Target pseudo-labels are filtered with a confidence threshold and non-maximum suppression before supervising the Student.
- Pseudo-Label Optimization: Unsupervised losses exclude bounding-box regression because confidence scores on unlabeled data reflect category confidence rather than localization confidence.
- Adversarial Learning: The Student aligns source and target feature distributions with a domain discriminator and adversarial optimization, helping the Teacher produce more precise pseudo-labels after EMA updates.
- Experiments: Evaluation tables report average precision on Clipart1k and Watercolor2k using ResNet-101, with comparisons against adaptation methods, source-only, and oracle baselines.
- Adversarial Learning: The design applies adversarial learning to the Student because it processes both domains, while target-only Teacher inputs avoid introducing additional source-domain bias.
4. Experiment
Experiments evaluate Adaptive Teacher across cross-domain detection and unseen-domain generalization settings, using AP or mAP with source-only and oracle benchmarks. Results show strong gains from adversarial loss, weak-strong augmentation, and mutual learning, with AT outperforming existing methods and oracle models in reported settings.
- Experimental settings: AT is evaluated on five public datasets spanning real-to-artistic, adverse-weather, and unseen-domain generalization settings.Cityscapes, Foggy Cityscapes, PASCAL VOC, Clipart1k, and Watercolor2k are used.
- Real to Artistic Adaptation: 49.3% mAP on Clipart1k surpasses UMT by 5.2% and makes AT the only method exceeding the oracle model in that comparison.The experiments use PASCAL VOC as source and Clipart1k as target for real-to-artistic adaptation.
- Adverse Weather Adaptation: AT outperforms all state-of-the-art approaches by more than 9% and exceeds the oracle model on Cityscapes-to-Foggy Cityscapes adaptation.Cityscapes provides labeled normal-weather images, while Foggy Cityscapes provides unlabeled adverse-weather images.
- Domain generalization: AT achieves superior unseen-domain generalization compared with Oracle and MT without observing target images during training.The reported setting trains on labeled PASCAL VOC and unlabeled Clipart1k, then tests on Watercolor2k using six overlapped classes.
- Ablation studies: Removing adversarial loss causes 8.7% and 4.4% drops on Clipart1k and Watercolor2k, while weather adaptation drops by 2.2%.The adversarial loss also reduces false positives in teacher-generated pseudo labels, and increasing its weight improves performance and learning stability.
- Ablation studies: Excluding weak-strong augmentation causes around 4% to 5% performance drops, while removing mutual learning produces a significant performance drop.The augmentation pipeline also reduces false positives, and the ablation attributes the main gain to mutual learning with target-domain pseudo labels.
5. Conclusion
Adaptive Teacher addresses cross-domain object detection with target-domain Teacher and cross-domain Student models, proper augmentation, adversarial learning, and mutual learning. Experiments and ablations support its effectiveness, including performance exceeding fully supervised Oracle training.
- Adaptive Teacher uses target-domain Teacher and cross-domain Student models to generate correct target-domain pseudo labels through mutual learning.The framework combines proper augmentation strategies and adversarial learning to address source-domain bias in both models.
- The training pipeline combines augmentation strategies and adversarial learning to resolve bias toward the source domain in both Teacher and Student models.
- AT outperforms the Oracle model trained with full supervision without seeing target-domain labels or images.
- Experiments on two benchmarks and extensive ablation studies confirm the effectiveness and superiority of the proposed framework for cross-domain object detection.