Source-linked AI summary
A Free Lunch for Unsupervised Domain Adaptive Object Detection without Source Data
Xianfeng Li, Weijie Chen, Di Xie, Shicai Yang, Peng Yuan, Shiliang Pu, Yueting Zhuang
TL;DR
The paper addresses object detection adaptation when labeled source data cannot be accessed. It proposes SFOD, which uses a pretrained source model, self-entropy descent for pseudo-label thresholding, and false-negative simulation; the framework is reported to achieve strong performance across adaptation tasks and to surpass most source-data-based UDA methods.
Problem
Source data-free unsupervised domain adaptive object detection lacked a solution despite privacy and transmission constraints on accessing labeled source data.
Method
SFOD models adaptation as noisy-label learning, generates target pseudo labels with a pretrained source model, selects thresholds using self-entropy descent, and simulates false negatives with augmentation.
Results
The framework is reported to achieve strong performance without source data and surpass most source-data-based UDA methods.
Takeaways & Limitations
The results suggest that existing object-detection UDA methods do not fully exploit labeled source data.
Takeaways & Limitations
During adaptation, supervision comes from the pretrained source model rather than direct access to source-domain data.
Abstract
from arXiv · showhide
Unsupervised domain adaptation (UDA) assumes that source and target domain data are freely available and usually trained together to reduce the domain gap. However, considering the data privacy and the inefficiency of data transmission, it is impractical in real scenarios. Hence, it draws our eyes to optimize the network in the target domain without accessing labeled source data. To explore this direction in object detection, for the first time, we propose a source data-free domain adaptive object detection (SFOD) framework via modeling it into a problem of learning with noisy labels. Generally, a straightforward method is to leverage the pre-trained network from the source domain to generate the pseudo labels for target domain optimization. However, it is difficult to evaluate the quality of pseudo labels since no labels are available in target domain. In this paper, self-entropy descent (SED) is a metric proposed to search an appropriate confidence threshold for reliable pseudo label generation without using any handcrafted labels. Nonetheless, completely clean labels are still unattainable. After a thorough experimental analysis, false negatives are found to dominate in the generated noisy labels. Undoubtedly, false negatives mining is helpful for performance improvement, and we ease it to false negatives simulation through data augmentation like Mosaic. Extensive experiments conducted in four representative adaptation tasks have demonstrated that the proposed framework can easily achieve state-of-the-art performance. From another view, it also reminds the UDA community that the labeled source data are not fully exploited in the existing methods.
Introduction
The paper introduces source data-free unsupervised domain adaptive object detection, using a pretrained source model and target-domain pseudo labels instead of source data. It addresses noisy pseudo labels with self-entropy descent and false-negative simulation.
- Introduction: Source data-free object detection addresses adaptation to unlabeled target domains when source data cannot be accessed because of privacy and transmission constraints.
- Introduction: The proposed SFOD framework models source data-free adaptation as learning with noisy labels using reliable target pseudo labels generated from a pretrained source model.
- Introduction: Object detection makes pseudo-label learning difficult because negative samples are numerous and hard positives can be confused with negatives.
- Introduction: Self-entropy descent searches confidence thresholds by selecting the first local minimum of mean self-entropy as scores decrease.Lower self-entropy indicates more confident predictions, while noisier labels are harder to fit.
- Introduction: False negatives dominate the generated noisy labels, motivating false-negative simulation through Mosaic-style augmentation.The simulation exploits easy positive samples to mimic false negatives.
- Introduction: The framework is presented as the first source data-free unsupervised domain adaptive object detection method and is reported to surpass most source-data-based UDA methods.
Related Works
Prior work includes feature-, image-, and pseudo-label-based UDA methods, while source data-free adaptation had mainly advanced in classification. The paper positions SFOD within noisy-label learning for object detection and uses self-entropy to guide threshold selection.
- Related Works: Earlier UDA detectors aligned image or instance distributions using domain-invariant features, image-space transformations, or pseudo-label similarity.
- Domain Adaptive Object Detection: The toy analysis relates noise degree to mean self-entropy, with sample imbalance producing local minima at both curve ends.
- Domain Adaptive Object Detection: SED searches an appropriate positive-negative confidence threshold from higher to lower scores using self-entropy.
- Related Works: Source data-free domain adaptation had progressed in classification, but source data-free unsupervised domain adaptive object detection remained unaddressed.
- Learning with Noisy Labels: Existing noisy-label research largely targets classification and often assumes manually specified, identically distributed noise.
- Learning with Noisy Labels: Object-detection noisy labels include hard positive and negative objects, and prior classification-based treatment cannot handle objects that are difficult to box out.
Source free Domain Adaptive Object Detection
Source-data-free object detection adapts a source-pretrained detector using unlabeled target data and pseudo labels, with SED selecting a confidence threshold. Because pseudo labels remain noisy and false negatives dominate, SFOD simulates missed objects through Mosaic augmentation.
- Source-free UDA optimizes the detector using unlabeled target data and a source-pretrained model rather than directly accessing labeled source data.
- Self-entropy descent: SED searches confidence thresholds from high to low and selects the first local minimum after mean self-entropy descends.Mean self-entropy is used as an indicator of pseudo-label reliability.
- Self-entropy descent: The detector assigns the highest-scoring foreground class when its confidence exceeds the threshold; otherwise, it assigns background.
- False negatives simulation: More than 50% of positive samples can remain difficult to box out near a zero threshold, while false positives account for a relatively small proportion.These missed positives behave as false negatives during training.
- False negatives simulation: Mosaic augmentation simulates small and obscured false negatives by randomly scaling and cutting detected positives.Its construction mixes four training images and is intended to suppress false-negative effects without harming true negatives.
- False negatives simulation: The SFOD pipeline combines SED-based threshold selection with false-negative simulation for target-domain training.
Experimental Setup
Experiments use five public datasets and evaluate four domain-adaptation settings under a common Faster-RCNN-based protocol. The setup uses resized images, ImageNet-pretrained VGG-16, SGD, and source data only during pretraining.
- Datasets: Five public datasets support the experiments, including KITTI, Sim10k, Cityscapes, and BDD100k.The supplied dataset descriptions identify these datasets and their roles in the adaptation benchmarks.
- Datasets: The experiments include adaptation from synthetic Sim10k images to real Cityscapes images.Table 2 reports this synthetic-to-real setting.
- Training Protocol: The common protocol resizes training and testing images to a 600-pixel short side and uses ImageNet-pretrained VGG-16 in Faster-RCNN.The detector is trained with SGD at a learning rate of 0.001 and batch size 1.
- Datasets: The large-scale adaptation benchmark transfers from Cityscapes to BDD100k daytime data.Table 3 identifies this as the Cityscapes-to-BDD100k daytime setting.
Comparison Results
Across four adaptation tasks, SFOD achieves competitive results without source data, with improvements reported for camera, synthetic-to-real, large-scale, and foggy-weather shifts. Performance remains constrained in the foggy setting relative to traditional UDA methods.
- KITTI to Cityscapes: 43.6% to 44.6% AP: Mosaic improves SFOD on KITTI-to-Cityscapes car detection and exceeds DA-Detection by 0.7%.SED alone already produces AP close to the ideal threshold selected with labeled target validation data.
- Sim10k to Cityscapes: SFOD achieves superior or comparable results to DA-Faster, Noise Labeling, and AT-Faster on Sim10k-to-Cityscapes adaptation.The source-data-free setting is described as more challenging than the source-data-based comparison methods.
- Cityscapes to BDD100k: 26.9% to 27.6% mAP: SED or SED+Mosaic improves over CR-DA-DET on Cityscapes-to-BDD100k daytime adaptation.The task evaluates seven shared object categories across a large unlabeled target domain.
- Cityscapes to Foggy Cityscapes: 22.3% to 33.5% mAP: SED with Mosaic improves foggy-weather adaptation, although it remains below traditional UDA performance.The comparison covers eight common categories from Cityscapes to Foggy Cityscapes.
- Cityscapes to Foggy Cityscapes: Defogging improves SFOD performance by approximately 3% on Cityscapes-to-Foggy-Cityscapes adaptation.The authors conclude that fog aggravates pseudo-label noise and affects detection performance.
Discussion and Analysis
The analysis attributes SFOD’s effectiveness to suppressing noisy-label effects, especially false negatives, so more objects are detected without source-domain data. The reported results are comparable or superior to source-data-based UDA methods.
- Discussion and Analysis: SED and false-negative simulation suppress noisy-label effects, enabling more objects to be detected.Figure 8 visualizes detection results for SED alone and SED combined with false-negative simulation.
- Discussion and Analysis: SFOD achieves comparable or superior results to existing source-data-based UDA methods, suggesting that source data are not fully exploited by those methods.This conclusion is presented as an implication of the comparative results.
Conclusion
The paper presents SFOD as a new learning paradigm for unsupervised domain adaptive object detection using only a pretrained source model. It frames noisy-label removal as a critical unresolved challenge and identifies this work as a first step in the direction.
- SFOD introduces a new learning paradigm for unsupervised domain adaptive object detection.
- The approach uses a pretrained source-domain model instead of source data to provide supervision signals.
- Completely removing false positives and false negatives from noisy labels remains very difficult without supervision.
- The work is presented as the first attempt in this direction and as a source of inspiration for the UDA community.