Source-linked AI summary
Instance Relation Graph Guided Source-Free Domain Adaptive Object Detection
Vibashan VS, Poojan Oza, Vishal M. Patel
TL;DR
Object detectors can generalize poorly to unfamiliar visual domains, while UDA requires source data that may be restricted. This paper adapts source-trained detectors without source data using graph-guided contrastive learning and student-teacher distillation, and reports that it outperforms prior adaptation methods across multiple benchmarks.
Problem
Object detectors can generalize poorly to unfamiliar visual domains, and UDA requires source data that may be restricted for privacy, transmission, or proprietary reasons.
Source: §1 · p. 1Method
The method uses an Instance Relation Graph to model relations among target-image proposals and guide contrastive learning, alongside student-teacher distillation for noisy pseudo-labels.
Source: §1 · p. 2, §3.1 · p. 4, §3.2.1 · p. 5Results
Across multiple detection benchmarks, the method outperforms state-of-the-art source-free domain adaptation and many unsupervised domain adaptation methods.
Source: §5 · p. 9Takeaways & Limitations
The approach efficiently adapts a source-trained object detector to a target domain without source data.
Source: §5 · p. 9
Abstract
from arXiv · showhide
Unsupervised Domain Adaptation (UDA) is an effective approach to tackle the issue of domain shift. Specifically, UDA methods try to align the source and target representations to improve the generalization on the target domain. Further, UDA methods work under the assumption that the source data is accessible during the adaptation process. However, in real-world scenarios, the labelled source data is often restricted due to privacy regulations, data transmission constraints, or proprietary data concerns. The Source-Free Domain Adaptation (SFDA) setting aims to alleviate these concerns by adapting a source-trained model for the target domain without requiring access to the source data. In this paper, we explore the SFDA setting for the task of adaptive object detection. To this end, we propose a novel training strategy for adapting a source-trained object detector to the target domain without source data. More precisely, we design a novel contrastive loss to enhance the target representations by exploiting the objects relations for a given target domain input. These object instance relations are modelled using an Instance Relation Graph (IRG) network, which are then used to guide the contrastive representation learning. In addition, we utilize a student-teacher based knowledge distillation strategy to avoid overfitting to the noisy pseudo-labels generated by the source-trained model. Extensive experiments on multiple object detection benchmark datasets show that the proposed approach is able to efficiently adapt source-trained object detectors to the target domain, outperforming previous state-of-the-art domain adaptive detection methods. Code and models are provided in \href{https://viudomain.github.io/irg-sfda-web/}{https://viudomain.github.io/irg-sfda-web/}.
1. Introduction
The introduction frames source-free adaptation as a practical need when source data cannot be shared, while noisy pseudo-labels and weak target representations complicate detection adaptation. It proposes an Instance Relation Graph-guided contrastive strategy and reports strong benchmark comparisons.
- Source-free adaptation addresses settings where privacy, transmission, or proprietary concerns restrict source-data access during adaptation.The introduction contrasts transmitting a source-trained detector model (∼500-1000MB) with source data (∼10-100GB).Source: §1 · p. 1
- Noisy source-model pseudo-labels and the absence of source data make adaptation challenging, motivating target-information distillation and stronger target representations.Source: §1 · pp. 1–2
- For a FoggyCityscapes bus, proposals with high overlap to the ground truth predict car, truck, or bus inconsistently, illustrating feature sensitivity to small proposal shifts.Source: §1 · p. 2
- RPN proposals provide multiple views of an object for contrastive learning, but their class-agnostic predictions make positive and negative pair assignment difficult.The paper notes that detection-oriented contrastive learning also faces high GPU and memory costs from large batches and multiple views.Source: Figure 3 · p. 2, §1 · p. 2
- A graph convolution network models relations among RPN proposals and uses them to identify positive and negative pairs that guide contrastive learning.Each graph node corresponds to a proposal, and its edges encode proposal similarity relations.Source: §1 · p. 2
- The method is evaluated on multiple visually distinct detection benchmarks and outperforms existing source-free and many unsupervised domain adaptation methods.Source: §1 · p. 2
2. Related works
Related work covers object-detection adaptation with source access and source-free alternatives, then discusses contrastive learning and graph convolution as foundations for the proposed approach.
Unsupervised Domain Adaptation
- Object-detection UDA methods use feature alignment, pseudo-label self-training, image generation, or object-center-aware training, while assuming access to labeled source and unlabeled target data.Source: §2 · p. 3
Source-Free Domain Adaptation
- Prior SFDA work spans classification, segmentation, detection, and video segmentation; detection approaches include pseudo-label self-training and self-supervised feature learning.Source: §2 · p. 3
Contrastive Learning
- Contrastive learning pulls positive-pair representations together and separates negatives, but detection extensions are difficult because models are computationally expensive and large batches are costly.Source: §2 · p. 3
Graph Convolution Neural Networks
- IoU-based graphs can miss relations between nonoverlapping proposals, such as two sidecar proposals that should be treated as a positive pair for contrastive learning.The paper motivates a learnable graph convolution network to model inter-instance relations instead.Source: §2 · p. 3
3. Proposed method
The method adapts a source-trained detector using unlabeled target data, combining student-teacher self-training with graph-guided contrastive learning. An Instance Relation Graph models proposal relations, supplies pairwise labels, and is trained with distillation alongside the detector.
3.1. Preliminaries
- SFDA uses a source-trained model and unlabeled target data; mean-teacher self-training supervises the student with filtered teacher pseudo-labels, which remain noisy under domain shift.The teacher is progressively updated by exponential moving average of student weights.Source: §3.1 · pp. 3–4
3.2. Graph-guided contrastive learning
- RPN proposals provide multiple views of target instances, while IRG models their relations to identify positive and negative proposal pairs for contrastive learning.Without target ground-truth boxes, class-agnostic RPN predictions do not reveal which proposals belong to the same instance.Source: §3.2 · pp. 4–5
3.2.1 Instance Relation Graph (IRG)
- IRG represents RPN proposal RoI features as graph nodes and learns interproposal relations; Graph Distillation Loss aligns class logits across graph-transformed and original features and student-teacher pipelines.Both networks share IRG, which is constructed from teacher proposals.Source: §3.2.1 · p. 5
3.2.2 Graph Contrastive Loss (GCL)
- Graph Contrastive Loss uses thresholded IRG relations to label proposal pairs, then combines those labels with key-query pairwise logits to train the student on target features.The teacher parameters are updated via EMA, while GCL updates only the student.Source: §3.2.2 · pp. 5–6
3.3. Overall objective
- The overall SFDA objective combines student pseudo-label loss, graph distillation loss, and graph contrastive loss.These terms are denoted SL, LGDL, and LGCL, respectively.Source: §3.3 · p. 6
4. Experiments and Results
Experiments evaluate source-free adaptation across adverse-weather, artistic, synthetic-to-real, and cross-camera shifts, alongside ablations of augmentation, distillation, and graph-guided contrastive learning.
4. Experiments and Results
- Experiments compare UDA and SFDA methods across four shifts: adverse weather, real-to-artistic, synthetic-to-real, and cross-camera.UDA has access to source and target data during adaptation, whereas SFDA uses only a source-trained model and target data.Source: §4 · p. 6
4.1. Implementation details
- Experiments use Faster R-CNN with ImageNet-pretrained ResNet-50; evaluation reports target-domain mAP at IoU 0.5.Images are resized to a 600-pixel shorter side, and the teacher EMA momentum and pseudo-label confidence threshold are 0.9.Source: §4.1 · p. 6
4.2.1 Adaptation to adverse weather
- The method improves over SFOD by 4.3 mAP and HCL by 3.2 mAP on Cityscapes→FoggyCityscapes, while remaining 2.5–3.5 mAP below two UDA methods that use labeled source data.It also improves 3.5 mAP over the mean-teacher self-training baseline.Source: §4.2.1 · pp. 6–7
4.2.2 Realistic to artistic data adaptation
- The method outperforms ADDA by 4.7 mAP and DANN by 0.3 mAP on PASCAL-VOC→Clipart, and SWDA by 2.6 mAP and I3Net by 1.5 mAP on PASCAL-VOC→Watercolor.It also consistently outperforms pseudo-label self-training and mean-teacher baselines in every category for both adaptations.Source: §4.2.2 · p. 7
4.2.3 Synthetic to real-world adaptation
- On Sim10K→Cityscapes, the method achieves state-of-the-art SFDA performance, improving by approximately 1 mAP over SFOD.The adaptation uses only the car category; the authors report that GCL learns discriminative positive pairs for different cars.Source: §4.2.3 · p. 7
4.2.4 Cross-camera adaptation
- For KITTI→Cityscapes, the authors report that their method outperforms existing state-of-the-art UDA methods.The supplied results text gives no specific margin or comparison with other SFDA methods.Source: §4.2.4 · p. 7
4.3. Ablation analysis
- Strong-student/weak-teacher augmentation consistently improves results over weak-weak and strong-strong variants; graph distillation raises performance from 34.3 to 35.9 mAP, and graph-guided contrastive learning adds 1.9 mAP.The authors attribute the augmentation result to student training on strong views and teacher pseudo-labels from weak views.Source: §4.3 · p. 8, Table 5 · p. 8
5. Conclusion
The paper presents graph-guided contrastive learning for source-free domain adaptive detection, using proposal relations to improve target representations. Experiments across multiple detection benchmarks show performance above state-of-the-art SFDA and many UDA methods.
5. Conclusion
- The approach uses a contrastive graph loss and an instance relation graph network to model proposal-instance relations and guide positive/negative proposal pairing for contrastive learning.The graph network is built on graph convolution, and the learned relations guide contrastive learning on target representations.Source: §5 · p. 8
- Experiments on multiple detection benchmarks show adaptation performance above state-of-the-art SFDA and many UDA methods.Source: §5 · p. 9
Supplementary material: Instance Relation Graph Guided Source-Free Domain Adaptive Object Detection
Supplementary ablations compare contrastive-loss choices and IRG variants, while visualizations illustrate proposal relations and detection outcomes. The reported findings favor graph-guided contrastive learning and a learnable IRG over the examined alternatives.
Ablation analysis for different loss functions
- In Cityscapes→FoggyCityscapes, supervised contrastive learning on mean-teacher improves performance by 1.9 mAP; distillation lowers it from 36.2 to 35.9 mAP; graph-guided contrastive learning adds 1.9 mAP on top of GDL.Strong augmentation uses color jitter, grayscale, Gaussian blur, and erasing; weak augmentation uses horizontal flips. Strong and weak augmentation simulate the domain gap between student and teacher.Source: Supplementary material: Instance… · p. 13
Ablations analysis on IRG network
- Learnable IRG modeling of positive relations performs considerably better than K-means pairing, while freezing IRG and using original edge weights yields 36.2 mAP.Source: Supplementary material: Instance… · p. 13, Table 7 · p. 13
Positive proposals visualization
- In the proposal visualization, blue marks the query, green marks positive proposals, and red marks negative proposals.The figure visualizes pairs generated by IRG.Source: Figure 7 · p. 14, Supplementary material: Instance… · p. 14
Detection visualizations
- Detection visualizations report that the model tackles negative transfer and produces high-confidence boxes, while RPN may fail to propose tiny or occluded objects under heavy domain shifts.Instances without proposals cannot benefit from the proposed contrastive learning strategy.Source: Figure 9 · p. 16, Figure 10 · p. 17
Related papers
This paper cites
- Attention Is All You Need2017
- Microsoft COCO: Common Objects in Context2014
- Semi-Supervised Classification with Graph Convolutional Networks2016
- A Simple Framework for Contrastive Learning of Visual Representations2020
- Representation Learning with Contrastive Predictive Coding2018
- SSD: Single Shot MultiBox Detector2015