Source-linked AI summary
Exploiting the Intrinsic Neighborhood Structure for Source-free Domain Adaptation
Shiqi Yang, Yaxing Wang, Joost van de Weijer, Luis Herranz, Shangling Jui
TL;DR
The paper addresses source-free domain adaptation when source data are unavailable and target features shift from the source domain. It proposes Neighborhood Reciprocity Clustering to exploit target neighborhoods and reports state-of-the-art performance across 2D image and 3D point-cloud datasets.
Problem
Source-free domain adaptation seeks to adapt a source pretrained model without source data, which may be inaccessible because of privacy or intellectual-property restrictions.
Method
Neighborhood Reciprocity Clustering encourages label consistency among target neighbors using affinity-weighted local, reciprocal, and expanded neighborhoods with self-regularization against noisy neighbors.
Results
The method achieves state-of-the-art performance on several 2D image and 3D point-cloud recognition datasets, including a 4% margin over PointDA on PointDA-10.
Takeaways & Limitations
Intrinsic local structure in target features is an important source of information for source-free domain adaptation.
Takeaways & Limitations
The method focuses on closed-set source-free domain adaptation, where both domains have the same classes.
Abstract
from arXiv · showhide
Domain adaptation (DA) aims to alleviate the domain shift between source domain and target domain. Most DA methods require access to the source data, but often that is not possible (e.g. due to data privacy or intellectual property). In this paper, we address the challenging source-free domain adaptation (SFDA) problem, where the source pretrained model is adapted to the target domain in the absence of source data. Our method is based on the observation that target data, which might no longer align with the source domain classifier, still forms clear clusters. We capture this intrinsic structure by defining local affinity of the target data, and encourage label consistency among data with high local affinity. We observe that higher affinity should be assigned to reciprocal neighbors, and propose a self regularization loss to decrease the negative impact of noisy neighbors. Furthermore, to aggregate information with more context, we consider expanded neighborhoods with small affinity values. In the experimental results we verify that the inherent structure of the target features is an important source of information for domain adaptation. We demonstrate that this local structure can be efficiently captured by considering the local neighbors, the reciprocal neighbors, and the expanded neighborhood. Finally, we achieve state-of-the-art performance on several 2D image and 3D point cloud recognition datasets. Code is available in https://github.com/Albert0147/SFDA_neighbors.
1 Introduction
Source-free domain adaptation adapts a pretrained source model without source data, using intrinsic neighborhood structure preserved among target features. Neighborhood Reciprocity Clustering encourages label consistency among weighted local, reciprocal, and expanded neighbors.
- Source-free domain adaptation adapts a source pretrained model without source data, which may be inaccessible because of privacy or intellectual-property restrictions.
- Despite covariance shift, same-class target samples are expected to form clusters in embedding space, motivating nearest-neighbor structure as adaptation information.On Ar→Rw, around 75% of nearest neighbors have the correct label, so ordinary neighbors can be noisy.
- Neighborhood Reciprocity Clustering assigns different supervision weights by encouraging reciprocal neighbors to agree in label prediction and using weaker connections to non-reciprocal neighbors.Affinity values describe connectivity and support class consistency among neighboring target features.
- Self-regularization reduces the negative impact of potential noisy neighbors while affinity-weighted neighborhood supervision promotes local class consistency.
- Expanded neighbors provide wider local context through neighbors of nearest neighbors, receiving small affinity values as auxiliary supervision.The method uses expanded neighbors because they are expected to be closer on the target data manifold than features added by simply enlarging the nearest-neighbor set.
- Experiments on three 2D image datasets and one 3D point cloud dataset report state-of-the-art performance compared with related methods.
2 Related Work
Related work covers distribution-alignment approaches, source-free adaptation methods, and graph-clustering methods that use neighborhood information. The paper distinguishes its approach by assigning affinity through reciprocity rather than training a graph network with labeled data.
- Domain Adaptation: Domain adaptation research includes moment matching and adversarial training methods that align source and target feature distributions.
- Source-free Domain Adaptation: Source-free adaptation methods operate without source data, with prior work addressing universal, open-set, and closed-set domain adaptation settings.
- Graph Clustering: Graph-clustering methods use neighborhood information, but they require labeled data to train graph networks for affinity estimation.
- Graph Clustering: Unlike the cited graph-clustering approaches, this method assigns affinity using reciprocity rather than a labeled graph network.
3 Method
NRC adapts a source-pretrained model without source data by enforcing prediction consistency over target neighborhoods, weighting neighbors by affinity and extending neighborhoods cautiously.
- Neighborhood-based adaptation: NRC uses target-feature neighborhoods to encourage similar predictions among semantically connected data points during source-free adaptation.The objective pushes data toward semantically close neighbors by encouraging similar predictions.
- Neighborhood-based adaptation: The method retrieves target neighbors from feature and score memory banks, which store features and corresponding prediction scores.The banks are updated for the current mini-batch before nearest-neighbor retrieval.
- Neighborhood affinity: Affinity-weighted supervision emphasizes strongly connected neighbors while assigning weaker influence to neighbors with weaker connectivity.Affinity values weight the prediction-similarity objective, with strong-connectivity neighbors receiving greater influence.
- Neighborhood affinity: Reciprocal nearest neighbors receive higher affinity than non-reciprocal neighbors because they have greater potential to belong to the same cluster.The method distinguishes reciprocal and non-reciprocal nearest neighbors when assigning affinity.
- Noise reduction: Self-regularization retains the ego feature’s current prediction to reduce the impact of noisy reciprocal neighbors from different classes.The stored prediction is treated as constant, while backpropagation updates only the current prediction.
- Prediction balancing: Prediction diversity loss discourages degenerate solutions in which the model assigns all target data to only a subset of classes.The loss encourages balanced predictions using the empirical label distribution and a uniform class distribution.
- Expanded neighborhoods: Expanded neighborhoods aggregate additional target information through neighbors’ nearest neighbors, using a small affinity value because they may be farther and noisier.Duplicate expanded neighbors are retained because repeated appearances indicate compact clusters and potential semantic closeness.
4 Experiments
Experiments evaluate NRC across 2D image and 3D point-cloud benchmarks, with ablations examining affinity weighting, expanded neighborhoods, self-regularization, and neighborhood choices. NRC achieves state-of-the-art results on Office-Home, VisDA, and PointDA-10, while analyses show that reciprocal and expanded neighborhoods improve adaptation when weighted appropriately.
- 2D image datasets: NRC achieves state-of-the-art performance on Office-Home and VisDA, surpassing SHOT by 3% and RWOT by 1.9% on VisDA.On Office-31, NRC is similar to 3C-GAN and below RSDA-MSTN.
- 3D point cloud dataset: NRC outperforms PointDA by 4% on the PointDA-10 3D point-cloud benchmark.PointDA requires source data during adaptation and uses extra attention modules.
- Ablation studies: Affinity weighting improves neighbor supervision: Office-Home accuracy rises from 69.1 with equal weighting to 71.1 with higher affinity for reciprocal neighbors.The expanded-neighborhood configuration reaches 72.2 when combined with affinity values.
- Ablation studies: Expanded neighborhoods help only with affinity weighting, achieving 72.2 on Office-Home versus 65.2 without affinity.Removing duplicated expanded neighbors significantly degrades performance, and simply increasing K performs worse than the expanded-neighborhood design.
- Ablation studies: Self-regularization improves performance with nearest-neighbor and all-neighbor supervision, while reasonable r values produce similar results and varying K and M demonstrates robustness.The analysis attributes the self-regularization gain to reducing the negative impact of noisy neighbors.
5 Conclusions
The paper introduces an SFDA method that uncovers intrinsic target structure by encouraging label consistency among local, reciprocal, and expanded neighbors. Experiments on 2D image and 3D point-cloud datasets support the efficacy of this approach.
- The method adapts source-free models by uncovering the intrinsic structure of target data.
- It encourages label consistency among local target features while distinguishing nearest, reciprocal, and expanded neighborhoods.
- Experiments verify the importance of target-feature local structure across 2D image and 3D point-cloud datasets.
Checklist
The checklist reports that the paper addresses contribution and scope questions, provides reproducibility materials and training details, and reports repeated-run averages. It also records several non-applicable human-subject items and notes that limitations were not discussed.
- The paper’s main claims accurately reflect its contributions and scope.
- The authors indicate that the paper does not discuss its limitations or potential negative societal impacts.
- The authors provide code, data-related instructions, training details, and hyperparameter information needed to reproduce the experiments.
- Main results are averaged over three runs with random seeds, and compute resources are reported.
- The paper cites existing assets but does not mention their licenses.
- The checklist marks participant instructions, risks, institutional review, compensation, and consent items as not applicable or unanswered where specified.