$d$-SNE: Domain Adaptation using Stochastic Neighborhood Embedding

1. Introduction

• Domain shift can make source-trained features meaningless for a target domain, requiring classifier boundaries to be redrawn.The introduction focuses on adapting a model when the target dataset is smaller than the source dataset and the label spaces match.Source: §1 · p. 1
• d-SNE combines stochastic neighborhood embedding with a modified-Hausdorff distance and min–max formulation in a shared latent space.The method seeks inter-class margins and intra-class compactness across domains, using a max-margin nearest-neighbor form that is learnable end to end.Source: §1 · p. 2
• d-SNE is reported to outperform state-of-the-art methods with few labeled samples, including in semi-supervised learning, and to generalize across domains without additional training.The authors also report cases where it outperforms unsupervised methods using all target-domain samples.Source: §1 · p. 2
• The paper identifies learning a domain-agnostic latent space and extending d-SNE to semi-supervised learning as key contributions.It also presents domain generalization and benchmark results as contributions.Source: §1 · p. 2

2. Related Works

• Domain-transformation methods learn a generative mapping, often using unlabeled data, to create a joint dataset for training a feature extractor and classifier [13].Some methods transform source data to target data, while SBADA-GAN also considers the reverse direction.Source: §2 · p. 2
• GAN-based examples include paired generators, PixelDA’s source-to-target image mapping, and CyCADA’s cycle-consistent and semantic losses [13].The surveyed work reports that CyCADA improved on the state of the art [13].Source: §2 · p. 2
• SBADA-GAN uses class consistency when mapping between domains and combines predictions from independent domain classifiers.It generates images in both domains and learns a classifier for each.Source: §2 · pp. 2–3
• Latent-space methods include adversarial learning, which seeks discriminative domain-invariant features, and multi-task learning, which uses shared features and multiple losses.Other surveyed approaches include ADDA and label-relaxed adversarial learning.Source: §2 · p. 3
• Multi-task approaches also use knowledge graphs, related-but-distinct network weights, or source–target association.CCSA and FADA use contrastive losses; other surveyed strategies include DIRT-T and self-ensembling.Source: §2 · p. 3

• Separate source and target neural networks map inputs into a shared d-dimensional latent space; they may share weights when input dimensionalities match.Source: §3 · pp. 3–4
• Target samples select source neighbors with distance-weighted probabilities, and training maximizes the probability that a target sample's predicted class is correct.Source: §3 · p. 3
• The final loss minimizes the largest same-class distance and maximizes the smallest different-class distance, replacing global distance-ratio optimization.Source: §3 · p. 4
• The supervised objective adds domain-specific cross-entropy losses as regularization, creating a multi-task setup that trains the networks in parallel.Source: §3 · p. 4
• The semi-supervised extension adds unlabeled target data through an unsupervised embedding network and an L2 consistency loss against target embeddings.Source: §3 · p. 4
• Mean-Teacher training initializes a target network from the trained target model, uses stochastic sibling augmentations, and updates target weights by exponential moving average.Source: §3 · p. 4

4. Experiments and Results

• The experiments cover four digit domains, three Office-31 domains, and synthetic-to-real VisDA-C transfers.The digit datasets include MNIST, MNIST-M, USPS, and SVHN; VisDA-C uses synthetic source images and real target images.Source: §4 · pp. 4–5

4.1. Digit Datasets

• For MNIST-to-USPS, d-SNE outperformed CCSA and FADA at every tested target-label count, including one-shot learning.As the number of target samples per class increased, the algorithms’ performance converged.Source: §4.1 · p. 5, Table 1 · p. 5
• Across five digit transfers, d-SNE outperformed supervised benchmarks in every experiment; on MNIST↔USPS, it also exceeded unsupervised methods.The authors report that semi-supervised d-SNE brought supervised performance closer to unsupervised results, while noting the comparison is unfair because unsupervised methods use all target images.Source: §4.1 · p. 5, Table 2 · p. 6
• After adaptation, the network improved its original accuracy on the source dataset, demonstrating domain generalization without retraining or fine-tuning.The reported evaluation applies the adapted model artifacts to the source dataset.Source: §4.1 · pp. 5–6, Table 3 · p. 6

4.2. Office31 Dataset

• On Office-31, d-SNE with a ResNet-101 base network significantly outperformed state-of-the-art benchmarks with only three samples in the target domain; margins were larger on the extremely large shifts A→W, W→A, A→D, and D→A.With VGG-16, d-SNE outperformed CCSA and FADA in the majority of cases, although its non-adaptation baseline performed worse than theirs.Source: §4.2 · pp. 6–7, Table 4 · p. 7

4.3. VisDA-C dataset

• With 10 target samples per class, d-SNE established itself as the clear state-of-the-art on VisDA-C, outperforming CCSA and unsupervised G2A.Its supervised setting did not match semi-supervised SE, but the authors report that comparison as unfair; the semi-supervised extension achieved the strongest result.Source: §4.3 · p. 7, Table 5 · p. 7

5. Conclusions

• Across standard benchmarking datasets, d-SNE establishes a clear state of the art on most datasets.The paper also reports that its semi-supervised extension pushes performance further.Source: §5 · p. 7
• d-SNE applies stochastic neighborhood embedding to supervised domain adaptation and can train neural networks end to end.Source: §5 · p. 7
• A semi-supervised extension of d-SNE pushes performance further.Source: §5 · p. 7