Continuously Indexed Domain Adaptation

1. Introduction

• Categorical domain adaptation treats domains as labels, but tasks such as adapting disease prediction across ages require accounting for relationships among domains.The introduction also gives underwater depth and viscosity as examples of continuous indices.Source: §1 · p. 1
• CIDA adapts adversarial domain adaptation by modeling the domain index distribution, rather than treating continuously indexed domains as categorical labels.The paper argues this captures underlying relations among domains and provides better theoretical guarantees for domain alignment.Source: §1 · pp. 1–2
• CIDA is extended with probabilistic and multi-dimensional variants to improve performance and handle multi-dimensional domain indices.The variants are described as requiring minimal overhead.Source: §1 · p. 2
• CIDA and its probabilistic and multi-dimensional variants significantly improve performance over state-of-the-art domain adaptation methods on synthetic and real-world medical datasets.Source: §1 · p. 2

2. Related Work

• Adversarial domain adaptation aligns source and target distributions by making encoded data indistinguishable to a discriminator trained to classify domains.The paper builds on this approach to address continuously indexed domains.Source: §2 · p. 2
• Incremental domain adaptation assumes domain shifts smoothly over time and adapts across multiple target domains using categorical methods for each domain pair.The cited work includes optimal transport, adversarial loss, generative adversarial networks, and linear transforms.Source: §2 · p. 2
• Incremental pairwise adaptation can propagate failures across later domain pairs, and its usual single shifting dimension contrasts with CIDA's multidimensional generalization.The paper says these differences are empirically verified in Sec. 5.Source: §2 · p. 2

3. Methods

• The methods consider source and target domains indexed by sets in a metric space and state that the approach extends directly to multidimensional indices.The paper says multidimensional indices bring further performance gains.Source: §3 · p. 2

3.1. Continuously Indexed Domain Adaptation (CIDA)

• CIDA learns an encoder and shared predictor while its discriminator regresses the continuous domain index, with a domain-index loss that is a monotonic function of the metric over indices.The encoder aims to align encodings across domains while supporting accurate label prediction.Source: §3.1 · pp. 2–3

3.2. Variants of CIDA

• Vanilla CIDA predicts the domain index directly with an L2 loss, which guarantees matching only the mean of p(u|z).Source: §3.2 · p. 3
• Probabilistic CIDA predicts a distribution for p(u|z), using a Gaussian discriminator that outputs its mean and variance and is trained with negative log-likelihood.The paper says this variant has guarantees for matching both mean and variance and can extend to higher-order moments.Source: §3.2 · p. 3
• PCIDA can use a Gaussian mixture model by replacing the discriminator with a mixture density network and using the corresponding negative log-likelihood.Source: §3.2 · p. 3

4.1. Analysis for the Simplified Game

• CIDA’s simplified-game equilibrium aligns the conditional domain-index mean E[u|z] with the marginal mean E[u].With the encoder fixed, the optimal discriminator outputs the conditional mean of the domain index.Source: §4.1 · pp. 3–4
• With the Gaussian model, PCIDA’s global optimum matches both the conditional and marginal means and variances; higher moments are a proposed extension.The paper suggests more complex probabilistic models, such as skew-normal distributions, for matching higher-order moments.Source: §4.1 · pp. 4–5
• For both CIDA and PCIDA, total alignment of encodings across continuously indexed domains achieves the simplified-game global optimum.Source: §4.1 · p. 4

4.2. Analysis of the Three-player Game

• In the full game, the optimal encoder preserves label information while aligning domains when model capacity is sufficient and training reaches optimum.When the domain index is relevant to the label, continuous adaptation can retain that information rather than discretizing the index; when it is a nuisance independent of the label, the encoding captures input information relevant to prediction while aligning domains.Source: §4.2 · p. 5

5. Experiments

• Across experiments, categorical adaptation yields poor alignment or marginal gains, while CIDA improves performance; PCIDA’s distribution prediction and multidimensional indices provide further benefits.The reported findings also associate PCIDA’s distribution prediction with avoiding bad equilibria.Source: §5 · p. 5

5.1. Baselines and Implementations

• Baselines either merge indexed domains or partition the continuous spectrum for adaptation; all use both input x and index u in the encoder for comparison with CIDA.CUA adapts target domains one-by-one from the closest to the farthest.Source: §5.1 · p. 6

5.3. Rotating MNIST

• On Rotating MNIST, CIDA and PCIDA adapt across rotation angles, while ADDA and DANN provide little or sometimes negative accuracy improvement and CUA struggles on distant targets.The source covers 0°–45° rotations; target images span 45°–360°.Source: Table 1 · p. 7, §5.3 · p. 7

5.2. Toy Datasets

• On Circle and Sine, CIDA captures the evolving decision boundary more accurately than baselines, while CUA’s pairwise failure on Sine propagates to later domains.CUA is the strongest baseline on Circle, but remains inferior to CIDA there.Source: §5.2 · p. 7

5.4. Healthcare Datasets

• For within-dataset age transfer, CIDA variants improve over baselines in extrapolation; interpolation can leave little room to improve when Source-Only is already accurate.For SHHS@Outside → SHHS@(52,75], Source-Only accuracy is 82.4%.Source: Table 2 · p. 8, §5.4 · p. 8
• Across SHHS, MESA, and SOF, CIDA and PCIDA improve cross-dataset transfer, with PCIDA a clear winner for the harder SOF-to-more-diverse-age-range transfers.On easier transfers to the older SOF population, CIDA is marginally better, while PCIDA performs well across scenarios.Source: Table 3 · p. 8, §5.4 · p. 8
• With multidimensional indices combining age and other patient variables, CIDA and PCIDA substantially improve accuracy over Source-Only on the SHHS task.The task transfers from SHHS ages [44,52] to (52,90].Source: Table 4 · p. 8, §5.4 · p. 9

• The paper identifies continuous-domain adaptation as its problem and proposes a series of methods supported by theoretical analysis and experiments on synthetic and real-world medical data.Source: §6 · p. 9
• The work demonstrates the viability of efficient adaptation across continuously indexed domains and its potential impact on real-world applications.Source: §6 · p. 9
• Future work could investigate matching higher- or infinite-order moments and applying the methods to robotics or other medical datasets.Source: §6 · p. 9

• The supplementary-materials document is identified as arXiv:2007.01807v2, dated 30 Aug 2020.Source: Supplementary Materials: Continuously… · p. 11

• If the conditional expectation of u given z is constant in z, it equals the marginal expectation of u.Source: §1 · p. 11
• If both the conditional expectation and variance of u given z are constant in z, they equal their respective marginal values.Source: §1 · p. 11
• The proof section states an optimal-discriminator result for PCIDA with E fixed and defines z as E(x, u).Source: §1 · p. 11

2 Experiments

• The supplementary experiments provide additional details, and the paper's code is available online.The code URL is https://github.com/hehaodele/CIDA.Source: §2 · p. 12

2.1 Experiment on the Healthcare Datasets

• For the healthcare experiments, the supplement describes three publicly available medical datasets, lists domain-index variables, and gives shared model and training details.The models use the same architecture; the supplement specifies λd = 2.0 for all methods and an Adam learning rate of 10^-4.Source: §2.1 · p. 12

2.2 Experiment on the Rotating MNIST

• Rotating MNIST spans rotation angles evenly across [0, 360°), with seven times as much target-domain data as source-domain data.The image encoder adds an STN taking the image and domain index as input; dropout enables deterministic or probabilistic encoding.Source: §2.2 · pp. 12–13

2.3 Experiments on the Toy Datasets

• Toy-dataset decision boundaries are visualized by fitting an RBF-kernel SVM to each neural network's predictions at equal fitting accuracy.The resulting boundaries are approximate, so some data points may appear on the wrong side.Source: §2.3 · p. 13

3.1 Categorical Domains versus Continuously Indexed Domains

• CIDA assumes p(x, y|u) and p(y|x, u) vary continuously with domain index u, and aims to predict across a continuous range of target domains.It seeks to capture the relation between p(y|x, u) and u.Source: §3.1 · p. 13
• Unlike categorical adaptation, CIDA's regression losses encode distances between domain indices rather than treating every pair of domains as equally distant.The passage identifies L2 and Gaussian or Gaussian Mixture Model losses as ways to regress domain indices.Source: §3.1 · p. 13

3.2 Matching p(u|z) versus Matching p(z|u)

• Matching the mean and variance of p(u|z) is stronger than matching those of p(z|u) when z has much higher dimension than u.In the passage's example, the former requires 2 × 4^100 parameters, versus 400 for the latter.Source: §3.2 · p. 14