PaperlayerView exact revision on arXiv

Source-linked AI summary

Learning Domain Trajectories with Flow Matching for Gradual Domain Adaptation

Yubo Huang, Yushe Cao, Zixi Wang, Jingzehua Xu, Xin Lai, Xiongyu Hu, Shuai Zhang

arXiv:2610.30685v1cs.LG

TL;DR

Large shifts make direct domain alignment unreliable, while existing gradual adaptation often uses discrete steps that may accumulate errors. FlowGDA learns continuous, simulation-free flows with source-anchored semantic consistency. The paper reports that it consistently outperforms existing gradual adaptation methods across multiple benchmarks.

  • Problem

    Direct source-to-target alignment becomes unreliable under large distribution gaps, and existing gradual adaptation methods commonly use discrete steps that may accumulate errors.

    Source: §1 · pp. 1–2

  • Method

    FlowGDA models gradual adaptation as continuous transport on a statistical manifold using flow-matched velocity fields and source-anchored semantic consistency.

    Source: §1 · p. 2, §4.1 · p. 3

  • Results

    FlowGDA consistently outperforms existing gradual adaptation methods across multiple benchmarks.

    Source: §1 · p. 2, §7 · p. 10

  • Takeaways & Limitations

    The results demonstrate the effectiveness of continuous representation transport for large domain shifts.

    Source: §7 · p. 10

    FlowGDA relies on ordered intermediate domains forming a meaningful gradual path and largely preserved class semantics; extra velocity networks and sequential optimization may increase complexity and training cost.

    Source: Limitations · p. 12

Abstract

from arXiv · show

Large distribution shifts remain a primary obstacle for domain generalization and unsupervised adaptation. Although gradual domain adaptation alleviates this difficulty by introducing ordered intermediate domains, most existing approaches still operate in a discrete, step-wise manner, which can accumulate errors and often relies on unstable Min--Max optimization. We propose \textbf{FlowGDA}, a flow-matching framework that models gradual adaptation as \emph{continuous} transport on a statistical manifold in a representation space. FlowGDA learns time-dependent velocity fields that induce smooth domain trajectories, using a simulation-free regression objective rather than adversarial training. To prevent semantic drift, we couple left-to-right transport between consecutive domains with a source-anchored transport and a semantic consistency loss, and we further stabilize learning via a two-stage sequential optimization scheme. Extensive experiments demonstrate that our method outperforms state-of-the-art techniques, validating flow matching as an effective alternative for domain adaptation.

1 Introduction

Large distribution shifts make direct source-to-target adaptation unreliable, motivating gradual adaptation through ordered intermediate domains. FlowGDA addresses limitations of discrete, potentially error-accumulating methods with continuous flow matching and semantic-preserving transport.

1 Introduction

  • Direct alignment becomes unreliable under large distribution gaps, while gradual adaptation decomposes the shift into smaller transitions through ordered intermediate domains.Intermediate domains have been shown to improve generalization bounds over direct adaptation.Source: §1 · pp. 1–2
  • Existing gradual methods are often discrete and may accumulate errors, while adversarial or optimal-transport optimization can be unstable or costly in high-dimensional representations.These limitations motivate learning continuous domain trajectories.Source: §1 · p. 2
  • FlowGDA models gradual adaptation as continuous transport on a statistical manifold, learning smooth velocity fields that preserve semantic information instead of adversarially aligning domains.Flow matching directly regresses velocity fields and avoids simulation-heavy or adversarial objectives.Source: §1 · p. 2
  • FlowGDA combines dual source-to-intermediate and intermediate-to-target flows with semantic consistency regularization, and reports improved accuracy, stability, and efficiency over existing GDA methods.The paper also provides a theoretical target-error bound.Source: §1 · p. 2

2 Related Work

Prior gradual domain adaptation methods use intermediate domains but usually treat their path discretely. FlowGDA instead models domain evolution as continuous representation-space flow, retaining geometric transport while avoiding costly transport computation or adversarial alignment.

2.1 Gradual Domain Adaptation

  • Gradual adaptation methods exploit intermediate domains through optimal transport, robust optimization, graph-based adaptation, and domain osmosis, but typically model the path as discrete steps that may accumulate errors.Prior theory analyzes when intermediate domains help and how adaptation paths affect bounds.Source: §2.1 · p. 2

2.2 Optimal Transport for Domain Adaptation

  • Optimal-transport methods provide distributional geometry for domain alignment but can require costly transport computation or adversarial optimization with difficult Lipschitz constraints.FlowGDA retains the geometric transport perspective while using simulation-free velocity-field regression.Source: §2.2 · p. 2

3 Problem Formulation

The problem formulation assumes an ordered chain of source, unlabeled intermediate, and unlabeled target domains, decomposing a large shift into smaller local shifts. The goal is to learn a classifier with low target-domain error using source labels, without adding supervision on intermediate or target domains.

3 Problem Formulation

  • The formulation follows the standard GDA assumption that source, intermediate, and target domains lie in an ordered gradual path, decomposing the large shift into smaller local shifts rather than adding intermediate or target supervision.The domains are ordered so consecutive shifts are milder than the direct source-to-target shift.Source: §3 · pp. 2–3
  • GDA consists of a labeled source domain, a sequence of unlabeled intermediate domains, and an unlabeled target domain arranged as D0 through DT.Source labels are available; target labels are used only for evaluation.Source: §3 · p. 3
  • The learning objective is a classifier f = h ◦ ϕ mapping inputs through representation space Z to labels, evaluated by target-domain error.The target error uses input-label pairs from the target distribution.Source: §3 · p. 3

4 Methodology

FlowGDA treats domain evolution as continuous transport in representation space, learning time-dependent velocity fields by flow matching rather than simulating trajectories or using adversarial optimization. Its manifold-aware dual flows, source-anchored semantic consistency, and sequential training are designed to support gradual transport across consecutive domains.

4 Methodology

  • FlowGDA replaces potentially error-accumulating discrete adaptation with a smooth trajectory on a statistical manifold, governed by a time-dependent velocity field.The framework learns velocity fields through flow matching, avoiding adversarial optimization and trajectory simulation.Source: §4 · p. 3, Figure 2 · p. 3

4.1 Flow Matching for Domain Transport

  • Conditional flow matching regresses the velocity along an interpolation path from a current-domain representation to its paired next-domain representation.The path starts at the left representation at t = 0 and reaches the right representation at t = 1; its objective compares predicted and ideal velocities.Source: §4.1 · pp. 3–4

4.2 Manifold-Aware Flow Matching

  • Manifold-aware velocity fields are conditioned on destination representations and normalized domain coordinates, situating each local transport step along the global domain path.The domain coordinate τi indicates the relative position of Di along the gradual path.Source: §4.2 · p. 4

4.3 Dual Flow Architecture

  • Each adaptation step uses a left-to-right flow for neighboring-domain shift and a source-to-left flow to provide a semantic anchor.The transport map ψa→b approximates the endpoint of transport from departure domain a to destination domain b.Source: §4.3 · p. 5

4.4 Training Objective

  • Training proceeds over consecutive domain pairs by first learning velocity fields with the encoder frozen, then updating the encoder with those fields frozen as curriculum strength rises from 0 to 1.The curriculum-controlled semantic loss uses a fixed source classifier, while source labels are available only for source samples.Source: §4.4 · pp. 5–6

5 Experiments

FlowGDA is evaluated on four benchmarks under ordered, unlabeled intermediate-domain adaptation and outperforms gradual and direct adaptation baselines. Ablations show that semantic consistency, source-anchored transport, manifold-aware conditioning, and sequential optimization contribute differently across datasets.

5.1 Experimental Setup

  • Evaluation covers Rotated MNIST, Color-Shift MNIST, Portraits, and Cover Type, with domains ordered along each benchmark’s natural shift parameter.Only the source domain is labeled; intermediate and target domains are unlabeled, and methods are trained sequentially along the domain chain.Source: §5.1 · pp. 6–7

5.2 Results

  • +6.4 points on Rotated MNIST, +7.4 on Color-Shift MNIST, and +2.8 on Portraits versus the strongest reported gradual baseline; on Cover Type, FlowGDA beats GST by +2.6 points.FlowGDA achieves the best performance across all four benchmarks, while direct UDA baselines perform substantially worse under large shifts.Source: §5.2 · p. 7

5.3 Ablation Study

  • Removing semantic consistency causes the largest average degradation, especially on Portraits and Cover Type, while single-flow variants drop more on real-world and tabular data.Manifold-aware conditioning improves over time-only velocity parameterization across datasets, and two-stage optimization yields more stable final performance than joint optimization.Source: §5.3 · p. 7

6 Theoretical Analysis

The analysis relates gradual adaptation to bounded risk changes across consecutive representation distributions and to transport energy under flow matching. It also formalizes semantic preservation by source-anchored transport and contrasts flow matching’s regression objective with adversarial optimization.

6.1 Generalization Error Bounds for Flow-Based Gradual Adaptation

  • The target-risk bound depends on source risk, the sum of consecutive representation-distribution discrepancies, and an ideal joint-error term; continuous flow aims to reduce stepwise mismatch.The analysis assumes a bounded, Lipschitz surrogate loss and explains that discrete independently estimated alignments can enlarge local mismatch and accumulate error.Source: §6.1 · p. 8

6.2 Flow Matching Controls Wasserstein Transport Cost

  • Flow matching links low-energy velocity fields to small Wasserstein transport cost, which controls per-step representation discrepancy and can in turn bound target risk.The stated control holds up to approximation and estimation errors in the empirical flow-matching objective.Source: §6.2 · p. 9

6.3 Semantic Consistency via Source-Anchored Transport

  • The source-anchored classification loss bounds misclassification after source-to-intermediate transport, formalizing its role in preserving semantic structure.The loss evaluates the transported source representation with the fixed source classifier.Source: §6.3 · p. 9

6.4 Why Flow Matching is More Stable than Adversarial Alignment

  • Flow matching uses a single-level supervised regression objective, unlike adversarial alignment’s min–max optimization, which can exhibit saddle-point instability and sensitivity to critic capacity.The analysis connects this optimization difference to the method’s empirically observed stability and faster convergence.Source: §6.4 · p. 9

7 Conclusions

FlowGDA models gradual domain adaptation as continuous transport on a statistical manifold, using simulation-free regression to learn velocity fields across domains. Experiments on multiple benchmarks show consistent outperformance of existing gradual adaptation methods.

7 Conclusions

  • FlowGDA learns representation-transport velocity fields through a simulation-free regression objective, avoiding adversarial optimization.The paper reports consistent outperformance of existing gradual adaptation methods across multiple benchmarks.Source: §7 · p. 10

Limitations

FlowGDA’s limitations include dependence on a meaningful ordered path and largely preserved class semantics, while its sequential optimization may add complexity and cost. The appendix also states the shared-labeling assumption and outlines the proofs and notation used in its theoretical analysis.

Limitations

  • FlowGDA assumes ordered intermediate domains form a meaningful path and class semantics remain largely preserved; severe label or class-appearance changes can weaken consistency.Extra velocity networks and sequential optimization over domain pairs may also increase implementation complexity and training cost versus simpler direct-adaptation baselines.Source: Limitations · p. 12

A Theoretical Proofs

  • The appendix provides proofs for a Wasserstein-based risk bound, the interpolation-path continuity equation, marginal-velocity learning, and a semantic-consistency bound.These proofs support the theoretical claims in Section 6.Source: A Theoretical Proofs · p. 13

A.1 Notation and Standing Assumptions

  • The theoretical setup uses an ordered chain from labeled source through unlabeled intermediate domains to an unlabeled target, with a shared conditional label distribution across domains.This assumption permits representation marginals to shift while the semantic labeling rule given a representation stays shared.Source: §A.1 · p. 13

A.2 Proof of Lemma 6.2

The proof expresses domain risk through representation distributions and a shared conditional labeling rule, then uses Lipschitz continuity to relate risk differences to Wasserstein distance.

A.2 Proof of Lemma 6.2

  • Domain risk is expressed by integrating the conditional risk over the domain’s representation distribution.The conditional risk averages the loss over the shared label distribution given each representation.Source: §A.2 · p. 14
  • The shared-labeling assumption makes the conditional label distribution independent of the domain index.This lets the proof compare domains through their representation distributions.Source: §A.2 · p. 14
  • Lipschitz continuity of the conditional risk yields a bound on risk differences using Wasserstein-1 distance between representation distributions.The proof normalizes the Lipschitz function before applying the definition of Wasserstein-1 distance.Source: §A.2 · p. 14

A.3 Proof of Theorem 6.3

The appendix develops risk and flow-matching arguments, then reports additional experiments on intermediate-domain quality and quantity. These experiments favor midpoint-like intermediate domains and finer trajectories, with gains diminishing once the path is sufficiently fine-grained.

A.3 Proof of Theorem 6.3

  • The proof applies a consecutive-domain risk decomposition recursively along the chain and accounts for irreducible source-target mismatch with an ideal joint-error term.The source and target risks bound the chain’s endpoints.Source: §A.3 · pp. 14–15

A.4 Proof of Theorem 6.7

  • The flow-matching proof connects conditional velocity regression to the marginal velocity field, whose kinetic energy upper-bounds squared Wasserstein-2 distance.It establishes the interpolation path as a valid transport path between the two representation distributions.Source: §A.4 · pp. 16–17

A.5 Proof of Corollary 6.8

  • The corollary proof defines per-step transport energy for consecutive-domain interpolation paths and uses it in the target-risk bound.The supplied proof fragments specify a finite-second-moment condition for the energy bound.Source: §A.5 · p. 17

A.6 Proof of Lemma 6.9

  • The semantic-consistency proof uses a fixed source classifier and cross-entropy to relate its predictions on transported representations to the source labels.The proof applies its cross-entropy argument to the transported source representation-label pair.Source: §A.6 · p. 17, §A.7 · p. 18

A.7 Proof of Theorem 6.10

  • FlowGDA does not optimize a JKO objective; it uses conditional flow matching to regress velocity fields between neighboring representation distributions.The appendix motivates the continuity equation and Benamou–Brenier formulation as theoretical tools aligned with that objective.Source: §A.7 · p. 18

B Additional Experimental Results

  • The appendix’s additional experiments examine intermediate-domain quality and count, domain ordering, early transport steps, and hyperparameter sensitivity.The overview frames these as questions specific to gradual domain adaptation rather than a repeat of the main comparison table.Source: B Additional Experimental Results · p. 19

B.1 Effect of Intermediate-Domain Quality

  • On Rotated MNIST, an intermediate domain near π/6 gives the best performance; choices near either endpoint yield slightly lower accuracy and larger variance.Direct source-to-target adaptation performs worse than using an intermediate bridge.Source: §B.1 · p. 19

B.2 Effect of the Number of Intermediate Domains

  • Increasing the number of intermediate domains improves performance across datasets, with larger gains on Portraits and Cover Type; gains gradually saturate as trajectories become sufficiently fine-grained.Direct adaptation is consistently weaker, but arbitrarily dense domain sequences are not required once local shifts are sufficiently small.Source: §B.2 · p. 19

B.3 Robustness to Domain Ordering

FlowGDA tolerates small local changes in domain ordering, but random or reversed orderings reduce performance. Experiments also find early transport steps more influential than late ones and report stability across reasonable hyperparameter ranges.

B.3 Robustness to Domain Ordering

  • FlowGDA is robust to mild local ordering changes, but random or reversed orders reduce performance, especially on Portraits and Cover Type.The results suggest estimating a reasonable ordering or neighborhood graph when domain order is noisy.Source: §B.3 · p. 19

B.4 Impact of Early Transport Steps

  • Skipping early transport pairs causes the largest degradation among single-block removals, while training only early pairs outperforms training only late pairs.The authors interpret early steps as establishing a stable trajectory and later steps as refining aligned features.Source: §B.4 · pp. 19–20

B.5 Hyperparameter Sensitivity

  • FlowGDA is stable across reasonable hyperparameter ranges; on Portraits, the best setting uses λcls = 0.1, hidden dimension 512, four curriculum levels, and batch size 128.The semantic-consistency weight is non-monotonic: omitting the loss loses label information, while excessive weight constrains adaptation.Source: §B.5 · p. 20

Related papers

Loading 2610.30685v1…