PaperlayerView exact revision on arXiv

Source-linked AI summary

Locality Preserving Joint Transfer for Domain Adaptation

Li Jingjing, Jing Mengmeng, Lu Ke, Zhu Lei, Shen Heng Tao

arXiv:1906.07441v1cs.CV

TL;DR

Domain adaptation must transfer knowledge from labeled source domains despite distribution differences and limited target labels. The paper jointly matches feature distributions, selects landmark samples, preserves locality, and uses domain-specific projections, achieving strong results across homogeneous and heterogeneous benchmarks, including gains over conventional and deep baselines.

  • Problem

    Domain adaptation seeks to transfer knowledge from well-labeled source domains to poorly labeled targets despite distribution divergence, while prior work often adapts only features or samples and may omit joint factors or heterogeneous settings.

  • Method

    The method jointly matches marginal and conditional distributions, selects landmarks to re-weight samples, preserves locality, applies label propagation, and learns domain-specific projections into a shared latent space.

  • Results

    Across five visual datasets, the approach outperformed conventional methods and end-to-end deep models; on CMU PIE, it reached 82.36% accuracy, 19.62% above JGSA.

  • Takeaways & Limitations

    The experiments indicate that heterogeneous adaptation can use handcrafted SURF features to reinforce deep VGG-FC6 features and can complement one deep feature with another.

Abstract

from arXiv · show

Domain adaptation aims to leverage knowledge from a well-labeled source domain to a poorly-labeled target domain. A majority of existing works transfer the knowledge at either feature level or sample level. Recent researches reveal that both of the paradigms are essentially important, and optimizing one of them can reinforce the other. Inspired by this, we propose a novel approach to jointly exploit feature adaptation with distribution matching and sample adaptation with landmark selection. During the knowledge transfer, we also take the local consistency between samples into consideration, so that the manifold structures of samples can be preserved. At last, we deploy label propagation to predict the categories of new instances. Notably, our approach is suitable for both homogeneous and heterogeneous domain adaptation by learning domain-specific projections. Extensive experiments on five open benchmarks, which consist of both standard and large-scale datasets, verify that our approach can significantly outperform not only conventional approaches but also end-to-end deep models. The experiments also demonstrate that we can leverage handcrafted features to promote the accuracy on deep features by heterogeneous adaptation.

I. INTRODUCTION

Domain adaptation addresses scarce target labels by transferring knowledge across domains whose distributions differ. The paper motivates jointly aligning features, adapting samples, and preserving local structure, including for heterogeneous feature spaces.

  • Motivation: Labeled samples are costly or difficult to collect, motivating transfer from well-labeled source domains to poorly labeled target domains.
  • Motivation: Domain adaptation must address distribution divergence because source knowledge cannot be directly transferred to the target domain.
  • Research gap: Prior joint methods often optimize only two of three latent factors and commonly support homogeneous rather than heterogeneous domain adaptation.
  • Proposed direction: The proposed approach jointly matches feature distributions, selects landmarks, and preserves sample manifold structures during knowledge transfer.
  • Prior approaches: Existing methods typically adapt features, samples, or classifiers, while distribution matching learns shared latent subspaces and landmark selection bridges domains through representative samples.
  • Supporting mechanisms: Graph embedding preserves sample relationships through graph weights and Laplacians, while label propagation uses neighborhood similarity to classify unlabeled data.

III. LOCALITY PRESERVING JOINT TRANSFER

The method formulates domain adaptation as learning a shared subspace that uncovers common latent features, preserves manifolds, and minimizes domain shifts. Its objective combines marginal and conditional distribution alignment with intra-class distance preservation.

  • Problem formulation: Given labeled source and unlabeled target samples with differing distributions, the method learns projections A and B into a shared subspace.
  • Problem formulation: The learned subspace is designed to uncover common latent features, preserve data manifold structures, and minimize domain shifts.
  • Unified objective: The objective combines marginal distribution shift, conditional distribution shift, and intra-class sample distance terms.
  • Unified objective: The method handles adaptation at both feature and sample levels while preserving local manifold structures through intra-class and inter-class distance terms.

1) Distribution Matching:

The method aligns source and target domains through domain-specific projections, matching marginal and conditional distributions while weighting landmark samples. Conditional matching also incorporates neighborhood information and iteratively updated target pseudo-labels.

  • Distribution Matching: Domain-specific projections A and B map source and target data into a shared latent space for alignment.The two mappings are learned jointly rather than using one shared projection.
  • Distribution Matching: Marginal and conditional MMD terms match cross-domain distributions, with equal weighting for the two terms.EMG matches marginal distributions, while ECD matches conditional distributions.
  • Distribution Matching: Conditional matching compares class-wise means and preserves neighborhood relationships between same-category samples.Target-domain pseudo-labels initialize the conditional term and are updated during optimization; ground-truth labels replace them in semi-supervised settings.
  • Landmark Selection: Sample-level adaptation learns source and target weight vectors α and β, whose constraints prevent degenerate one-sample landmark solutions.The parameter δ controls the ratio of selected landmarks among source or target samples.
  • Distribution Matching: The weighted distribution terms are reformulated into matrix expressions using coefficient matrices and cross-domain blocks.These matrix forms support the subsequent joint optimization over projections and sample weights.

3) Locality Preservation:

Locality preservation uses graph-based constraints to keep same-class neighbors close while separating nearby samples from different classes. Intrinsic and penalty graphs encode these two complementary goals.

  • Locality Preservation: Two Laplacian graph terms preserve local consistency separately in the source and target domains.The graphs enforce closeness for same-category samples and separability for different-category samples.
  • Graph Construction: Each graph uses a Laplacian L = D−W, with D containing row sums of the corresponding weight matrix.Ww and Wb are the intrinsic and penalty weight matrices.
  • Intrinsic Graph: The intrinsic graph connects nearest-neighbor pairs that share the same label.Its weights encourage same-class samples to become more compact while preserving local manifold structure.
  • Penalty Graph: The penalty graph connects k-nearest pairs whose samples belong to different classes.This graph preserves local discriminative information by making different-class samples more separable.

4) Overall Objective Function:

The overall objective combines the model’s transfer components with locality preservation and target-domain variance control. Trade-off parameters regulate the locality and target variance terms.

  • Overall Objective Function: The overall objective includes a locality-preserving term and a target variance term alongside the transfer objective.The locality and target variance contributions are controlled by γ and µ, respectively.

C. Problem Optimization

Optimization alternates between solving for the projection, updating target pseudo-labels, and refining landmark weights and objective matrices. The projection is obtained from a generalized eigen-decomposition.

  • Constrained Optimization: The Lagrangian formulation introduces a diagonal multiplier matrix whose entries are the largest eigenvalues of the associated eigendecomposition problem.The resulting eigenvectors determine P, after which the subspaces spanned by A and B are recovered.
  • Projection Optimization: The transformation P is obtained by solving a generalized eigen-decomposition and selecting d eigenvectors associated with the largest eigenvalues.P is partitioned into domain-specific transformations A and B.
  • Embedding Construction: The algorithm maps source, labeled target, and unlabeled target data into their respective subspaces using A and B.The embeddings are Zs = ATXs, Zl = BTXl, and Zu = BTXu.
  • Pseudo-Label Updating: Label propagation updates pseudo-labels in the target domain after the data are embedded.Initialization uses labeled and unlabeled target data, and the updates occur within the iterative procedure.
  • Alternating Optimization: The iterative procedure updates landmark weights α and β and recomputes distribution and locality-related matrices.The loop continues while the optimization has not converged.

2) Optimizing sample weights α and

The method optimizes sample-related variables through a quadratic-programming formulation and then uses label propagation to classify target instances.

  • 2) Optimizing sample weights α and: The sample-weight optimization is converted into an equivalent problem solvable with Quadratic Programming solvers.The formulation defines coefficients for source-source and source-unlabeled sample relationships before applying QP optimization.
  • 2) Optimizing sample weights α and: The resulting learned space is used to train a classifier through label propagation for predicting target-domain labels.The procedure is summarized in Algorithm 1.

D. Computational Complexity

Algorithm 1 has complexity O(Tdm^2 + Tn^3 + TCn^2), while the experiments span standard and large-scale domain-adaptation datasets.

  • D. Computational Complexity: O(Tdm^2 + Tn^3 + TCn^2) is the overall computational complexity of Algorithm 1.The three terms arise from eigendecomposition, quadratic programming, and computation of MMD and graph-embedding matrices.
  • D. Computational Complexity: The algorithm typically uses fewer than 10 iterations, with subspace dimensionality d generally no greater than 200.These settings are reported as sufficient for convergence and smaller than the relevant sample and feature dimensions.
  • A. Datasets Description: CMU PIE includes 41,368 images from 68 people under varying poses, illumination conditions, and facial expressions.The experiments use five selected poses after grayscale conversion, cropping, and resizing to 32 × 32 pixels.
  • A. Datasets Description: USPS and MNIST provide handwritten-digit benchmarks with 9,298 USPS images and 60,000 MNIST training examples.USPS includes 7,291 training and 2,007 test images, while MNIST also has 10,000 test examples.
  • A. Datasets Description: Office-Home has 15,500 images in 65 categories across four domains, while VisDA2017 has over 280K images across 12 categories.Office-Home uses VGG-F fc7 deep features, and VisDA2017 distinguishes synthetic training, real validation, and unlabeled test domains.

B. Protocols

The experiments evaluate homogeneous and heterogeneous adaptation across multiple benchmarks, comparing the proposed method with traditional and deep baselines. Results report strong performance on CMU PIE, Office+Caltech, and heterogeneous feature-transfer tasks.

  • B. Protocols: The evaluation covers homogeneous adaptation on Office+Caltech, CMU PIE, MNIST, and USPS, plus heterogeneous adaptation on Office+Caltech.Large-scale tests additionally use Office-Home and VisDA, with comparisons against traditional and end-to-end deep methods.
  • B. Protocols: The study forms 12 Office+Caltech tasks, 12 Office-Home tasks, and 20 CMU PIE tasks by pairing domains.Data are normalized with z-score; the average weight is fixed at δ = 0.5, graph neighborhoods use five neighbors, and T = 5.
  • C. Results and Discussions: 82.36% average classification accuracy on CMU PIE exceeds the best baseline JGSA by 19.62% across all 20 evaluations.The method attributes the gains to jointly addressing features and samples while preserving local consistency with intrinsic and penalty graphs.
  • C. Results and Discussions: 3.22% improvement over CDLS and 2.12% over DAN is achieved on Office+Caltech with DeCAF6 features.The proposed approach outperforms both traditional methods and the reported end-to-end deep models, DDC and DAN.
  • C. Results and Discussions: SURF-to-VGG-FC6 transfer performs better with transfer-learning algorithms than with PCA+SVM and SVMt baselines.The results support using handcrafted SURF features to reinforce deep VGG-FC6 features.
  • C. Results and Discussions: DeCAF6-to-VGG-FC6 and VGG-FC6-to-DeCAF6 experiments show that one deep feature can complement another.The paper reports that its approach is more effective than previous works in these heterogeneous transfers.

D. Large-scale Evaluations

The method is evaluated on large-scale Office-Home and VisDA benchmarks against traditional and deep adaptation methods, alongside analyses of landmark selection, parameter sensitivity, convergence, and MMD reduction.

  • Large-scale benchmark results: 2% accuracy improvement was achieved against DHN on Office-Home and MCD on VisDA.Both comparison methods are described as state-of-the-art deep neural network approaches.
  • Large-scale benchmark results: The method performs better than both traditional and deep methods on Office-Home and VisDA.These results support applicability to large-scale datasets and favorable accuracy.
  • Landmark selection: Selected landmarks are more similar to target samples, helping identify relevant samples and filter outliers to avoid negative transfer.The comparison uses samples with larger and smaller weights in the landmark-selection analysis.
  • Parameter sensitivity: γ is best set to a small value, µ generally falls in [10^-3,1], and latent-space dimensionality d is not sensitive on tested evaluations.The detailed sensitivity analysis reports γ from 10^-5 to 10^-3 on Office+Caltech and describes d as insensitive.
  • Convergence and discrepancy: The objective decreases monotonically and becomes steady after about 5 iterations, while MMD distance decreases during optimization.The analysis characterizes convergence as fast and reports progressively smaller distribution divergence.
  • Overall evaluation: Experiments on five visual datasets verify the superiority of the proposed method across homogeneous and heterogeneous adaptation settings.The experiments also examine unsupervised and semi-supervised adaptation and feature reinforcement across handcrafted and deep features.

APPENDIX

The appendix extends the linear projection formulation to nonlinear problems through kernel mappings, preserving a similar optimization structure.

  • Kernelization Extension: The method is extended to nonlinear problems by mapping features into a Reproducing Kernel Hilbert Space.The linear projections are replaced through kernel feature mappings.
  • Kernelization Extension: In the kernelized formulation, P and Q are defined as Φ(X)A and Φ(X)B, respectively.Variables involving X_s and X_u are likewise replaced by their mapped feature representations.
  • Kernelization Extension: The kernelized MMD matrix uses Φ(X_s)(H_sm + H_sc)Φ(X_s)^T for M_ss.This replaces the corresponding feature-space expression with mapped source features.
  • Kernelization Extension: The kernelized formulation remains very similar to the linear formulation and can be optimized by the same method.

B. Multi-domain Extension

The model can be extended to multi-source domain adaptation by jointly minimizing MMD across multiple source domains and the target domain, with analogous extensions for other objective terms.

  • Multi-domain Extension: For two source domains, the extension jointly minimizes the MMD metric across the source domains and target dataset.The source domains are denoted by subscripts 1 and 2, with the target domain represented separately.
  • Multi-domain Extension: The multi-source MMD matrix assigns within-domain and cross-domain weights according to source- and target-sample membership.The displayed construction specifies positive within-domain terms, negative source-target terms, and zero otherwise.
  • Multi-domain Extension: Other parts of the objective can be rewritten in the same manner to support additional source or target domains.The equations become more complex as the number of domains increases.
  • Multi-domain Extension: An alternative multi-source solution, CLGA, addresses domain adaptation at both global and local levels.It jointly mitigates marginal and conditional distribution gaps globally and considers local structure.
Loading 1906.07441v1…