PaperlayerView exact revision on arXiv

Source-linked AI summary

CPCANet: Deep Unfolding Common Principal Component Analysis for Domain Generalization

Yu-Hsi Chen, Abd-Krim Seghouane

arXiv:2605.05136v3cs.CV

TL;DR

Domain generalization seeks representations that transfer to unseen domains despite distribution shifts, but structured invariant subspaces based on second-order statistics remain underexplored. CPCANet unfolds the Flury-Gautschi algorithm for Common Principal Component Analysis into differentiable neural layers and achieves state-of-the-art zero-shot transfer across four benchmarks.

  • Problem

    Integrating CPCA’s structured, second-order statistical subspace learning into gradient-based deep learning for domain generalization remains underexplored.

  • Method

    CPCANet unfolds the Flury-Gautschi CPCA solver into differentiable Riemannian optimization layers that guide representation learning with shared domain structure.

  • Results

    CPCANet achieves state-of-the-art zero-shot transfer across multiple domain-generalization benchmarks and performs best on average among methods using the same ResNet-50 backbone.

  • Takeaways & Limitations

    CPCANet provides an architecture-agnostic framework with minimal computational overhead and no task-specific hyperparameter tuning for generalization under distribution shift.

  • Takeaways & Limitations

    The current formulation relies on domain labels to estimate latent covariances, leaving unsupervised and self-supervised extensions for future work.

Abstract

from arXiv · show

Domain Generalization (DG) aims to learn representations that remain robust under out-of-distribution (OOD) shifts and generalize effectively to unseen target domains. While recent invariant learning strategies and architectural advances have achieved strong performance, explicitly discovering a structured domain-invariant subspace through second-order statistics remains underexplored. In this work, we propose CPCANet, a novel framework grounded in Common Principal Component Analysis (CPCA), which unrolls the iterative Flury-Gautschi (FG) algorithm into fully differentiable neural layers. This approach integrates the statistical properties of CPCA into an end-to-end trainable framework, enforcing the discovery of a shared subspace across diverse domains while preserving interpretability. Experiments on four standard DG benchmarks demonstrate that CPCANet achieves state-of-the-art (SOTA) performance in zero-shot transfer. Moreover, CPCANet is architecture-agnostic and requires no dataset-specific tuning, providing a simple and efficient approach to learning robust representations under distribution shift. Code is available at https://github.com/wish44165/CPCANet.

1 Introduction

CPCANet addresses domain generalization under distribution shifts by unfolding Common Principal Component Analysis into a differentiable deep-learning framework. It combines CPCA’s shared-subspace modeling with nonlinear representation learning and evaluates the approach on four standard DG benchmarks.

  • Motivation: Domain Generalization seeks representations that remain effective when deployment data differ from training distributions, where neural networks can suffer severe performance degradation.The motivation follows from deep networks’ reliance on identically distributed training and testing data.
  • Common Principal Component Analysis: CPCA discovers a common subspace across diverse covariance matrices by estimating a shared orthogonal transformation over second-order statistics.This provides a mathematically rigorous approach to identifying invariant structures across multiple distributions.
  • Common Principal Component Analysis: Standard CPCA is limited by its linear formulation, hindering its ability to model complex nonlinear visual data and integrate its statistical guarantees with gradient-based deep learning.These limitations motivate bridging classical statistical subspace learning with modern deep representation learning.
  • Contributions: CPCANet provides a CPCA-based DG framework that isolates domain-invariant structure from domain-specific correlations.The framework is designed to connect CPCA’s statistical perspective with deep representation learning.
  • Contributions: CPCANet uses differentiable deep-unfolded Riemannian optimization with Cayley retraction and hypernetwork-driven step sizes for stable optimization on the Stiefel manifold.The method integrates the CPCA objective into an end-to-end differentiable framework.
  • Contributions: The study evaluates CPCANet on four standard domain-generalization benchmarks.This contribution establishes a comprehensive experimental validation program for the proposed framework.

2 Related Work

This section situates CPCANet within Domain Generalization, Common Principal Component Analysis, and Deep Unfolding Networks. It contrasts DG’s zero-shot objective with Domain Adaptation and summarizes CPCA’s shared-basis formulation and algorithmic foundations.

  • Domain Generalization: Domain Generalization learns representations that generalize to unseen target domains under out-of-distribution shifts, unlike Domain Adaptation, which uses target-domain samples during training.DG therefore operates in a zero-shot transfer setting and includes baselines such as Empirical Risk Minimization (ERM).
  • Common Principal Component Analysis: Common Principal Component Analysis identifies a common basis across distinct datasets modeled with group-specific means and positive definite covariance matrices.The formulation considers K groups, with each group containing independently drawn p-variate normal samples.
  • Common Principal Component Analysis: CPCA simultaneously diagonalizes group covariance matrices, identifying common principal components while allowing group-specific variances along shared components.The diagonal matrices Λk contain the component-specific variances for each group.
  • Common Principal Component Analysis: The iterative Flury-Gautschi algorithm estimates the common basis, after which each data matrix is projected as Uk = Xkβ to obtain sample CPC representations.The projection is defined for groups k = 1, . . . , K.
  • Common Principal Component Analysis: CPCA extensions include partial sharing, robust estimation, state-space formulations, efficient stepwise optimization, and applications across diverse statistical modeling settings.These developments build on CPCA’s foundational statistical studies.
  • Deep Unfolding Networks: Deep Unfolding Networks convert iterative algorithms into trainable neural networks while preserving interpretability, originating with LISTA’s learnable reformulation of sparse-coding solvers.LISTA reformulated classical solvers including ISTA and FISTA as learnable network layers.

3 Methodology

The methodology introduces a CPCA-based perspective on domain generalization, develops a differentiable CPCA solver for end-to-end integration, and then specifies training and inference procedures.

  • 3.1 CPCA Perspective: Section 3.1 introduces a CPCA-based perspective on domain generalization.
  • 3.2 Differentiable CPCA Solver: Section 3.2 develops a differentiable CPCA solver to integrate CPCA into a deep learning framework.The solver addresses CPCA’s incompatibility with end-to-end training.
  • 3.3 Training and Inference: Section 3.3 describes the training objective and inference procedure.

3.1 Problem Formulation: Domain Generalization via CPCA

CPCANet formulates domain generalization as learning a shared CPCA subspace from latent source covariances, with predictions transferred to an unseen target environment. The framework combines neural feature extraction, bottleneck projection, and orthogonal projection to suppress domain-specific correlations while preserving shared structure.

  • 3.1 Problem Formulation: Domain Generalization via CPCA: Domain generalization uses K distinct source environments to learn a predictive model minimizing expected risk on a strictly unseen target environment.Each environment provides samples drawn from its own joint distribution over inputs and labels.
  • 3.1 Problem Formulation: Domain Generalization via CPCA: A neural backbone and linear bottleneck map each source input into a d-dimensional latent representation whose unbiased sample covariance captures environment-specific geometry.The backbone maps inputs from R^p to R^D, followed by a bottleneck from R^D to R^d.
  • 3.1 Problem Formulation: Domain Generalization via CPCA: CPCA seeks a shared orthogonal matrix β that simultaneously diagonalizes the K latent source covariance matrices and defines a structured domain-invariant subspace.Source feature matrices are projected onto the common basis as Uk = Zkβ.
  • 3.1 Problem Formulation: Domain Generalization via CPCA: The orthogonal projection suppresses domain-specific spurious correlations while preserving shared invariant structures, and target samples undergo the same backbone, bottleneck, and CPCA-subspace projection.For an unseen target sample, the final representation is u(te) = β⊤z(te).
  • 3.1 Problem Formulation: Domain Generalization via CPCA: Direct classification in the low-dimensional CPCA subspace enforces domain-invariant predictions but creates a severe information bottleneck addressed by feature modulation.The naive classifier uses parameters Wcls ∈ R^d×C and bcls ∈ R^C.

3.2 Derivation of the Deep Unfolded CPCA Solver

CPCANet replaces the non-differentiable FG solver with a deep unfolded CPCA solver that optimizes an orthogonal basis through Cayley-retracted Riemannian updates. The resulting T-layer procedure uses covariance-conditioned step sizes and produces a strictly orthogonal, differentiable basis adapted to each forward pass.

  • Solver motivation: The deep unfolded solver addresses FG’s incompatibility with computational graphs, enabling gradient backpropagation and end-to-end optimization of the common orthogonal matrix β.Classical FG estimation is iterative and cannot support backpropagation within modern computational graphs.
  • Orthogonal retraction: Cayley retraction maps an unconstrained skew-symmetric matrix A ∈ so(d) onto O(d), preserving β’s orthogonality without computationally prohibitive SVD-based orthogonalization.Optimization occurs on the orthogonal manifold rather than directly under β⊤β = I_d.
  • Riemannian gradient: The CPCA objective is differentiated through the orthogonal-group geometry by projecting the Euclidean gradient onto the skew-symmetric Lie algebra so(d).The basis-transformed variances are defined from the diagonal entries of β⊤S_kβ, and the resulting tangent gradient is implemented using domain-specific skew-symmetric weight matrices and Hadamard products.
  • Dynamic unfolding: A hypernetwork H_ϕ generates covariance-conditioned step-size vectors η ∈ R^T for the unfolded layers, with sigmoid scaling constraining each step size to (0, 0.5).Dynamic step sizes address varying mini-batch covariance statistics and improve optimization stability under mini-batch noise.
  • Dynamic unfolding: After T tangent-space updates initialized at A_0 = 0, the final projection β_T is strictly orthogonal, fully differentiable, and adapted to the current forward pass’s statistical structure.The updates use the normalized tangent gradient and iteratively produce the corresponding orthogonal projections.

3.3 Training Objective and Inference

CPCANet trains the backbone, step-size hypernetwork, and unfolded CPCA solver end-to-end with a CPCA regularization that encourages joint diagonalization. At inference, the learned invariant basis conditions feature modulation rather than replacing high-dimensional classification features, while zero initialization preserves the ERM starting point.

  • Training Objective: CPCA regularization penalizes covariance structure in the learned basis to make backbone representations amenable to joint diagonalization.The regularization is applied to the covariance matrix in the learned basis for each domain.
  • Training Objective: The full objective includes a structural-alignment term weighted by λcpca, and all major components are trained end-to-end.The trainable components are the feature backbone, step-size hypernetwork, and unfolded CPCA solver.
  • Inference: The orthogonal basis captures mini-batch domain-invariant geometry and conditions high-dimensional backbone features instead of serving as a low-dimensional classification bottleneck.Direct classification in the CPCA bottleneck would discard fine-grained, class-discriminative information from the ambient feature space.
  • Inference: Two lightweight MLPs map the invariant signal back to dimension D to produce affine parameters for channel-wise feature modulation.The modulation is inspired by Feature-wise Linear Modulation (FiLM).
  • Inference: Zero-initializing the final MLP layers yields γ = 1 and ∆f = 0, starting from the standard ERM baseline and gradually introducing modulation during training.The modulated feature is then passed to a linear classifier, preserving parity with the standard readout.

4 Experiments

CPCANet is evaluated on four DG benchmarks under controlled, reproducible training settings. It achieves the best average performance among ResNet-50 methods, remains computationally comparable to ERM, and is robust to its key hyperparameters while benefiting from stronger backbones.

  • Benchmark datasets: CPCANet is evaluated on PACS, VLCS, OfficeHome, and TerraIncognita, with dataset statistics and exact download sources provided for reproducibility.The paper highlights inconsistencies in prior dataset names and image counts that can hinder fair comparisons.
  • Experimental setup: All methods and datasets use fixed training settings to control for differences in training duration, batch size, optimizers, learning-rate schedules, and dataset-specific tuning.These choices are summarized in Table 2 to support fair and reproducible evaluation.
  • Overall results: CPCANet achieves the best average performance among methods using the same ResNet-50 backbone under the controlled training setup.The study also reports computational costs for all reproduced methods.
  • Hyperparameter sensitivity: Performance remains stable across the coarse grid search over projection dimension d and unfolding stages T, indicating robust extraction of domain-invariant representations.The optimal configuration, d = 256 and T = 3, is fixed for subsequent backbone experiments.
  • Backbone comparison: More advanced backbone architectures improve performance, often at the cost of increased computational resources, while CPCANet remains comparable to the ERM baseline.CPCANet is also combined with DeiT and VMamba backbones.

5 Conclusion

CPCANet integrates CPCA-based statistical structure into deep neural networks through differentiable unfolded Riemannian optimization and achieves SOTA zero-shot transfer performance across multiple DG benchmarks. The framework also motivates extensions beyond domain-labeled training and toward online distribution-shift settings.

  • CPCANet integrates CPCA-based statistical structure into deep neural networks through a differentiable, unfolded Riemannian optimization scheme.
  • CPCANet achieves SOTA zero-shot transfer performance across multiple domain-generalization benchmarks while remaining architecture-agnostic.
  • Broader Impacts and Future Directions: Because the current formulation relies on domain labels to estimate latent covariances, extending it to unsupervised or self-supervised settings is identified as a future direction.
  • Broader Impacts and Future Directions: Applying the differentiable framework to distribution-shift scenarios such as data streams and online learning is another proposed research direction.

NeurIPS Paper Checklist

The checklist affirms that the paper’s claims, limitations discussion, reproducibility disclosures, and compute-resource reporting are adequate, while noting that formal proofs and publicly available code are absent at submission.

  • Claims: The abstract and introduction accurately summarize the CPCA-based framework, deep unfolded approach, and reported performance gains.
  • Limitations: The paper discusses limitations, including reliance on domain labels and lack of validation in settings such as online learning.
  • Theory assumptions and proofs: Formal theoretical results requiring standalone proofs are not introduced because the paper focuses on methodology and model design based on established principles.
  • Experimental result reproducibility: The paper reports sufficient model, training, dataset, evaluation, and hyperparameter details to reproduce the main experiments.
  • Open access to data and code: The checklist marks open access as [No] because code was unavailable at submission, although an open-source implementation with training and evaluation details was planned.
  • Experiments compute resources: Experiments use an NVIDIA A100 GPU (80GB), with peak GPU usage and total GPU time reported in Table 3.
Loading 2605.05136v3…