Source-linked AI summary
Angle-Based Joint and Individual Variation Explained
Qing Feng, Meilei Jiang, Jan Hannig, J. S. Marron
TL;DR
Integrating disparate data blocks from common experimental subjects calls for separating variation shared across blocks from variation specific to each block. AJIVE uses score subspaces and perturbation theory to perform that separation; in the toy example it ran faster than JIVE and COBE.
Problem
Existing approaches did not provide a simultaneous decomposition that highlights joint and individual variation, and earlier methods faced issues including iterative computation and unsupported separation thresholds.
Source: §1 · p. 4Method
AJIVE uses score subspaces to describe variation and perturbation theory to guide separation of joint and individual components.
Source: §1 · p. 2Results
10.8 seconds for AJIVE was about 11 times faster than JIVE (121 seconds) and 39 times faster than COBE (422 seconds) in the toy example.
Source: §1 · p. 2Takeaways & Limitations
AJIVE captures joint and individual variation without separate normalization, including across data blocks with substantial scale heterogeneity.
Source: §1 · p. 2When initial ranks make the Wedin bound larger than the random-direction bound, its inference can be too conservative to be useful; this can be caused by over-specification.
Source: §2.3.1 · p. 13
Abstract
from arXiv · showhide
Integrative analysis of disparate data blocks measured on a common set of experimental subjects is a major challenge in modern data analysis. This data structure naturally motivates the simultaneous exploration of the joint and individual variation within each data block resulting in new insights. For instance, there is a strong desire to integrate the multiple genomic data sets in The Cancer Genome Atlas to characterize the common and also the unique aspects of cancer genetics and cell biology for each source. In this paper we introduce Angle-Based Joint and Individual Variation Explained capturing both joint and individual variation within each data block. This is a major improvement over earlier approaches to this challenge in terms of a new conceptual understanding, much better adaption to data heterogeneity and a fast linear algebra computation. Important mathematical contributions are the use of score subspaces as the principal descriptors of variation structure and the use of perturbation theory as the guide for variation segmentation. This leads to an exploratory data analysis method which is insensitive to the heterogeneity among data blocks and does not require separate normalization. An application to cancer data reveals different behaviors of each type of signal in characterizing tumor subtypes. An application to a mortality data set reveals interesting historical lessons. Software and data are available at GitHub <https://github.com/MeileiJiang/AJIVE_Project>.
1. Introduction
AJIVE addresses the challenge of separating shared and block-specific variation in heterogeneous data by analyzing score subspaces and using perturbation theory to guide segmentation. Its toy example shows that AJIVE captures joint and individual signals despite large differences in scale and feature count, where several conventional methods fail.
1. Introduction
- AJIVE uses score subspaces to define joint and individual variation and perturbation theory to segment them, yielding an identifiable decomposition even when individual spaces are correlated.The method avoids a tuning parameter for distinguishing joint from individual variation.Source: §1 · p. 2
- 10.8 seconds for AJIVE was about 11 times faster than JIVE (121 seconds) and 39 times faster than COBE (422 seconds) in the toy example.The paper attributes this efficiency to built-in singular value decomposition replacing lengthy iterative algorithms.Source: §1 · p. 2
- Existing approaches lacked a simultaneous decomposition of joint and individual variation, while JIVE and COBE were iterative and lacked a theoretical basis for their separation threshold.The paper also notes that some earlier methods recommend normalization for scale heterogeneity without consistent criteria.Source: §1 · pp. 2, 4
1.1. Toy example
- In the toy example, X has 100 features and Y has 10,000, their values differ by almost four orders of magnitude, and their individual score subspaces are not orthogonal.The example uses two 100-column blocks, with joint and individual structures plus noise.Source: §1.1 · p. 4
- AJIVE captures the toy example's signal variations and distinguishes joint from individual structure, whereas concatenation, PLS, CCA, and original JIVE fail to provide a meaningful joint analysis.The example is presented as challenging because of differences in block magnitude and feature count, as well as correlated individual signals.Source: §1.1 · pp. 4, 6, Figure 3 · p. 6
2. Proposed method
The proposed-method section presents the population model and the estimation approach, with estimation details organized across Sections 2.2, 2.3, and 2.4.
2.1. Population model
AJIVE models shared and block-specific variation through score subspaces, with the joint component sharing a row space across blocks and individual components constrained relative to it. Its three-step procedure extracts signal spaces, segments joint and individual variation, and produces the final decomposition.
2.1. Population model
- Each data block is modeled as a low-rank signal plus additive noise, with the signal decomposed into joint and individual variation.The blocks share experimental subjects as columns, while their feature counts may differ.Source: §2.1 · pp. 6–7
- The joint matrices share a common score subspace, defined by their row spaces across blocks.The paper uses row-space vectors as score vectors and calls the row space a score subspace.Source: §2.1 · p. 7
- Individual score subspaces are orthogonal to the joint space and have no nontrivial intersection across blocks.The individual subspaces need not be mutually orthogonal.Source: §2.1 · p. 7
- Under the stated row-space conditions, the decomposition is unique; AJIVE then extracts signal spaces, segments variation, and forms final outputs.The model also assumes isotropic additive noise whose singular values are smaller than the smallest signal singular values.Source: §2.1 · p. 7
2.2. Step 1: Signal space initial extraction
AJIVE begins by thresholding singular values in each block to estimate low-rank signal spaces, then uses perturbation bounds and resampling to assess how accurately those spaces are estimated. The resulting angle bounds estimate well in the toy examples, though they can be conservative.
2.2.1. Initial low-rank approximation
- Thresholding each block’s singular values produces a low-rank signal estimate, but thresholds trade off excluding joint signal against including individual or noise components.The toy scree plots select ranks 2 for X and 3 for Y based on prominent singular-value jumps.Source: §2.2.1 · p. 8
2.2.2. Approximation accuracy estimation
- The largest principal angle measures the distance between signal and estimated row spaces, and the Wedin bound limits this subspace deviation under noise.When the estimated rank is correct, the bound depends on projected noise energy and the smallest estimated signal singular value.Source: §2.2.2 · pp. 9–10
- Resampling random subspaces orthogonal to the estimated signal provides estimates of projected noise energy used to estimate Wedin bounds.The random-subspace scheme has lower variance than sampling remaining singular vectors.Source: §2.2.2 · p. 10
- For square X, Wedin bounds are relatively tight with under- or correctly specified rank, but are conservative in the other cases; resampling estimates the bounds effectively across cases.For high-dimensional, low-sample-size Y, empirical coverage was 100% at the reported quantiles, reflecting conservative bounds.Source: §2.2.2 · pp. 10, 12, Figure 5 · p. 11
2.3. Step 2: Score space segmentation
In the two-block case, AJIVE uses principal angles between estimated score subspaces to distinguish joint from individual variation; for multiple blocks, it uses squared singular values and corresponding bounds. Rank selection compares these measures with random-direction and Wedin cutoffs.
2.3.1. Two-block case
- For two blocks, AJIVE treats small principal angles between estimated score subspaces as candidate joint directions and uses their associated vectors to estimate the joint space.The angles are ordered from smallest to largest.Source: §2.3.1 · pp. 12–13
- A two-block direction is selected as joint only when its angle is below both the random-direction 5th percentile and the resampled Wedin-bound 95th percentile.The random-direction cutoff screens angles consistent with random subspaces; the Wedin cutoff bounds angles that may arise from perturbations.Source: §2.3.1 · p. 13
- With initial ranks 2 and 2, both observed angles exceed the Wedin bound but remain below the random-direction cutoff, indicating correlated individual spaces rather than joint variation.The true joint signal is mostly in the third component of Y, which is omitted at these ranks.Source: §2.3.1 · p. 13
- At correctly specified ranks 2 and 3, the smallest angle falls below the Wedin bound, while the approximately 45° second angle remains above it and is not classified as joint.The 45° angle corresponds to the individual spaces in this simulated example.Source: §2.3.1 · p. 13
- Over-specifying both initial ranks as 3 loosens the Wedin bound, so the 45° angle can no longer be ruled out as joint; an additional angle is indistinguishable from pure noise.The extra angle is consistent with the first data block having true rank 2.Source: §2.3.1 · p. 13
- For initial ranks 2 and 4, the Wedin bound exceeds the random-direction bound and is too conservative to be useful; over-specification may cause this, but not always.Further analysis of this behavior is identified as an open problem.Source: §2.3.1 · p. 13
2.3.2. Multi-block case
- For multiple blocks, AJIVE uses squared singular values of concatenated score-space estimates: values near K suggest corresponding directions are close across blocks and may represent common joint scores.Random-direction and Wedin bounds are used to segment joint from individual components; the random-direction bound is estimated by simulation.Source: §2.3.2 · pp. 14–15
- The multi-block Wedin lower bound is independent of variation magnitudes, making joint-variation extraction insensitive to scale heterogeneity across blocks.Figure 7 generalizes the two-block diagnostic because principal angles between multiple subspaces are not defined.Source: §2.3.2 · p. 15
2.4. Step 3: Final decomposition and outputs
AJIVE reconstructs each block’s joint variation from the estimated joint row space, then separates individual signal from residual noise. Its outputs offer full-matrix, block-specific, and common-normalized representations for different analysis needs.
2.4. Step 3: Final decomposition and outputs
- AJIVE reconstructs each block’s joint variation by projecting it onto the estimated joint space, then removes any joint direction that fails a block’s Step 1 signal threshold.A direction can be removed even when joint structure is present in all but a few blocks.Source: §2.4 · p. 15
- After projecting away the joint space, AJIVE rethresholds each block’s residual with SVD, retaining above-threshold components as individual signal and treating the remainder as noise.The individual structure is orthogonal to the joint row space.Source: §2.4 · p. 15
- Full-matrix joint and individual outputs are suited to analyses focused on original features, such as identifying driving genes.These representations retain the original feature-by-subject dimensions.Source: §2.4 · p. 16
- Block-specific score representations reduce dimensionality for analyses focused on relationships between subjects, with no information loss when rotation-invariant methods are used.The representations include block-specific joint and individual score matrices and corresponding loading matrices.Source: §2.4 · p. 17
- Common Normalized Scores provide a shared basis across blocks when component correspondence matters; their loadings are linked by common scores but are not guaranteed to be orthogonal.The Common Normalized Representation is the software’s default output.Source: §2.4 · p. 17
3. Data analysis
The analysis applies AJIVE to TCGA breast-cancer measurements and Spanish male and female mortality data, whose data blocks differ in type and scale. It finds subtype-related shared and block-specific cancer variation, and joint and individual mortality patterns associated with historical changes.
3.1. TCGA Data
- DiProPerm z-score 26.54 and AUC 0.878 quantify the four-block joint component’s separation of Luminal A from the other subtypes.The second AJIVE step suggested two joint components, but the projection threshold retained only one across all four data types.Source: §3.1 · pp. 18–19, Figure 9 · p. 19
- Joint-component loadings identify TP53, TTN, and PIK3CA in Mutation, GATA3 in RPPA, and GE genes not flagged by earlier subgroup-focused studies.The authors connect GATA3’s RPPA loading with its large mutation loading.Source: §3.1 · p. 19
- DiProPerm z-score 31.60 and AUC 0.996 quantify a second joint component specific to GE, CN, and RPPA for Basal-like versus other subtypes.Its AUC is close to 0.999 from all original GE features; the component is not shared with Mutation.Source: §3.1 · p. 19, Figure 9 · p. 19
3.2. Spanish mortality data
- With rank choices 3 for males and 2 for females, AJIVE yields two joint components and one male-specific individual component.Figure 10 gives the rank-choice rationale; the authors note that loadings help show age-group effects.Source: §3.2 · p. 20, Figure 10 · p. 21
- The first joint mortality mode captures improvement in both sexes, especially around the 1950s, with different age-group patterns for males and females.Male loadings suggest gradually increasing improvement from older to younger ages; female loadings show large benefits among child-bearing-age groups.Source: §3.2 · p. 20
- The second gender-specific component contrasts years around 1950, when automobile penetration started, with years around 1980; automobile events more strongly influenced males aged 20–45, with larger loading values and a second kernel-density peak.The female loading plot shows the contrast too, but less strongly.Source: §3.2 · pp. 20–21
- The male individual component captures a Spanish Civil War mortality spike that affected young-to-middle-aged groups more than others.Source: §3.2 · p. 21
4. Optimization perspective
This section compares AJIVE with PLS, CCA/PAA, and COBE through their optimization formulations. It highlights AJIVE’s principal-angle thresholding for separating joint from individual variation and its SVD-based relationship to COBE.
4. Optimization perspective
- The comparison frames AJIVE, PLS, CCA, and COBE through optimization problems using row spaces, with the data matrices assumed row-centered in the PLS and CCA sections.Source: §4 · p. 21
- The mortality plots associate joint variation with broad improvement and automobile fatalities, while male-specific variation reflects the Spanish civil war’s impact on young-to-middle-aged men.The automobile event has a stronger impact on males aged 20–45.Source: Figure 12 · p. 22, Figure 11 · p. 22, Figure 13 · p. 23
4.1. Partial least squares
- PLS maximizes sample covariance between linear combinations of block rows, so strong individual structure can be found ahead of real joint structure and misclassified as joint.The authors illustrate this phenomenon with a toy example.Source: §4.1 · pp. 22–23
4.2. Canonical correlation analysis/ Principal angle analysis
- AJIVE uses principal angles on low-rank approximations and averages paired principal vectors into joint vectors, with a theoretically grounded threshold to segment joint and individual variation.When d1,d2 > n, CCA can be driven by noise; low-rank approximations address this issue.Source: §4.2 · p. 24
- AJIVE extends naturally to multiple blocks and is closely related to one multiset CCA formulation, although there is no consensus on which mCCA extension is preferable.Source: §4.2 · p. 24
4.3. Common orthogonal basis extraction
- AJIVE’s SVD gives the same result as COBE’s optimization, runs much faster than its heuristic iteration, and supplies a principled threshold for selecting the joint space.On TCGA data, COBE’s default threshold of 0.01 found no joint component; raising it to 1 left COBE unable to finish on the authors’ computer.Source: §4.3 · p. 25
Appendix A. Proofs
The proofs characterize joint variation through shared row-space directions and individual variation through orthogonal complements. They also use sequential projection optimization and perturbation bounds to support joint-component identification.
Appendix A. Proofs
- The joint row subspace is defined as the intersection of the data blocks’ row spaces.The proof excludes trivial cases and specifies a nonzero joint subspace.Source: Appendix A · p. 25
- Projecting each block onto the shared row space and its orthogonal complement yields joint and individual matrices whose row spaces are orthogonal.The proof states these matrices sum to the original block and are uniquely defined under its assumptions.Source: Appendix A · p. 25
- The perturbation argument bounds the angle between estimated and common joint spaces by sin(θ1,1 + θ2,1).This bound follows from the two perturbation angles given in the proof.Source: Appendix A · p. 26
- Each singular-value component is selected sequentially by maximizing the Frobenius norm over rank-1 projections orthogonal to earlier optima.The maximizing projection is denoted Q_i.Source: Appendix A · p. 26
- The optimization continues through at least r_J steps when the singular values meet the stated condition for joint components.The proof’s condition concerns perturbations of the common joint row space in each block’s estimated row space.Source: Appendix A · p. 26
Appendix B. Details of the toy example
The toy example tests whether methods separate joint and individual signals across blocks with very different dimensions and correlated individual spaces. The comparisons show distinct failures for concatenation, PLS, and JIVE, while COBE segments the example correctly but takes longer than AJIVE.
Appendix B. Details of the toy example
- In Section 1.1, a toy example comprises two data blocks, X (100 × 100) and Y (10,000 × 100), with patterns corresponding to joint and individual structures.Source: Appendix B · p. 26
- Concatenation SVD fails to yield a meaningful joint analysis: noise from X dominates Y’s individual signal in the low-rank approximations.Across ranks 2, 3, and 4, the individual pattern in Y is not captured; Figure B.14 shows the approximations.Source: Appendix B · p. 27, Figure B.14 · p. 27
- PLS mixes joint and individual components rather than separating them, and increasing its component count adds more noise.Figure B.15 shows approximations for one, two, and three components.Source: Appendix B · p. 27, Figure B.15 · p. 28
- JIVE’s permutation-test ranks mix X’s individual component into the joint approximation, while its BIC ranks label true joint signal as noise.Manually specifying the correct joint and individual ranks produces the correct results.Source: Appendix B · p. 27, Figure B.16 · p. 29
- COBE correctly segments the toy example but takes 39 times longer than AJIVE.The captioned comparison concerns the toy example’s joint and individual variation structure.Source: Appendix B · p. 29