Source-linked AI summary
Joint and individual variation explained (JIVE) for integrated analysis of multiple data types
Eric F. Lock, Katherine A. Hoadley, J. S. Marron, Andrew B. Nobel
TL;DR
Existing analyses need to integrate multiple high-dimensional data types measured on common objects while preserving both shared and type-specific information. JIVE addresses this need by decomposing the data into joint, individual, and residual components. Applied to GBM gene-expression and miRNA data, it improves characterization of tumor types and biological interactions, with limitations involving outliers, missing values, and comparison settings.
Problem
Separate analyses of multiple data types can miss associations, although each type may provide unique information for inference about common objects.
Method
JIVE decomposes multiple data matrices into low-rank joint variation, low-rank variation individual to each data type, and residual noise.
Results
JIVE applied to GBM gene-expression and miRNA data provides better characterization of tumor types and better understanding of biological interactions between the data types.
Takeaways & Limitations
JIVE can analyze data regardless of whether dimensionality exceeds sample size and can extend to more than two data types while revealing joint and individual structures.
Takeaways & Limitations
JIVE estimates are not robust to outliers, and missing values must be imputed before estimation.
Abstract
from arXiv · showhide
Research in several fields now requires the analysis of data sets in which multiple high-dimensional types of data are available for a common set of objects. In particular, The Cancer Genome Atlas (TCGA) includes data from several diverse genomic technologies on the same cancerous tumor samples. In this paper we introduce Joint and Individual Variation Explained (JIVE), a general decomposition of variation for the integrated analysis of such data sets. The decomposition consists of three terms: a low-rank approximation capturing joint variation across data types, low-rank approximations for structured variation individual to each data type, and residual noise. JIVE quantifies the amount of joint variation between data types, reduces the dimensionality of the data and provides new directions for the visual exploration of joint and individual structures. The proposed method represents an extension of Principal Component Analysis and has clear advantages over popular two-block methods such as Canonical Correlation Analysis and Partial Least Squares. A JIVE analysis of gene expression and miRNA data on Glioblastoma Multiforme tumor samples reveals gene-miRNA associations and provides better characterization of tumor types. Data and software are available at https://genome.unc.edu/jive/
1. Introduction.
Biomedical research increasingly integrates multiple high-dimensional data types measured on common samples, but separate analyses miss cross-type associations. The paper motivates JIVE as an integrative decomposition that separates shared and data-type-specific variation, illustrated with GBM gene-expression and miRNA data.
- Motivation: Multiple technologies increasingly measure diverse high-dimensional information on common biological samples.Examples include public databases and large-scale projects such as TCGA.
- Motivation: Analyzing data types separately can miss associations between them, while each type may contain unique information useful for inference.The motivation is especially strong for integrated gene-expression and miRNA analysis.
- Proposed method: Gene-expression and miRNA data contain biologically related shared patterns, but each also has systematic variation unrelated to the other.Individual variation may reflect technical artifacts or biological information, and can obscure joint signals.
- Proposed method: JIVE decomposes integrated data into low-rank joint structure, low-rank individual structures, and residual noise.The decomposition is designed to separate shared effects from variation specific to each data type.
- GBM application: JIVE identifies complex shared structure in GBM samples and substantial gene-expression-specific variation, while joint structure appears more relevant to cancer biology.The heatmap uses the same sample ordering for both data types; individual structure explains more variability than joint structure in this analysis.
2. Model and estimation.
JIVE models multiple data matrices as shared low-rank structure, data-type-specific low-rank structure, and residual noise, estimating these components under rank and orthogonality constraints. An iterative SVD-based procedure quantifies joint and individual variation and recovers simulated signals before applying the framework to GBM data.
- Model: JIVE represents joint structure with one low-rank matrix and individual structure with a separate low-rank matrix for each data type.For GBM, the matrices contain gene expression and miRNA measurements for the same 234 tissue samples.
- Model: The unified model decomposes each data matrix into joint structure, individual structure, and residual error.The error matrices have independent entries with mean zero.
- Model: Joint and individual components are constrained to specified ranks, with orthogonality ensuring unique separation of shared and data-type-specific sample patterns.The orthogonality requirement does not constrain the model because equivalent decompositions can be written with this property.
- Estimation: Estimation minimizes squared residual error by alternating updates of individual structures and joint structure until convergence.Each update uses a truncated SVD after removing the other component: rank-r for the joint structure and rank-ri for each individual structure.
- Estimation: The iterative procedure is monotone, decreasing residual norm at every step and converging to a coordinate-wise minimum.Further convergence properties of the algorithm remain under study.
- GBM application: Permutation testing selected rank 5 for joint structure, rank 33 for gene-expression structure, and rank 13 for miRNA structure.The test used α = 0.01 and 1000 permutations after scaling the two data types for differences in dimension and variability.
3. Model factorization.
JIVE factorizes each data type into shared low-rank structure, data-type-specific low-rank structure, and residual noise. Its joint scores reveal subtype patterns across miRNA and gene expression, while individual scores capture patterns specific to each data type.
- Relation to PCA: The joint structure is factorized as US, with U containing variable loadings and S containing sample scores.This parallels the PCA representation of a low-rank matrix.
- Model factorization: JIVE represents each data type as joint structure, individual structure, and residual noise.The joint and individual terms are low-rank approximations.
- Interpretation: Joint scores summarize sample patterns shared across miRNA and gene expression, whereas individual scores summarize data-type-specific patterns.Loading matrices describe how these score patterns are expressed across variables.
- GBM data: Subtype distinctions are clearly visible in the joint-score scatterplot but not visually apparent in either individual-score scatterplot.Samples are displayed by the first two joint, individual miRNA, and individual gene-expression components.
- GBM data: The joint clustering indicates miRNA involvement in subtype differentiation, despite a small fraction of gene-expression variation being explained jointly.Individual variation appears to reflect other biological components.
- GBM data: A permutation test found the four subtypes significantly more distinguished on estimated joint structure than on the original gene-expression and miRNA data (p < 0.001; 10,000 permutations).SWISS scores quantify within-subtype variability relative to total variability; lower scores indicate more subtype distinction.
4. Comparison with existing methods.
The paper contrasts JIVE with methods that either mix joint and individual variation or are limited by dimensionality, overfitting, or pairwise design. Simulations show JIVE more directly recovers common signals while its individual components distinguish data-type-specific groups.
- Existing methods: Variable-by-variable association studies do not identify the global modes of variation that drive cross-data-type associations.The paper distinguishes this focus from large-scale pairwise correlation and network analyses.
- Existing methods: Consensus PCA and related concatenation approaches synthesize data types but do not distinguish joint effects from individual effects.PCA of concatenated data is driven by all variation, joint or individual.
- CCA: Standard CCA is typically unsuitable for high-dimensional data because directions are undefined when p1 > n or p2 > n and overfitting can occur otherwise.In the simulation, CCA scores were highly correlated but showed nearly no association with the common or individual structures.
- PLS: PLS handles high-dimensional data, but structured variation unrelated to the other data type can drastically alter its scores and complicate interpretation.O2-PLS addresses this issue but remains restricted to pairwise comparisons.
- Existing methods: mCCA extends the PLS idea to two or more data types, while JIVE explicitly separates shared and data-type-specific structure.The supplied passages characterize mCCA as a natural extension of PLS; the JIVE decomposition provides the separate components.
- Illustrative example: In simulations, JIVE joint scores closely tracked the common signal, and individual scores distinguished groups specific to X and Y.Joint loadings reflected effects on the first 25 variables, while individual loadings reflected effects across all 50 variables.
5. Variable sparsity.
Sparse JIVE selects associated subsets of variables across data types, enabling joint components to expose sample patterns and gene–miRNA relationships. In GBM data, the first two components separated tumor subtypes and revealed associations beyond simple correlations.
- Variable sparsity: Sparse JIVE uses penalties to shrink weak variable loadings exactly to zero, producing sparse joint and individual structures.The implementation uses an L1 penalty, though other sparsity-inducing penalties may be substituted.
- Variable sparsity: Inducing sparsity identifies subsets of variables within each data type that are associated, while joint sample scores reveal patterns driving those associations.
- GBM application: The first joint component’s nonzero gene–miRNA loading pairs closely match significant correlation pairs, with good agreement in their signs.The sparse loading pattern forms a Cartesian product of genes and miRNAs with nonzero loadings.
- GBM application: The first joint component distinguishes Mesenchymal and Proneural subtypes, suggesting these sample differences drive the component and influence the correlation structure.
- GBM application: The second joint component distinguishes Neural and Classical subtypes and captures gene–miRNA associations not immediately apparent from correlations alone.These associations appear to be masked by variation captured in the first joint component.
- GBM application: The interaction network links predicted targets supported by at least two databases among miRanda, Pictar, RNA22, and TargetScan.It focuses on genes and miRNAs with large absolute loadings in the first two joint components.
- GBM application: POSTN has the largest gene loading in the first joint component, while mir-124a has the largest miRNA loading in both components.
6. Summary and discussion.
JIVE addresses the limited availability of general methods for integrated high-dimensional data by separating coordinated and data-type-specific variation. The GBM application improved tumor-type characterization and understanding of gene–miRNA interactions, while several methodological boundaries remain.
- Summary and discussion: JIVE identifies coordinated activities across multiple data types alongside features unique to particular data types.The authors report that accounting for joint structure can improve estimation of individual structure, and vice versa.
- Summary and discussion: In GBM tumor samples, JIVE provided better characterization of tumor types and better understanding of biological interactions between gene and miRNA data.
- Limitations: JIVE estimates are not robust to outliers, and missing values must be imputed before estimating joint and individual structures.A robust JIVE version and an estimation approach that explicitly accounts for missing values are identified as potential extensions.
- Limitations: The statistical properties of the algorithm need further attention, including useful confidence measures for variation explained by joint and individual structures.Bootstrapping may help, but the discrete nature of the ranks requires careful consideration.
- Extensions: Although demonstrated on vertically integrated biomedical data, JIVE may also apply to horizontally integrated data and other contexts such as finance.Those applications are described as currently under study.
SUPPLEMENTARY MATERIAL
The supplementary article provides additional theoretical, computational, and empirical validation materials for JIVE.
- SUPPLEMENTARY MATERIAL: The supplement includes a proof of decomposition existence and uniqueness, permutation-based rank selection, and algorithm pseudocode.
- SUPPLEMENTARY MATERIAL: It also discusses computing time, efficiency, invariance properties, and applications to diverse simulated data sets.