Source-linked AI summary
A Variational Information Bottleneck Approach to Multi-Omics Data Integration
Changhee Lee, Mihaela van der Schaar
TL;DR
Multi-omics samples often have missing views, while existing multi-view methods typically assume all views are available, complicating flexible integration and task-relevant learning. DeepIMV is a deep variational information bottleneck approach for incomplete multi-view observations, consisting of four network components. DeepIMV consistently gains from data integration and significantly outperforms state-of-the-art benchmarks in prediction performance on real-world datasets.
Problem
Multi-omics samples often have missing views, while existing multi-view methods typically assume all views are available, complicating flexible integration and task-relevant learning.
Source: §1 · p. 1Method
DeepIMV is a deep variational information bottleneck approach for incomplete multi-view observations, consisting of four network components.
Source: §1 · p. 1Results
Takeaways & Limitations
DeepIMV provides a unified approach for flexibly integrating available views while focusing representations on target-relevant intra-view and inter-view interactions.
Source: §6 · p. 8
Abstract
from arXiv · showhide
Integration of data from multiple omics techniques is becoming increasingly important in biomedical research. Due to non-uniformity and technical limitations in omics platforms, such integrative analyses on multiple omics, which we refer to as views, involve learning from incomplete observations with various view-missing patterns. This is challenging because i) complex interactions within and across observed views need to be properly addressed for optimal predictive power and ii) observations with various view-missing patterns need to be flexibly integrated. To address such challenges, we propose a deep variational information bottleneck (IB) approach for incomplete multi-view observations. Our method applies the IB framework on marginal and joint representations of the observed views to focus on intra-view and inter-view interactions that are relevant for the target. Most importantly, by modeling the joint representations as a product of marginal representations, we can efficiently learn from observed views with various view-missing patterns. Experiments on real-world datasets show that our method consistently achieves gain from data integration and outperforms state-of-the-art benchmarks.
1 Introduction
Incomplete omics observations make integration difficult: existing multi-view methods generally assume every view is present, while discarding incomplete samples or imputing missing data has drawbacks. DeepIMV uses an information-bottleneck approach with marginal and joint representations and reports stronger prediction performance on real-world datasets.
1 Introduction
- Missing omics views are common, while discarding incomplete samples reduces sample sizes and mean imputation can distort marginal and joint distributions.These constraints arise from experimental designs or differences among data platforms.Source: §1 · p. 1
- Existing multi-view methods typically assume complete observations, motivating a framework that handles varied missing-view patterns while learning task-relevant within- and across-view interactions.Source: §1 · p. 1
- DeepIMV uses view-specific encoders and predictors, a product-of-experts module, and a multi-view predictor to integrate observed views.Its joint representation is modeled as a product of marginal representations.Source: §1 · p. 1
- DeepIMV consistently gains from data integration and significantly outperforms state-of-the-art benchmarks on prediction performance.The approach targets task-relevant intra-view and inter-view interactions.Source: §1 · p. 2
2 Related Works
Prior multi-view work includes methods for complete observations and approaches to incomplete views, but gaps remain around supervised, flexible integration. Related information-bottleneck work also does not address the paper’s full setting.
2.1 Multi-View Learning
- Complete-view methods learn shared representations, sometimes using labels to make them target-discriminative, but assume all views are available.Source: §2.1 · p. 2
- Some incomplete-view methods use imputation or auxiliary generation, while flexible latent-factorization and matrix-factorization approaches are unsupervised and may miss target-relevant information.Source: §2.1 · p. 2
- DeepIMV differs from CPM-Nets by using posterior factorization and task-relevant latent representations; the paper identifies reconstruction-based testing and joint sample updates as CPM-Nets limitations.The authors also note CPM-Nets is designed for classification, with regression extensions potentially losing cluster-friendly representations.Source: §2.1 · p. 2
2.2 Information Bottleneck
- The information bottleneck balances concise representations with prediction power, but prior multi-view IB approaches do not cover supervised integration of incomplete views containing common and complementary information.One cited approach cannot handle missing views during training or testing; another is an unsupervised two-view method.Source: §2.2 · p. 2
3 Incomplete Multi-View Problem
The paper frames incomplete multi-view learning as supervised prediction when samples may lack views in arbitrary patterns. Its central challenges are using marginal and joint information and integrating the available views flexibly.
3 Incomplete Multi-View Problem
- A view is missing when its observation is absent, and the observed-view set specifies the sample’s view-missing pattern.An observation is incomplete when that set is a proper subset of all views; otherwise it is complete.Source: §3 · p. 3
- An incomplete multi-view problem involves predicting a target for a new observation with an arbitrary pattern of missing views.The definition covers classification and regression.Source: §3 · p. 3
- The problem requires representations that use both marginal and joint aspects of observed views and flexibly integrate different missing-view patterns.Source: §3 · p. 3
4 Method: DeepIMV
DeepIMV integrates incomplete multi-view observations through a product-of-experts joint representation and uses the information bottleneck principle to focus marginal and joint representations on target-relevant information. View-specific and multi-view predictors support learning view-specific and cross-view aspects of the target.
4 Method: DeepIMV
- DeepIMV combines view-specific encoders and predictors, a product-of-experts module, and a multi-view predictor to integrate incomplete multi-view observations.The encoders map each view into a common latent space; the product of marginal representations forms the joint representation used for multi-view prediction.Source: §4 · p. 3
4.1 Toward Task-Relevant Representations
- The information bottleneck principle encourages marginal and joint representations to retain task-relevant information while discarding input information irrelevant to prediction.The joint representation encodes the observed views, and the multi-view predictor estimates the target from it.Source: §4.1 · p. 4
4.2 Product-of-Experts for Incomplete Views
- A product of marginal posteriors forms the joint posterior, allowing available views to be integrated across missing-view patterns without generating missing views.The joint posterior can be computed from the available marginal posteriors; PoE can produce a sharper distribution that combines common and complementary information.Source: §4.2 · p. 4
4.3 Building View-Specific Expertise
- View-specific predictors apply the information bottleneck principle to marginal representations so each encoder can learn view-specific expertise for predicting the target.The authors introduce these predictors to address the difficulty of training a product-of-experts while allowing views to capture potentially complementary target information.Source: §4.3 · pp. 4–5
4.4 Training
- Training minimizes a combination of marginal and joint information-bottleneck losses, with α controlling the trade-off between them.The training procedure updates view-specific encoder-predictor pairs and the final predictor.Source: §4.4 · p. 5
5 Experiments
Experiments evaluate DeepIMV on TCGA mortality and CCLE drug-sensitivity prediction, including incomplete-view integration, ablations, and robustness to missing views. Across these settings, the reported comparisons show gains from integrating incomplete observations and competitive performance as views or missingness vary.
5 Experiments
- Experiments evaluate multi-view learning methods on TCGA6 for 1-year mortality and CCLE for drug sensitivity, comparing DeepIMV with 2 baselines and 6 state-of-the-art methods.For TCGA, analyses use 7,295 samples across 4 views; 3,282 have incomplete multi-view observations.Source: §5 · p. 5, §5.1 · pp. 5–6
5.1.1 Multi-Omics Data Integration
- DeepIMV outperforms benchmarks when integrating incomplete TCGA samples across observed-view counts, and performs best with partially observed views after training only on complete samples, except at one view.Table 2 reports AUROC across |Vn|=1–4 and distinguishes training on complete-only versus complete and incomplete samples.Source: §5.1.1 · p. 6, Table 2 · p. 6
- DeepIMV outperforms benchmarks using MVAE view completion in place of mean imputation on TCGA at |Vn|=3.Table 3 compares AUROC across view-completion methods.Source: §5.1.1 · p. 6, Table 3 · p. 6
- TCGA latent projections become more discriminative as DeepIMV incorporates views, while using views {1, 2, 3} yields projections nearly identical to using all four.The authors suggest this could inform which omics layers need not be measured for cost-efficient predictions.Source: §5.1.1 · pp. 6–7, Figure 2 · p. 7
5.1.2 Ablation Study
- The ablation study finds that PoE, view-specific predictors, and marginal IB losses contribute to DeepIMV’s performance improvement.The study compares these components on TCGA using the same setting as the preceding integration analysis.Source: §5.1.2 · p. 7, Table 4 · p. 7
5.2.1 Multi-Omics Data Integration
- As more views are integrated, DeepIMV usually improves drug-sensitivity discrimination across the tested CCLE datasets; DCCA and DCCAE saturate at two views.GCCA improves as it generalizes to multiple views, while MVAE and MOFA representations focus on input information for reconstruction.Source: §5.2.1 · p. 8, Figure 3 · p. 8
5.2.2 Robustness to Missing Views
- Across missing-view rates, DeepIMV outperforms all benchmarks on Irinotecan and Panobinostat and is comparable to the best benchmark on Lapatinib and PLX4720.The comparison varies the missing-sample rate from R=0.0 to R=1.0 with six views.Source: §5.2.2 · p. 8
6 Conclusion
DeepIMV identifies target-relevant intra-view and inter-view interactions while flexibly integrating available views across missing-view patterns. On real-world multi-omics datasets, it significantly outperforms existing multi-view learning methods in prediction performance.
6 Conclusion
- DeepIMV significantly outperforms existing multi-view learning methods in prediction performance across the evaluated real-world multi-omics datasets.The authors also report consistent gains from data integration.Source: §6 · p. 8
- The method flexibly integrates available views regardless of their missing patterns in a unified framework.Source: §6 · p. 8
- DeepIMV finds intra-view and inter-view interactions relevant to predicting target labels.Source: §6 · p. 8
Supplementary Material for A Variational Information Bottleneck Approach to Multi-Omics
The supplementary derivation expresses the joint variational information bottleneck loss as a trade-off between predictive information and information retained about the input. Variational approximations yield the loss, with the label entropy omitted because it does not affect optimization.
A.1 Derivation of Variational IB Loss in (1) and (2)
- The joint IB loss balances negative target information, −I(Z; Y), against input information retained in Z, βI( X̄; Z).β ≥ 0 controls the balance between the two information quantities.Source: §A.1 · p. 11
- Variational approximations qθ(Z| X̄) and qψ(Y|Z) are used to derive the information terms in the joint IB loss.Source: §A.1 · p. 11
- The label entropy H(Y) is ignored in the approximate loss because it is independent of the optimization procedure.Source: §A.1 · p. 11
- The marginal representation IB loss is derived analogously to the joint representation loss.Source: §A.1 · p. 11
A.2 Training DeepIMV via IB Losses
DeepIMV trains shared encoders through both joint and marginal IB losses, pairing them with a multi-view predictor and view-specific predictors, respectively. Its MLP architecture and balancing coefficients are selected or set as described in the implementation details.
A.2 Training DeepIMV via IB Losses
- Joint IB loss updates the view-specific encoders and multi-view predictor, while marginal IB losses update the encoders and view-specific predictors.Figure S.1 illustrates the architecture for three views with the second view missing.Source: §A.2 · p. 11, Figure S.1 · p. 12
B Implementation Details
- DeepIMV uses MLPs for its view-specific encoders and predictors and multi-view predictor; cross-validation on the validation set selects hidden units and layers by minimum total loss Lθ,ψ in (4).Candidate hidden-unit counts were 50, 100, or 300; layer counts were 1, 2, or 3; latent representation dimensions were 50 or 100.Source: B Implementation Details · p. 12
- Training uses Adam with learning rate 1 × 10^-4 and dropout probability 0.7; L1 regularization is used for CCLE but not TCGA.The L1 coefficient is 1 × 10^-4 for CCLE.Source: B Implementation Details · p. 12
- The implementation sets α = 1.0 and β = 0.01, assuming equal βv across views.Source: B Implementation Details · p. 12
C Details of the Benchmarks
The benchmark comparison combines direct MLP baselines with representation-learning methods followed by downstream prediction, while handling missing views differently across methods. The TCGA dataset uses four omics views and clinical labels, including a binary one-year mortality label.
C Details of the Benchmarks
- The benchmark set compares two baselines and six state-of-the-art multi-view methods, including GCCA, DCCA, DCCAE, MVAE, CPM-Nets, and MOFA.Source: C Details of the Benchmarks · p. 12
- Unsupervised methods first learn a common latent representation and then train an MLP for target prediction; Base1 concatenates views, while Base2 averages predictions from observed views.Source: C Details of the Benchmarks · p. 13
- DCCA and DCCAE use only two views, so experiments with more than two available views select the two best-performing views for those methods.Source: C Details of the Benchmarks · p. 13
- MVAE, CPM-Nets, and MOFA handle incomplete observations during training, whereas other benchmarks except Base2 use mean imputation for missing views.Source: C Details of the Benchmarks · p. 14
D.1 TCGA Dataset
- The TCGA dataset combines DNA methylation, microRNA expression, mRNA expression, and RPPA with clinical labels.Source: §D.1 · p. 14
- Time to death or censoring in TCGA clinical labels is converted into a binary label for one-year mortality.Source: §D.1 · p. 14
D.2 CCLE Dataset
The CCLE experiments use six omics views and drug-response labels, with within-view missing values imputed and each view reduced to 100 features. In additional experiments, MVAE-based imputation did not consistently improve benchmark performance, and DeepIMV outperformed benchmarks except for Lapatinib on CCLE.
D.2 CCLE Dataset
- CCLE experiments used six omics views: DNA copy number, DNA methylation, microRNA and mRNA expression, RPPA, and metabolites.Drug response was binarized by ActArea quartiles: the top 25% were labeled sensitive and the rest non-sensitive.Source: §D.2 · p. 14
- Within observed views, missing values were mean-imputed, and kernel PCA with polynomial kernels reduced each view to 100 features.The authors state that the reduction focused experiments on integration and avoided the curse of dimensionality.Source: §D.2 · p. 14
E.1 Additional Experiments with Multi-View Imputations
- With MVAE-based imputation, DeepIMV still outperformed benchmarks on all tested datasets except Lapatinib on CCLE.Benchmarks using MVAE reconstructions did not always improve over mean imputation, which can fail to preserve information relevant to prediction.Source: §E.1 · p. 15
E.2 Sensitivity Analysis – Effects of α and β
The sensitivity analysis examines how α and β affect DeepIMV's prediction performance using complete- and incomplete-view samples. Both parameters have failure modes at extreme values, and the experiments use different α settings for TCGA and CCLE.
The Effect of α
- At α = 10.0, performance drops as DeepIMV emphasizes view-specific aspects at the expense of joint aspects; too-small α impairs task-relevant learning from individual views.The analysis uses TCGA data and reports AUROC across α and β settings.Source: §E.2 · p. 15, Figure S.2 · p. 16
The Effect of β
- At β = 0.1, classification performance drops because excessive β blocks input information needed for prediction; small β can promote overfitting.The authors use early stopping in the small-β cases to prevent overfitting.Source: §E.2 · pp. 15–16, Figure S.2 · p. 16
E.2 Sensitivity Analysis – Effects of α and β
- Throughout all experiments, they set the parameter pair (α, β) to (1.0, 0.01) for the TCGA dataset and (0.1, 0.01) for the CCLE dataset.Source: §E.2 · p. 16