PaperlayerView exact revision on arXiv

Source-linked AI summary

ARO: Aligned Representation learning for multi-Omics data

Amogh Singh, Yash Shah, Chiara D'Ercoli, Arash Mehrjou, Patrick Schwab, Timothy Jones, Pietro Liò

arXiv:2610.06443v1cs.LGcs.AI

TL;DR

High assay costs and limited paired, complete multi-omics data leave cross-modal inference an open problem. ARO learns shared representations to reconstruct missing modalities from transcriptomic input. Its embeddings support cancer classification, while the authors note that validation on external cohorts is still needed.

  • Problem

    High assay costs and limited paired, complete multi-omics data make comprehensive profiling difficult, and cross-modal inference remains open.

    Source: §1 · p. 1

  • Method

    ARO uses a masked autoencoder to learn shared representations and reconstruct proteomic and metabolomic data from transcriptomic input without requiring paired modality samples.

    Source: §1 · p. 1, §4 · p. 3, §5 · p. 3

  • Results

    ARO’s learned embeddings support cancer/non-cancer discrimination and laterality classification, reaching 100% accuracy on test data and 72.4% accuracy on laterality.

    Source: §5 · p. 4, §7 · p. 5

  • Takeaways & Limitations

    Reconstruction-trained embeddings can be used with lightweight MLPs for cancer-status and laterality classification.

    Source: §7 · p. 5

    The limited number of training samples may constrain generalization to unseen distributions, motivating validation on external cohorts.

    Source: §6 · p. 4

Abstract

from arXiv · show

The high cost of functional molecular assays, and prevalence of missing modalities and unmatched samples in computational biology, create significant barriers to comprehensive multi-omic profiling, essential for capturing and reasoning over molecules, cells, tissues, and organisms. This work proposes a model that learns meaningful representations from multi-omics cancer data supporting the reconstruction of missing and unpaired modalities. Contrary to increasingly complex, larger models, e.g. Foundation Models (FMs), ARO prioritizes practical applicability in limited or incomplete data settings. ARO optimally reconstructs missing modalities (MSE of $0.15$ on the validation and test data in the Unmasked settings), with its learned latent embeddings enabling a downstream cancer classification task. Our findings indicate that analyzing diverse molecular layers as a single integrated system offers a reliable and cost-efficient approach, reducing dependence on large-scale experimental testing, while still supporting multi-omic exploration in limited data settings.

1. Introduction

The introduction frames scarce complete, paired multi-omics data and open cross-modal inference as key gaps. ARO is proposed to reconstruct modalities without paired samples, learn a shared representation, and support downstream classification.

  • Limited availability of paired, complete multi-omics datasets and the high cost of assays constrain comprehensive characterization of biological processes.Source: §1 · p. 1
  • Transcriptomic measurements are accessible but provide only a partial view of cellular function, motivating integration of multiple omic layers.Source: §1 · p. 1
  • Existing integration methods may assume complete, aligned data or address missing subsets rather than complete modality recovery, while cross-modal inference remains open.Source: §1 · p. 1
  • ARO is presented as processing transcriptomic, proteomic, and metabolomic layers without paired modality samples and mapping transcriptomics to predicted proteomic and metabolomic spaces.Source: §1 · p. 1
  • The work proposes a shared latent representation from which ARO can infer data characteristics.Source: §1 · p. 1
  • ARO achieves 100% accuracy on classifying tumorous samples and 72.4% on laterality classification.Source: §1 · pp. 1–2

2. Background and Preliminaries

The background describes varied omics modeling approaches and gaps in alignment and modality recovery. ARO uses a shared autoencoder on cancer datasets, with preprocessing and normalization choices framed as a methodological rather than rigorous bioinformatics harmonization effort.

2. Background and Preliminaries

  • Prior approaches include foundation models, task-specific multimodal methods, domain-prior methods, and modality-specialized autoencoders, with differences in data types and integration focus.Source: §2 · p. 2

3. Data Extraction and Pre-processing

  • ARO reconstructs missing and unpaired modalities from one observed input, using a shared autoencoder without large external pretraining, biological priors, pathway information, or auxiliary imaging.Source: §3 · p. 2
  • The study uses public glioblastoma and renal cell cancer cohorts with tumor and normal paired samples and multiple transcriptomic, proteomic, and metabolomic measurements.Source: §3 · p. 2
  • The cohorts use different transcriptomic quantification pipelines, and expression was normalized per feature rather than harmonized to a common metric.The stated aim was methodological: to test representation learning on heterogeneous omics data under minimal domain-specific preprocessing assumptions.Source: §3 · p. 2

4. Architecture

ARO is a masked autoencoder with an MLP-based encoder and decoder. Training uses feature masking and reconstruction, while separate masked and unmasked setups define whether transcriptomics alone is used to infer the other modalities.

  • ARO uses an MLP-based encoder and decoder with a fully connected hidden-layer stack and ReLU nonlinearities.Source: §4 · p. 3
  • The masked setup leaves transcriptomics as input to reconstruct the other omic modalities, while the unmasked setup omits input masking.Source: §4 · p. 3
  • Training drops a random 10% of features and reconstructs the original input using Huber loss, excluding missing values from the loss with a binary mask.Source: §4 · p. 3

5. Experiments and Results

The experiments assess reconstruction in masked and unmasked settings and the downstream utility of ARO’s learned embeddings. ARO’s unmasked embeddings performed strongly on cancer classification, while PCA reconstructed more accurately but performed more weakly on downstream tasks.

Reconstruction

  • ARO’s reconstructed outputs retained distributional structure rather than collapsing to mean-like predictions, with loss computed only over observed values and masking varying across samples and batches.These design choices discourage trivial median-like solutions.Source: §5 · p. 3
  • PCA achieved superior reconstruction accuracy to ARO, despite substantially weaker performance on downstream tasks.Tables 1 and 2 report masked and unmasked reconstruction comparisons.Source: §5 · p. 3, Table 1 · p. 4, Table 2 · p. 4

Comparison with MODIS

  • MODIS scaled to approximately 22B parameters and produced out-of-memory errors, preventing complete training; limited training yielded MSE losses of 2.53 for transcriptomics, 0.55 for proteomics, and 0.43 for metabolomics.The authors report ARO remained computationally tractable across approximately 50M–700M parameters.Source: §5 · p. 4

Shared representation

  • The latent-space visualizations showed distinct clusters of cancerous and non-cancerous samples and of cancer laterality.Figure 2a uses PCA to visualize the latent space.Source: §5 · p. 4, Figure 2 · p. 4

Downstream analysis

  • ARO’s UNMASKED embeddings reached 100% accuracy on test data for binary cancer classification and 72.4% accuracy for cancer laterality, versus PCA’s 72.41% and 58.62%, respectively.Figure 2 presents the latent-space and downstream-task views.Source: §5 · p. 4, Figure 2 · p. 4

6. Limitations and Future Work

The authors identify limits in sample size, data scope, and biological interpretation, while describing masked reconstruction and a unified latent space as promising directions for multi-omics representation learning. They also propose external validation and generative extensions as future work.

6. Limitations and Future Work

  • Despite high feature dimensionality, the training sample count remains limited, motivating validation on external cohorts and potentially other diseases.The authors note this setting may constrain generalization to unseen data distributions.Source: §6 · p. 4
  • ARO focuses on aligned multi-omics matrices rather than heterogeneous settings such as imaging or spatial transcriptomics, and its learned associations are statistical regularities without established biological plausibility.Further analysis is needed to substantiate potential biological meaning.Source: §6 · pp. 4–5
  • The authors suggest variational autoencoders as a possible extension for synthesizing proteomic and metabolomic profiles conditioned on transcriptomic inputs.Source: §6 · p. 5

7. Conclusion

  • Masked-input reconstruction learns shared representations whose projections show clustering and whose embeddings support cancer discrimination with lightweight classifiers.The conclusion reports cancer and non-cancer discrimination and qualitative performance on laterality classification.Source: §7 · p. 5
  • ARO learns one latent representation across omic modalities rather than combining separate modality representations after training; the authors call for evaluation across datasets and disease types.They also identify generative extensions as a future direction.Source: §7 · p. 5

Software and Data

The study uses public glioblastoma and renal cell carcinoma multi-omics data to evaluate cross-omic translation and latent representation alignment. It presents ARO as a way to reconstruct proteomics and metabolomics from transcriptomic data, while cautioning that reconstructions are approximations.

Software and Data

  • Public glioblastoma and renal cell carcinoma cohorts are used to evaluate cross-omic translation and latent representation alignment.The source code is available on GitHub.Source: Software and Data · p. 5

Impact Statement

  • Reconstructed proteomic and metabolomic modalities from transcriptomics are approximations, requiring validation in clinical or other high-stakes applications.The approach addresses paired-dataset scarcity and may expand the utility of existing data resources.Source: Impact Statement · p. 5

Contributions

  • Amogh Singh contributed to data selection, model development, metric collection, and downstream-task validation; Yash Shah implemented the primary autoencoder and explored alternatives.Chiara D’Ercoli contributed to data selection and preparation, sourcing, literature review, drafting, writing, and finalization.Source: Contributions · p. 5

Appendix

ARO uses a fully connected feedforward autoencoder to compress modality inputs into a 2,048-dimensional representation and reconstruct the original feature space. Training uses Huber loss, with Bayesian optimization for hyperparameter tuning.

A. Model Architecture

  • The encoder projects inputs from 71,094 to 2,048 dimensions and uses three 2,048-dimensional hidden layers, each with ReLU activation and dropout.Input dropout is p = 0.1; hidden-layer dropout is p = 0.0.Source: A. Model Architecture · p. 7
  • The decoder projects the latent representation back to the original feature dimensions and applies softplus to enforce non-negative outputs.This follows preprocessing and log-space normalization.Source: A. Model Architecture · p. 7

B. Optimization Details and Hyperparameter Tuning

  • Training uses AdamW at a 10^-5 learning rate without weight decay, for 200 epochs with batch size 80 and a ReduceLROnPlateau scheduler.The scheduler uses reduction factor 0.1 and patience 10; mixed bfloat16 precision is used with fp32 fallback on unsupported hardware.Source: B. Optimization Details and… · p. 7
  • Bayesian optimization tunes dropout, hidden dimensions, learning rate, modality-specific masking, layer count, batch normalization, and weight decay; training optimizes Huber loss.The tuning results are shown in Figure 3.Source: B. Optimization Details and… · p. 7

C. Autoencoders

An autoencoder learns a compressed latent representation by reconstructing its input. Its encoder maps input data into latent space, and its decoder maps that representation back to the original input space.

  • An autoencoder learns an efficient encoding by reconstructing its input from a compressed latent representation.Source: C. Autoencoders · p. 7
  • The encoder maps input x in R^n to latent representation h in R^m, and the decoder maps h back to reconstruction y in R^n.Source: C. Autoencoders · pp. 7–8
  • The autoencoder objective minimizes reconstruction loss Lδ over N samples.Here the reconstruction loss is Huber loss.Source: C. Autoencoders · p. 8

D. Results

The results section includes reconstruction-performance tables reporting five-run averages under MSE and Huber loss; the supplied captions do not provide the values.

  • Table 4 reports reconstruction performance using MSE loss averaged over five runs.Source: Table 4 · p. 8
  • Together, the tables present five-run averages for reconstruction performance under two loss measures, MSE and Huber loss.Source: Table 4 · p. 8, Table 5 · p. 8
  • Table 5 reports reconstruction performance using Huber loss averaged over five runs.Source: Table 5 · p. 8

E. Comparison

This block documents downstream PCA confusion matrices and outlines dataset details, hyperparameter tuning, and learned-embedding visualization.

F. Dataset Breakdown

  • Table 6 provides details of the multi-omics datasets considered and analyzed in the study.Source: Table 6 · p. 8
  • Figure 3 shows validation-loss hyperparameter tuning across dropout, hidden dimensions, learning rate, modality-specific masking, hidden layers, batch normalization, and weight decay.The lowest-validation-loss configuration is selected from each setup.Source: Figure 3 · p. 9
  • Figure 4 depicts a t-SNE clustering of the learned ARO embeddings.Source: Figure 4 · p. 10

E. Comparison

  • Figure 5 presents confusion matrices for downstream tasks using PCA projections.The text confirms that the downstream analysis confusion matrix is given in Figure 5, but no outcomes are reported here.Source: E. Comparison · p. 8, Figure 5 · p. 10
Loading 2610.06443v1…