PaperlayerView exact revision on arXiv

Source-linked AI summary

Subcellularly Resolved Single-Cell Embedding Learning with Transcriptomic data, Protein Structure and Localization Information

Zhen Zhou, Jiachen Li, Yuan Liu, Xiaoyong Pan, Hong-Bin Shen

arXiv:2609.02344v1q-bio.GNcs.AI

TL;DR

Existing cell embeddings often represent cells from expression without subcellular spatial heterogeneity or protein structure. This paper uses cross-attention to integrate RNA expression, protein sequence and structural features, and localization; evaluations report accurate cell identification and improved multimodal integration.

  • Problem

    Many cell embeddings represent cells through gene-expression values and neglect intracellular spatial heterogeneity, while transcript abundance alone incompletely describes protein function.

    Source: §1 · p. 2

  • Method

    The framework uses cross-attention to integrate RNA expression, protein features including structural information, and subcellular localization into cell representations.

    Source: §1 · p. 2

  • Results

    Across cell-type identification and multimodal integration evaluations, the authors report accurate identification, superior integration performance, and robust transferability across diverse datasets.

    Source: Multimodal RNA–Protein representations… · p. 8, Cross-modal Representation Learning… · p. 10

  • Takeaways & Limitations

    The results support using cross-modal RNA–protein interaction modeling for cell-type identification and transfer across diverse single-cell datasets.

    Source: Multimodal RNA–Protein representations… · p. 8

    The model relies on existing protein annotations and structure-derived embeddings, uses coarse-grained localization priors, and does not explicitly model temporal cell dynamics.

    Source: Discussion · p. 11

Abstract

from arXiv · show

Existing cell embedding methods predominantly rely on transcriptomic or proteomic measurements and represent each cell as a holistic entity, thereby overlooking the subcellular localization of individual molecules. Moreover, they rarely incorporate protein structural information, despite its fundamental role in determining molecular interactions and functions. In this work, we propose a multimodal framework for learning subcellularly resolved cell embeddings by jointly leveraging RNA expression profiles, protein sequence representations, and protein structural information. Specifically, we employ a cross-attention architecture to integrate transcriptomic, sequence, and structural modalities and model their interactions within distinct subcellular compartments. The resulting embeddings represent each cell through its fine-grained subcellular organization, capturing both molecular expression patterns and the functional properties of the associated proteins. By learning cell representations at subcellular resolution, our framework preserves spatially organized biological information while integrating complementary signals across multiple molecular levels. To the best of our knowledge, this is the first framework that produces subcellularly resolved cell embeddings by jointly incorporating transcriptomic information, protein sequence representations, and protein structural knowledge within a unified cross-modal learning paradigm.

Introduction · 1 Introduction

The introduction argues that cell embeddings should go beyond transcriptomic profiles to capture protein structure and intracellular spatial organization. It proposes a cross-attention framework that integrates RNA expression, protein features, and subcellular localization.

1 Introduction

  • Transformer-based single-cell foundation models, including Geneformer1, scGPT2, and scFoundation3, learn representations from millions of transcriptomic profiles for downstream tasks.The introduction names cell type annotation, trajectory inference, and perturbation prediction as example applications.Source: §1 · p. 1
  • Transcriptome-centric embeddings may miss molecular mechanisms because mRNA abundance alone does not capture the functional execution of biological processes.These methods represent cells as collections of gene expression values, implicitly treating mRNA abundance as sufficient for cellular identity.Source: §1 · p. 2
  • Protein function depends on three-dimensional structure as well as expression, so RNA profiles alone can miss functional differences between proteins with similar expression patterns.The introduction points to folding topology, binding interfaces, and structural domains as sources of functional variation.Source: §1 · p. 2
  • Subcellular localization shapes cellular processes and disease-associated states, yet most embedding approaches treat cells as homogeneous and overlook intracellular spatial heterogeneity.The passage cites roles in gene expression, RNA transport, protein trafficking, and signal transduction.Source: §1 · p. 2
  • Advances in imaging, spatial transcriptomics, and localization datasets motivate jointly modeling transcriptomic signals, protein abundance, protein structure, and subcellular localization.The passage presents these modalities as components of a comprehensive cellular representation.Source: §1 · p. 2
  • The proposed cross-attention framework integrates RNA expression, protein features, protein structural information, and subcellular localization to learn biologically informed cell embeddings.The authors state that the embeddings are intended to better reflect cellular complexity and enable more accurate downstream biological inference.Source: §1 · p. 2

Material & Methods · Preprocessing and Pre-training · Protein Structure and RNA Foundation Embedding

The method preprocesses RNA counts and aligns genes to UniProt protein entries, then builds biologically informed RNA and protein representations for downstream multimodal fusion. Protein features combine sequence, structure, and localization context, while RNA features capture contextual gene and cell information.

Preprocessing and Pre-training

  • After filtering genes with fewer than 20 expressed counts, the method selects 3,000 highly variable genes, scales each cell’s counts to a fixed target such as 10,000, and log-transforms them, following prior strategies [10, 11] and preprocessing practice [10, 12–15].The log transformation is described as reducing dynamic range and extreme values while preserving relative differences among lowly expressed genes.Source: Preprocessing and Pre-training · p. 3
  • Gene symbols are mapped to corresponding UniProt protein entries to create an RNA–protein correspondence dictionary linking transcript-level measurements with protein representations when available.UniProt supplies protein sequences and associated annotations; structural and functional annotations are used when available.Source: Preprocessing and Pre-training · p. 3

Protein Structure and RNA Foundation Embedding

  • FoldExplorer17 processes UniProt protein sequences and annotations into embeddings that integrate sequence and structural priors, capturing sequence identity, structural topology, and functional similarity.FoldExplorer is described as a structure-aware protein representation model using sequence-enhanced graph embedding.Source: Protein Structure and RNA Foundation… · p. 3
  • A seven-dimensional UniProt localization vector records dominant localization probabilities for cytoplasm, nucleus, plasma membrane, mitochondria, endoplasmic reticulum, Golgi apparatus, and endosome.The vector is concatenated with the structural protein embedding to combine molecular properties with spatial context.Source: Protein Structure and RNA Foundation… · p. 3
  • scGPT18 receives gene identifiers and their expression values to generate contextualized gene and cell embeddings that capture gene dependencies and cell-state-specific transcriptional programs.These RNA embeddings reflect higher-order regulatory relationships learned from large-scale single-cell datasets, beyond count-based representations.Source: Protein Structure and RNA Foundation… · p. 4
  • FoldExplorer-derived protein embeddings and scGPT-derived RNA embeddings are aligned as modality-specific representations for subsequent cross-attention-based multimodal fusion.After preprocessing and alignment, RNA and protein features are projected into a shared embedding space for direct interaction in downstream cross-attention modules.Source: Preprocessing and Pre-training · p. 3, Protein Structure and RNA Foundation… · p. 4

The Framework of RNA&Protein SubCell Embedding

The framework uses cross-attention to integrate transcriptomic and proteomic inputs into a joint cell embedding. Separate transformer-based encoders process the modalities, with RNA embeddings guiding protein representation learning.

The Framework of RNA&Protein SubCell Embedding

  • The framework uses cross-attention model19 to process prior gene knowledge and expression values alongside protein tokens.These inputs enter dedicated modality-specific embedding spaces and are encoded with separate transformer-based architectures.Source: The Framework of RNA&Protein SubCell… · p. 4
  • A masked-attention transformer inspired by scGPT captures gene–gene and gene–cell dependencies from partially observed expression profiles.It encodes the transcriptomic pathway after modality-specific projection.Source: The Framework of RNA&Protein SubCell… · p. 4
  • RNA embeddings guide cross-attention in the proteomic encoder to model protein abundance from corresponding cell surface protein tokens.This RNA-informed interaction provides contextual guidance for aligning RNA and protein representations.Source: The Framework of RNA&Protein SubCell… · p. 4
  • The RNA-guided proteomic reasoning yields a unified multi-omic joint embedding that integrates information from both modalities for downstream analytical tasks.The embedding combines complementary transcriptomic and proteomic information.Source: The Framework of RNA&Protein SubCell… · p. 4

RNA&Protein Encoder Module

The module encodes RNA with masked self-attention and proteins with a Transformer, then uses cross-attention to produce an RNA-conditioned protein representation for downstream tasks.

RNA&Protein Encoder Module

  • A masked-attention Transformer models gene–gene interactions and uses known and masked gene subsets to prevent queries from attending to unknown genes.This enables autoregressive-style prediction on non-sequential transcriptomic data.Source: RNA&Protein Encoder Module · pp. 4–5
  • FlashAttention supports gene panels with tens of thousands of genes by reducing quadratic memory complexity while maintaining exact attention computation.The paper specifies that the gene count M can reach tens of thousands.Source: RNA&Protein Encoder Module · p. 5
  • A randomly initialized Transformer encodes proteins, with stacked self-attention capturing structural and contextual dependencies among protein tokens.Blocks include residual connections, layer normalization, and positionwise feed-forward networks.Source: RNA&Protein Encoder Module · p. 5
  • RNA embeddings act as queries and keys while protein embeddings provide values, producing an RNA-conditioned protein representation.The output is projected back to the original embedding dimension, then passed through a residual connection and layer normalization for downstream tasks.Source: RNA&Protein Encoder Module · pp. 5–6

Cell Representation Learning Via Cross-Modal Fusion

The framework builds a unified cell representation by combining protein structure modeling with RNA-guided cross-modal fusion. A pooled fusion representation captures protein organization, transcriptomic context, and global cellular state.

Cell Representation Learning Via Cross-Modal Fusion

  • A hierarchical embedding framework integrates protein-level structure modeling with RNA-guided cross-modal fusion to construct a unified multimodal cell representation.The protein encoder prepends a ⟨cls⟩ token to aggregate global protein-level information.Source: Cell Representation Learning Via… · p. 6
  • Self-attention produces a global protein representation that integrates intra-protein structural and contextual dependencies.The model obtains this representation from the hidden state of the prepended ⟨cls⟩ token.Source: Cell Representation Learning Via… · p. 6
  • Cross-attention uses protein representations as queries and RNA embeddings as keys and values, enriching protein tokens with transcriptomic context while preserving structural information.The protein representations have first passed through self-attention; RNA embeddings are denoted H.Source: Cell Representation Learning Via… · p. 6
  • The fusion block’s ⟨cls⟩ hidden state serves as the final cell representation, encoding protein structure, transcriptomic context, and global cellular state.These signals derive respectively from self-attention, cross-attention, and ⟨cls⟩ pooling.Source: Cell Representation Learning Via… · p. 6

Loss Function

The model is pretrained with a composite objective combining masked gene-expression modeling and RNA-conditioned protein-expression prediction. These objectives capture gene-level dependencies and transcript–protein relationships.

Loss Function

  • The composite pretraining objective pairs masked gene-expression modeling with protein-expression prediction conditioned on transcriptomic features.Together, the components target intra-modal gene-regulatory structure and inter-modal transcript–protein relationships.Source: Loss Function · p. 7

Masked Gene Expression Modeling

  • Masked gene-expression modeling randomly hides genes and uses an MLP to reconstruct their expression, minimizing mean squared error to learn gene co-expression and latent regulatory dependencies.Source: Masked Gene Expression Modeling · p. 7

Cross-Modal Protein Prediction Loss

  • Cross-modal protein prediction estimates each cell’s protein abundance from its RNA embedding using cross-attention fusion and a regression head.The objective is described as aligning transcriptomic and proteomic representations.Source: Cross-Modal Protein Prediction Loss · p. 7

Overall Objective

  • The final objective combines three losses, with λ1 = 0.6 and λ2 = 0.4 weighting masked gene-expression modeling and cross-modal prediction, respectively.Source: Overall Objective · p. 7

Results

The multimodal framework separated immune cell types and subpopulations, and subcellular embeddings improved classification. Across heterogeneous single-cell datasets, it achieved strong integration performance while preserving biological structure.

Multimodal RNA–Protein representations improve cell type identification in PBMC datasets

  • On 10x Multiome PBMC, the model scored 0.962 accuracy, 0.961 precision, 0.937 recall, and 0.947 macro-F1; on the multi-tissue benchmark, its scores exceeded Seurat’s on all four metrics.For multi-tissue cells, model versus Seurat scores were accuracy 0.922 vs. 0.790, precision 0.898 vs. 0.848, recall 0.920 vs. 0.732, and macro-F1 0.905 vs. 0.761. The PBMC embeddings resolved immune populations and closely related T-cell states; confusion-matrix results showed strong agreement with annotations and minimal misclassification.Source: Multimodal RNA–Protein representations… · p. 8

Effect of Subcellular Cell Embedding on Cell Type Identification

  • Adding seven-dimensional subcellular embeddings improved 10x Multiome PBMC accuracy from 0.92 to 0.94 and macro-F1 from 0.90 to 0.92, with similar improvements on the multi-tissue dataset.The ablation also improved CD4 TCM discrimination and reduced its misclassification with cDC2, indicating that compact localization features complemented transcriptomic information.Source: Effect of Subcellular Cell Embedding… · p. 9

Cross-modal Representation Learning Enables Robust Batch Integration while Preserving Cellular Identity

  • On the Human PBMCs Covid dataset, the model achieved the highest AvgBIO (0.657), with NMIcell 0.742 and ARIcell 0.691, outperforming scGPT, TotalVI, and Scanpy on AvgBIO.AvgBIO was 0.626 for scGPT, 0.607 for TotalVI, and 0.519 for Scanpy.Source: Cross-modal Representation Learning… · pp. 9–10
  • In a separate integration comparison, the model achieved the highest AvgBIO (0.795), exceeding scGPT (0.723), Harmony (0.627), and Seurat (0.541).The embedding visualizations showed cells aligned across batches while retaining distinct immune identities and major lineage structure.Source: Cross-modal Representation Learning… · p. 10
  • The integration results indicate that RNA, protein, and subcellular-scale features supported batch correction while preserving cell identity across heterogeneous datasets.Protein features helped resolve populations with overlapping transcriptional profiles, while subcellular embeddings added cellular-organization context that stabilized cell-state representations; localization features also improved CD4+ TCM identification versus cDC2.Source: Cross-modal Representation Learning… · p. 10
  • On the TEA-seq, ECCITE-seq, and CITE-seq benchmark, the model achieved AvgBIO 0.567 and Overall 0.728, exceeding scGPT on biological conservation, cell-type clustering, and batch correction.Compared with scGPT, AvgBIO was 0.567 vs. 0.517, NMIcell 0.638 vs. 0.548, ARIcell 0.408 vs. 0.339, and AvgBATCH 0.968 vs. 0.951.Source: Cross-modal Representation Learning… · p. 10, Table 1 · p. 11

Discussion

The framework combines transcriptomic and proteomic information, including protein structure and subcellular context, to learn biologically informed single-cell representations. The discussion describes its cross-attention and fusion design, then identifies limitations and directions for future work.

Discussion

  • The framework incorporates protein structure and subcellular spatial context alongside gene expression to characterize cellular states beyond expression-only representations.It uses structure-aware protein embeddings and localization priors with transcriptomic information.Source: Discussion · p. 11
  • Cross-attention aligns RNA and protein modalities, combining FoldExplorer-derived structure-aware embeddings and UniProt localization priors with scGPT-inspired transcriptomic representations.The model uses cross-attention to learn interactions between transcriptomic programs and proteomic functions.Source: Discussion · p. 11
  • A CLS-token fusion strategy forms a cell-level representation, trained with masked gene modeling and cross-modal prediction to learn intra-modal structure and inter-modal alignment.The representation encodes protein structure, transcriptomic context, and spatial organization.Source: Discussion · p. 11
  • Limitations include reliance on existing protein annotations and structure embeddings, coarse-grained localization priors, and no explicit modeling of temporal cell dynamics.These constraints may miss dynamic conformational changes, context-specific protein behavior, finer spatial organization, and trajectories such as differentiation or perturbation responses.Source: Discussion · p. 11
  • Future work could incorporate dynamic protein structures, spatially resolved omics, perturbation-aware modeling, and larger multimodal datasets across diverse biological conditions.These extensions are intended to improve intracellular characterization, enable cellular response prediction, and enhance generalization and biomedical applicability.Source: Discussion · p. 11

DECLARATION OF INTERESTS

The authors declare no competing interests and report that GPT-5 was used only for language editing, with the authors responsible for the manuscript’s scientific content and conclusions.

DECLARATION OF INTERESTS

DECLARATION OF GENERATIVE AI AND AI-ASSISTED TECHNOLOGIES

MAIN FIGURE TITLES AND LEGENDS

The figures present a cross-attention model that combines RNA expression with protein sequence, structure, and localization information to produce subcellularly resolved cell embeddings. They show cell-type discrimination and improved biological conservation, including benefits from subcellular features.

Figure 1

  • The framework uses FoldExplorer sequence-and-structure representations with seven-compartment localization features, masked-attention RNA encoding, and cross-attention fusion into a CLS-token cell embedding.RNA embeddings guide protein representation learning; a Transformer processes fused protein features, and FlashAttention improves efficiency for large gene sets.Source: MAIN FIGURE TITLES AND LEGENDS · pp. 13–14

Figure 2

  • The RNA–protein fusion embeddings distinguish PBMC immune subtypes, with UMAP, a confusion matrix, and comparisons against scGPT and Seurat across four annotation metrics.The described evaluations cover a multi-tissue immune cell benchmark and the 10x Multiome PBMC dataset; no metric values are provided in these passages.Source: MAIN FIGURE TITLES AND LEGENDS · p. 15, Figure 2 · p. 15

Figure 3

  • A seven-dimensional subcellular representation consistently improved annotation on the 10x Multiome PBMC and multi-tissue immune benchmarks across four metrics.The metrics were accuracy, precision, recall, and macro-F1; the ablation compares models with and without subcellular embeddings.Source: MAIN FIGURE TITLES AND LEGENDS · p. 16, Figure 3 · p. 16

Figure 4

  • AvgBIO was highest for our method at 0.795, indicating improved biological conservation in the healthy PBMC integration benchmark.The supplied passages give inconsistent scGPT, Seurat, and Harmony score assignments, so those comparator values are omitted.Source: Figure 4 · p. 17, MAIN FIGURE TITLES AND LEGENDS · p. 18
  • Subcellular localization improved discrimination of CD4+ TCM cells from closely related cDC2 populations in healthy PBMC embeddings.The visualizations also show separation of major immune populations while preserving relationships among related cell states.Source: Figure 4 · p. 17, MAIN FIGURE TITLES AND LEGENDS · p. 18

Related papers

Loading 2609.02344v1…