PaperlayerView exact revision on arXiv

Source-linked AI summary

Subcellularly Resolved Single-Cell Embedding Learning with Transcriptomic data, Protein Structure and Localization Information

Zhen Zhou, Jiachen Li, Yuan Liu, Xiaoyong Pan, Hong-Bin Shen

arXiv:2609.02344v1q-bio.GNcs.AI

TL;DR

Existing cell embeddings often omit protein structural information and intracellular spatial organization. This work introduces a cross-attention multimodal framework that integrates RNA, protein, structural, and localization information, achieving strong cell identification and integration results across single-cell benchmarks while retaining stated scope limitations.

  • Problem

    Existing embeddings are largely transcriptome-centric, whereas protein structure and subcellular localization provide additional functional and spatial information.

  • Method

    A cross-attention multimodal framework jointly integrates RNA expression with protein sequence, structure-aware embeddings, and subcellular localization information.

  • Results

    The model improved cell-type identification and multimodal integration across evaluated benchmarks, including AvgBIO 0.567 and Overall 0.728 on TEA-seq, ECCITE-seq, and CITE-seq.

  • Takeaways & Limitations

    Subcellularly informed multimodal embeddings preserve molecular and spatial signals while supporting accurate cell identification and transferability across diverse single-cell datasets.

  • Takeaways & Limitations

    The framework relies on existing annotations and structure-derived embeddings, uses coarse-grained localization priors, and does not explicitly model temporal cell dynamics.

Abstract

from arXiv · show

Existing cell embedding methods predominantly rely on transcriptomic or proteomic measurements and represent each cell as a holistic entity, thereby overlooking the subcellular localization of individual molecules. Moreover, they rarely incorporate protein structural information, despite its fundamental role in determining molecular interactions and functions. In this work, we propose a multimodal framework for learning subcellularly resolved cell embeddings by jointly leveraging RNA expression profiles, protein sequence representations, and protein structural information. Specifically, we employ a cross-attention architecture to integrate transcriptomic, sequence, and structural modalities and model their interactions within distinct subcellular compartments. The resulting embeddings represent each cell through its fine-grained subcellular organization, capturing both molecular expression patterns and the functional properties of the associated proteins. By learning cell representations at subcellular resolution, our framework preserves spatially organized biological information while integrating complementary signals across multiple molecular levels. To the best of our knowledge, this is the first framework that produces subcellularly resolved cell embeddings by jointly incorporating transcriptomic information, protein sequence representations, and protein structural knowledge within a unified cross-modal learning paradigm.

1 Introduction

Existing cell embeddings are largely transcriptome-centric and treat cells as spatially homogeneous, while protein structure and subcellular localization provide additional functional and organizational information. The proposed framework jointly models transcriptomic, protein, structural, and localization signals to learn biologically informed subcellular cell representations.

  • Most existing cell embeddings rely on gene expression and may miss molecular mechanisms underlying cellular behavior.
  • Protein structure complements transcript abundance because folding topology, binding interfaces, and structural domains influence protein function.
  • Subcellular localization regulates processes including gene expression, RNA transport, protein trafficking, and signal transduction, but many embeddings neglect intracellular spatial heterogeneity.
  • A comprehensive cellular representation is proposed to combine transcriptomic signals, protein abundance, structural properties, and spatial organization.
  • The framework jointly integrates RNA expression, protein features, protein structure, and subcellular localization through cross-attention-based multimodal learning.

Material & Methods

The method preprocesses and aligns RNA and protein modalities, then derives biologically informed embeddings from pretrained transcriptomic and structure-aware protein models. Localization priors are combined with protein structure before both modalities are projected into a shared space for cross-attention fusion.

  • RNA preprocessing filters low-count genes, selects the top 3,000 highly variable genes, and normalizes and log-transforms expression counts.Counts are scaled to a target sum such as 10,000 before log transformation.
  • Protein representations combine UniProt sequences and annotations with optional embeddings from pretrained structure or sequence encoders.
  • UniProt gene-to-protein mapping aligns transcript-level measurements with corresponding protein entities when available.
  • FoldExplorer generates structure-aware protein embeddings, while UniProt localization priors add spatial cellular context.
  • The localization prior is concatenated with the structural embedding to represent molecular properties and spatial organization jointly.
  • scGPT produces contextualized RNA embeddings that capture gene dependencies and cell-state-specific transcriptional programs beyond raw expression counts.

The Framework of RNA&Protein SubCell Embedding

The framework uses separate transcriptomic and proteomic encoders before allowing RNA representations to guide protein representation learning through cross-attention. Their interaction produces a unified joint embedding for downstream analyses.

  • Transcriptomic and proteomic inputs are projected into modality-specific spaces and encoded with separate transformer-based architectures.
  • The transcriptomic pathway uses masked attention to model gene–gene and gene–cell dependencies from partially observed expression profiles.
  • RNA embeddings act as contextual guidance for proteomic representation learning through cross-attention over protein tokens.
  • The RNA-guided proteomic pathway yields a unified multi-omic joint embedding that combines complementary information from both modalities.

RNA&Protein Encoder Module

The encoder module applies masked self-attention to RNA and self-attention to protein tokens, then uses RNA–protein cross-attention to produce a unified multimodal representation. Attention projections and masking define how modality-specific information is modeled and aligned.

  • RNA Encoder: RNA embeddings are encoded with a multi-layer Transformer to model gene–gene interactions, using query, key, and value projections.The embedding dimension is D and the number of genes is M.
  • RNA Encoder: Masked attention partitions genes into known and unknown subsets to support autoregressive-style prediction on non-sequential expression data.The mask prevents query tokens from attending to future or unknown genes.
  • RNA Encoder: FlashAttention is used to reduce quadratic memory complexity while maintaining exact attention computation for large gene panels.
  • Protein Encoder: The protein encoder applies stacked self-attention layers to capture structural and contextual dependencies among protein tokens.Transformer blocks are followed by residual connections, layer normalization, and feed-forward networks.
  • Cross-modal Fusion: Cross-attention uses RNA embeddings as queries and keys and protein embeddings as values to create an RNA-conditioned protein representation.The resulting representation is projected back to the original embedding dimension.
  • Cross-modal Fusion: The final multimodal representation encodes transcriptomic context and proteomic structural information for downstream tasks.

Cell Representation Learning Via Cross-Modal Fusion

The framework hierarchically integrates protein structure with RNA-guided cross-modal fusion to construct a unified multimodal cell representation. The resulting embedding combines structural organization, transcriptomic context, and global cellular state.

  • The hierarchical embedding framework integrates protein-level structure modeling with RNA-guided cross-modal fusion.
  • A special ⟨cls⟩ token is prepended to protein tokens to aggregate global protein-level information.The sequence contains N protein tokens.
  • Self-attention transforms protein tokens into representations whose ⟨cls⟩ state integrates intra-protein structural and contextual dependencies.
  • Cross-attention uses protein representations as queries and RNA embeddings as keys and values, enabling RNA-guided modulation of protein representations.
  • Each protein token is enriched with transcriptomic context while preserving structural information from the protein encoder.
  • The final multimodal cell representation is extracted from the fusion block’s ⟨cls⟩ hidden state.It encodes intra-protein structural organization, transcriptomic context, and global cellular state.

Loss Function

The model is pretrained with masked gene expression modeling and cross-modal protein-expression prediction, combining their losses into a composite objective. These objectives learn gene-regulatory structure and transcript–protein relationships.

  • The composite pretraining objective combines masked gene expression modeling with transcript-conditioned protein-expression prediction.
  • Masked gene expression modeling randomly masks genes and reconstructs their observed expression values.
  • Optimizing masked expression reconstruction encourages learning gene–gene co-expression structure and latent regulatory dependencies from incomplete observations.
  • The cross-modal prediction objective estimates protein expression from RNA-derived representations using cross-attention-based fusion and a regression head.
  • Cross-modal prediction enforces consistency between transcriptomic and proteomic representations.
  • The final training objective is a weighted combination of the three losses, with λ1 = 0.6 and λ2 = 0.4 controlling two component contributions.

Results

The multimodal framework resolved immune cell identities and subpopulations while improving classification, batch integration, and preservation of biological structure across heterogeneous datasets.

  • Cell type identification: UMAP embeddings preserved global immune organization and resolved closely related T cell states, including CD4 and CD8 naive, TCM, and TEM populations.The results indicate that transcriptomic and protein-level information captured subtle phenotypic differences beyond transcriptomic information alone.
  • Cell type identification: 0.962 accuracy, 0.961 precision, 0.937 recall, and 0.947 macro-F1 were achieved on the 10x Multiome PBMC dataset.The method improved consistently across most metrics compared with scGPT.
  • Cell type identification: 0.922 accuracy, 0.898 precision, 0.920 recall, and 0.905 macro-F1 were achieved on the multi-tissue immune cell benchmark, outperforming Seurat.The reported recall and macro-F1 improvements indicate better generalization across heterogeneous immune populations.
  • Overall interpretation: Cross-modal attention integrating RNA, protein, and subcellular information enabled accurate cell-type identification and robust transferability across diverse single-cell datasets.Protein features and subcellular constraints supplied complementary information for closely related populations, including improved CD4+ TCM identification and reduced cDC2 confusion.
  • Subcellular ablation: Adding subcellular embeddings increased 10x Multiome PBMC accuracy from 0.92 to 0.94 and macro-F1 from 0.90 to 0.92.Similar improvements were observed on the multi-tissue immune cell dataset, despite the subcellular representation having only seven dimensions.
  • Batch integration: On the TEA-seq, ECCITE-seq, and CITE-seq benchmark, the method achieved 0.567 AvgBIO and 0.728 Overall scores, with improved batch correction and clustering consistency versus scGPT.UMAP visualization showed aligned batches while retaining major immune identities and fine-grained heterogeneity.

Discussion

The framework combines transcriptomic, protein structural, and subcellular localization information through cross-attention to produce compact, biologically informed cell representations. The authors identify limitations in dynamic protein behavior, coarse localization priors, and static cell snapshots, while proposing dynamic and spatial extensions.

  • Discussion: The framework jointly integrates transcriptomic and proteomic information, explicitly adding protein structure and subcellular spatial context to characterize cellular states.This design extends beyond representations based primarily on gene expression profiles.
  • Discussion: Cross-attention aligns RNA and protein modalities, using FoldExplorer structure-aware protein embeddings, UniProt localization priors, and an scGPT-inspired RNA Transformer.A CLS-token fusion strategy produces a compact cell representation encoding structural, transcriptomic, and spatial information.
  • Discussion: A composite objective combining masked gene modeling and cross-modal prediction supports intra-modal structure learning and inter-modal alignment.
  • Limitations: The model relies on existing protein annotations and structure-derived embeddings that may not fully capture dynamic conformational changes or context-specific protein behavior.This limits how completely the representation reflects changing protein states.
  • Limitations: Subcellular localization is represented as a coarse-grained prior, and the framework models static cell snapshots rather than temporal differentiation or perturbation responses.Future directions include dynamic protein structures, spatially resolved omics, perturbation-aware modeling, and larger multimodal datasets.

DECLARATION OF INTERESTS

The manuscript declares no competing interests and states that GPT-5 assisted only with language editing.

  • DECLARATION OF INTERESTS: The authors declare no competing interests.
  • DECLARATION OF INTERESTS: GPT-5 was used solely for language editing, while the authors developed, verified, and remain responsible for the scientific content and conclusions.

MAIN FIGURE TITLES AND LEGENDS

The framework combines RNA, protein sequence and structure, and subcellular localization through cross-modal fusion to produce cell embeddings. Its evaluations examine immune-cell identification, subcellular ablation effects, and batch integration using UMAP visualizations and classification comparisons.

  • Figure 1: Framework overview: Protein sequences and annotations are encoded with structure-aware representations, while seven-dimensional localization vectors capture major cellular compartments.The compartments include cytoplasm, nucleus, plasma membrane, mitochondria, endoplasmic reticulum, Golgi apparatus, and endosome.
  • Figure 1: Framework overview: RNA expression is processed with a masked-attention Transformer, and RNA embeddings guide protein representation learning through cross-attention.The fused representations are aggregated with a CLS token into a unified multimodal cell embedding.
  • Figure 2: RNA–protein fusion: The RNA–protein fusion model preserves major PBMC populations and supports classification of closely related immune cell subtypes.The confusion matrix and UMAP visualizations compare annotated identities with predictions from the learned embeddings.
  • Figure 3: Subcellular-scale embedding: Subcellular-enhanced embeddings improve cell-type annotation across evaluation metrics despite using only seven dimensions.The comparison covers the 10x Multiome PBMC and multi-tissue immune-cell benchmarks, using accuracy, precision, recall, and macro-F1.
  • Figure 3: Subcellular-scale embedding: Subcellular localization improves discrimination of CD4+ TCM cells and reduces confusion with closely related cDC2 populations.The learned representation separates major immune populations while preserving relationships among related cell states.
  • Figure 4: Integration visualization: Healthy PBMC integration is assessed by donor-label mixing and preservation of coherent cell-type separation across the proposed method, scGPT, Seurat, and Harmony.The figure’s two UMAP views color the same cells by batch and reference cell-type annotations, respectively.
Loading 2609.02344v1…