PaperlayerView exact revision on arXiv

Source-linked AI summary

Language as the Interface: Foundation-Model Contrastive Learning Links Transcriptomes and Electrophysiology

Junbo Shen, Jinying Gao, Bo Lei

arXiv:2609.37024v1cs.AI

TL;DR

Paired Patch-seq data offer a basis for aligning molecular and electrophysiological measurements, but such alignment remains comparatively limited in pretrained frameworks. LANGPATCH uses one frozen language interface to align the modalities, and reports strong prediction, alignment, and transfer results across cohorts.

  • Problem

    Pretrained frameworks have comparatively limited joint modeling of transcriptomic and functional data, despite paired Patch-seq measurements that can support cross-modal learning.

    Source: §1 · pp. 1–2

  • Method

    LANGPATCH embeds gene descriptions and verbalized electrophysiology with one frozen text encoder, then uses a cohort-conditioned adapter and paired contrastive learning to align them.

    Source: §1 · p. 2

  • Results

    Across cohorts, LANGPATCH leads forward prediction; mouse FOSCTTM is 0.107/0.135 vs. 0.208/0.222 for JAMIE, and transfer extends across cortical areas and species.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Cross-modal maps can impute physiology for atlas cells never patched and assign type, layer, and marker expression to recorded cells without sequencing.

    Source: §1 · p. 2

    Evidence covers two mouse cortical areas and one primary human cohort; transfer mainly preserves relative variation, and new datasets require an API for the text embedder.

    Source: §6 · p. 9

Abstract

from arXiv · show

Integrating transcriptomic and electrophysiological data is essential for building multimodal foundation models for neuroscience. Patch-seq provides paired measurements of gene expression and intrinsic electrophysiology from the same neuron, establishing a basis for training cross-modal models. Here we introduce LangPatch, a foundation-model-based contrastive learning framework that uses paired Patch-seq data to align pretrained GenePT representations with electrophysiological phenotypes through a language-based interface. Gene descriptions and verbalized electrophysiological profiles are embedded by the same frozen text encoder. A context adapter and projection modules connect the modalities through paired contrastive learning. Across mouse visual, mouse motor, and human cortical cohorts, LangPatch achieves the highest mean transcriptome-to-electrophysiology prediction correlation among the evaluated foundation-model and representation-learning methods. It also improves held-out cross-modal alignment in the two mouse cohorts (FOSCTTM 0.107/0.135 vs. 0.208/0.222 for JAMIE, an existing cross-modal Patch-seq imputation method). It predicts transcriptomic family, type, cortical layer, and marker-gene expression from electrophysiology, exceeding other baselines on most endpoints. More importantly, the method transfers across brain areas and species: a model trained on mouse visual cortex predicts electrophysiology in motor cortex with approximately 70% correlation retention and in human cortex with 47% (58% on acute-slice recordings). Together, these results demonstrate alignment between molecular and functional representations of neurons, providing a building block for multimodal foundation models in neuroscience.

1 INTRODUCTION

Patch-seq pairs molecular and electrophysiological measurements, but their alignment within pretrained frameworks remains comparatively limited. LANGPATCH uses a shared frozen language interface and paired contrastive learning, with strong prediction and cross-area and species transfer results.

1 INTRODUCTION

  • Paired Patch-seq measurements support cross-modal learning and evaluation, while pretrained-framework alignment between transcriptomic and functional data remains comparatively limited.The central question is how to use paired measurements to align pretrained molecular representations with electrophysiological phenotypes.Source: §1 · pp. 1–2
  • Forward maps can impute physiology for atlas cells never patched, while reverse maps can assign type, layer, and marker expression to recorded cells without sequencing.The paper presents these as uses of cross-modal maps beyond the cohort that generated them.Source: §1 · p. 2
  • LANGPATCH embeds gene descriptions and verbalized electrophysiology with one frozen text embedder, adapting the gene prior with cohort statistics and aligning paired cells contrastively.The 0.4M-parameter adapter and affine projections are trained without updating embedder weights.Source: §1 · p. 2
  • FOSCTTM 0.107/0.135 vs. 0.208/0.222 for JAMIE; LANGPATCH also leads forward prediction and reverse family and marker readouts among evaluated methods.These comparisons span two mouse cohorts and a human cortex cohort.Source: §1 · p. 2
  • From mouse to human cortex, 18 of 25 electrophysiological features transfer better than chance, retaining 47% of in-domain correlation overall and 58% on acute slices.Across mouse cortical areas, the method retains 70%/71% of its in-domain correlation.Source: §1 · pp. 2–3

2 RELATED WORK

Prior work has related Patch-seq modalities, represented genes and cells through language, and pretrained single-cell models. LANGPATCH combines a frozen language coordinate system with paired cross-modal alignment.

2 RELATED WORK

  • Patch-seq methods include statistical associations, coupled autoencoders, manifold regularization, and joint-latent imputation; LANGPATCH focuses on reusing pretrained gene representations.JAMIE (Cohen Kalafut et al., 2023) is identified as the strongest published comparison.Source: §2 · p. 3
  • GenePT represents cells by expression-weighted gene-description embeddings, while CellWhisperer aligns transcriptomes with text annotations; neither uses electrophysiology.TabLLM (Hegselmann et al., 2023) serializes numeric rows as text, a precedent for verbalized records.Source: §2 · p. 3
  • Single-cell foundation models update pretrained weights, whereas LANGPATCH adapts a frozen language embedder’s prior rather than a domain model’s weights.Evaluations cited in the paper report embeddings no better than principal components or linear baselines (Bendidi et al., 2024).Source: §2 · p. 3
  • Unlike scCLIP and scPairing, LANGPATCH joins its modality towers through linear maps over one shared frozen embedder.Its adapter is conditioned on cohort statistics, and the framework evaluates both prediction directions and transfer across areas and species.Source: §2 · p. 3

3 METHOD

LANGPATCH expresses genes and electrophysiology in a shared frozen text-embedding space, then aligns paired cells with lightweight trained components. Its representation supports prediction in both directions and transfer to new cohorts.

3 METHOD

  • Both modalities use the same frozen general-purpose text embedder; only a cohort-statistics context adapter and two affine projections are trained.Gene descriptions enter through GenePT, and electrophysiology enters through deterministic verbalization.Source: §3 · p. 4, Figure 1 · p. 4

3.1 GENE SIDE: A FROZEN LANGUAGE PRIOR, ADAPTED BY DATASET STATISTICS

  • The gene-side adapter uses a bounded gated residual on frozen gene embeddings, and each cell is the expression-weighted mean of adapted genes followed by a linear projection.Because the cell representation is linear in expression, the method can attribute predictions to genes.Source: §3.1 · pp. 4–5

3.2 ELECTROPHYSIOLOGY SIDE: VERBALIZE, THEN EMBED

  • Electrophysiology feature vectors become short verbalized records embedded by the same frozen encoder, making the representation vocabulary-agnostic in principle.The records include readable feature names, units, values, and within-cohort quintile labels.Source: §3.2 · p. 5

3.3 ALIGNMENT

  • Paired cells train the adapter and projections jointly with symmetric InfoNCE and auxiliary regularizers, while the text embedder remains frozen.Auxiliary regression of standardized electrophysiological features anchors the shared space to measurable quantities.Source: §3.3 · p. 5

3.4 READOUTS

  • Feature-specific regressors predict electrophysiology from transcriptomic embeddings, while a multitask head reads transcriptomic labels and marker expression from electrophysiology.Transfer uses the adapted gene matrix, projections, and heads unchanged on new cohorts; feature correspondences are needed only for scoring.Source: §3.4 · p. 5

4 EXPERIMENTAL SETUP

The experiments benchmark three Patch-seq cohorts with held-out prediction, alignment, and transfer evaluations. Cohort-stratified folds and task-specific metrics define the comparisons.

Cohorts

  • The in-domain cohorts comprise 3,654 mouse visual, 1,208 mouse motor, and 612 human cortical cells, with 39, 29, and 75 electrophysiology features, respectively.The zero-shot human target includes 704 cells with electrophysiology, including 247 acute-slice cells.Source: §4 · pp. 5–6

Protocol

  • Each cohort uses five type-stratified train/validation/test folds, with means over five held-out test folds and paired tests across features or folds.The protocol notes that folds are partitions, not replicates; scalar-endpoint tests across five folds have minimum attainable p = 0.06.Source: §4 · p. 6

Baselines

  • Forward baselines include frozen GenePT, Geneformer, UCE, and Nicheformer embeddings, while joint-space and reverse-readout tasks compare additional alignment and supervised methods.JAMIE is re-run on the same folds; reverse baselines include direct electrophysiology classifiers and regressors.Source: §4 · p. 6

Transfer scoring

  • Transfer is scored on 11 matched mouse concepts and 25 human concepts; retention is target mean |r| divided by in-domain mean |r| on the same concepts.Concepts pass when their expected-sign correlation exceeds a permutation null; absolute scale is assessed after affine calibration on paired target cells.Source: §4 · p. 6

Evaluation measures

  • Figure 2 reports reverse family readout, held-out alignment, family label transfer, and training-time and peak-memory comparisons for one fold.FOSCTTM is lower-is-better, with 0.5 as chance.Source: Figure 2 · p. 7

5 RESULTS

LangPatch leads learned representations in forward transcriptome-to-electrophysiology prediction and improves held-out shared-space alignment and reverse readouts. It also transfers from mouse visual cortex to motor cortex and human cortex without target training.

5.1 FORWARD PREDICTION

  • 0.522±0.018 (visual), 0.520±0.021 (motor), and 0.414±0.030 (human) are LangPatch’s mean per-feature r values, highest among learned representations on each cohort.It also has the highest AUROC and lowest raw MAE on all three cohorts; Table 1 reports held-out prediction across five folds.Source: §5.1 · p. 7

5.2 SHARED SPACE: ALIGNMENT AND REVERSE READOUTS

  • FOSCTTM is 0.107 ± 0.004 and 0.135 ± 0.010 on visual and motor cohorts, versus 0.208/0.222 for JAMIE; family label transfer is 0.926/0.848.Reverse family macro-F1 is 0.779/0.710, and marker-gene r is 0.539/0.572; the method exceeds direct electrophysiology heads on 4/5 visual and 5/5 motor endpoints.Source: §5.2 · p. 7

5.3 TRANSFER ACROSS CORTICAL AREAS AND SPECIES

  • 70% (visual→motor) and 71% (motor→visual) of in-domain correlation are retained, with all 11 matched cross-area concepts above permutation null in both directions.About 20 paired target cells suffice for affine calibration to approach the error of calibration on all target cells.Source: §5.3 · p. 8
  • 47% of mouse in-domain correlation is retained on human cortex without human training, rising to 58% on acute-slice cells.The model passes the null on 18/25 human concepts overall and 20/25 acute-slice concepts; after calibration, 56% and 60% beat a constant predictor, respectively.Source: §5.3 · p. 8

6 DISCUSSION

The shared space organizes transcriptomic and electrophysiological embeddings by family, while gene attributions indicate which genes the predictors rely on. The evidence is limited in cohort and feature scope, and the authors describe the framework as a building block for multimodal neuroscience models.

6 DISCUSSION

  • The shared space has family-organized transcriptomic and electrophysiological embeddings, and gene attributions recover cell-type markers while identifying predictor-relevant genes.The attribution analyses are hypothesis-generating; two readouts favor the Plain GenePT MLP.Source: §6 · p. 9
  • Evidence covers two mouse cortical areas and one primary human cohort; only the motor cohort contains excitatory neurons, and recording-sensitive features such as threshold voltage transfer poorly even after calibration.Zero-shot transfer primarily preserves relative variation, and a small number of paired target cells may still be needed for calibration.Source: §6 · p. 9
  • A lightweight contrastive framework aligns a pretrained gene-language prior with verbalized electrophysiology, with reported prediction, alignment, reverse-readout, and attribution results across three cohorts.The authors present this representation as a building block for multimodal foundation models in neuroscience.Source: §6 · p. 10

A HYPERPARAMETERS, PARAMETER COUNTS AND TRAINING DETAILS

The appendix specifies cohort definitions, training settings for both modeling stages, baseline configurations, and the deterministic template used to verbalize electrophysiology.

A HYPERPARAMETERS, PARAMETER COUNTS AND TRAINING DETAILS

  • Stage 1 computes fold-specific expression context and trains the alignment model with full-batch AdamW; nonlinear projectors did not improve validation InfoNCE in preliminary runs.Only the electrophysiology auxiliary regression is active during the first 200 epochs, after which contrastive and cell-tower auxiliary losses are ramped in.Source: A HYPERPARAMETERS, PARAMETER COUNTS… · p. 16
  • Stage 2 trains a separate MLP for each electrophysiological feature, selecting the epoch by validation raw-scale MSE and the head width by cohort-specific validation.The selected width is 512–128 for both mouse cohorts and 256 for human cortex; transfer and attribution analyses instead use the default 2048–512 heads.Source: A HYPERPARAMETERS, PARAMETER COUNTS… · p. 16
  • Human forward heads use nested validation because the mouse configuration overfits with 397 training and 92 validation cells per fold.The selected 256-unit head scores 0.414 ± 0.030 on test folds.Source: A HYPERPARAMETERS, PARAMETER COUNTS… · p. 16
  • Baseline configurations vary by model, and pretrained single-cell models receive CPM-normalized human expression instead of their pretraining raw-count input.The fine-tuned Nicheformer baseline visibly overfits; the GenePT–ephys VAE does not train on motor or human cortex under its mouse configuration.Source: A HYPERPARAMETERS, PARAMETER COUNTS… · p. 16

B VERBALIZATION OF ELECTROPHYSIOLOGY

  • Electrophysiology is rendered deterministically as named, unit-bearing feature values with within-cohort quintile labels and four summary profile tokens, then embedded with text-embedding-3-large.Human records use the same template with a human feature-name map, requiring no mouse–human feature correspondence for embedding.Source: B VERBALIZATION OF ELECTROPHYSIOLOGY · p. 17

C INTEGRATION METRICS AND PROTOCOL

The section defines held-out integration metrics and protocols, compares frozen input representations and the full adapter, and reports aligner cost.

C INTEGRATION METRICS AND PROTOCOL

  • Held-out FOSCTTM measures the fraction of opposite-modality cells closer than the true match, while label-transfer accuracy uses a five-nearest-neighbor classifier across modalities.Values are aggregated as mean ± s.d. over five test folds; each method is scored on its own embedding pair.Source: C INTEGRATION METRICS AND PROTOCOL · pp. 17–18
  • JAMIE's held-out FOSCTTM is 0.208±0.006 on visual cortex and 0.222±0.013 on motor cortex; its released notebook's transductive 0.002 is not comparable.The notebook fits and transforms all cells jointly with full correspondence available.Source: C INTEGRATION METRICS AND PROTOCOL · p. 17
  • FOSCTTM is 0.107/0.135 for the full method, versus 0.133/0.186 for Plain GenePT without the adapter and 0.187/0.235 for Nicheformer embeddings.The comparison supports that the substrate matters and the adapter adds alignment on top of the text prior.Source: C INTEGRATION METRICS AND PROTOCOL · pp. 17–18, Table 4 · p. 18
  • Stage 1 trains in 63 s with 1,430 MB on one visual-cortex fold, about three times faster than JAMIE but using about four times its GPU memory.The comparison uses one NVIDIA H100 and excludes data loading and one-off embedding.Source: C INTEGRATION METRICS AND PROTOCOL · p. 18

D REVERSE READOUT DETAILS

The reverse readout maps electrophysiology into an aligned transcriptomic representation and uses a multitask head to predict transcriptomic labels and expression targets.

D REVERSE READOUT DETAILS

  • The readout targets family, fine type, cortical layer, marker-gene expression, and marker programs; absent panel markers are skipped.The evaluated label sets include 6/9 family classes, 60/76 fine types, and 6/4 layer classes for visual/motor cortex, respectively.Source: D REVERSE READOUT DETAILS · pp. 18–19
  • A feature-vector regressor predicts the aligned transcriptomic embedding, whose leading principal components are concatenated with electrophysiology for multitask prediction.The final head predicts family, type, layer, markers, and programs without updating stage-1 parameters.Source: D REVERSE READOUT DETAILS · p. 19
  • The canonical readout exceeds the per-task supervised control on 9 of 10 endpoints and is 1.4–2.1× JAMIE on family F1.It exceeds the supervised control on 4/5 visual and 5/5 motor endpoints.Source: D REVERSE READOUT DETAILS · p. 19, Table 5 · p. 19

E CROSSWALKS, PERMUTATION NULLS AND CALIBRATION PROTOCOL

The transfer protocol fixes biophysical crosswalks before scoring, evaluates correlation against a permutation null, and tests whether limited paired-cell calibration reduces errors.

Crosswalks

  • The visual↔motor crosswalk fixes 11 matched biophysical concepts and expected signs before any cross-cohort score; sag, AHP, and adaptation are sign-uncertain.The mouse-to-human crosswalk also freezes matching rules before scoring, excludes features without counterparts, and covers 25 Lee concepts and 26 Chartrand concepts.Source: E CROSSWALKS, PERMUTATION NULLS AND… · pp. 19–20

Scoring, null and retention

  • Transfer passes the null when signed Pearson r exceeds the 95th percentile of 1,000 cell-permutation nulls with the preregistered sign.Retention is target |r| divided by an in-domain reference; verdict A requires R ≥0.5 and at least two thirds of concepts to pass.Source: E CROSSWALKS, PERMUTATION NULLS AND… · p. 20

Few-shot calibration

  • Few-shot calibration fits an affine map on 5–100 sampled target cells and evaluates median absolute error over 200 resamples.On Lee, 20-cell calibration met the error-ratio criterion (1.025) but only 56% of concepts beat the MAD; acute-slice calibration met both criteria.Source: E CROSSWALKS, PERMUTATION NULLS AND… · p. 20

F HUMAN IN-DOMAIN BENCHMARK (LEE 2023)

On the human Lee cohort, the selected LangPatch representation leads the compared methods in mean feature correlation, while mouse-frozen representations are strongest with at most 50 labeled human cells.

Design

  • The benchmark uses 612 complete-case cells and 75 features from a 704-cell cohort, with subclass-stratified five-fold splits.Nested validation selects the human forward head because the 92-cell validation fold is too small to select both per-feature epochs and the configuration.Source: F HUMAN IN-DOMAIN BENCHMARK (LEE 2023) · p. 20

Results

  • 0.414±0.030 mean r for the proposed representation versus 0.406±0.034 for JAMIE; 43/75 features were better in r (p = 0.25), and 50/75 in AUROC (p = 5 × 10−3).The margin over JAMIE is not significant in r, so this is not a claim of human state of the art.Source: F HUMAN IN-DOMAIN BENCHMARK (LEE 2023) · p. 20, Table 7 · p. 21

Low-resource curve

  • With at most 50 labeled human cells, the mouse-frozen representation is the best representation for a ridge head.The mouse stage 1 model had not seen human electrophysiology before head training.Source: F HUMAN IN-DOMAIN BENCHMARK (LEE 2023) · p. 20

Nested-validation head selection

  • The 256-unit head used as “Ours” is selected by validation performance among four configurations; test scores are reported for completeness.The table also reports Pearson r, AUROC, scaled MSE, raw MAE, and comparisons with Plain GenePT MLP.Source: F HUMAN IN-DOMAIN BENCHMARK (LEE 2023) · p. 21

G ATTRIBUTION

The attribution analysis compares gene-level explanations across models using biological-plausibility, stability, and perturbation readouts, while treating attribution as hypothesis-generating rather than causal.

Method

  • Gene attributions use gradient×input difference from the training-fold mean expression profile, with integrated gradients as corroboration.The analysis compares the proposed model, Plain GenePT MLP, and a reproduced JAMIE gene-to-electrophysiology path on the same held-out cells and folds.Source: G ATTRIBUTION · p. 21

Readouts

  • Readouts include marker recovery, gene-set enrichment, ranking stability, gene deletion and keep-only tests, and changes in joint-space family-transfer accuracy.Single-gene occlusion is negligible at roughly 1/1,300 of the pooled cell vector, so set-scale perturbations are used.Source: G ATTRIBUTION · pp. 21–22, Table 9 · p. 22

Interpretation

  • Attribution describes trained-model reliance and is hypothesis-generating, not causal.The proposed model’s joint space depends on its top genes, but the analysis does not establish that any gene causes an electrophysiological feature.Source: G ATTRIBUTION · pp. 22–23

Reading

  • The proposed model recovers markers faster and is more enriched for markers and ion-channel genes on motor cortex, while tying Plain GenePT MLP on visual-cortex marker recovery.JAMIE’s marker recovery is at or below random-ranking level, and its top-50 sets overlap by only 0.11–0.13 across folds.Source: G ATTRIBUTION · p. 23

H FULL TRANSFER TABLES

The transfer tables report cross-area and cross-species prediction correlations, null-passing concepts, and retention, with performance varying across target cohorts and cell compositions.

Mouse visual↔motor

  • 70% and 71% correlation retention across mouse visual↔motor cortex, with all 11 concepts above the permutation null in both directions.Resting potential transfers worst at 17–35% retention; adaptation has opposite sign because the two pipelines orient its index differently.Source: Table 10 · p. 23, H FULL TRANSFER TABLES · pp. 23–24

Mouse→human

  • On the full Lee human cohort, transfer reaches mean r 0.282 with 18/25 concepts above null and R = 0.47; acute slices reach 0.335, 20/25, and R = 0.58.Chartrand’s L1 cells score R = 0.17, and the paper attributes the collapse to cell composition rather than the crosswalk; removing subclass means lowers Lee mean r from 0.30 to 0.17.Source: Table 10 · p. 23, H FULL TRANSFER TABLES · p. 24

Scoring and verdicts

  • Retention is mean target |r| divided by the mouse in-domain held-out mean |r| over the same concepts; verdict A requires R ≥0.5 and at least two thirds above null.The visual-model score on cells it was trained on is an upper reference, not a transfer score.Source: H FULL TRANSFER TABLES · p. 23
Loading 2609.37024v1…