PaperlayerView exact revision on arXiv

Source-linked AI summary

SpaFactor: Lightweight Spatial Context-Aware Gene Program Modeling for Histology-to-Transcriptomics Inference

Shiting Ruan, Xitong Ling, Qiming He, Ziyou Yan, Huaitian Yuan, Tian Guan, Ying Xiao, Xu Guan, Yonghong He

arXiv:2609.28563v1cs.LGq-bio.QM

TL;DR

H&E-to-ST prediction could provide a scalable alternative to spatial transcriptomics, but compact models must handle tissue context and coordinated gene outputs. SpaFactor combines neighborhood context with shared gene-program decoding and outperforms strong comparators across its reported cohorts, with gains particularly evident for spatially variable genes.

  • Problem

    H&E-to-ST methods must capture tissue context and coordinated gene outputs, while richer spatial models can add optimization and deployment costs.

    Source: Introduction · p. 1

  • Method

    SpaFactor combines central-spot morphology with local and regional context, maps it through a residual MLP to latent gene programs, and decodes expression through shared gene loadings.

    Source: Introduction · p. 2, Factor Decoder · p. 3

  • Results

    SpaFactor outperforms the strongest published comparator on all five primary-benchmark metrics, including 36.36% Gene PCC, and SpaFactor or SpaFactor-Cal exceeds selected prior models on 18 of 20 external-cohort metrics.

    Source: Main Results · p. 4

  • Takeaways & Limitations

    The reported validation indicates that the gains extend to biologically organized spatial function, including biomarker maps and held-out pathway activities.

    Source: Introduction · p. 2

Abstract

from arXiv · show

Spatial transcriptomics (ST) profiles gene expression within tissue architecture, but its cost and experimental complexity limit routine use. Predicting spatial expression from routinely available hematoxylin and eosin (HE) images therefore offers a scalable alternative. However, conventional methods often fit high-dimensional gene outputs as independent targets, overlooking the biological coordination among genes while remaining vulnerable to high-dimensional noise and overfitting. Existing attempts to address this limitation often rely on computationally heavy graph networks or complex auxiliary supervision. We therefore introduce SpaFactor, a lightweight and efficient low-rank morphology-program-gene factorization framework. At the input, SpaFactor efficiently fuses the visual representation of the central spot with multiscale local and regional neighborhood context, yielding a histologic representation that captures cellular morphology and microenvironmental heterogeneity. For modeling, a residual MLP stably learns a nonlinear mapping from the tissue microenvironment to low-dimensional latent gene programs. These activities are decoded through shared gene loadings into coordinated multi-gene expression predictions. Across five public cohorts, SpaFactor achieves the best aggregate performance, with particularly clear improvements for spatially variable genes, and more faithfully recovers biologically organized spatial patterns. These results demonstrate that lightweight joint modeling of tissue context and gene programs can improve both predictive accuracy and biological fidelity.

Introduction

Spatial transcriptomics provides spatially localized molecular profiles, but cost and experimental complexity limit routine adoption. SpaFactor combines tissue context with shared gene-program modeling in a lightweight predictor.

Introduction

  • Spatial transcriptomics localizes gene expression to tissue regions and neighborhoods, but assay cost, processing complexity, sequencing burden, and platform-specific workflows constrain its broader adoption.H&E images offer a routinely collected complementary source for mapping expression in retrospective cohorts and samples without matched spatial transcriptomics.Source: Introduction · p. 1
  • H&E-to-ST modeling must address both tissue context and coordinated gene outputs while keeping the downstream predictor compact.The paper frames this as connecting spatial tissue evidence to gene programs after fixing a strong pathology representation.Source: Introduction · pp. 1–2
  • SpaFactor concatenates central-spot morphology with parameter-free local and regional summaries, then maps the combined representation to latent programs and shared gene loadings.The method uses a four-block residual MLP, with cached embeddings and neighborhood summaries enabling batched dense operations.Source: Introduction · p. 2
  • The paper’s contributions combine lightweight spatial aggregation, factorized gene prediction, and validation of accuracy, efficiency, and biological organization across five cohorts.The cohorts total 421 slides and 799,085 spots; validation includes leakage-controlled evaluation, matched-feature controls, ablations, efficiency analysis, biomarker maps, and held-out pathway activities.Source: Introduction · p. 2

Related Work

Prior H&E-to-expression work has expanded from patch- or slide-level regression toward richer spatial interactions and image–expression alignment. SpaFactor fixes the pathology representation and uses compact structures for both spatial context and gene outputs.

Histology-to-Expression Prediction

  • Earlier H&E-to-expression studies span patch- and slide-level regression, inter-spot transformers and graph relations, contrastive alignment, and cross-modal generation; SpaFactor fixes the foundation representation to isolate downstream task structures.The passage notes that backbone choice, spatial fusion, and downstream predictors often vary together across studies.Source: Histology-to-Expression Prediction · p. 2

Spatial Context and Structured Gene Modeling

  • Prior methods enrich spatial context or gene dependencies, often using attention-heavy interaction, external gene semantics, pathway supervision, or expression foundation models; SpaFactor instead uses parameter-free context and a data-learned factor space.A matched direct GigaPath-MLP baseline and exact-configuration ablations are used to isolate the value of the two structures.Source: Spatial Context and Structured Gene… · p. 2

Method

SpaFactor predicts spot-level gene panels from H&E images and coordinates, using same-slide neighborhood summaries and a residual encoder to infer latent programs. Shared gene loadings produce coordinated outputs, while optional training-time calibration leaves the inference path H&E-only.

Problem Formulation: H&E-to-ST Prediction

  • Given paired histology and spatial transcriptomics during training, SpaFactor predicts a selected gene panel at each spot; inference requires only H&E images and spot coordinates.Gene-panel construction, normalization, HVG ranking, and checkpoint selection are performed within the training fold.Source: Problem Formulation: H&E-to-ST… · p. 2

Frozen Histology Feature Extraction

  • A frozen GigaPath embedding of the central spot is concatenated with mean embeddings from local and regional neighborhoods before the residual MLP predicts program activities.Figure 1 specifies local k = 4 and regional k = 16 neighborhoods; the embedding and neighborhood summaries are precomputed.Source: Figure 1 · p. 3

Spatial Context Aggregation

  • Coordinates select neighbors only within the same slide, and the context operator uses local and regional mean evidence without central-minus-neighbor contrasts.Validation selected the simpler mean-context design.Source: Spatial Context Aggregation · p. 3

Residual Morphology-to-Program Encoder

  • The context vector is normalized and projected to a 1024-dimensional hidden state, then refined by four residual MLP blocks before a factor head produces 256 activities.The inner width is 2048 and dropout is 0.10.Source: Residual Morphology-to-Program Encoder · p. 3

Factor Decoder

  • A trainable loading matrix and gene bias decode the expression panel, coupling genes through shared spot-level activities rather than gene-specific output weights alone.The shared program space regularizes gene-specific fluctuations unsupported by morphology and provides an interface to functional gene sets; for 2,000 genes, the output module is 62.1% smaller than a direct 1024-to-gene module.Source: Factor Decoder · pp. 3–4

Variance-Aware Correlation-Aligned Learning

  • HVGs ranked from outer-training data receive tiered weights, and a small correlation term aligns spatial variation without replacing pointwise reconstruction.The specified loss is L_SpaFactor = L_WMSE + 0.1L_PCC.Source: Variance-Aware Correlation-Aligned… · p. 4

Optional Training-Time Calibration

  • SpaFactor-Cal adds a gated, out-of-fold calibration loss during training, then removes teachers, anchor genes, and the gate for H&E-only validation and test inference.The calibration loss weight is α = 0.07.Source: Optional Training-Time Calibration · p. 4

Experiments

SpaFactor is evaluated across five public cohorts using matched slide-level fivefold splits and comparisons with published methods. It outperforms strong prior models, with gains in spatially variable gene recovery and evidence that regional context contributes most to the contextual improvement.

Experimental Setup

  • Evaluation covers five public cohorts totaling 421 sections and 799,085 spots, using matched slide-level fivefold splits and training-fold-only preprocessing and checkpoint selection.The primary cohort has 108 breast sections; external evaluation spans breast, bowel, brain, and skin cohorts.Source: Experimental Setup · p. 4

Main Results

  • 18 of 20 displayed metrics across four external cohorts favor SpaFactor or SpaFactor-Cal over the selected strongest prior, with VR-PCC advantages consistent across all four tissues.The framework primarily improves spatially organized expression rather than uniformly shifting every spot-level score.Source: Main Results · p. 4
  • 36.36% Gene PCC and 63.41% VR-PCC@200; SpaFactor outperforms the strongest published comparator on all five primary-benchmark metrics.SpaFactor-Cal remains close to the teacher-free model, so optional calibration is complementary rather than necessary for the overall gain.Source: Main Results · p. 4, Table 1 · p. 5
  • Gains from VR-PCC@50 through VR-PCC@200 indicate broader spatial discrimination rather than an advantage driven by a few favorable markers.Gene PCC and VR-PCC assess each gene’s spatial ordering across spots, unlike Spot PCC, which emphasizes agreement among genes within each spot.Source: Main Results · pp. 4–5
  • Against matched GigaPath-MLP, SpaFactor wins 20–23 of 25 dataset–fold comparisons per metric, with a 5.61% relative gain for VR-PCC@50 and paired-bootstrap intervals excluding zero.The advantage isolates downstream spatial-context and gene-program structure; it requires neither graph propagation nor cross-spot attention nor end-to-end foundation-model tuning.Source: Main Results · p. 5

Ablation and Sensitivity Analyses

  • Removing both neighborhood summaries lowers Gene PCC by 2.37 percentage points and VR-PCC by about four points; regional context contributes most, with local context adding complementary detail.Regional context alone recovers most of the full model’s performance.Source: Structural Component Ablations · p. 5

K Spot PCC Gene PCC VR-PCC@50 VR-PCC@100 VR-PCC@200

Sensitivity analyses favor a compact, transferable architecture, while biomarker and pathway evaluations show recovery of biologically organized spatial patterns.

K Spot PCC Gene PCC VR-PCC@50 VR-PCC@100 VR-PCC@200

  • Removing both neighborhood summaries causes the largest interpretable decline: Gene PCC falls by 2.37 percentage points and VR-PCC by about four percentage points.Regional context recovers most full-model performance; local context adds a smaller improvement, while residual skips enable stable optimization and factorized decoding provides targeted refinement.Source: K Spot PCC Gene PCC VR-PCC@50… · p. 6
  • Performance remains stable over K = 64–1024; K = 512 improves the primary cohort by at most 0.33 percentage points over K = 256 but transfers inconsistently across external cohorts.The results support K = 256 as the more transferable accuracy–capacity trade-off.Source: Sensitivity to Latent Factor Capacity · p. 6, Table 3 · p. 6
  • Residual depth has a nonmonotonic effect, with a broad high-performing region around L = 8–12 and no consistent metric improvement beyond it.L = 10 and L = 12 have 2.13× and 2.51× as many trainable parameters as L = 4, with up to 38% longer training; L = 4 is retained as the lightweight configuration.Source: Figure 3 · p. 6, Residual Depth and Efficiency · p. 6, Table 4 · p. 6
  • Across λPCC = 0–0.20, the selected 0.10 weight improves reported gene-level measures by at most 0.16 percentage points over removing the term.The sensitivity analyses identify architecture as more consequential than capacity or loss tuning, with K = 256 and L = 4 selected for transferability and computational economy.Source: Table 5 · p. 6, Training Objective Sensitivity · p. 6

Biomarker and Functional Pathway Recovery

Conclusion

SpaFactor is a compact downstream model for connecting spatial tissue context with coordinated gene programs on frozen pathology representations. Across five public cohorts, it improves H&E-to-ST prediction, particularly for spatially variable genes, and recovers biologically organized patterns.

Conclusion

  • SpaFactor combines central morphology with local and regional context, then decodes latent programs through shared gene loadings.The model connects spatial tissue context with coordinated gene programs using frozen pathology representations.Source: Conclusion · p. 8
  • Across five public cohorts, SpaFactor improves H&E-to-ST prediction, with particularly clear gains for spatially variable genes.The conclusion characterizes the framework as computationally efficient and biologically faithful.Source: Conclusion · p. 8
  • Matched controls and ablations identify spatial context as the primary performance source, while residual mapping and factorized decoding support stable, coordinated prediction.Biomarker and held-out pathway analyses further confirm biologically organized spatial recovery.Source: Conclusion · p. 8

A.1 Cohorts and evaluation splits

The appendix describes five-cohort evaluation with matched slide-level splits, frozen histology features, fold-specific gene processing, and spot- and gene-level correlation metrics.

A.1 Cohorts and evaluation splits

  • The study evaluates five public cohorts, using matched fivefold slide-level splits with held-out test slides and validation sampled from the remaining slides.Each outer fold holds out one fifth of slides; the approximate train/validation/test partition is 70%/10%/20%.Source: §A.1 · p. 10

A.2 Histology preprocessing

  • Registered spot-centered RGB tiles are cropped to 224 × 224 pixels and encoded once per spot by a frozen 1,536-dimensional Prov-GigaPath tile encoder.The same cached representation is used by SpaFactor, SpaFactor-Cal, and the matched GigaPath-MLP.Source: §A.2 · p. 10

A.3 Expression preprocessing

  • Targets are log(1 + c) values, and each fold constructs a gene panel from training slides only, with at most 2,000 genes and a specified mix of HVGs and stable highly expressed genes.Filtering, ranking, and standardization use training-fold expression statistics applied unchanged to validation and test slides.Source: §A.3 · p. 10

A.4 Evaluation metrics

  • Evaluation reports Spot PCC for within-spot profile agreement, Gene PCC for across-spot spatial agreement per gene, and VR-PCC over the K most variable held-out genes.Values are fivefold means and standard deviations on a percentage scale.Source: §A.4 · p. 10

A.5 Matched feature control

The appendix reports paired matched-feature comparisons, describes SpaFactor-Cal's training-only calibration procedure, and measures initialization stability across seeds.

A.5 Matched feature control

  • All five mean differences favored SpaFactor with 95% confidence intervals entirely above zero; the largest gain was VR-PCC@50 (+3.37 percentage points).The matched control shares frozen features, folds, gene panels, loss weights, and evaluation code; win counts across 25 paired comparisons range from 20 to 23 by metric.Source: §A.5 · pp. 10–11, Table A.3 · p. 11

B SpaFactor-Cal

  • SpaFactor-Cal uses out-of-fold teacher predictions and a gated, weighted calibration term during training; teachers, anchors, and the gate are discarded before evaluation.Inference therefore uses the same H&E tiles and spatial coordinates as SpaFactor and requires no expression measurements.Source: B SpaFactor-Cal · p. 11

C.1 Initialization stability

  • Across five seeds, standard deviations were 0.06, 0.26, 0.31, 0.37, and 0.41 percentage points for Spot PCC, Gene PCC, and VR-PCC@50/100/200, respectively; the maximum range was 1.00 percentage points for VR-PCC@200.The split seed was fixed while model-training seeds varied across five folds of the primary cohort.Source: §C.1 · p. 11

C.2 Compute environment

Experiments ran in a specified software and hardware environment, with one GPU per process and cached embeddings and neighborhood summaries reused.

Compute environment

  • Each process used one GPU, and cached tile embeddings and neighborhood summaries were reused.The reported hardware included AMD EPYC 9655 CPUs, 377 GiB RAM, and 32 GB NVIDIA RTX 5090 GPUs.Source: §C.2 · p. 11
  • The software environment used Ubuntu 22.04.2, Python 3.10.20, PyTorch 2.13.0/CUDA 13.0, and NumPy 2.2.6.The reported machine configuration included AMD EPYC 9655 CPUs, 377 GiB RAM, and 32 GB NVIDIA RTX 5090 GPUs.Source: §C.2 · p. 11
  • The compute setup paired AMD EPYC 9655 CPUs and 377 GiB RAM with 32 GB NVIDIA RTX 5090 GPUs.Each process used one GPU; cached tile embeddings and neighborhood summaries were reused.Source: §C.2 · p. 11

D Complete benchmark results

Across five cohorts, the SpaFactor family led every reported VR-PCC comparison and achieved the highest Gene PCC in three cohorts; SpaFactor-Cal also led Spot PCC in two. The appendix reports complete cohort benchmarks, a shared evaluation setup, and initialization stability across five seeds.

D Complete benchmark results

  • All methods used the same folds, training-data-derived gene panels, frozen histology features, source normalization, and evaluation code.Tables D.1–D.5 report complete comparisons for the five formal cohorts.Source: D Complete benchmark results · p. 11
  • Across five cohorts, the SpaFactor family led all 15 VR-PCC comparisons and achieved the highest Gene PCC in three cohorts; SpaFactor-Cal also led Spot PCC in two.The authors describe this as consistent gene-wise recovery with complementary calibration gains.Source: D Complete benchmark results · p. 11, Table D.1 · p. 12, Table D.2 · p. 12
  • Initialization stability is reported across five training seeds using PCC mean ± standard deviation over five folds (%).Table C.1 reports this stability evaluation.Source: Table C.1 · p. 12
Loading 2609.28563v1…