PaperlayerView exact revision on arXiv

Source-linked AI summary

LEAF-Mamba: Feature Adaptation and Fusion for Whole Slide Image Analysis

Lubin Gan, Jing Zhang, Mingjun Ma, Linhao Qu, Dachun Kai, Xiaoyan Sun

arXiv:2610.29746v1cs.CV

TL;DR

Weakly supervised WSI analysis must adapt pretrained patch representations and combine local with long-range context under slide-level supervision. LEAF-Mamba pairs reconstruction-regularized adaptation with parallel Mamba and convolutional streams and input-dependent fusion. It achieves the highest mean diagnostic AUC and survival C-Index among compared methods with each of three frozen encoders.

  • Problem

    Task supervision of frozen-feature adaptation does not explicitly constrain recovery of input patch features, while standard Mamba does not retain its local convolutional output as a separate representation for fusion.

    Source: §1 · p. 2

  • Method

    LEAF-Mamba jointly trains a reconstruction-regularized latent adapter and aggregates adapted tokens through parallel Mamba and convolutional streams with content-dependent gated fusion.

    Source: §1 · p. 3

  • Results

    LEAF-Mamba achieves the highest mean diagnostic AUC and mean survival C-Index among compared methods with each of three frozen encoders.

    Source: §1 · pp. 2–3

  • Takeaways & Limitations

    The component analyses support jointly designing feature adaptation and contextual aggregation for frozen-feature WSI prediction.

    Source: §1 · p. 3, §6 · p. 20

    The study focuses on frozen patch representations; transfer across institutions remains an important next step for characterizing when the benefits generalize.

    Source: §5.2 · pp. 19–20

Abstract

from arXiv · show

Whole slide image (WSI) analysis enables computational diagnosis and prognosis, but gigapixel images and limited annotations make slide-level learning challenging. Multiple instance learning addresses this task by aggregating patch representations from frozen pretrained encoders. Recently, Mamba has enabled efficient modeling of long patch sequences. However, task supervision alone does not explicitly constrain information recovery during feature adaptation, while Mamba couples local processing with long-range propagation rather than retaining separate representations for fusion. Motivated by these challenges, we introduce LEAF-Mamba, a framework that jointly addresses latent evidence adaptation and fusion for frozen-feature WSI analysis. A reconstruction-regularized adapter learns compact, task-specific representations while reconstructing the input features. Parallel Mamba and convolutional streams then capture long-range dependencies and sequence-local interactions, with an input-dependent gate modulating their joint representation before slide-level pooling. Experiments across feature extractors demonstrate improved average performance over strong MIL baselines in both diagnostic classification and survival prediction. Component analyses further support the contributions of adaptation and fusion. These findings highlight the value of jointly designing feature transformation and contextual aggregation to translate pretrained representations into effective slide-level predictions.

1. Introduction

Weakly supervised WSI analysis must aggregate heterogeneous patch evidence from gigapixel slides without dense region annotations. LEAF-Mamba addresses frozen-feature adaptation and contextual fusion, and reports the highest mean diagnostic AUC and survival C-Index among compared methods for each of three encoders.

1. Introduction

  • Weakly supervised WSI models learn from slide- or patient-level labels because dense region-level annotation demands extensive pathologist effort.Slides contain heterogeneous tissue evidence that must be aggregated for diagnostic or prognostic prediction.Source: §1 · p. 1
  • The reconstruction-regularized latent adapter jointly trains compact task-specific codes with slide prediction and patchwise input reconstruction.The decoder is removed at inference.Source: §1 · pp. 2–3
  • Parallel Mamba and sequence-local convolutional representations are projected together and modulated by an input-dependent gate before pooling.This retains separate short- and long-range representations for fusion.Source: §1 · pp. 2–3
  • LEAF-Mamba achieves the highest mean diagnostic AUC and survival C-Index among compared methods with each of three frozen encoders.The evaluated encoders are ResNet-50, PLIP, and CONCH v1.5.Source: §1 · pp. 2–3

2. Related Work

Related work spans slide-level aggregation, long-sequence modeling, frozen-feature adaptation, and combinations of local and global operators. The cited work frames LEAF-Mamba against distinct choices in representation, ordering, and aggregation.

2.1. MIL for whole slide image analysis

  • MIL work ranges from attention and clustering to dual-stream selection, pseudo-bag distillation, probabilistic prediction, spatial order restoration, and feature re-embedding.Jigsaw learns order restoration from spatial correlations, while RRTMIL re-embeds extracted patch features before MIL aggregation.Source: §2.1 · pp. 3–4

2.2. Long-sequence modeling in pathology

  • State-space and Mamba models offer linear-time long-sequence alternatives, while WSI methods study sequence modeling, semantic ordering, and retrieval-guided aggregation.MambaMIL is identified as the closest sequence-side comparator.Source: §2.2 · p. 4

2.3. Pathology foundation models and representation mismatch

  • Pathology foundation models provide reusable patch representations, while downstream adaptation addresses feature mismatch under a frozen-feature protocol.The passage distinguishes feature mismatch from generalization to a new acquisition domain.Source: §2.3 · p. 4

2.4. Adapters and hybrid sequence operators

  • Prior adapter and hybrid-operator work motivates adaptation and local-global combinations, but does not settle the value of a chosen WSI serialization.RRTMIL is the closer protocol-level reference for re-embedding offline patch features before MIL aggregation.Source: §2.4 · p. 4, Figure 2 · p. 5

2.5. Structural information and distributed aggregation

  • Related work on structural information and distributed aggregation considers how local evidence is preserved and how information sources are organized collectively.The cited examples include graph representations and multi-agent communication; the passage connects them conceptually to patch aggregation under slide-level supervision.Source: §2.5 · p. 5

3. Method

LEAF-Mamba transforms frozen patch features, models them with long-range and sequence-local streams, and fuses the resulting representations before slide-level pooling. Its method combines a reconstruction-regularized adapter with gated evidence fusion for weakly supervised WSI prediction.

3.1. Problem Setup

  • The model maps frozen patch features to slide-level predictions, with the patch encoder fixed and the adapter and aggregator trained jointly from slide-level supervision.The setup covers diagnostic labels and survival targets comprising observed time and event indicator.Source: §3 · p. 5, §3.1 · p. 6

3.2. Latent Evidence Adaptation

  • LEA adapts frozen patch features into lower-dimensional codes before aggregation, while an auxiliary reconstruction objective penalizes input-recovery error alongside the task loss.A nonlinear encoder and linear skip projection form the latent representation; a decoder reconstructs the original feature matrix.Source: §3.2 · p. 6

3.3. Global-Local Evidence Modeling

  • The global stream models long-range dependencies, while a separate convolutional stream captures sequence-local interactions in the coordinate-ordered tokens.For coordinate-bearing bags, raster serialization sorts by y and then x, and the default convolution spans seven consecutive tokens.Source: §3.3 · pp. 6–7

3.4. Content-Dependent Evidence Fusion

  • Content-dependent fusion concatenates the two streams and uses a sigmoid gate to modulate their projected joint representation before pooling.A residual feed-forward update refines the representation, and gated attention pools the final normalized tokens into a bag representation for the task head.Source: §3.4 · pp. 7–8

4. Experiments

Experiments evaluate LEAF-Mamba under a shared frozen-feature protocol across diagnostic classification and survival prediction, then test its performance and component choices. The results favor the complete design across encoders and tasks, with controlled comparisons probing adaptation, fusion, and a qualitative case study.

4.1. Datasets and Protocol

  • The unified evaluation covers three diagnostic datasets and five TCGA survival cohorts, using shared features, splits, stopping rules, and metrics for reproduced baselines.BRACS uses both five-fold cross-validation and its official split; Mamba-based models share a fixed coordinate-derived patch ordering.Source: §4.1 · p. 8, §4.2 · p. 8

4.2. Implementation Details

  • The implementation uses two evidence-fusion blocks, with local convolution and global Mamba outputs concatenated, projected, gated, and pooled into a slide representation.The LEA decoder reconstructs frozen features during training, while gated attention pooling feeds a linear task head.Source: §4.2 · pp. 8–9

4.3. Comparison with Recent WSI Baselines

  • Across all three frozen encoders, LEAF-Mamba leads mean diagnostic AUC and survival C-Index.Diagnostic AUCs are 87.7, 88.5, and 89.4 with ResNet-50, PLIP, and CONCH v1.5, exceeding Jigsaw by 1.1, 1.9, and 1.0 points. Mean survival C-Index ×100 is 67.0 with ResNet-50 and 67.3 with PLIP and CONCH v1.5, exceeding Jigsaw by 0.1, 0.2, and 0.3 points, respectively.Source: Figure 4 · p. 9, §4.3 · pp. 9–10

4.4. Ablation Study

  • With LEA enabled on BRACS official using CONCH v1.5, gated fusion scores 78.6 AUC, 40.0 ACC, and 32.7 F1, exceeding global-only aggregation by 1.8, 3.6, and 3.5 points.It also gains 0.8 AUC, 2.1 ACC, and 1.4 F1 points over the learned static gate.Source: Table 5 · p. 16, §4.4 · p. 17
  • Removing reconstruction lowers AUC from 76.8 to 75.8, ACC from 36.4 to 35.8, and F1 from 29.2 to 28.1; the decoder is removed at inference.Full LEA uses 1,250.0K adapter parameters during training and 723.5K during inference.Source: Table 6 · p. 16, §4.4 · pp. 17–18
  • Across three classification settings, the complete model has the highest AUC and ACC; on TCGA-BRCA, LEA raises global-only C-Index from 67.2 to 68.1.On Camelyon16, gated aggregation without LEA has slightly higher F1 than the complete model, 92.7 versus 92.5; with PLIP, LEA’s effect on AUC differs by aggregator.Source: Table 7 · p. 17, §4.4 · p. 18

4.5. Qualitative Case Study

  • On one BRACS malignant carcinoma example, LEAF-Mamba predicts the correct label while the baseline predicts benign hyperplasia; the heatmap comparison is qualitative, not evidence of localization accuracy or clinical utility.The displayed top-k patches differ between models.Source: §4.5 · p. 18, Figure 5 · p. 19

5. Discussion and Limitations

The component analyses support the value of adaptation and fusion while showing that their contributions vary by task and aggregator. The study is scoped to frozen patch representations, with encoder fine-tuning, spatial alternatives, and cross-institutional transfer left for future work.

5.1. Interpretation of the Component Analyses

  • Parallel global–local processing improves on either branch alone, input-dependent fusion improves on a learned static gate, and removing reconstruction reduces performance.The analyses link adaptation to the features available for aggregation and fusion to how complementary sequence representations are combined.Source: §5.1 · p. 19
  • Component contributions depend on task and aggregator: adaptation helps gated global–local BRACS with PLIP but not global-only AUC, while adapted global-only slightly exceeds the complete model on TCGA-BRCA C-Index.The complete framework’s strongest average benchmark performance does not imply that each component improves every setting.Source: §5.1 · p. 19

5.2. Scope and Limitations

  • The study focuses on frozen patch representations, so its comparisons concern adaptation and aggregation without updating the encoder.The information available to the model depends on the pretrained encoder.Source: §5.2 · p. 19
  • Future work should test selective encoder fine-tuning, alternative spatial orderings and explicit 2D relations, and transfer across institutions.Independent cohorts with different patient populations and slide preparation practices would help characterize when benefits generalize.Source: §5.2 · p. 20

6. Conclusion

LEAF-Mamba combines reconstruction-regularized token adaptation with long-range sequence modeling, local convolution, and input-dependent fusion for frozen-feature WSI analysis. The paper reports strong overall performance across feature extractors and prediction tasks, and identifies cross-institutional evaluation and multimodal learning as future directions.

6. Conclusion

  • LEAF-Mamba combines reconstruction-regularized token adaptation, long-range sequence modeling, sequence-local convolution, and input-dependent fusion in a frozen-feature WSI framework.The framework targets adaptation of pretrained patch representations while bringing together complementary local and global information.Source: §6 · p. 20
  • Across feature extractors and prediction tasks, LEAF-Mamba achieves strong overall performance in diagnostic classification and survival prediction.The component analyses further highlight the value of jointly designing feature adaptation and contextual aggregation.Source: §6 · p. 20
  • Future work will extend the approach to cross-institutional evaluation and multimodal learning with complementary patient information.The paper frames this as a direction toward more comprehensive disease characterization.Source: §6 · p. 20

Related papers

Loading 2610.29746v1…