Source-linked AI summary
CPathOGen: Spatially and Morphologically Controlled H&E Counterfactuals for Probing Pathology Models
Samarth Singhal, Varang Rai
TL;DR
Pathology models can use intertwined tissue and slide-preparation signals, complicating interpretation of their predictions. CPathOGen generates controlled H&E counterfactuals and uses verified interventions to probe model responses. The reported fidelity and probing results show recoverable requested changes and distinct patterns of predictive performance and intervention sensitivity.
Problem
Tissue morphology and slide-preparation or digitization cues can be intertwined, making it difficult to determine which features shape pathology-model predictions.
Source: §1 · p. 1Method
CPathOGen generates matched H&E counterfactuals by controlling cellular organization, nuclear morphology, and stain appearance, then measures pathology-model responses to verified interventions.
Source: §1 · p. 2Results
Independent analyzers recover requested spatial changes, while probing finds that predictive performance and intervention sensitivity differ and that dominant biological responses vary across encoders.
Source: §4.3 · p. 6, §5 · p. 8, §4.5 · p. 8Takeaways & Limitations
The framework supports fidelity-audited evaluation of pathology models’ sensitivity to biological factors and nuisance variation, including potential reliance on stain or orientation signals.
Source: §5 · p. 8Counterfactuals are condition-matched synthetic samples rather than exact reconstructions, and prediction shifts under these interventions do not establish patient-level biological causality.
Source: §5 · p. 8
Abstract
from arXiv · showhide
Computational pathology models infer biologically and clinically meaningful outcomes from histology, but their predictions are shaped by complex, intertwined tissue signals whose roles are important to understand. Common pixel- and feature-space perturbations can produce implausible tissue, making model responses difficult to interpret. We introduce CPathOGen, a conditional latent-diffusion framework for generating paired H\&E counterfactuals with explicit controls over cellular spatial organization, nuclear morphology, and stain appearance. Cellular maps condition spatial structure through a spatial encoder, while a morphology/appearance vector modulates denoising through blockwise feature-wise linear modulation (FiLM). On held-out H\&E tiles, CPathOGen generates visually plausible tissue with improved distributional agreement after spatially guided selection, as reflected by lower Fr\'echet Inception Distance (FID) and Kernel Inception Distance (KID); generated cells track requested abundance and position, and measured morphology and color vary monotonically with their controls. We use these verified interventions to probe pathology encoders with endpoint heads, task-specific classifiers, and survival models. Responses are quantified using total variation distance and prediction-flip rate. We further introduce the Biology-Nuisance Sensitivity Ratio, a metric that contrasts model sensitivity to biologically motivated morphology and spatial factors with sensitivity to non-biological, stain-related nuisance variation. CPathOGen provides a practical, fidelity-audited framework for evaluating robustness and controlled feature sensitivity in computational pathology model. \href{https://github.com/a12dongithub/PathOGen}{GitHub} and \href{https://huggingface.co/a12donhf/CPathOGen}{Hugging~Face}.
1. Introduction
CPathOGen generates matched H&E counterfactuals from requested tissue interventions, then measures how pathology models respond. The approach addresses the difficulty of separating tissue features from preparation and scanning cues when interpreting model predictions.
1. Introduction
- Histopathology supports clinically meaningful prediction, and pretrained pathology encoders broaden the tasks that can be studied with limited labels, but their evidence remains difficult to interpret.The cited examples include CTransPath, UNI, Virchow, and CONCH.Source: §1 · p. 1
- Slide preparation and digitization can influence apparent tissue features, so a model may partly use preparation or scanning clues even on held-out data.The passage notes that standard normalization and augmentation do not fully remove clues about slide origin.Source: §1 · p. 1
- A highlighted region does not reveal whether nuclear size, immune density, spatial mixing, stain, or scanner artifacts drove a prediction.Some saliency maps can remain visually similar after model parameters or training labels are randomized [1].Source: §1 · pp. 1–2
- Prior classifier-guided counterfactual optimization can change several correlated attributes, complicating attribution of a model response to one requested factor.The passage contrasts CPathOGen’s requested interventions with methods that optimize edits toward a target prediction [10] [3] [4] [18] [8].Source: §1 · p. 2
- CPathOGen varies cellular organization, nuclear morphology, or stain appearance while preserving the remaining modeled factors, then records pathology-model prediction changes.The paper also presents controllable H&E synthesis and counterfactual probing as its two technical contributions.Source: §1 · p. 2
- The study trains its diffusion model on 1.4 million 512×512 H&E tiles from 1,114 one-per-patient WSIs and probes PAM50 classification and overall survival models.CellViT++ supplies weak cellular labels, while HoVer-Net and StarDist provide independent measurements [11] [16] [30].Source: §1 · p. 2
2. Related Work
Prior work uses saliency, classifier-guided counterfactuals, and generative models to study pathology images. CPathOGen combines cellular-map conditioning with morphology and appearance controls to support verified, single-factor interventions.
2.1. Explainability and counterfactual model probing
- Saliency localizes influential regions, while counterfactual methods compare inputs with different predictions; CPathOGen instead uses a conditional generator to test external pathology models.Related approaches include discriminative comparisons, diffusion-guided edits, and interventions on diffusion activations [10] [3] [18] [4] [35].Source: §2.1 · p. 2
- Pathology counterfactuals have linked synthetic class transitions to molecular labels and edited morphology toward target predictions, while CPathOGen requires verification of the requested intervention before probing.The prior approaches described include class transitions [8] and MoPaDi [27].Source: §2.1 · p. 2
2.2. Controllable histopathology generation
- Existing histopathology generators advance realism, scale, conditioning, or synthetic-data augmentation, but are not designed for separable controls over cell organization, nuclear morphology, and appearance.The cited methods include PathologyGAN, PathLDM, PixCell, gigapixel WSI synthesis, Spatial Diffusion, and TopoCellGen [41] [12] [22] [40].Source: §2.2 · p. 2
- CPathOGen pairs a spatial encoder for cellular maps with FiLM conditioning for morphology and appearance, allowing one control family to vary while other modeled conditions remain fixed.The generated change is checked for recovery before the image is used to probe a model.Source: §2.2 · p. 2
3. Method
CPathOGen generates matched H&E counterfactuals by separately controlling cellular spatial organization and global morphology/appearance, then records how frozen pathology models respond. Its conditions come from weak cellular labels, and candidate images are selected using spatial-map agreement before model probing.
3.1. Framework overview
- CPathOGen modifies a requested control, generates a matched H&E counterfactual, and records the response of a frozen downstream model.The framework combines a spatial encoder for cellular maps with blockwise FiLM conditioning for morphology and appearance.Source: §3.1 · pp. 2–3
3.2. Weakly supervised condition construction
- CellViT++ weak labels provide nucleus contours, centroids, and cell types for spatial maps, while a 16-feature vector summarizes nuclear morphology and RGB appearance.The five spatial-map channels represent tumor, inflammatory, epithelial, stromal, and necrotic compartments; stored preprocessing settings preserve feature ordering and standardization at inference.Source: §3.2 · p. 3
3.3. Conditional latent-diffusion model
- The generator adapts pretrained Stable Diffusion 2.1 weights to H&E tiles and trains a conditional denoising U-Net on noisy image latents, spatial maps, and morphology vectors.Text conditioning remains fixed because synthesis is governed by structured pathology controls rather than natural-language prompts.Source: §3.3 · pp. 3–4
3.4. Latent spatial conditioning
- A trainable convolutional encoder maps cellular maps to latent-resolution spatial features, which are concatenated with noisy latents to condition denoising.The first U-Net convolution expands from four to eight input channels; added spatial weights start at zero while original latent weights retain pretrained initialization.Source: §3.4 · p. 4
3.5. Morphology and Stain modulation
- A learned mapping turns the morphology vector into channel-wise scale and shift parameters that modulate residual-block activations throughout the denoising U-Net.The identity offset preserves unmodulated activations when the predicted scale is zero; bounded parameters and the spatial map provide complementary controls over geometry/appearance and cellular location.Source: §3.5 · p. 4
3.6. Prediction-independent fidelity guidance
- For each condition, candidate seeds are ranked by agreement between the requested spatial map and cellular organization recovered from each generated image.CellViT++ supplies the primary agreement signal, and selection is fixed before evaluation without observing the probed model’s predictions, representations, or gradients.Source: §3.6 · p. 4
3.7. Paired interventions and model probing
- Paired interventions reuse the same initial noise and hold non-target conditions fixed, then record downstream outputs for biological endpoints, survival risk, or image embeddings.Interventions can vary nuclear properties, spatial composition/density/distance/mixing, or color controls; generated changes are verified after synthesis.Source: §3.7 · p. 5
4. Experiments
The experiments evaluate distributional realism, recovery of spatial and morphology controls, and pathology-model responses to verified counterfactual interventions. Results show improved realism after spatial selection, recovered requested features, and greater average model sensitivity to the evaluated biological interventions than to stain and rotation controls.
4.1. Experimental setup
- The study uses patient-separated TCGA breast-cancer tiles, with tile-level PAM50 evaluation and fixed 16-tile survival bags; generated images are not used to train downstream models.The diffusion model was trained on approximately 1.4 million tiles from 1,114 whole-slide images.Source: §4.1 · p. 5
4.2. Distributional realism
- Spatially guided eight-candidate selection improves FID and KID against an independent real-tile reference, while Figure 3 provides qualitative examples rather than pixelwise reconstructions.The selected candidates better express requested cellular structure and more closely follow the held-out H&E distribution.Source: §4.2 · p. 5, Figure 3 · p. 6
4.3. Spatial fidelity
- All three analyzers recover total cellular burden consistently; lower per-type agreement reflects classification and taxonomy differences, while the substantially weaker random-tile control supports map-specific agreement.StarDist has the highest position-only F1 but is not penalized for type disagreement; agreement across analyzers indicates fidelity is not specific to the selection model.Source: §4.3 · p. 6
4.4. Morphology and appearance fidelity
- Agreement is strongest for size, gradient, and color, and within-image measurements generally follow the requested ordering when a single control is varied.Within-image tests preserve the spatial map, diffusion seed, and other morphology/appearance entries.Source: §4.4 · p. 6
4.5. Counterfactual model probing
- The probing contrasts nuclear enlargement, shape irregularity, immune burden, and tumor–immune mixing with stain brightness and rotation, measuring responses using total variation distance and prediction-flip rate.The Biology-to-Nuisance Ratio compares mean sensitivity across those biological interventions with sensitivity to the stain and rotation controls; it is not predictive performance or biological correctness.Source: §4.5 · pp. 6–8
- UNI2-h has the strongest PAM50 discrimination but is not the least sensitive model; dominant biological responses vary by encoder, and tumor–immune mixing is smallest across all three.Rotation generally moves outputs less than stain brightness, although nonzero flip rates indicate residual orientation sensitivity.Source: §4.5 · p. 8
- Patient-clustered 95% confidence intervals for every model and task remain above one, supporting greater average sensitivity to evaluated biological interventions than to stain brightness and image rotation.The survival responses are attenuated because each intervention replaces one tile in a fixed 16-tile patient representation.Source: §4.5 · p. 8, Table 4 · p. 8
5. Conclusion
CPathOGen offers a structured framework for generating controlled H&E counterfactuals and auditing computational pathology models. Fidelity experiments recover requested changes, while probing distinguishes predictive performance from intervention sensitivity; the conclusions are bounded by synthetic interventions and training-data requirements.
5. Conclusion
- CPathOGen provides a structured framework for spatially and morphologically controlled H&E counterfactuals and computational pathology model audits.The framework is intended to examine sensitivity to biological factors, nuisance variation, and potential spurious signals such as stain or orientation.Source: §5 · p. 8
- Fidelity experiments recover requested cellular organization and appearance changes across independent analyzers, while predictive performance and intervention sensitivity describe different model properties.The framework supports safety-oriented evaluation, and its evidence may help identify models needing refinement or inform model selection based on performance and feature reliance.Source: §5 · p. 8
- The framework requires a large, representative training corpus with sufficiently reliable cellular annotations or weak labels.Its counterfactuals are condition-matched synthetic samples, not exact reconstructions; unmodeled properties may change, and prediction shifts characterize model behavior rather than patient-level biological causality.Source: §5 · p. 8
- The authors identify external-cohort and expert validation, richer cellular conditions, and more efficient candidate selection as future work.They also suggest that controlled generation and paired probing could be adapted to imaging fields where spatial organization and global attributes jointly influence predictions.Source: §5 · p. 8