Source-linked AI summary
SCORPION: Addressing Scanner-Induced Variability in Histopathology
Jeongun Ryu, Heon Song, Seungeun Lee, Soo Ick Cho, Jiwon Shin, Kyunghyun Paeng, Sérgio Pereira
TL;DR
Scanner variability can change model predictions for the same tissue, while conventional unseen-scanner evaluation does not directly measure this consistency. The paper introduces scanner-paired SCORPION data and SimCons, which combines style augmentation with consistency loss; SimCons improves scanner consistency without compromising task performance.
Problem
Existing domain-generalization evaluations do not directly assess whether predictions remain consistent when only the scanner changes and tissue stays the same.
Method
SCORPION provides spatially aligned patches from identical tissue scanned with five scanners, while SimCons combines style-based augmentation with consistency loss.
Results
SimCons enhances model consistency across scanners without compromising task performance.
Takeaways & Limitations
SCORPION and SimCons provide a dataset and methodology for evaluating and improving reliability across scanners in computational pathology.
Takeaways & Limitations
The consistency-loss weight λ must be controlled because setting it too high can hinder primary task performance.
Abstract
from arXiv · showhide
Ensuring reliable model performance across diverse domains is a critical challenge in computational pathology. A particular source of variability in Whole-Slide Images is introduced by differences in digital scanners, thus calling for better scanner generalization. This is critical for the real-world adoption of computational pathology, where the scanning devices may differ per institution or hospital, and the model should not be dependent on scanner-induced details, which can ultimately affect the patient's diagnosis and treatment planning. However, past efforts have primarily focused on standard domain generalization settings, evaluating on unseen scanners during training, without directly evaluating consistency across scanners for the same tissue. To overcome this limitation, we introduce SCORPION, a new dataset explicitly designed to evaluate model reliability under scanner variability. SCORPION includes 480 tissue samples, each scanned with 5 scanners, yielding 2,400 spatially aligned patches. This scanner-paired design allows for the isolation of scanner-induced variability, enabling a rigorous evaluation of model consistency while controlling for differences in tissue composition. Furthermore, we propose SimCons, a flexible framework that combines augmentation-based domain generalization techniques with a consistency loss to explicitly address scanner generalization. We empirically show that SimCons improves model consistency on varying scanners without compromising task-specific performance. By releasing the SCORPION dataset and proposing SimCons, we provide the research community with a crucial resource for evaluating and improving model consistency across diverse scanners, setting a new standard for reliability testing.
1 Introduction
Scanner differences can alter whole-slide images of the same tissue, making computational pathology predictions inconsistent and clinically consequential. SCORPION provides paired data and SimCons provides a consistency-focused framework to evaluate and address this problem.
- Scanner-induced differences in color, contrast, and texture can make computational models produce inconsistent predictions for the same tissue, potentially affecting diagnoses and treatment recommendations.
- Existing benchmarks such as Camelyon17 and MIDOG2021 evaluate domain generalization but lack scanner-paired images for directly measuring inter-scanner consistency.
- SCORPION contains 480 tissue regions scanned with five scanners, producing 2,400 spatially aligned patches that isolate scanner variability from tissue heterogeneity.
- SimCons combines style-based augmentation with a consistency loss to encourage consistent predictions while maintaining task-specific performance.
2 SCORPION dataset
SCORPION pairs scans of identical tissue across five scanners, enabling controlled analysis of scanner-induced variability. Its paired evaluation protocol reveals differences that unpaired analyses can obscure and quantifies consistency across scanner pairs.
- 2 SCORPION dataset: Each SCORPION sample contains five patches sharing the same tissue content, differing only in scanner, across 480 tissue regions and 2,400 aligned patches.The dataset uses five scanners and spatial alignment to control tissue differences.
- 2.2 Dataset Analysis.: Input-level RGB statistics and ResNet50 feature embeddings are analyzed with density contours and UMAP to characterize scanner-induced differences.
- 2.2 Dataset Analysis.: Paired analysis separates scanner distributions in feature space and most RGB channels, whereas unpaired analysis shows substantial overlap that can mask inter-scanner differences.
- The evaluation computes consistency for each of 10 scanner pairs, then reports their average and minimum scores to summarize overall and worst-case consistency.
3 Method
SimCons combines style-based augmentation with consistency training to make predictions less sensitive to scanner-like style changes while preserving supervised task learning. Its objective balances supervised and consistency losses, with the balance controlled by λ.
- SimCons generates style-altered images while preserving content, then aligns predictions for original and altered images with a consistency loss.
- Style-based augmentation improves robustness to scanner-related style variation but does not alone ensure consistent predictions for identical tissue content.
- The framework combines supervised learning with consistency regularization so predictions remain style-invariant while task-specific learning continues.
- The consistency-loss weight λ controls the trade-off: increasing it can reduce scanner-induced prediction variability, but excessive weighting can hinder primary task performance.
4 Experiments
Experiments evaluate SimCons for tissue segmentation using scanner-paired Dice consistency on SCORPION and primary-task Dice on HTS. Style augmentation improves scanner consistency, while adding consistency loss further improves consistency and can improve task performance, with λ controlling the trade-off.
- 4 Experiments: The evaluation trains models on HTS and tests scanner-paired consistency on SCORPION, whose scanners extend beyond the subset represented in HTS.HTS contains 8,327 patches from 3,399 H&E-stained WSIs, while SCORPION provides the paired scanner evaluation.
- 4.2 Main Results: Style-augmented methods improve scanner-paired consistency over the Baseline, and adding consistency loss further boosts the consistency metric.The experiments use scanner-paired Dice on SCORPION as the consistency measure and primary-task Dice on HTS.
- 4.2 Main Results: Consistency loss also improves primary-task performance, especially on the test set, while reducing scanner-induced prediction variability.The reported effect suggests improved generalization beyond the consistency objective.
- 4.2 Main Results: SimCons combines style-based augmentation with a consistency loss, using supervised loss for the primary task and consistency loss for predictions under style variation.The loss coefficient λ controls the influence of the consistency term.
- 4.3 Trade-off: Scanner Robustness vs Task Performance: Consistency improves as λ increases, but primary-task performance degrades for λ > 0.3 because mode collapse can produce trivial consistent predictions.Within the reported experiment setting, the optimal λ range is 0.3 to 0.5.
5 Conclusion
The paper presents SCORPION as a scanner-paired dataset and evaluation protocol for isolating scanner variability, alongside SimCons for reducing inter-scanner prediction differences. The reported results show improved consistency without compromising task performance, supporting more reliable scanner-generalization evaluation.
- 5 Conclusion: SCORPION provides scanner-paired patches from five scanners to evaluate model consistency while controlling for tissue differences.The dataset supports an evaluation protocol that quantifies inter-scanner variability.
- 5 Conclusion: SimCons integrates style-based augmentation with consistency loss to mitigate scanner-induced prediction variability.The framework is presented as complementary to SCORPION’s dataset and evaluation contribution.
- 5 Conclusion: The reported results show enhanced cross-scanner consistency without compromising task performance.The conclusion frames these findings as relevant to reliable model predictions in clinical decision-making.