PaperlayerView exact revision on arXiv

Source-linked AI summary

Towards Reliable AI-Based Histological Staining: A Systematic Study of Scaling and Uncertainty in Unpaired Generative Models

Qasim Siddiqui, Adrian Friebel, Maiju Myllys, Zaynab Hobloss, Daniela Gonzalez, Ahmed Ghallab, Stefan Hoehme

arXiv:2608.24626v1cs.CV

TL;DR

Sirius Red provides the standard quantitative readout of liver fibrosis, but systematic evaluation of unsupervised H&E-to-SR models and their uncertainty is limited. This paper benchmarks six architectures across 54 configurations and finds that perceptual quality, task-specific error, and ensemble uncertainty capture largely independent aspects of model fitness.

  • Problem

    Sirius Red provides the standard quantitative fibrosis readout, while comparative evaluation and uncertainty analysis of unsupervised H&E-to-SR models remain limited.

    Source: Introduction · p. 2, §8 · p. 14

  • Method

    The study benchmarks six unsupervised architectures across 54 H&E-to-SR configurations, evaluating perceptual, distributional, and task-specific performance alongside deep-ensemble uncertainty.

    Source: §8 · p. 14

  • Results

    Task-specific error, perceptual quality, and ensemble uncertainty measure largely independent axes of model fitness, so no single axis gives a complete picture.

    Source: §8 · p. 14

  • Takeaways & Limitations

    Model choice depends on deployment objective: CycleGAN’s medium generator at full data minimizes fibrosis-quantification error, DCLGAN’s small generator balances accuracy and ensemble agreement, and MUNIT is the safest default across variable data budgets.

    Source: §8 · p. 14

    The reader study covers only two configurations and three readers, precluding inter-rater agreement statistics and per-family rankings.

    Source: S6 Expert Reader Study · p. 35

Abstract

from arXiv · show

Liver fibrosis, the principal predictor of long-term outcome in chronic liver disease, is staged from histological estimates of collagen content. Sirius Red (SR) provides the standard quantitative readout (collagen proportionate area, CPA) but is not acquired at every clinical centre and consumes tissue, time, and reagent cost beyond the routine Hematoxylin and eosin (H&E) stain. AI-based virtual staining can generate SR directly from H&E, yet systematic benchmarks of unsupervised models are scarce and their predictive uncertainty has not been quantified, even though visually plausible outputs may not faithfully reproduce the underlying tissue structure. We therefore benchmark six unsupervised image-to-image architectures (GAN-based and diffusion-based) across 54 scaling configurations on a newly released paired H&E to SR mouse liver dataset, the first open resource for this translation task. Each configuration is evaluated jointly on perceptual, distributional, and task-specific axes plus a blinded expert reader study; the best per family is then retrained as a deep ensemble, the first systematic comparison of epistemic uncertainty across unsupervised stain-to-stain architectures. Across families, perceptual quality, task-specific error, and ensemble agreement measure largely independent axes of model fitness: GAN-based methods cluster tightly on perceptual metrics yet differ substantially on task error and ensemble agreement, while the diffusion-based method (CycleDiffusion) is qualitatively different on all three. No single metric captures these differences, so reliable virtual staining requires reporting and selecting on all three jointly. The dataset, tiling pipeline, models, and evaluation code are released publicly.

Introduction · 2 Related Work · 3 Dataset

The paper motivates virtual Sirius Red (SR) staining as a way to derive a quantitative fibrosis readout from routine H&E without additional tissue or laboratory resources. It introduces a scaling benchmark, ensemble uncertainty analysis, and an open mouse-liver dataset for unpaired H&E-to-SR translation.

Introduction

  • SR provides the standard quantitative fibrosis readout as collagen proportionate area, while virtual SR from H&E could avoid the tissue, time, cost, and alignment burdens of additional stains.A full tissue-characterisation panel may require four or more serial sections from a limited biopsy.Source: Introduction · p. 2
  • Prior unsupervised H&E-to-SR comparisons in liver are absent, while existing benchmarks focus on other stains or tissues and fix model scale and data budget.Supervised translation requires pixel-aligned adjacent-section pairs, which are impractical at scale and vulnerable to tissue-deformation bias.Source: Introduction · p. 2
  • The study evaluates 54 configurations across six GAN- and diffusion-based architectures, three generator capacities, and three data fractions using CPA MAE, FID, LPIPS, and patch-SSIM.CPA MAE uses a frozen collagen segmentation model on real and virtual SR; the four metrics jointly reveal where perceptual and task-level rankings diverge.Source: Introduction · pp. 2–3
  • Deep ensembles of independently seeded best-per-family models produce spatially resolved disagreement maps as an indicator of tile-level epistemic uncertainty.In the unpaired setting, no tile-level ground-truth SR is available, so ensemble disagreement complements aggregate WSI task evaluation.Source: Introduction · pp. 2–3

2 Related Work

  • Benchmarks find quantitative rankings can disagree with pathologists’ visual assessments [27], and standard image metrics may not preserve pathology-relevant structure.These findings motivate task-specific evaluation of whether diagnostically relevant content is retained [10, 38].Source: §2 · p. 3
  • Uncertainty estimation in unpaired stain translation remains unexplored; UGAC [43] estimates MRI uncertainty through a Gaussian loss, unlike ex-post ensemble disagreement.In supervised virtual cell staining, ensemble uncertainty correlated with per-tile translation error (Pearson ρ≈0.83).Source: §2 · p. 4

3 Dataset

  • The open dataset contains 70 mouse-liver WSIs from 35 mice, with serial H&E and SR sections treated as unpaired and tiles downsampled to 256×256 pixels.The 512×512 tiles were acquired at 20-fold magnification and downsampled to an effective resolution of 0.442µm/pixel; samples cover sham controls and four post-surgical stages.Source: §3 · p. 4
  • An animal-level split reserves five animals for testing, but the most advanced fibrosis stage is absent because its day-63 specimen could not be co-registered.Thirty animals contribute 201,257 H&E and 196,415 SR training tiles; five test animals contribute 16,290 tiles per domain. The release includes WSIs, preprocessing and tiling pipelines, and evaluation code.Source: §3 · p. 4

Models

The study benchmarks six unsupervised image-to-image architectures spanning GAN-based and diffusion-based approaches, at three generator capacities. All models are trained from scratch under a fixed data and compute budget to compare capacity and data effects fairly.

Models

  • CycleGAN [52], DCLGAN [16], UNIT [36], MUNIT [20], and UVCGAN [42] represent GAN-based approaches with distinct strategies for content alignment, latent sharing, or higher-capacity stability.DCLGAN uses dual patch-level contrastive loss; UNIT uses a shared VAE bottleneck; MUNIT uses domain-specific content and style codes; UVCGAN uses a Vision Transformer block and masked-image pretraining.Source: Models · p. 5
  • CycleDiffusion [47] departs from adversarial training, coupling domains in a shared diffusion noise space through DDIM [41] inversion and sampling.Its generative process is deterministic.Source: Models · p. 5
  • Each model is trained at three generator capacities (∼10, ∼50, and ∼100 million parameters) to measure how translation quality responds to added capacity within each architecture family.All models are trained from scratch under a fixed data and compute budget to compare capacity and data effects fairly across families.Source: Models · p. 5

5 Scaling Study: Experimental Design and Results

Across 54 configurations, perceptual quality, collagen-quantification error, and ensemble agreement distinguish different aspects of virtual-stain performance. No single metric captures these differences, and results vary with model family, generator size, and training-data fraction.

5 Scaling Study: Experimental Design and Results

  • Scaling comparisons covered 54 independently trained configurations: six architectures, three generator sizes, and nested 25%, 50%, and 100% training-data fractions.All configurations used the same five held-out paired specimens; models trained for 750,000 steps, except UVCGAN’s 250,000-step pretraining plus 500,000-step finetuning schedule.Source: §5 · pp. 5–6

5.1 Evaluation Metrics

  • Because H&E and SR sections are non-adjacent, the study used co-registration and a frozen SR-trained nnU-Net to compare paired collagen proportionate area, with CPA MAE as the primary task metric.The authors caution that FID can miss collagen-specific structure because its InceptionV3 features are trained on natural images; readers also assessed generated-tile detection and biological plausibility.Source: §5.1 · pp. 6–7

5.2 Results: Scaling Study

  • CPA MAE ranged from 0.008 to 0.171 while perceptual metrics saturated for well-behaved GANs; CycleGAN-M achieved the study-lowest CPA MAE, 0.008.CycleDiffusion had systematically lower Patch-SSIM across scales, illustrating that visual and task-specific performance can diverge.Source: Figure 2 · p. 7, §5.2 · pp. 7–8, Table 1 · p. 8
  • Scaling effects were family-specific: DCLGAN’s large model collapsed at reduced data, UVCGAN’s medium and large models degraded across data fractions, while MUNIT remained stable.CycleDiffusion-S CPA MAE exceeded 0.15 at both 25% and 50% data, whereas its medium model achieved its best CPA MAE at 25% data (0.035).Source: §5.2 · p. 8
  • Among 54 configurations, CPA MAE rankings agreed only moderately with Patch-SSIM (rs=0.63), LPIPS (rs=0.64), and FID (rs=0.55).Patch-SSIM and LPIPS agreed most strongly (rs=0.82); all reported metric-pair correlations were significant at p<0.001.Source: §5.2 · p. 8, Table 2 · p. 9
  • Readers identified 38% of best-configuration tiles and 62% of worst-configuration tiles as generated, and rated the best plausible but the worst implausible.The best configuration received plausibility scores of 2.7, 3.2, and 3.7 of 4; the worst received 1.1, 1.4, and 1.7.Source: §5.2 · p. 9

6 Epistemic Uncertainty Analysis

Deep ensembles reveal that virtual-stain uncertainty varies across model families and is not a measure of correctness. Reading uncertainty alongside CPA MAE distinguishes models that perceptual quality alone does not separate.

6 Epistemic Uncertainty Analysis

  • Each family’s full-data generator with lowest CPA MAE (FID as tiebreak) was retrained with ten independent seeds; ensemble variance signals disagreement, not correctness.The study used deep ensembles rather than MC dropout, citing poor MC-dropout calibration in unpaired translation and convergence near ten members in prior work.Source: §6 · p. 9

6.1 Uncertainty Quantification

  • Uncertainty maps sum RGB-channel sample variances across ensemble members, and per-tile uncertainty is their mean standard deviation over tissue pixels.High map values mark regions where independently seeded models disagree about the translation.Source: §6.1 · pp. 10–11

6.2 Results: Ensemble Variance

  • CycleDiffusion had the lowest median uncertainty (23.35) but widest IQR (5.89), versus GAN medians of 37.68–41.42 and IQRs of 1.48–3.80.CycleGAN-M paired the study-low CPA MAE (0.008 ± 0.007) with median uncertainty 39.09 and IQR 3.39; DCLGAN paired CPA MAE 0.028 with the narrowest IQR (1.48), while CycleDiffusion paired CPA MAE 0.051 with the lowest median and widest spread.Source: Figure 4 · p. 11, §6.2 · pp. 11–12
  • GAN uncertainty maps moderately predicted tile-level cycle-reconstruction error (Pearson ρ = 0.50–0.66), but CycleDiffusion uncertainty was uncorrelated (ρ = −0.03).DCLGAN led with ρ = 0.66 and ECE 0.055; no family achieved meaningful within-tile spatial calibration.Source: §6.2 · p. 12

7 Discussion

Across the 54 configurations, perceptual quality, task-specific error, and ensemble agreement captured different aspects of model fitness, so no single metric supports reliable virtual-staining selection. The discussion also emphasizes family-specific scaling behavior and limits on generalizing the findings beyond this fixed-budget mouse-liver study.

Toward a multi-axis evaluation protocol

  • Perceptual quality, CPA MAE, and ensemble agreement capture distinct model-fitness axes, making joint evaluation necessary for reliable virtual-staining selection.The optimal scaling point is family-specific and cannot be predicted from perceptual metrics alone; uncertainty alone does not show whether an output is correct.Source: §7 · pp. 12–13

Perceptual saturation and the case for task-specific evaluation

  • Across 37 stable GAN configurations, Patch-SSIM 0.298–0.349, LPIPS 0.038–0.040, and FID 190–253 correlated only modestly with CPA MAE (rs=0.63, 0.64, and 0.55).CycleGAN’s medium and large generators at 100% data had Patch-SSIM 0.334 vs. 0.342 but CPA MAE 0.008 vs. 0.085; expert readers unanimously separated the two configurations on detection and plausibility.Source: §7 · p. 12

Minimum viable configurations across scaling

  • With the full training set, medium generators were most reliable across families, while MUNIT’s CPA MAE stayed 0.040–0.059 across nine conditions.DCLGAN’s small generator remained stable across data fractions (CPA MAE 0.019–0.049), whereas CycleGAN and UNIT were less reliable with scarce data; the apparent medium-capacity optimum may reflect overfitting, and nested subsets cannot separate data quantity from diversity.Source: §7 · p. 12

Training instability

  • DCLGAN’s large generator collapsed at reduced data fractions (FID 327–363, Patch-SSIM 0.13–0.14), while UVCGAN’s medium and large variants failed across most fractions (FID up to 395).The authors hypothesize insufficient patch density for DCLGAN’s dual InfoNCE buffer and poor transfer of UVCGAN’s masked-image pretraining to larger generators; both failures were abrupt, making FID monitoring important for early detection.Source: §7 · pp. 12–13

CycleDiffusion as a distinct generative paradigm

  • CycleDiffusion’s Patch-SSIM 0.200–0.293 versus 0.298–0.342 for stable GANs reflects its DDIM inversion bias, which trades pixel sharpness for domain-invariant structure.Its small generator’s CPA MAE rose from 0.051 at 100% data to 0.171 at 25%, while the medium generator reached 0.035 at 25% versus 0.082 at 100%; the small model was selected at 100%, and the medium model under data reduction.Source: §7 · p. 13

Epistemic uncertainty agreement across families

  • DCLGAN paired the lowest GAN-family IQR (1.48) with across-tile calibration Pearson ρ = 0.66 and ECE 0.055, while CycleDiffusion had median uncertainty 23.35, IQR 5.89, and uncertainty-cycle-error correlation ρ = −0.03.MUNIT combined the highest GAN median uncertainty (41.42) with stable CPA MAE through style variation; CycleGAN’s selected medium model paired study-low CPA MAE with moderate spread. CycleDiffusion was conditionally consistent, not uniformly more reliable, under the fixed-budget, from-scratch protocol.Source: §7 · p. 13

Limitations and future work

  • The study’s rankings are limited by a single mouse tissue, disease model, and staining laboratory, a five-specimen test set, and a reader study of two configurations and three readers.Adjacent-section registration limits perceptual metrics, and the uncertainty framework omits aleatoric variation; proposed next steps include cross-cohort and human-biopsy validation, pre-trained priors, broader pathology metrics, and methods to narrow CycleDiffusion’s wide ensemble spread.Source: §7 · pp. 13–14

8 Conclusion

The study benchmarks unsupervised H&E-to-Sirius Red models and finds that deployment choices depend on the downstream objective, because task error, perceptual quality, and ensemble uncertainty capture largely independent aspects of model fitness.

8 Conclusion

  • The study benchmarks six unsupervised architectures across 54 scaling configurations, jointly assessing perceptual, distributional, task-specific, and per-tile ensemble uncertainty outcomes.The authors describe this as the first systematic comparison of epistemic uncertainty across unsupervised stain-to-stain architectures.Source: §8 · p. 14
  • Deployment choice varies by objective: CycleGAN minimizes fibrosis task error, DCLGAN balances task accuracy with per-tile ensemble agreement, MUNIT is the variable-budget default, and CycleDiffusion is conditionally consistent.CycleGAN’s medium generator at full data has the lowest task-specific error; DCLGAN’s small generator offers the task-accuracy and ensemble-agreement trade-off. CycleDiffusion is not uniformly more reliable than GAN alternatives.Source: §8 · p. 14
  • Task-specific error, perceptual quality, and ensemble uncertainty measure largely independent axes, so benchmarks should report all three and select models by the axis closest to downstream use.Reporting any single axis gives an incomplete picture.Source: §8 · p. 14
  • The dataset, tiling pipeline, models, and evaluation code are released as I2I-Stain-Zoo; the next priority is validation across independent mouse models and human liver biopsies.The release is available at https://github.com/HoehmeLab/I2I-Stain-Zoo.Source: §8 · p. 14

Supplementary Material · S1 Dataset Details

S1 documents a 35-animal mouse liver dataset spanning fibrosis stages and details its histology, preprocessing, leakage-resistant splits, and public release. H&E and Sirius Red sections share macroscopic morphology but are non-adjacent, so they are not pixel-level paired.

S1 Dataset Details

  • The dataset uses liver sections from 35 male C57BL/6N mice across four post-surgical fibrosis stages and BDL or sham conditions, with four to five animals per stage and condition.Stages are days 3, 21, 42, and 63; the cohort originated from, and the stages represent early through advanced fibrosis.Source: S1 Dataset Details · p. 20
  • Only vehicle-treated sections were included, excluding pharmacologically treated arms to avoid intervention-driven morphological variation and isolate BDL disease progression.H&E and Sirius Red are used, with Sirius Red serving as the diagnostic gold standard for hepatic fibrosis.Source: S1 Dataset Details · p. 20
  • Serial 4 µm sections from each tissue block were stained with H&E and Sirius Red; their non-adjacent locations preserve macroscopic consistency but preclude pixel-level correspondence.Sirius Red selectively binds fibrillar collagen types I and III.Source: S1 Dataset Details · p. 20
  • Slides were digitised at 20× and 0.221 µm/pixel, then tiled into 512×512 patches, downsampled to 256×256, and normalised channel-wise to [−1, 1].Tissue masks used QuPath intensity thresholds; training tiles with under 50% tissue were discarded, while test tiles were not tissue-filtered.Source: S1 Dataset Details · p. 20
  • Animal-level, stage- and treatment-stratified splitting prevents leakage; the shared test set has five held-out paired specimens, but no day-63 BDL specimen.The only eligible day-63 BDL candidate could not be co-registered across stains and was excluded; all 54 configurations use the same test set.Source: S1 Dataset Details · p. 21
  • Training uses 30 paired specimens and nested 25%, 50%, and 100% data-fraction subsets, so comparisons vary training-set size rather than slide composition.The subsets are strict contiguous-folder ranges; Table S2 summarizes split counts and stage distribution.Source: S1 Dataset Details · p. 21, Table S2 · p. 21
  • The CC BY 4.0 I2I-Stain-Zoo release includes tissue masks, tile metadata, split assignments, image patches, pretrained checkpoints, and training scripts.Release is subject to the data-sharing terms of the source study.Source: S1 Dataset Details · p. 21

S2 Model Details

The section describes six unpaired translation architectures, distinguished by their generator designs, latent representations, and training objectives. It also specifies three model scales and notes CycleDiffusion’s higher inference cost relative to GAN generators.

CycleGAN

  • CycleGAN [52] uses bidirectional ResNet generators, 70×70 PatchGAN discriminators, LSGAN training, and a cycle-consistency loss.An optional identity loss uses λid=0.5; the cycle-consistency weight is λcyc=10.Source: S2 Model Details · p. 22

UNIT

  • UNIT [36] encodes both domains into a shared latent space, uses VAE-sampled codes, and combines adversarial, L1 reconstruction, and KL-divergence losses.Its bottleneck has domain-private and shared residual blocks; λGAN=1, λrecon=10, and λKL=0.01.Source: S2 Model Details · p. 22

MUNIT

  • MUNIT [20] separates domain-invariant content from domain-specific style, injects target style through AdaIN, and optimizes adversarial, reconstruction, content, and style consistency.Its style vector has dimension ds=8, and the objective weights are λimg=10, λc=1, and λs=1.Source: S2 Model Details · p. 22

DCLGAN

  • DCLGAN [16] augments CycleGAN with symmetric InfoNCE contrastive learning over sampled generator features, alongside cycle consistency, while disabling identity loss.It samples 256 locations per feature map and projects them into 256-dimensional embeddings; λcyc=10 and λDCL=1.Source: S2 Model Details · p. 23

UVCGAN

  • UVCGAN [42] replaces CycleGAN’s ResNet generator with a UNet–Vision Transformer hybrid and first pretrains masked-patch reconstruction before cycle-consistent GAN finetuning.Pretraining masks four 32×32 patches per image; finetuning uses λcyc=10 and λid=0.5.Source: S2 Model Details · p. 23

CycleDiffusion

  • CycleDiffusion [47] trains independent domain-specific DDPMs without adversarial or cycle-consistency losses, then translates through a shared Gaussian latent using deterministic DDIM inversion and sampling.The approximate round-trip reconstruction is an inference-time property; translation uses 200 inversion and 200 sampling steps.Source: S2 Model Details · pp. 23–24

Model Size Configurations

  • Each architecture is trained at three scales, but CycleDiffusion is substantially slower at inference because each tile requires 200 inversion and 200 sampling steps.GAN generators use a single forward pass; CycleDiffusion’s training step is lower-cost because its DDPMs train independently without cycle coupling. Table S3 gives the generator-size configurations.Source: S2 Model Details · p. 24, Table S3 · p. 25

S3 Implementation Details

Models were trained with shared step-based optimization settings but architecture-specific schedules. At inference, GAN-based models used one generator pass per tile, whereas CycleDiffusion used sequential DDIM inversion and sampling passes.

Shared training hyperparameters

  • All models used Adam [29] at 2×10−4 with β1=0.5, β2=0.999, batch size 1, no weight decay, and linear learning-rate decay from halfway to the final step.Training was step-based, with one optimizer update per step, to enable fair comparison across data-fraction configurations with different tile counts.Source: S3 Implementation Details · p. 25

Per-model training steps and schedules

  • CycleGAN, UNIT, MUNIT, and DCLGAN each trained for 750,000 steps, alternating a single Adam update between generator and discriminator.Source: S3 Implementation Details · p. 25
  • UVCGAN used two stages: 250,000 masked-autoencoding pretraining steps with ReZero, then 500,000 cycle-consistent finetuning steps initialized from the Stage 1 generator checkpoint.Stage 2 applied the full cycle-consistency and adversarial losses.Source: S3 Implementation Details · p. 25
  • CycleDiffusion trained both domain DDPMs simultaneously for 750,000 steps using ε-prediction and a linear noise schedule with T=1000 diffusion steps.It used no fine-tuning phase or post-training step-count adjustment.Source: S3 Implementation Details · p. 25

Inference details

  • GAN-based models generated each test tile with one GA→B generator pass, while CycleDiffusion used DDIM inversion for 200 steps followed by DDIM sampling for 200 steps.Generated tiles were denormalised from [−1,1] to [0,1] and saved with original tile identifiers for downstream metric computation.Source: S3 Implementation Details · p. 26

S4 Full Scaling Study Results

The full scaling study reports four evaluation metrics across 54 configurations spanning six models, three generator sizes, and three training-data fractions. It flags collapsed configurations using joint FID and Patch-SSIM criteria and separately illustrates ensemble-based uncertainty estimation.

S4 Full Scaling Study Results

  • Patch-SSIM, LPIPS, and CPA MAE are reported as mean ± std over five held-out paired specimens, while FID is a set-level statistic without per-sample variance.This matches the main paper’s reporting for the three per-specimen metrics.Source: S4 Full Scaling Study Results · p. 26
  • Configurations are flagged as collapsed only when FID > 300 and Patch-SSIM < 0.20; high FID alone is insufficient.Collapsed configurations are marked with an asterisk in Tables S5–S6.Source: S4 Full Scaling Study Results · p. 26, Table S5 · p. 27, Table S6 · p. 28
  • Tables S5–S6 report all four evaluation metrics for 54 configurations across six models, three generator sizes, and three training-data fractions.The tables cover CycleGAN, UNIT, MUNIT, DCLGAN, UVCGAN, and CycleDiffusion.Source: S4 Full Scaling Study Results · p. 26, Table S5 · p. 27, Table S6 · p. 28
  • Figure S1 estimates epistemic uncertainty by forwarding each H&E tile through K=10 independently trained generators and computing pixel-wise sample variance across their SR outputs.The variance map is denoted U(x).Source: Figure S1 · p. 29

S5 Uncertainty Calibration Analysis

Ensemble uncertainty provides only a weak tile-level triage signal for most GAN-based families, no meaningful within-tile spatial calibration, and no useful signal for CycleDiffusion. Calibration is measured against cycle-reconstruction error rather than true translation error, so its results require that qualification.

S5.1 Uncertainty Quantification

  • A ten-generator ensemble yields a pixel-wise mean prediction and spatial uncertainty map, with channel-wise sample variances summed to retain total disagreement across RGB channels.Tile-level uncertainty is the mean standard deviation over tissue pixels.Source: S5.1 Uncertainty Quantification · pp. 29–30

S5.2 Calibration Methodology

  • Calibration compares uncertainty with each model’s own cycle-reconstruction error proxy, using within-tile Spearman, across-tile Spearman and Pearson, and tile-level ECE.The proxy is not ground-truth translation error; within-tile metrics assess spatial co-location, while across-tile correlations and ECE assess tile-level calibration.Source: S5.2 Calibration Methodology · pp. 30–31

S5.3 Per-Model Uncertainty and Calibration Results

  • Within-tile Spearman correlations are weak across all six families (rs = 0.058–0.165), ruling out pixel-level uncertainty as a reliable quality gate.MUNIT (rs = 0.165 ± 0.173) and DCLGAN (rs = 0.157 ± 0.084) lead, but MUNIT is inconsistent across tiles; UVCGAN (rs = 0.069) and CycleDiffusion (rs = 0.058) are near zero.Source: Table S7 · p. 31, S5.3 Per-Model Uncertainty and… · p. 31, Figure S3 · p. 33
  • GAN families show moderate across-tile Pearson correlations (ρ = 0.50–0.66), but weaker, inconsistent Spearman correlations (rs = −0.18–0.21) reveal non-monotonic relationships.CycleDiffusion is uncorrelated across tiles (Pearson ρ = −0.03, Spearman rs = −0.04); the GAN Pearson pattern is driven by a small number of high-leverage tiles.Source: Table S7 · p. 31, S5.3 Per-Model Uncertainty and… · p. 32, Figure S4 · p. 34
  • DCLGAN has the lowest tile-level ECE (0.055), followed by MUNIT (0.058), while CycleDiffusion has the highest (0.214).All six reliability curves lie predominantly below the identity line, meaning ensembles generally assign more uncertainty than normalized cycle error warrants; in every family, the lowest-uncertainty bin is an exception where error exceeds uncertainty.Source: Table S7 · p. 31, S5.3 Per-Model Uncertainty and… · p. 32, Figure S2 · p. 32
  • CycleDiffusion’s uncertainty is uninformative: its error remains approximately constant (En ≈0.43–0.49) across uncertainty bins and its cycle error varies too little across tiles to be predicted from ensemble variance.The reported cause is DDIM inversion cycle error, whose tile-mean p1–p99 range is 30.0–38.6, versus 0.7–6.7 for CycleGAN, UNIT, DCLGAN, and UVCGAN and 4.9–30.6 for MUNIT.Source: S5.3 Per-Model Uncertainty and… · pp. 32–33

S5.4 Interpretation

  • DCLGAN and MUNIT are the most defensible choices for soft uncertainty-based pathologist triage, but calibration rankings only partly align with CPA MAE rankings.CycleGAN achieves the study-best CPA MAE (0.008) despite only moderate calibration, indicating that perceptual quality, task accuracy, and uncertainty calibration must each be considered.Source: S5.4 Interpretation · p. 33

S5.5 Summary

  • GAN-based ensembles offer a weak but non-trivial tile-level triage signal (across-tile Pearson ρ = 0.50–0.66), but uncertainty screening cannot replace task-specific CPA MAE evaluation.No family achieves meaningful within-tile spatial calibration, and CycleDiffusion uncertainty is wholly uninformative.Source: S5.5 Summary · p. 33

S6 Expert Reader Study

In a blinded study, readers found the best virtual-staining configuration difficult to distinguish from real Sirius Red and judged its collagen plausible, unlike the worst configuration. The study covered only three readers and two configurations, so it cannot establish inter-rater statistics or rankings across model families.

S6 Expert Reader Study

  • Three blinded readers assessed held-out tiles independently, without knowing the real-to-generated proportion or how many images came from each source.Two readers were external clinical liver experts and one was a co-author experienced with histological images; none had participated in model development or evaluation.Source: S6 Expert Reader Study · pp. 34–35
  • The study compared the scaling study’s best CycleGAN-M at 100% data (CPA MAE 0.008) with its worst CycleDiffusion-S at 25% data (CPA MAE 0.171).These configurations had the lowest and highest CPA MAE among all 54 configurations.Source: S6 Expert Reader Study · p. 34
  • For Task 2, readers judged collagen plausibility against H&E tissue architecture in 30 paired comparisons, rating each generated image independently on a four-point scale without a neutral option.The genuine SR section came from a non-adjacent plane, so H&E—not pixel-level correspondence—served as the reference for plausible collagen distribution.Source: S6 Expert Reader Study · p. 35
  • Only 38% of best-configuration tiles, versus 62% of worst, were correctly identified as generated; all three readers rated the best plausible and the worst implausible.For Task 2, there was no overlap between the two configurations’ plausibility ratings for any reader.Source: Table S8 · p. 35, S6 Expert Reader Study · p. 35
  • Reader separation tracked the configurations’ CPA MAE ranking rather than their more similar perceptual scores, independently of the automated pipeline; the small study cannot rank families or estimate inter-rater agreement.It included only two configurations and three readers; extending the protocol across all six families and multiple centres was left to future work.Source: S6 Expert Reader Study · p. 35

Related papers

Loading 2608.24626v1…