Source-linked AI summary
Domain-Adapted Latent Diffusion Models for CT-to-PET Image Translation
Yann Bourdé, Lea Bogensperger, Arnela Hadzic, Michael Krauthammer
TL;DR
Paired CT/PET data are scarce, limiting diffusion-based translation, while PET remains less accessible than CT. The paper adapts a pretrained latent diffusion model with a signed power transform and evaluates tumor-focused metrics alongside full-image metrics. It reports better pixel-wise reconstruction and localized tumor fidelity than CPDM, while identifying limitations of standard full-image metrics for sparse PET.
Problem
Diffusion-based CT-to-PET methods are limited by scarce paired training data, while PET’s cost and infrastructure requirements make it less accessible than CT.
Source: §1 · p. 1Method
The paper adapts natural-image latent diffusion priors for CT-to-PET synthesis using supervised finetuning and a signed power transform for PET intensity sparsity.
Source: §1 · p. 2, §2.2 · p. 3Results
The approach reports superior pixel-wise reconstruction and localized tumor fidelity compared with CPDM.
Source: §1 · p. 2, §3.3 · p. 6Takeaways & Limitations
The paper concludes that background-filtered and tumor-mask-restricted metrics provide more meaningful assessments of sparse PET synthesis than standard full-image metrics alone.
Source: §3.4 · p. 7, §4 · p. 9The study had access only to PCLT20K because the paired CT/PET dataset used in CPDM was not publicly available.
Source: §3.1 · p. 4
Abstract
from arXiv · showhide
Positron Emission Tomography (PET) provides important metabolic information for tumor detection and staging but remains significantly less accessible than Computed Tomography (CT) due to its high cost, infrastructure requirements, and reliance on injected radioactive tracers. CT-to-PET image translation using generative models aims to address this disparity, yet state-of-the-art diffusion-based methods are limited by the scarcity of paired training data. To overcome this, we propose a domain-adapted latent diffusion framework that transfers natural-image priors using a relatively small dataset of 21,930 paired CT/PET slices, incorporating a signed power transform to resolve extreme intensity sparsity in PET data. Compared to CPDM, a state-ofthe-art baseline for CT-to-PET image translation, our approach yields superior performance in pixel-wise reconstruction and localized tumor fidelity, demonstrating effective transfer of natural-image priors to the medical domain. We further provide a systematic analysis of evaluation metric reliability for sparse PET data, showing that standard full-image metrics can be misleading indicators of translated image quality, and propose background-filtered and mask-restricted metrics as more clinically meaningful alternatives.
1 Introduction
CT and PET provide complementary anatomical and metabolic information, but PET is less accessible and paired CT/PET data are scarce. The paper adapts natural-image latent diffusion priors for CT-to-PET translation and evaluates reconstruction, tumor fidelity, and metrics for sparse PET data.
1 Introduction
- PET adds metabolic information to CT’s anatomical detail, but its costly infrastructure and radiotracer requirements make it less accessible.Together, the modalities offer a more complete diagnostic picture than either alone.Source: §1 · p. 1
- Scarce public paired CT/PET data constrain diffusion methods trained from scratch and may limit performance and generalization.Source: §1 · p. 2
- The paper introduces domain-adapted latent diffusion, using transfer learning to adapt natural-image priors with paired CT/PET data.The authors report that this approach mitigates data scarcity and enhances PET synthesis fidelity.Source: §1 · p. 2
- The framework adds a signed power transform for PET intensity imbalance and outperforms CPDM in pixel-level reconstruction and localized tumor fidelity.Source: §1 · p. 2
- For sparse PET evaluation, the paper proposes background-filtered and mask-restricted metrics as alternatives to relying on standard full-image metrics alone.Source: §1 · p. 2
2 Method
The method performs conditional CT-to-PET synthesis in latent space, adapts a pretrained natural-image model to PET, and evaluates both full-image and tumor-region fidelity.
2.1 Conditional CT-to-PET Synthesis
- The conditional synthesis process adds Gaussian noise to the PET latent and concatenates it with the clean CT latent for the U-Net input.The framework uses a low-dimensional latent space to reduce the computational cost of diffusion while preserving perceptual quality.Source: §2.1 · pp. 2–3
2.2 Domain Adaptation and Finetuning Strategy
- Stable Diffusion v1.4 pretrained on natural images initializes the framework; its first U-Net convolution is expanded for CT conditioning, and the U-Net is supervised-finetuned.The pretrained VAE reconstructs PCLT20K sufficiently accurately, so it remains frozen while the denoising U-Net is finetuned.Source: §2.2 · p. 3, Table 1 · p. 3
- A signed power transform compresses PET’s dynamic range and spreads near-zero values before VAE encoding; its inverse restores the original intensity scale at inference.Source: §2.2 · pp. 3–4
- The study compares fully unfrozen U-Net finetuning with a variant that freezes selected blocks while keeping attention, normalization, and time-embedding layers trainable.The variants assess the trade-off between adaptation flexibility and preservation of pretrained priors.Source: §2.2 · p. 4
2.3 Evaluation Metrics
- Evaluation combines full-reference metrics with tumor-mask metrics; mask-restricted SSIM and LPIPS use a tight bounding box to retain patch context.Mask-restricted metrics do not penalize uptake predicted outside the tumor region, so the authors interpret them alongside full-image metrics.Source: §2.3 · p. 4
3 Experiments
Experiments evaluate transfer learning and signed power preprocessing for CT-to-PET synthesis on PCLT20K, then examine metric caveats and robustness across model variants and corrupted inputs.
3.1 Dataset
- PCLT20K provides 21,930 paired 512 × 512 CT/PET slices and tumor masks from 605 patients; the CPDM paired dataset was unavailable to the authors.The predefined split is 80%/20%, with an 80-sample validation set.Source: §3.1 · pp. 4–5
3.2 Training Details
- All models were finetuned from Stable Diffusion v1.4, with four variants crossing base or signed-power inputs and frozen or unfrozen U-Net weights.The power variants use p = 0.25; all models use L1 loss. Training uses AdamW, a 1 × 10−4 learning rate, cosine scheduling, EMA, and a 1,000-step DDPM schedule.Source: §3.2 · p. 5
3.3 Conditional CT-to-PET Results
- The power model outperforms both CPDM checkpoints on MAE, SSIM, and PSNR, while CPDM performs better on LPIPS.Masked tumor-region metrics are substantially lower than full-image metrics, and the models perform similarly across the three masked metrics; the large full-image SSIM gap between base and power largely disappears within the mask.Source: Table 2 · p. 5, Table 3 · p. 6, §3.3 · p. 6
3.4 Metric Caveats and Background Filtering
- Full-image metrics can mislead on sparse PET: background filtering largely explains the apparent base-versus-power gap as background noise rather than tumor-region fidelity.The filter zeros pixels below τ = 0.05; the authors recommend background-filtered or mask-restricted metrics alongside full-image metrics.Source: §3.4 · pp. 7–8, Table 4 · p. 7
3.5 Ablations and Extended Experiments
- Across power coefficients, metrics are similar; under increasing CT noise, LPIPS worsens consistently while masked metrics degrade more consistently at high noise.The paired Wilcoxon test found all reported comparisons significant at p < 0.05 except full-image PSNR between base frozen and power. Coefficient variants had SSIM approximately 0.88, PSNR approximately 29, MAE approximately 0.015, and LPIPS between 0.12 and 0.15; the best p = 0.25 power variant remained top overall, though other variants often fell within its standard deviation. OOD tests used 16 samples per noise level; PSNR showed no consistent trend and high standard deviation.Source: Power transform · p. 8, Out-of-distribution experiment · p. 8, Statistical significance · p. 8
4 Conclusion
The framework adapts natural-image priors for CT-to-PET translation with 21,930 paired slices, outperforming CPDM in reconstruction accuracy and localized tumor synthesis. The conclusion also identifies evaluation concerns and directions for testing the approach’s data-efficiency limits.
4 Conclusion
- The approach yields superior reconstruction accuracy and localized tumor synthesis compared with the CPDM baseline.The framework adapts natural-image priors using 21,930 paired slices and incorporates a signed power transform to address PET intensity imbalance.Source: §4 · p. 9
- Background-pixel dominance limits standard metrics for sparse PET, while background-filtered and tumor-mask-restricted metrics provide more clinically meaningful assessments.Source: §4 · p. 9
- Reducing the training set could identify the minimum paired-sample count before transfer learning degrades; image-space losses may better align training with pixel-level clinical fidelity.The proposed future work would also clarify how loss choice interacts with the signed power transform.Source: §4 · p. 9