Source-linked AI summary
Zero-shot Medical Image Translation via Frequency-Guided Diffusion Models
Yunxiang Li, Hua-Chieh Shao, Xiao Liang, Liyuan Chen, Ruiqi Li, Steve Jiang, Jing Wang, You Zhang
TL;DR
Diffusion-based medical image translation can lose source-image structural detail, while paired images with matching anatomy are difficult to obtain. FGDM guides diffusion with low- and high-frequency information, allowing target-domain-only training for zero-shot translation. The paper reports that FGDM outperforms state-of-the-art methods across its evaluated metrics, while noting that its frequency-similarity premise limits the translation tasks to similar modalities or acquisition settings.
Problem
Diffusion forward processes lose structural details that reverse diffusion cannot fully recover, and paired source-target images with matching anatomy are difficult to obtain.
Source: §I · p. 2, A. Diffusion Models · p. 4Method
FGDM uses low- and high-frequency source information to guide diffusion generation of intermediate frequencies and trains on target-domain images alone.
Source: §I · p. 2Results
FGDM is reported to outperform state-of-the-art models in all metrics, and its head-and-neck-trained model translated lung images without fine-tuning.
Source: §VI · p. 13Takeaways & Limitations
The model supports zero-shot translation without source-domain training when source and target images share similar low- and high-frequency information.
Source: §I · p. 2FGDM assumes source-target differences are mainly intermediate-frequency, limiting current applicability to similar modalities or same-modality acquisition settings; it may not perform well for MR-to-CT translation.
Source: §V · p. 12
Abstract
from arXiv · showhide
Recently, the diffusion model has emerged as a superior generative model that can produce high quality and realistic images. However, for medical image translation, the existing diffusion models are deficient in accurately retaining structural information since the structure details of source domain images are lost during the forward diffusion process and cannot be fully recovered through learned reverse diffusion, while the integrity of anatomical structures is extremely important in medical images. For instance, errors in image translation may distort, shift, or even remove structures and tumors, leading to incorrect diagnosis and inadequate treatments. Training and conditioning diffusion models using paired source and target images with matching anatomy can help. However, such paired data are very difficult and costly to obtain, and may also reduce the robustness of the developed model to out-of-distribution testing data. We propose a frequency-guided diffusion model (FGDM) that employs frequency-domain filters to guide the diffusion model for structure-preserving image translation. Based on its design, FGDM allows zero-shot learning, as it can be trained solely on the data from the target domain, and used directly for source-to-target domain translation without any exposure to the source-domain data during training. We evaluated it on three cone-beam CT (CBCT)-to-CT translation tasks for different anatomical sites, and a cross-institutional MR imaging translation task. FGDM outperformed the state-of-the-art methods (GAN-based, VAE-based, and diffusion-based) in metrics of Frechet Inception Distance (FID), Peak Signal-to-Noise Ratio (PSNR), and Structural Similarity Index Measure (SSIM), showing its significant advantages in zero-shot medical image translation.
I. INTRODUCTION
The paper addresses structural detail loss in diffusion-based medical image translation and the difficulty of obtaining paired, anatomically matched images. FGDM uses low- and high-frequency information to guide generation of intermediate frequencies, enabling target-domain-only training and zero-shot translation.
I. INTRODUCTION
- Diffusion forward processes lose structural details that reverse diffusion cannot fully recover, while paired source-target images with matching anatomy are scarce and may weaken robustness to out-of-distribution inputs.Such errors can distort, shift, or remove anatomical structures and tumors.Source: §I · pp. 1–2
- CBCT and CT differences are concentrated mainly at intermediate frequencies, motivating FGDM to use low- and high-frequency information to guide generation of the intermediate frequencies.The low-pass filter captures intensity and overall semantic information, while the high-pass filter captures anatomical details.Source: §I · p. 2, Figure 2 · p. 2
- FGDM trains only on target-domain data and can translate source images without fine-tuning when the domains share similar low- and high-frequency information.The model is presented as a zero-shot medical image translation framework at the anatomical level.Source: §I · p. 2
- Its high- and low-pass filter thresholds can be adjusted at test time to accommodate varying domain-invariant frequency regions across translation tasks.Source: §I · p. 2
- FGDM achieves the best performance in CBCT-to-CT translation, surpassing GAN-, VAE-, and diffusion-based state-of-the-art benchmark models.Source: §I · p. 2
II. RELATED WORKS
Medical image translation maps between domains while aiming to preserve anatomy, and prior work includes GAN-, VAE-, and diffusion-based approaches. FGDM instead uses filtered source-image frequencies as conditions and generates intermediate-frequency information through reverse diffusion.
A. Medical Image Translation
- Medical image translation maps images across domains while preserving anatomy, with applications including diagnosis, image harmonization, image-quality improvement, and treatment planning.Prior approaches include CycleGAN, GCGAN, RegGAN, UNIT, and MUNIT, with different mapping, consistency, registration, and latent-space strategies.Source: A. Medical Image Translation · pp. 3–4
B. Diffusion Models for Image Translation
- Diffusion methods offer realistic image generation, but medical translation can trade anatomical faithfulness for image quality; FGDM is presented as a framework to improve faithfulness while retaining realism.Unlike SynDiff, FGDM does not rely on CycleGAN and trains using target-domain images alone.Source: B. Diffusion Models for Image… · p. 4
III. METHODS
- FGDM filters CBCT images into low- and high-frequency diffusion conditions, then generates intermediate-frequency information through reverse diffusion to produce CT images.Source: Figure 3 · p. 3, §III · p. 4
A. Diffusion Models
Forward diffusion adds Gaussian noise and progressively loses structural detail. The paper views this process as a low-pass filter, with higher-frequency information affected earlier.
A. Diffusion Models
- Forward diffusion adds Gaussian noise over T steps, and the resulting structural detail loss cannot be recovered by conventional diffusion models.The reverse process is defined as denoising, but the supplied passage identifies the structural detail loss as unrecoverable by conventional models.Source: A. Diffusion Models · p. 4
B. Low Pass Filter
- FGDM is trained on CT to generate intermediate-frequency information from low- and high-frequency CT conditions, then uses CBCT frequency information as conditions for zero-shot translation.Training uses noise steps through T; zero-shot translation uses ˜T, which can be chosen according to source–target disparities.Source: Figure 4 · p. 5
- White Gaussian noise has equal power across frequencies, supporting the paper’s frequency-domain account of diffusion.The text derives this from the noise autocorrelation and its Dirac delta form.Source: B. Low Pass Filter · p. 5
- Higher-frequency signals are affected earlier as diffusion steps increase, and a threshold-dependent step ˜T marks loss of non-low-frequency information.At a given frequency, SNR depends on original signal power and decreases with increasing frequency.Source: B. Low Pass Filter · pp. 5–6
C. High Pass Filter
The high-pass filter uses Sobel gradients to extract high-frequency image information.
C. High Pass Filter
- The Sobel operator serves as a discrete differential high-pass filter that computes image-intensity gradients to obtain high-frequency information.Horizontal and vertical Sobel kernels are applied separately to form gradient images.Source: C. High Pass Filter · p. 6
- Convolving with horizontal and vertical Sobel kernels produces filtered images Gx and Gy, whose approximations are combined into gradient magnitude.The passage describes the horizontal and vertical convolutions as the first step in obtaining the high-frequency information.Source: C. High Pass Filter · p. 6
- A threshold η is used to obtain the high-frequency information H.Source: C. High Pass Filter · p. 6
D. Model Training
Model training uses a denoised diffusion GAN backbone and an adversarial objective to match conditional denoising transitions.
D. Model Training
- The denoised diffusion GAN backbone samples in fewer than 8 steps, versus over 1000 for the original DDPM, with performance close to DDPM and inference hundreds of times faster.The model uses softened reverse Kullback–Leibler divergence (SRKL), described as a special f-divergence instance.Source: D. Model Training · p. 6
- Training matches the conditional denoising model pθ(st−1|st, Hη) to q(st−1|st) using an adversarial loss that minimizes divergence Dadv per denoising step.Source: D. Model Training · p. 6
- The SRKL discriminator is time-dependent and is trained through its specified equation.Source: D. Model Training · p. 6
E. Zero-Shot Image Translation
The zero-shot formulation assumes source and target images have approximately matching low- and high-frequency information, with their main differences in intermediate frequencies. The algorithm uses the source image, low- and high-pass filtering, and a pre-trained denoiser to produce a translated image.
E. Zero-Shot Image Translation
- Zero-shot translation is premised on source and target domains sharing approximately the same low- and high-frequency information while differing mainly at intermediate frequencies.The model is trained only on target-domain data and must respond to source-domain samples not observed during training.Source: E. Zero-Shot Image Translation · p. 6
- The translation task is specified by the forward-process distribution in Eq. 29 and the reverse process in Eq. 30.The supplied passages identify these equations as part of the translation description but do not provide a complete prose account of each operation.Source: E. Zero-Shot Image Translation · p. 6
- Algorithm 1 takes a source image, pre-trained denoiser, low- and high-pass filters, constant variance, and a decreasing sequence as inputs, and outputs a high-faithfulness translated image.The listed inputs include the diffusion step count and the two frequency filters.Source: E. Zero-Shot Image Translation · p. 6
- The algorithm initializes the low-pass-filtered source as the current state and extracts a high-frequency component from the source before iterating from the selected step down to 1.The initialization also samples noise with variance σ^2 to construct the low-pass-filtered source.Source: E. Zero-Shot Image Translation · p. 6
A. Datasets and pre-processing
The evaluation uses CBCT-to-CT datasets from head and neck, lung, and OAR anatomies, plus a cross-center head MR T1 dataset. The CBCT reference CTs are deformably registered, while the MR data come from separate source and target repositories.
1) Head and Neck Dataset:
- The head and neck CBCT-to-CT dataset contains 70 training, 9 validation, and 20 test cases, with deformably registered CT images used as testing references.2D slices were resized to 192 × 192 after HU clipping and normalization; the CT and CBCT voxel spacings differed.Source: 1) Head and Neck Dataset · p. 7
2) Lung Dataset:
- The lung dataset contains CBCT and CT images from 18 patients and tests the head-and-neck-trained model directly on another anatomical site.Its CT images were deformably registered to corresponding CBCT images for evaluation.Source: 2) Lung Dataset · p. 7
3) Organs at Risk (OARs) Dataset:
- The OARs dataset contains paired CBCTs and CTs from 24 patients, including esophagus, spinal cord, heart, and lung anatomies.It is included to assess robustness and versatility under greater data-distribution variation.Source: 3) Organs at Risk (OARs) Dataset · p. 7
A. Datasets and pre-processing
- The cross-center Head MR T1 dataset includes images from NFBS1 and the MICCAI 2020 challenge’s ABCs2 repository, reflecting real-world image-translation needs; NFBS’s 125-participant collection served as source-domain data for FGDM training.The ABCs dataset comprised 60 patients diagnosed with glioblastoma.Source: 4) Head MR T1 Dataset · p. 7
4) Head MR T1 Dataset:
The paper describes preprocessing for the head MR T1 data, implementation choices for frequency guidance, and the metrics and method classes used for comparisons. The supplied comparison visuals and tables identify their datasets and scope but do not provide result values.
4) Head MR T1 Dataset:
- Head MR T1 slices were intensity-clipped between the 0th and 99th percentiles, normalized to [0,1], and resized to 256 × 256.The clipping is described as removing outliers and standardizing image dynamic range.Source: 4) Head MR T1 Dataset · p. 7
B. Implementation Details
- Testing uses η = 10 and T̃ = 4 after parameter study, while η is randomized from 1–25 during training to vary the extracted high-frequency conditioning signal.The authors state that test-stage parameter adjustment requires no model retraining.Source: B. Implementation Details · p. 7
C. Comparison Metrics and Methods
- The comparison evaluates realism with FID and faithfulness with PSNR and SSIM against both source CBCT and registered CT references, with frequency-domain MSE also calculated.The paper notes that registered CT references are not exact ground truth because of non-deformation-induced changes and registration errors.Source: C. Comparison Metrics and Methods · pp. 7–8
- The benchmark includes GAN-based, VAE-based, and diffusion-based methods; the listed tables cover head-and-neck in-distribution, lung and OAR out-of-distribution, and head T1 MR comparisons.The visual comparisons shown here cover head-and-neck in-distribution CBCT-to-CT and lung and OAR out-of-distribution CBCT-to-CT translation.Source: C. Comparison Metrics and Methods · pp. 8–9, Table I · p. 8, Table II · p. 9
D. Performance on the Head and Neck Dataset
On the head-and-neck benchmark, FGDM is reported to outperform the other methods across metrics while combining image realism with anatomical detail preservation. Visual comparisons describe remaining small structural mismatches despite FGDM’s stronger detail retention.
D. Performance on the Head and Neck Dataset
- CycleGAN and GCGAN score comparatively well on SSIM but poorly on FID, while RegGAN has slightly better FID than CycleGAN but worse SSIM and PSNR.UNIT outperforms GAN methods on FID but performs poorly on SSIM.Source: D. Performance on the Head and Neck… · p. 9
- FGDM outperforms the other methods in all metrics and in frequency-domain MSE, while retaining anatomical details through frequency-domain guidance.The authors report that it corrects intermediate-frequency mismatches and, to some extent, low- and high-frequency mismatches, including some high-frequency streak artifacts.Source: D. Performance on the Head and Neck… · p. 10
- SDEdit and EGSDE look acceptable at coarse scale, but zoomed views show structural loss and distortion around the nasal cavity and teeth.The passage also reports that VAE-based results do not preserve anatomical detail well.Source: D. Performance on the Head and Neck… · pp. 10–11
- FGDM preserves structural details closest to the source among the compared methods, although some small structure mismatches remain.The authors identify those residual mismatches as warranting further development and optimization.Source: D. Performance on the Head and Neck… · p. 11
E. Performance on the Lung Dataset
FGDM retained anatomical detail more robustly than benchmark methods under distribution shifts in CBCT-to-CT and cross-center MR translation. Its frequency-conditioning ablations and parameter tests also show that the guidance choices affect translation performance and source-structure retention.
E. Performance on the Lung Dataset
- FGDM maintained anatomical accuracy on lung images shifted from the head-and-neck training distribution, while most benchmark methods lost structural detail or hallucinated structures.The evaluation applied models trained on head-and-neck data directly to lung images without fine-tuning.Source: E. Performance on the Lung Dataset · p. 11
F. Performance on the Organs at Risk (OARs) Dataset
- FGDM was more robust across all metrics on the OARs dataset despite its substantial distribution shift from head-and-neck images.The OARs images often lacked the dark air background typical of the training images.Source: F. Performance on the Organs at Risk… · p. 11
G. Performance on the Head MR T1 Dataset
- FGDM consistently outperformed the other methods on FID, SSIM, and PSNR in translation between the NFBS and ABCs head-MR datasets.Because the datasets were unpaired, FID assessed realism while PSNR and SSIM compared translated images with source images for faithfulness.Source: G. Performance on the Head MR T1 Dataset · p. 11
1) Effectiveness of high-frequency and low-frequency conditioning:
- Removing either high-frequency or low-frequency conditioning significantly reduced performance, with SSIM falling substantially when high-frequency guidance was removed.The ablation tested the two frequency-guidance components on the head-and-neck dataset.Source: 1) Effectiveness of high-frequency and… · p. 11
2) Impact of the high-frequency ratio parameter
- η = 10 yielded optimal performance; larger values filtered out too much high-frequency guidance, while smaller values retained more artifacts.The high-pass filter parameter can be adjusted at test time for different translation tasks.Source: 2) Impact of the high-frequency ratio… · p. 11
- Reducing the diffusion-step count retained more low-frequency information and increased PSNR and SSIM relative to source CBCT images.The passage notes that fewer steps bring the filtered image closer to the original CBCT domain.Source: 3) Impact of the low-frequency ratio… · p. 11
3) Impact of the low-frequency ratio parameter
FGDM’s conclusion reports stronger metrics than other models and zero-shot application to shifted lung and OARs data. The low-frequency setting trades source-image similarity against target-domain matching, while the method’s scope and design remain bounded by stated assumptions and choices.
3) Impact of the low-frequency ratio parameter
- A low-frequency step count of 4 balances structural similarity to CBCT with intensity similarity to CT; fewer steps preserve more source detail but also streak artifacts.At 5 steps, image intensity deviated from the reference CT; retention factors can be adjusted during inference.Source: 3) Impact of the low-frequency ratio… · p. 12
V. LIMITATION
- FGDM is intended for domains with similar low- and high-frequency information, such as CBCT-to-CT or same-modality acquisition shifts; MR-to-CT may not perform well.The stated premise is that domain differences are concentrated mainly at intermediate frequencies.Source: §V · p. 12
- The study prioritizes a speed–quality balance with a diffusion-GAN backbone and leaves model-structure optimization and task-specific loss design for future work.The authors note that original-DDPM frequency guidance could be evaluated when inference speed is less critical.Source: §V · p. 13
VI. CONCLUSION
- FGDM outperformed other models across metrics and translated lung and OARs images without fine-tuning after training on head-and-neck data.The conclusion presents this as evidence of robustness to zero-shot translation and out-of-distribution data.Source: §VI · p. 13