Source-linked AI summary
ADN: Artifact Disentanglement Network for Unsupervised Metal Artifact Reduction
Haofu Liao, Wei-An Lin, S. Kevin Zhou, Jiebo Luo
TL;DR
Existing supervised MAR methods rely on paired or synthesized data that may not represent clinical artifact formation and can generalize poorly. This paper introduces ADN, which disentangles content and artifacts in latent space and trains from unpaired artifact-affected and artifact-free CT images. ADN matches supervised methods on synthesized data and generalizes better than them on clinical datasets.
Problem
Existing DNN-based MAR methods require paired CT images with and without artifacts, while synthesized artifacts may not accurately represent clinical CT artifact mechanisms.
Method
ADN separately encodes CT content and artifacts in latent spaces and uses specialized decoders and translation losses to learn MAR from unpaired images.
Results
ADN achieves comparable performance to supervised methods on synthesized data and significantly outperforms them on clinical datasets, where supervised methods generalize poorly.
Takeaways & Limitations
Unsupervised learning with artifact disentanglement is feasible for MAR and is more robust than supervised learning for the evaluated clinical CT datasets.
Abstract
from arXiv · showhide
Current deep neural network based approaches to computed tomography (CT) metal artifact reduction (MAR) are supervised methods that rely on synthesized metal artifacts for training. However, as synthesized data may not accurately simulate the underlying physical mechanisms of CT imaging, the supervised methods often generalize poorly to clinical applications. To address this problem, we propose, to the best of our knowledge, the first unsupervised learning approach to MAR. Specifically, we introduce a novel artifact disentanglement network that disentangles the metal artifacts from CT images in the latent space. It supports different forms of generations (artifact reduction, artifact transfer, and self-reconstruction, etc.) with specialized loss functions to obviate the need for supervision with synthesized data. Extensive experiments show that when applied to a synthesized dataset, our method addresses metal artifacts significantly better than the existing unsupervised models designed for natural image-to-image translation problems, and achieves comparable performance to existing supervised models for MAR. When applied to clinical datasets, our method demonstrates better generalization ability over the supervised models. The source code of this paper is publicly available at https://github.com/liaohaofu/adn.
I. INTRODUCTION
Metal artifacts degrade CT images and complicate medical analysis, while existing projection-based and supervised deep-learning approaches require unavailable or unrealistic paired data. The paper addresses this gap with unsupervised artifact disentanglement and reports comparable synthesized-data performance and stronger clinical generalization than supervised methods.
- Motivation: Metal implants create inconsistent X-ray projections that produce severe streaking and shading artifacts, degrading image quality and medical image analysis.The issue arises because metals attenuate X-rays strongly and non-uniformly across the spectrum.
- Motivation: Projection-completion methods can introduce secondary artifacts and are limited when manufacturers withhold projection data and reconstruction algorithms.Large implants make consistency with imaging content and projection geometry difficult to satisfy.
- Research gap: Existing DNN-based MAR methods are supervised and require anatomically identical CT pairs with and without metal artifacts.Their reliance on synthesized or paired data motivates an unsupervised alternative.
- Proposed approach: ADN reformulates MAR as latent-space artifact disentanglement, separating artifact and anatomical-content components so artifact removal follows by decoding without the artifact component.Grouping artifact-affected and artifact-free images provides the inductive bias for learning without paired images.
- Proposed approach: ADN uses specialized encoders and decoders to support artifact reduction, artifact synthesis, self-reconstruction, and related translations from unpaired inputs.The method exploits relationships among these translations for unsupervised learning.
- Findings: On synthesized data, ADN achieves comparable performance to supervised methods, while on clinical datasets it consistently outperforms supervised methods affected by domain shift.The paper positions clinical generalization as the central advantage of the unsupervised approach.
III. METHODOLOGY
The paper formulates MAR without paired CT images and learns it from separate artifact-affected and artifact-free image domains. ADN disentangles content and artifacts in latent space, enabling artifact removal through artifact-free decoding.
- Problem formulation: The method assumes no paired dataset is available and learns the MAR mapping from unpaired artifact-affected and artifact-free CT images.Artifact-affected images belong to Ia, artifact-free images to I, and paired correspondences are unavailable.
- Artifact disentanglement: ADN encodes artifact-affected images into separate content and artifact spaces, while artifact-free images are encoded into the content space.If disentanglement succeeds, the content code preserves anatomy without artifact information.
- Artifact disentanglement: Decoding an artifact-affected image's content code without its artifact code produces an artifact-free counterpart, while combining an artifact-free content code with an artifact code synthesizes artifacts.These latent manipulations provide the basis for unsupervised training and artifact translation.
A. Encoders and Decoders
ADN separates content and artifact encoding and uses domain-specific decoders to reconstruct, remove, or add artifacts. Its decoder combinations define the image translations used by the network.
- Architecture: ADN contains an artifact-free encoder-decoder pair and an artifact-affected encoder-decoder pair with separate content and artifact encoders.The artifact-affected decoder receives both content and artifact codes, whereas the artifact-free decoder receives only content codes.
- Encoders: The content encoders map artifact-affected and artifact-free images into the shared content space, while the artifact encoder maps artifacts into the artifact space.This division implements the network's artifact-disentanglement design.
- Decoders: The artifact-affected decoder combines content and artifact codes to reconstruct an affected image or add artifacts to an artifact-free image.Decoding cx with a reconstructs xa, while decoding cy with a produces an artifact-affected image.
- Decoders: The artifact-free decoder maps a content code to an artifact-free image, removing artifacts from an affected input or reconstructing an artifact-free input.Decoding cx removes artifacts from xa, whereas decoding cy reconstructs y.
- Image translations: An artifact-transferred image combines content from an artifact-free image with artifacts from an affected image, enabling synthesized artifact transfer.The resulting image can be re-encoded to support the network's self-reduction mechanism.
B. Learning
ADN learns metal artifact reduction without paired CT images by disentangling artifact and content components in latent space. Specialized translation and loss designs provide unsupervised learning signals for artifact reduction, synthesis, reconstruction, and self-reduction.
- Artifact disentanglement: ADN learns an MAR model by encoding artifact-affected images into content and artifact components, then decoding the content component into artifact-free images.The resulting composition f = GI ◦ EcIa serves as the MAR model, while other encoders and decoders provide additional learning signals.
- Learning objectives: ADN uses adversarial, reconstruction, artifact consistency, and self-reduction losses to encourage the intended translated outputs.The objective combines two adversarial losses with Lart, Lrec, and Lself using weighted terms controlled by hyperparameters.
- Image translations: Different encoder-decoder combinations support artifact reduction, artifact synthesis, self-reconstruction, and other image translations from unpaired inputs.These relationships are exploited to train ADN without paired data.
- Reconstruction loss: Reconstruction losses require the artifact and content encoder-decoder pairs to act as autoencoders, preserving information when encoding and decoding the same image.The reconstructed outputs correspond to the original artifact-affected and artifact-free inputs, with L1 loss used to encourage sharper outputs.
- Artifact consistency loss: Artifact consistency matches the artifact differences before and after translation, encouraging anatomical closeness without forcing artifact-containing inputs and outputs to match exactly.Directly minimizing the artifact-reduced image against the artifact-affected input would reintroduce artifacts and conflict with adversarial learning.
- Self-reduction loss: Self-reduction adds artifacts to an artifact-free image and then removes them, producing a paired synthetic relation for regression-based regularization.The generated artifact-affected image and original artifact-free image provide the pair used by the self-reduction loss.
C. Network Architectures
The network uses CNN-based encoders, decoders, and discriminators assembled from reusable blocks. Its key architectural novelty is artifact pyramid decoding, which merges artifact and content features across scales during artifact-affected image decoding.
- Building components: The encoders, decoders, and discriminators are formulated as CNNs built from stacked architectural blocks, with some structures inspired by image-translation methods.The blocks provide the basic components used throughout the network.
- Building blocks: Residual, downsampling, and upsampling blocks form the core encoder and decoder components.Downsampling uses strided convolution for dimensionality reduction, while residual connections preserve low-level features.
- Artifact pyramid decoding: Artifact pyramid decoding merges artifact and content codes at multiple feature scales while decoding artifact-affected images.The artifact encoder produces a feature pyramid, and the artifact-affected decoder combines features through residual, merge, upsampling, and final blocks.
- Artifact pyramid decoding: Merging blocks concatenate content and artifact feature maps along channels before locally combining them with a 1 × 1 convolution.This design is intended to recover artifact details using high-definition features at relatively low cost.
A. Baselines
The evaluation compares ADN with conventional, supervised, and unsupervised MAR or image-translation baselines. Implementations use official code when available and specified alternatives otherwise.
- Compared methods: The comparison includes nine methods spanning conventional MAR, supervised MAR, and unsupervised image-to-image translation.The conventional methods are LI and NMAR; supervised methods are CNNMAR, UNet, and cGANMAR; unsupervised methods are CycleGAN, DIP, MUNIT, and DRIT.
- Conventional baselines: LI and NMAR represent widely used conventional projection-based MAR approaches.They are included as the conventional baselines in the evaluation.
- Learning-based baselines: CNNMAR, UNet, and cGANMAR serve as supervised comparison methods, while CycleGAN, DIP, MUNIT, and DRIT represent unsupervised translation baselines.The latter group is described as state-of-the-art for unsupervised image-to-image translation problems.
- Implementations: Official released code is used when possible; LI and NMAR use CNNMAR implementations, and cGANMAR is trained with Pix2Pix code because their backends are identical.UNet uses a publicly available PyTorch implementation.
- Architecture reporting: Table I specifies the architecture configurations using channel, kernel, stride, and padding settings.These labels describe the convolution-layer configurations in the building blocks.
B. Datasets
The study evaluates ADN on one synthesized CT dataset and two clinical datasets, using paired and unpaired data arrangements to test supervised and unsupervised settings.
- SYN: SYN contains 4,118 synthesized artifact-free CT images, with 3,918 pairs used for training and validation and 200 pairs for testing.Artifacts are synthesized by simulating beam hardening and Poisson noise during polychromatic projection generation.
- CL1: CL1 is a Spineweb dataset split into artifact-affected and artifact-free CT groups, with 6,270 and 21,190 images, respectively.Metal regions are identified using HU thresholds, and 200 artifact-affected images are withheld for testing.
- CL2: CL2 contains 2,560 noisy clinical CBCT images, mostly with metal artifacts whose implants are generally outside the imaging field of view.The dataset is used to evaluate a challenging cross-modality setting; 200 images are withheld for testing.
- Training setup: For SYN, the unsupervised setting withholds paired counterparts by dividing training pairs into artifact-affected-only and artifact-free-only groups.This removes paired supervision while retaining separate artifact-affected and artifact-free inputs.
- Training setup: For CL1 and CL2, unsupervised training samples one artifact-affected image and one artifact-free image as inputs, while supervised CL2 models are trained on synthesized CL1 data.Raw X-ray projections are unavailable, so projection-based baselines use forward-projected data.
D. Performance on synthesized data
On the synthesized SYN dataset, ADN substantially outperforms other unsupervised image-translation methods and performs comparably to conventional and supervised MAR methods.
- Evaluation: PSNR and SSIM are used for quantitative SYN evaluation, with higher values indicating better performance.Table II and Fig. 6 provide the quantitative and qualitative comparisons, respectively.
- Unsupervised baselines: ADN performs significantly better than the other unsupervised methods on SYN.CycleGAN and DRIT preserve more content but also retain artifacts, while MUNIT prioritizes diverse outputs and DIP is not designed for structured metal artifacts.
- Conventional and supervised baselines: ADN performs on par with conventional and supervised methods on SYN.UNet and cGANMAR slightly outperform ADN on PSNR and SSIM, while ADN outperforms projection-interpolation approaches quantitatively and qualitatively.
E. Performance on clinical data
On clinical data, ADN generalizes better than supervised methods, including under the more challenging CL2 cross-modality setting; the ablation study links improvements to its loss components.
- Clinical evaluation: On CL1, supervised UNet and cGANMAR do not generalize well, whereas projection-based methods are more robust but introduce secondary artifacts.For CL2, all supervised methods fail because they were trained only on CT images and metal implants are outside the imaging field of view.
- Ablation study: M1 uses only adversarial loss, M2 adds reconstruction loss, M3 adds artifact consistency loss, and M4 uses all losses as ADN.The variants are compared on SYN using quantitative and qualitative evaluations.
- Ablation study: ADN variants progressively improve artifact suppression and anatomical preservation as reconstruction, artifact consistency, and self-reduction mechanisms are added.M2 preserves more anatomical detail than M1, M3 suppresses most metal artifacts, and M4 further improves the outputs.
G. Artifact Synthesis
ADN supports unsupervised artifact synthesis by transferring realistic metal artifacts into artifact-free CT images. The network also supports artifact reduction and broader post-processing applications without requiring paired training images.
- Artifact synthesis: ADN synthesizes metal artifacts by transferring artifacts from clinical images into artifact-free images, producing realistic results except for implant positioning.The synthesized artifacts merge naturally into artifact-free images and are difficult to distinguish from real artifacts.
- Artifact synthesis: Adversarial loss encourages synthesized outputs to resemble real artifact images, while artifact consistency preserves artifacts and suppresses transferred content.These two designs support unsupervised clinical artifact synthesis.
- Artifact reduction: ADN can reduce metal artifacts directly from CT images when raw projection data and reconstruction algorithms are unavailable.Manufacturers often withhold these data and algorithms, limiting projection-based methods.
- Artifact reduction: ADN may extend to deblurring, destreaking, and denoising because its formulation does not assume a specific artifact type.The authors report smoother outputs for noisy clinical inputs.
- Artifact synthesis: Realistic artifact synthesis could augment medical-image datasets with metal-affected examples for downstream tasks such as CT segmentation.The authors suggest this may improve segmentation-model performance when few metal-affected images are available.
- Overall findings: Across synthesized and clinical datasets, unsupervised ADN achieves comparable performance to supervised methods and is more robust than transferring synthesized-data knowledge to clinical images.The conclusion presents direct unsupervised learning on clinical CT images as the more feasible approach.