Source-linked AI summary
TomoGAN: Low-Dose Synchrotron X-Ray Tomography with Generative Adversarial Networks
Zhengchun Liu, Tekin Bicer, Rajkumar Kettimuthu, Doga Gursoy, Francesco De Carlo, Ian Foster
TL;DR
Low-dose or limited-projection tomography produces noisy reconstructions, compromising image quality. TomoGAN adapts a GAN-based approach to denoise high-resolution synchrotron images, and the authors report visible improvement across similar samples and facilities.
Problem
Short exposures and limited projection counts produce noisy reconstructions that significantly affect image quality.
Source: §1 · pp. 1–2Method
TomoGAN adapts a GAN-based method and U-Net architecture to improve high-resolution synchrotron tomography images.
Source: §1 · p. 3Results
TomoGAN visibly improves noisy reconstructions and works on similar samples from other facilities with different noise characteristics after training on one sample.
Source: §4 · pp. 16–17Takeaways & Limitations
The authors see potential for studying dynamic features through good-quality reconstructions from faster experiments using fewer projections or shorter exposures.
Source: §4 · p. 17The best adjacent-slice depth depends on dataset characteristics such as feature resolution, so d = 3 may not suit datasets whose feature sizes change slowly across slices.
Source: §3.1 · p. 8
Abstract
from arXiv · showhide
Synchrotron-based x-ray tomography is a noninvasive imaging technique that allows for reconstructing the internal structure of materials at high spatial resolutions from tens of micrometers to a few nanometers. In order to resolve sample features at smaller length scales, however, a higher radiation dose is required. Therefore, the limitation on the achievable resolution is set primarily by noise at these length scales. We present \TOMOGAN{}, a denoising technique based on generative adversarial networks, for improving the quality of reconstructed images for low-dose imaging conditions. We evaluate our approach in two photon-budget-limited experimental conditions: (1) sufficient number of low-dose projections (based on Nyquist sampling), and (2) insufficient or limited number of high-dose projections. In both cases the angular sampling is assumed to be isotropic, and the photon budget throughout the experiment is fixed based on the maximum allowable radiation dose on the sample. Evaluation with both simulated and experimental datasets shows that our approach can significantly reduce noise in reconstructed images, improving the structural similarity score of simulation and experimental data from 0.18 to 0.9 and from 0.18 to 0.41, respectively. Furthermore, the quality of the reconstructed images with filtered back projection followed by our denoising approach exceeds that of reconstructions with the simultaneous iterative reconstruction technique, showing the computational superiority of our approach.
1 Introduction
Low-dose and undersampled synchrotron tomography produces noisy reconstructions, while prior denoising approaches have not been thoroughly evaluated across experimental datasets. TomoGAN adapts a GAN-based approach for high-resolution synchrotron images and reports improved reconstructions across simulated and experimental settings.
1 Introduction
- Short exposures can avoid sample damage or capture dynamic phenomena, while fewer projections also yield noisy reconstructions that limit image quality.The target is to map noisy reconstructions caused by fewer projections, lower resolutions, or shorter imaging times toward an approximation of the ideal image.Source: §1 · pp. 1–2
- Prior synchrotron deep-learning studies addressed projection denoising or reconstruction denoising, but experimental evaluation across datasets was limited for one reconstruction method.The cited projection-denoising work reported a 10-fold increase in signal-to-noise ratio; the reconstruction-denoising work was evaluated primarily on simulations.Source: §1 · p. 2
- TomoGAN produces greatly improved low-dose reconstructions with visible features in simulated foam and experimental shale images.The examples use 64 of 1,024 projections; the method is also reported to work across experimental datasets from different instruments.Source: Figure 1 · p. 2, Figure 2 · p. 3, §1 · p. 3
- Medical-image methods may introduce nonexistent features in high-resolution synchrotron images because finely detailed structures contain high-frequency content.The passage contrasts high-frequency synchrotron features with the low-frequency content that medical-image methods are typically tailored to recognize.Source: §1 · p. 3
- TomoGAN adapts a U-Net architecture for high-resolution synchrotron tomography and is reported to work with limited training data and varied experimental datasets.The authors evaluate experimental shale datasets from different facilities and imaging conditions, training on one dataset and testing on others.Source: §1 · p. 3
2 Methods
TomoGAN uses a conditional GAN to enhance noisy tomographic reconstructions, with a generator trained against a discriminator and additional image losses. The method is evaluated using simulated and experimental datasets under reduced-projection and shorter-exposure conditions.
2.1 Model architecture
- TomoGAN’s generator maps noisy reconstructions to enhanced images, while the discriminator provides feedback during training.After training, only the generator is used to enhance reconstructions.Source: §2.1 · p. 4, Figure 4 · p. 6
2.1.1 Generator
- The generator adapts U-Net with three down- and up-sampling layers, zero-padded convolutions, and a stack of adjacent images as input.Copy operations carry high-resolution features from the down-sampling path into up-sampling layers.Source: §2.1.1 · p. 4, Figure 3 · p. 5
2.1.2 Discriminator
- The discriminator uses six 2D 3×3 convolutional layers and two fully connected layers, with Wasserstein distance used for generator-loss calculation.The discriminator’s convolutional layers use leaky ReLU activations.Source: §2.1.2 · p. 4
2.2.1 Discriminator loss
- The discriminator’s loss adds a gradient penalty to the Wasserstein critical loss to improve training stability.The reported penalty weight is λD = 10, balancing the Wasserstein distance and gradient penalty.Source: §2.2.1 · p. 5
2.2.2 Generator loss
- The generator minimizes a weighted combination of adversarial, VGG perceptual, and pixelwise MSE losses; the MSE term enforces low-frequency structural correctness.The perceptual loss compares features extracted by the first 16 layers of an ImageNet-pretrained VGG-19 network.Source: §2.2.2 · pp. 5–6
2.2 Model training
- Training randomly selects corresponding low-dose and normal-dose reconstruction slices, and the trained generator alone advances reconstructions.To reduce the chance of mode collapse, the generator is trained once every four discriminator training steps.Source: Figure 4 · p. 6, §2.2.2 · p. 6
2.3.1 Simulated datasets
- Simulated evaluation trains on one foam-feature dataset and tests on another, using 1024 projections and simulated reduced-projection or Poisson-noise conditions.The two datasets use different random seeds; reduced-projection tests retain 1/2, 1/4, 1/8, or 1/16 of the projections.Source: §2.3.1 · p. 7
2.3.2 Experimental datasets
- Experimental evaluation uses four shale datasets from two samples imaged at APS and SLS, training on DSB1 APS and testing on the other three datasets.The experimental tests also simulate limited projection counts and shorter exposures with Poisson noise.Source: §2.3 · p. 6, §2.3.2 · p. 7
3 Experimental Results
The experiments assess TomoGAN configurations and compare their image quality and computational requirements. For adjacent-slice inputs, depth d = 3 gives the best quality among the tested depths, though the best depth depends on dataset characteristics.
3 Experimental Results
- The results section evaluates configurations and datasets, then compares image quality and computational requirements across reconstruction approaches.Source: §3 · p. 7
3.1 Effectiveness of using adjacent slices in image enhancement
- d = 3 gives the best image quality among tested depths, especially where feature edges are not sharp.The comparison uses models trained with d = 1, 3, 5, and 7; Figures 5 and 6 show image and SSIM/PSNR comparisons.Source: §3.1 · p. 7, Figure 5 · p. 8, Figure 6 · p. 8
- Adjacent slices tend to contain highly correlated features, and stacking them acts like a multiple linear regressor for noise reduction.The slices from i −⌊d/2⌋ to i +⌊d/2⌋ are inputs for enhancing slice i.Source: §3.1 · p. 7
- The best depth depends on dataset characteristics such as feature resolution, so d = 3 may not be best when feature sizes change slowly across slices.Source: §3.1 · p. 8
3.2 Importance of the various loss terms
The loss-term experiments examine whether adversarial and perceptual losses improve reconstructions. Both help avoid artifacts and each improves SSIM and PSNR when used alone; using them together is only slightly better than adversarial loss alone.
3.2 Importance of the various loss terms
- Adversarial and perceptual losses help avoid artifacts in which small features merge or become difficult to see.The authors describe these artifacts as intolerable in practice.Source: §3.2 · p. 8
- Each of the adversarial and perceptual loss terms provides considerable SSIM and PSNR improvement when used in isolation.Source: §3.2 · p. 8
- Using adversarial and perceptual losses together is only slightly better than using adversarial loss alone.Source: §3.2 · p. 8
3.3 Simulated datasets
Simulated experiments evaluate TomoGAN under fewer-projection and shorter-exposure conditions. The enhanced reconstructions retain high quality with fewer projections and reach quality comparable to reconstructions based on ground-truth data under simulated exposure noise.
3.3 Simulated datasets
- The simulated evaluation covers both fewer projections and shorter exposure times, which produce noisy measurements and reconstructions.Source: §3.3 · p. 8
3.3.1 Fewer projections
- For the fewer-projection test, 1024 projections are subsampled to 512, 256, 128, or 64 to model limited-view step-scan CT.The scenarios represent reducing experiment time or x-ray dose.Source: §3.3.1 · p. 9
- With subsampled projections, TomoGAN improves SSIM consistently across slices, and reconstruction quality is comparable to that from 1024 projections.The experiments subsample 1024 projections to 512, 256, 128, or 64; Figure 9 illustrates the fewer-projection comparison.Source: §3.3.1 · p. 9, Figure 9 · p. 10
3.3.2 Shorter exposure times
- Under simulated shorter exposure times, TomoGAN-enhanced reconstructions reach quality comparable to reconstructions based on ground-truth data.The simulated intensities are 100, 500, 1000, and 10 000 photons per pixel; ground truth is reconstructed from noise-free simulation data.Source: §3.3.2 · p. 10
3.4 Experimental datasets
TomoGAN is tested on shale datasets from different samples and facilities, using one dataset for training and noisy versions of the others for evaluation. It improves experimental reconstructions under both fewer-projection and shorter-exposure conditions, though gains are smaller than on simulated data.
3.4 Experimental datasets
- TomoGAN is trained on DSB1 APS and evaluated on other shale datasets, including data from different samples and facilities, to mimic practical use.The full 1024-projection reconstructions serve as ground truth because experimental ground truth is unavailable.Source: §3.4 · pp. 10–11
3.4.1 Fewer projections
- With 64 projections, TomoGAN produces reconstructions comparable to ground truth, with higher contrast and considerably reduced feature-value variance.Using fewer projections also reduces experiment time, dataset size, and reconstruction computation by the subsampling fraction.Source: §3.4.1 · p. 12, Figure 13 · p. 13
- TomoGAN consistently improves SSIM across slices and subsampling levels on DSN1 SLS, although its scores are below those for simulated data.The authors attribute the difference to better ground-truth training data and simpler, circle-only features in the simulated dataset.Source: §3.4.1 · p. 12, Figure 14 · p. 13
- The SSIM scores are consistent across slices, which the authors identify as important for a black-box predictor..Source: §3.4.1 · p. 12
3.4.2 Shorter exposure time
- For shorter-exposure data, TomoGAN is trained on DSB1 APS projections with added Poisson noise and applied to the other three datasets.The comparison includes projections acquired at 1/16 of normal exposure time.Source: §3.4.2 · p. 13, Figure 15 · p. 14
- Experimental SSIM improvement under shorter exposure is smaller than in simulation, which the authors attribute to the greater diversity of experimental feature shapes, sizes, and contrast.The reported comparison uses per-slice SSIM scores.Source: §3.4.2 · p. 13, Figure 14 · p. 13
3.5 Comparison with iterative methods
Filtered back projection followed by TomoGAN is faster than SIRT in the reported timing comparison, and iterative reconstruction does not provide better image quality in the illustrated case.
3.5 Comparison with iterative methods
- Although iterative methods are more resilient to projection noise, the authors describe them as computationally demanding and prohibitively expensive for large data..Source: §3.5 · p. 14
- 70 ms per slice for filtered back projection plus TomoGAN, versus 550 ms for SIRT with 400 iterations; iterative reconstruction does not provide better image quality in the illustrated case.The timings were measured on one NVIDIA Tesla V100-SXM2 GPU.Source: Figure 16 · p. 14, §3.5 · p. 14
3.6 Comparison with other solutions on experimental datasets
On the shale dataset, TomoGAN is compared with FBPConvNet and WGAN-VGG. The reported comparison highlights low-contrast artifacts in FBPConvNet and cautions that WGAN-VGG’s brain-image tuning makes its shale result unfair to interpret as a general comparison.
3.6 Comparison with other solutions on experimental datasets
- TomoGAN uses adversarial loss and a pre-trained VGG network, while its U-Net-based generator has three boxes and no batch normalization to reduce inference computation and memory needs.On one NVIDIA Tesla V100 with batch size eight, TomoGAN processes a 1024×1024 image in 30 ms versus 90 ms for FBPConvNet.Source: §3.6 · pp. 14–15
- Unlike TomoGAN’s U-Net variant, WGAN-VGG uses a handcrafted generator; TomoGAN also incorporates adjacent-slice information, which the authors say appears beneficial in practice..Source: §3.6 · p. 15
- FBPConvNet introduces low-contrast artifacts on the shale image, while WGAN-VGG has lower SSIM and PSNR and does not work well on this dataset.The authors note that the WGAN-VGG comparison is not fair because its model and parameters were tuned for brain images, unlike high-frequency hard X-ray shale images.Source: §3.6 · pp. 15–16, Figure 18 · p. 15, Figure 17 · p. 15
3.7 Model interpretation
The authors inspect bottom-layer feature maps to interpret how TomoGAN processes noisy images. Different channels appear to respond to distinct sample structures, including small features that could be lost in noise.
3.7 Model interpretation
- The examined feature maps come from the last down-sampling layer, where the image is reduced to 1/8 its original size and the channel count rises from d to 128.The authors focus on this layer because they expect it to show refined features selected by preceding convolution kernels.Source: §3.7 · p. 16, Figure 19 · p. 16
- Different bottom-layer channels appear to capture distinct structures, including dense regions, air, the sample exterior, finer detail, and small holes.The authors suggest that different convolution kernels capture different feature types.Source: §3.7 · p. 16, Figure 19 · p. 16
4 Discussion
TomoGAN improves reconstructions from noisy x-ray projections and can transfer from one training sample to similar samples, including across facilities and noise conditions. The authors see potential for faster studies of dynamic features using fewer projections or shorter exposures.
4 Discussion
- After training on one sample, TomoGAN can improve reconstructions from noisy projections of similar samples, including across facilities and noise characteristics.The authors report visible improvement on noisy projection datasets.Source: §4 · pp. 16–17
- The reported transfer extends to similar samples collected at different facilities and with different noise characteristics.Source: §4 · p. 17
- TomoGAN may support faster studies of dynamic features by using fewer projections or shorter exposure times per projection.Source: §4 · p. 17
License
The license grants the U.S. Government broad rights to the manuscript and provides for public access to federally sponsored research results.
License
- The U.S. Government retains a paid-up, nonexclusive, irrevocable, worldwide license to the article.Source: License · p. 20
- The license covers reproducing the article, preparing derivative works, distributing copies, and public performance or display.Source: License · p. 20
- The Department of Energy will provide public access to these federally sponsored research results under its Public Access Plan.Source: License · p. 20