PaperlayerView exact revision on arXiv

Source-linked AI summary

Eformer: Edge Enhancement based Transformer for Medical Image Denoising

Achleshwar Luthra, Harsh Sulakhe, Tanish Mittal, Abhishek Iyer, Santosh Yadav

arXiv:2109.08044v2eess.IVcs.CV

TL;DR

Low-dose CT denoising must address noise and loss of detail associated with reduced X-ray dose, while prior conventional approaches are described as constrained by limited data. Eformer combines transformer-based denoising with edge enhancement and residual learning; the authors report state-of-the-art performance and better results from residual than direct prediction.

  • Problem

    Low-dose CT denoising addresses noise and reduced image detail from lowering X-ray dose, while conventional approaches are described as limited by data and accuracy.

    Source: §1 · p. 1, §2 · p. 2

  • Method

    Eformer is a transformer-based denoising architecture with edge enhancement, trained to predict residual noise rather than the denoised image directly.

    Source: §1 · p. 1, §3.4 · p. 3

  • Results

    The authors report state-of-the-art performance and that residual learning clearly outperforms direct denoised-image prediction for medical image denoising.

    Source: §1 · p. 2, §5 · p. 5

  • Takeaways & Limitations

    The authors conclude that their work supports using transformers for medical image denoising and plan to explore related tasks.

    Source: §5 · p. 5

Abstract

from arXiv · show

In this work, we present Eformer - Edge enhancement based transformer, a novel architecture that builds an encoder-decoder network using transformer blocks for medical image denoising. Non-overlapping window-based self-attention is used in the transformer block that reduces computational requirements. This work further incorporates learnable Sobel-Feldman operators to enhance edges in the image and propose an effective way to concatenate them in the intermediate layers of our architecture. The experimental analysis is conducted by comparing deterministic learning and residual learning for the task of medical image denoising. To defend the effectiveness of our approach, our model is evaluated on the AAPM-Mayo Clinic Low-Dose CT Grand Challenge Dataset and achieves state-of-the-art performance, $i.e.$, 43.487 PSNR, 0.0067 RMSE, and 0.9861 SSIM. We believe that our work will encourage more research in transformer-based architectures for medical image denoising using residual learning.

1. Introduction

The introduction motivates low-dose CT denoising as a way to reduce radiation exposure while addressing noise and loss of image detail. It presents Eformer as a transformer-based approach and reports that residual learning outperforms direct image prediction.

1. Introduction

  • Reducing CT X-ray dose can increase noise, reduce edge and feature contrast, and oversmooth images.The proposed method aims to preserve details while reducing noise in low-dose scans.Source: §1 · p. 1
  • Prior denoising methods are described as emphasizing global operations, while transformer patch embeddings may leverage local visual information.The introduction notes that vision transformers had not yet been applied to medical image datasets.Source: §1 · p. 1
  • The paper identifies its work, to the authors’ knowledge, as the first use of transformers for medical image denoising.It also describes experiments using residual learning.Source: §1 · p. 1
  • Eformer is introduced as an edge-enhancement architecture for medical image denoising using transformers and learnable Sobel filters.The authors report that it outperforms existing state-of-the-art methods.Source: §1 · p. 1
  • Residual learning clearly outperforms direct denoised-image prediction, which can resemble an identity mapping.The introduction also frames experiments with direct prediction as a comparison to test residual learning.Source: §1 · p. 2

2. Related Work

Related work spans convolutional, GAN-based, and transformer approaches to image denoising. The paper draws on transformer successes in other vision tasks and adapts a Uformer-like design for medical image denoising.

2. Related Work

  • Prior medical image denoising work includes encoder-decoder CNNs, fully convolutional networks, and GAN-based models.The passage notes limited data and consequent low accuracy of conventional approaches in low-dose CT.Source: §2 · p. 2
  • Transformers have been used for image restoration tasks including super-resolution, denoising, deraining, and colorization.Uformer uses non-overlapping window-based self-attention and depth-wise convolution to capture local context efficiently.Source: §2 · p. 2
  • Eformer combines an edge-enhancement module with a Uformer-like architecture for medical image denoising.The authors report state-of-the-art results for this approach.Source: §2 · p. 2

3. Our Approach

Eformer combines edge-enhanced features with a transformer encoder-decoder for medical image denoising. The approach uses residual prediction and a joint pixelwise and perceptual loss.

3.1. Sobel-Feldman Operator

  • The Sobel–Feldman block uses horizontal, vertical, and diagonal operators to emphasize edges, with edge-information feature maps concatenated into the network.Figure 2 shows input images on the left and edge-enhanced images on the right.Source: §3.1 · pp. 2–3, Figure 2 · p. 3

3.2. Transformer based Encoder-Decoder

  • Eformer applies non-overlapping window-based self-attention and a locally enhanced feed-forward network in transformer blocks across encoder and decoder stages.The transformer block also serves as the bottleneck; the supplied equation passage describes its W-MSA and LeFF stages with layer normalization and residual additions.Source: §3.2 · p. 3

3.3. Downsampling & Upsampling

  • Upsampling uses transpose convolution with a 4 × 4 kernel and stride 2 to avoid uneven overlap associated with checkerboard artifacts.The stated kernel-size choice follows the cited guidance that kernel size should be divisible by stride.Source: §3.3 · p. 3

3.4. Residual Learning

  • Residual learning predicts the noise residual from a noisy image, then subtracts it to obtain the denoised image.The paper motivates this formulation by stating that residual mappings are easier to optimize when the original mapping resembles an identity mapping.Source: §3.4 · p. 3

3.5. Optimization

  • The objective combines pixelwise MSE with multi-scale perceptual loss to address blur and oversmoothing from MSE alone.The perceptual loss compares ResNet-50 features from denoised and ground-truth images; the final objective weights MSE and MSP with predefined constants.Source: §3.5 · pp. 3–4

3.6. Overall Network Architecture

  • The full Eformer pipeline combines Sobel-derived features with LeWin transformer stages and decodes a residual image.Sobel features are concatenated with feature maps during encoding and decoding; the output projection maps the decoded C-channel feature map to a one-channel residual.Source: §3.6 · p. 4

4. Results and Discussions

On the AAPM-Mayo low-dose CT dataset, Eformer and Eformer-Residual outperform state-of-the-art methods on PSNR and MSE, with comparable SSIM performance. The authors interpret the SSIM result as indicating high visual quality and preservation of important details.

Results and Discussions

  • Eformer and Eformer-Residual outperform state-of-the-art methods on PSNR and MSE, while their SSIM performance is comparable.The authors say the PSNR and MSE results indicate efficient denoising, and comparable SSIM suggests high visual quality without losing important details.Source: §4 · p. 4

5. Conclusion

The paper presents a residual-learning image denoising model for the medical domain, combining transformers and edge enhancement to produce denoised images. It reports state-of-the-art performance using multi-scale perceptual and traditional MSE losses.

  • The model achieves state-of-the-art performance using a combination of multi-scale perceptual loss and traditional MSE loss.Source: §5 · p. 5
  • The denoising model uses residual learning and leverages transformers with an edge-enhancement module.Source: §5 · p. 5
  • The authors hope the work encourages transformer use in medical image denoising and plan to explore related tasks.Source: §5 · p. 5

Supplementary Material for Eformer: Edge Enhancement based

The supplementary material identifies the work as Eformer and lists its authors and institutional affiliation.

7. Dataset Details

The study uses paired low-dose and normal-dose CT scans from the AAPM-Mayo Clinic dataset, with low-dose scans generated by Poisson noise insertion.

  • The AAPM-Mayo Clinic Low-Dose CT Grand Challenge Dataset contains paired low-dose and normal-dose scans from 140 patients across abdomen, chest, and head CT.The stated patient counts for those scan types are 48, 49, and 42, respectively.Source: §7 · p. 7
  • Low-dose scans are synthetic, generated by inserting Poisson noise into projection data to reach 25% of full-dose noise.Source: §7 · p. 7
  • The scans are provided in DICOM format, with 512 × 512 pixel images; image data are extracted to NumPy arrays and scaled from 0 to 1 for training.Source: §7 · p. 7

8. Parameter Details and Network Training

Training uses resized CT images and an ADAM optimizer, while Sobel convolution filters follow a pattern with a learnable parameter.

  • The Sobel convolutional block follows the filter pattern shown in Figure 4, with α as a learnable parameter.Figure 4 presents four different sets of Sobel filters.Source: §8 · p. 7, Figure 4 · p. 7
  • Each 512 × 512 image is resized to 128 × 128 pixels for model input.Source: §8 · p. 7
  • Experiments use PyTorch, default convolutional initialization except for the Sobel block, and an ADAM optimizer with learning rate 0.00002.Training ran on a 16GB NVIDIA TESLA P100 GPU.Source: §8 · p. 7

9. LeWin Transformer

The LeWin Transformer Block combines non-overlapping window-based self-attention with a Locally-enhanced Feed-Forward Network (LeFF). Its attention module splits feature maps into windows before computing self-attention on each window’s flattened features.

9. LeWin Transformer

  • LeWin combines non-overlapping Window-based Multi-head Self-Attention (W-MSA) with a Locally-enhanced Feed-Forward Network (LeFF), which integrates convolution.The passage identifies these as the block’s two core designs.Source: §9 · p. 7
  • The block applies layer normalization before W-MSA and LeFF, with each module’s output added to its input through a residual connection.The passage defines LN as layer normalization.Source: §9 · p. 8
  • W-MSA splits a 2D feature map into non-overlapping M × M windows and performs self-attention on each window’s flattened features.For a feature map of size H × W, the number of windows is N = HW/M^2.Source: §9 · p. 8
  • Outputs from all attention heads are concatenated and linearly projected to produce the final results.The attention calculation follows the formulation in.Source: §9 · p. 8

Related papers

Loading 2109.08044v2…