Source-linked AI summary
CTformer: Convolution-free Token2Token Dilated Vision Transformer for Low-dose CT Denoising
Dayang Wang, Fenglei Fan, Zhan Wu, Rui Liu, Fei Wang, Hengyong Yu
TL;DR
LDCT denoising must address severe noise and artifacts, while transformers had been little explored for this task. CTformer is a convolution-free Token2Token Dilated Vision Transformer with dilated and shifted feature processing, interpretability tools, and overlapped inference; experiments report superior denoising performance and model efficiency.
Problem
LDCT images suffer severe noise and artifacts, while the potential of vision transformers for LDCT denoising had been little explored.
Method
CTformer uses convolution-free token rearrangement, dilation and cyclic shift, overlapped inference, and attention-map and explanatory-graph analysis.
Results
Experiments demonstrate superior denoising performance and model efficiency compared with competing methods.
Takeaways & Limitations
CTformer is presented as a pure transformer for LDCT denoising that addresses boundary artifacts and exposes denoising behavior through interpretation methods.
Abstract
from arXiv · showhide
Low-dose computed tomography (LDCT) denoising is an important problem in CT research. Compared to the normal dose CT (NDCT), LDCT images are subjected to severe noise and artifacts. Recently in many studies, vision transformers have shown superior feature representation ability over convolutional neural networks (CNNs). However, unlike CNNs, the potential of vision transformers in LDCT denoising was little explored so far. To fill this gap, we propose a Convolution-free Token2Token Dilated Vision Transformer for low-dose CT denoising. The CTformer uses a more powerful token rearrangement to encompass local contextual information and thus avoids convolution. It also dilates and shifts feature maps to capture longer-range interaction. We interpret the CTformer by statically inspecting patterns of its internal attention maps and dynamically tracing the hierarchical attention flow with an explanatory graph. Furthermore, an overlapped inference mechanism is introduced to effectively eliminate the boundary artifacts that are common for encoder-decoder-based denoising models. Experimental results on Mayo LDCT dataset suggest that the CTformer outperforms the state-of-the-art denoising methods with a low computation overhead.
I. INTRODUCTION
LDCT denoising addresses severe noise and artifacts, but transformers had been little explored for this task. CTformer is introduced as a convolution-free transformer using token rearrangement, dilation, cyclic shift, overlapped inference, and interpretability analysis.
- LDCT images contain severe noise and artifacts compared with normal-dose CT images.
- Transformers had been little investigated for LDCT denoising despite their reported success in computer vision.
- CTformer excludes convolution by using token rearrangement to complement local information and is presented as a pure transformer for LDCT denoising.
- Dilation and cyclic shift enlarge the receptive field, while overlapped inference addresses boundary artifacts in encoder-decoder denoising models.
- The paper develops visual attention maps and an explanatory graph to examine CTformer denoising patterns and hierarchical attention flow.
- Experiments report superior denoising performance with fewer trainable parameters and MACs than other state-of-the-art methods.
II. RELATED WORK
Prior LDCT denoising work used physical priors, iterative methods, and CNN-based models. These approaches motivate transformer-based modeling because CNNs have limited receptive fields and are less suited to whole-image structural similarity.
- Previous LDCT denoising studies are categorized into traditional algorithms and convolution models.
- Traditional algorithms: Traditional methods use physical priors within iterative reconstruction frameworks to suppress noise and artifacts.
- Convolution models: CNN-based approaches include autoencoders, residual convolutional networks, GANs, and multi-stage models for LDCT reconstruction or post-processing.
- Transformer-based direction: CTformer uses tokenization and detokenization within a residual encoder-decoder, with T2TD blocks in the encoder and symmetric IT2TD blocks in the decoder.
- Convolution models: CNN-based models typically have limited receptive fields, reducing their ability to capture global context and model structural similarity across an image.
III. METHODS
In supervised LDCT denoising, a model learns to map paired noisy LDCT inputs to clean NDCT targets. Training is formulated by minimizing mean square error between the network output and the target.
- The supervised task learns a mapping from a paired noisy LDCT image x to a clean NDCT image y.
- The neural network is trained by optimizing a mean square error loss between f(W; x) and y.
- Here, f(W; x) denotes the neural network and W denotes its collection of parameters.
A. Architecture of the CTformer
CTformer is a convolution-free residual encoder-decoder transformer that combines tokenization, self-attention, cyclic shifts, dilated unfolding, and overlapped inference to denoise LDCT images while addressing boundary artifacts.
- Backbone: CTformer uses tokenization/detokenization blocks, four CTformer modules, an intermediate transformer block, and residual encoder-decoder shortcuts.Encoder modules use transformer and Token2Token Dilation blocks; decoder modules symmetrically use inverse Token2Token Dilation and transformer blocks.
- Tokenization: The tokenization block unfolds a noisy CT image into 2D patches treated interchangeably as tokens.The resulting token sequence is represented as T0 ∈ R^b×n×d0.
- Transformer block: Transformer blocks map tokens into query, key, and value tensors and use multi-head attention, normalization, MLPs, and residual connections.The attention map is later used for post-hoc interpretability analysis.
- Token2Token dilation: Token2Token processing reshapes feature maps, cyclically shifts pixels, and uses overlapped dilated unfolding to preserve local correlations and capture longer-range context.Dilation enlarges the receptive field with less computational cost, while cyclic shifting extracts tokens from varied feature-map regions.
- Inference: Overlapped inference keeps only each patch’s central region to reduce boundary artifacts when patchwise denoised outputs are stitched.The method increases computation, so artifact elimination must be balanced against cost.
C. Interpretability of the CTformer
CTformer interpretability combines static attention-map probing with an explanatory graph that traces how attended image parts flow across layers.
- Attention-map analysis: Attention maps provide a self-interpretability signal for CTformer denoising, where conventional saliency maps are unsuitable for regression-based image-to-image tasks.The analysis focuses on structural and semantic information processed by the model.
- Explanatory graph: The explanatory graph represents attended object parts as nodes and attention flow across layers as edges.This complements static attention by describing hierarchical changes in attended regions.
- Node construction: TopK and local-maximum selection identify attended object parts from attention maps.TopK selects the K-highest activations, whereas local maximum selection detects local peak activations.
- Edge construction: Masked feature maps connect nodes between neighboring layers by forwarding each retained region and linking it to the next layer’s highest activation.Repeating this process recursively builds a graph showing how CTformer attention shifts.
IV. EXPERIMENTS
Experiments on the Mayo LDCT dataset evaluate CTformer’s denoising quality, efficiency, interpretability, and attention behavior. CTformer achieves strong visual and quantitative results while maintaining low computational cost.
- Experimental setup: The study trains and evaluates CTformer on the publicly available Mayo LDCT dataset, using patient L506 for evaluation and nine patients for training.The dataset contains 2,378 low-dose and normal-dose 3.0 mm CT slices from ten patients.
- Experimental setup: CTformer is compared with RED-CNN, WGAN-VGG, MAP-NN, and AD-NET using retrained implementations.The comparison includes established low-dose CT and natural-image denoising models.
- Denoising performance: CTformer produces the clearest perceptual denoising results, suppressing noise and artifacts while preserving spatial smoothness and structural details around lesions.The compared methods reduce noise to varying degrees, but some introduce shadows, tissues, or blotchy noise.
- Denoising performance: 0.9121 SSIM and 9.0233 RMSE are reported for CTformer, alongside the smallest parameter count and lowest MACs among the compared methods.The paper reports these as the best performance with the lowest computational cost on L506.
- Model efficiency: CTformer maintains superior efficiency across model sizes, with SSIM curves toward the top-left and RMSE curves toward the bottom-left of the parameter and MAC plots.At similar parameter and MAC counts, CTformer delivers better scores than RED-CNN, MAP-NN, and AD-NET.
- Interpretability: Attention maps emphasize edges and key object parts in early layers, become more semantically implicit later, and focus on noise-concentrated content in the fourth layer.The explanatory graphs further show recurring attention-flow patterns across slices and co-activation of object parts undergoing similar denoising.
V. ABLATION STUDY
Ablation experiments examine T2TD, cyclic shift, and transformer-block count. The results support cyclic shift and indicate that a shallow configuration can provide the best performance with lower training burden.
- Study design: The ablation study evaluates the T2TD block, cyclic shift operation, and number of intermediate transformer blocks.These experiments isolate major architectural choices in CTformer.
- T2TD block: CTformer converges faster than Sole-ViT and improves SSIM by 0.0235 and RMSE by 3.3362.Sole-ViT omits the T2TD module and uses sole convolution during tokenization.
- Cyclic shift: Cyclic shift improves SSIM by 0.0026 and RMSE by 0.1337 while producing greater spatial smoothness than CTformer without cyclic shift.Without cyclic shift, the model introduces additional noise components.
- Block number: The one-block CTformer achieves the best performance among models with 1, 2, 4, and 8 intermediate transformer blocks.Increasing block count raises computational cost slowly but increases actual training time dramatically.
VI. CONCLUSION
The paper presents CTformer as a convolution-free transformer for LDCT denoising, combining dilated tokenization, cyclic shift, interpretation methods, and overlapped inference. Experiments report superior denoising performance and model efficiency relative to competitors.
- Conclusion: CTformer is proposed as a convolution-free transformer empowered by dilated tokenization and cyclic shift for LDCT denoising.The paper identifies it as the first pure transformer model for this task.
- Conclusion: Interpretation methods decode CTformer’s hidden behavior, while overlapped inference addresses boundary artifacts common in encoder-decoder models.The conclusion presents both mechanisms as additional contributions beyond the denoising architecture.
- Conclusion: Experimental results demonstrate that CTformer outperforms competing methods in denoising performance and model efficiency.The paper suggests extending CTformer to other medical denoising problems in future work.