Source-linked AI summary
Fully Dense UNet for 2D Sparse Photoacoustic Tomography Artifact Removal
Steven Guan, Amir Khan, Siddhartha Sikdar, Parag V. Chitnis
TL;DR
Sparse PAT measurements produce artifact-laden reconstructions, creating a need for effective artifact removal. The paper proposes FD-UNet, a densely connected UNet used as a post-processing CNN, and reports superior artifact removal and image enhancement versus standard UNet across tested datasets. Its performance remains dependent on training data matching the features present at testing and on the quality of the initial reconstruction.
Problem
Sparse detector sampling causes severe artifacts in standard PAT reconstructions, while iterative alternatives can require difficult constraint selection and repeated computation.
Method
FD-UNet adds dense connectivity to UNet’s contracting and expanding paths and applies the CNN to time-reversal reconstructions for post-processing artifact removal.
Results
FD-UNet consistently outperforms standard UNet for artifact removal and image enhancement across synthetic and mouse-brain vasculature datasets.
Takeaways & Limitations
Dense feature reuse enables a more compact FD-UNet to learn relevant artifact-removal features and improve reconstruction quality within the post-processing approach.
Takeaways & Limitations
Performance depends strongly on training data matching testing features and on the quality of the time-reversal reconstruction, especially when features are obscured or missing.
Abstract
from arXiv · showhide
Photoacoustic imaging is an emerging imaging modality that is based upon the photoacoustic effect. In photoacoustic tomography (PAT), the induced acoustic pressure waves are measured by an array of detectors and used to reconstruct an image of the initial pressure distribution. A common challenge faced in PAT is that the measured acoustic waves can only be sparsely sampled. Reconstructing sparsely sampled data using standard methods results in severe artifacts that obscure information within the image. We propose a modified convolutional neural network (CNN) architecture termed Fully Dense UNet (FD-UNet) for removing artifacts from 2D PAT images reconstructed from sparse data and compare the proposed CNN with the standard UNet in terms of reconstructed image quality.
I. INTRODUCTION
Sparse spatial sampling in PAT makes standard reconstructions artifact-prone, motivating efficient learned post-processing. The paper proposes FD-UNet, which adds dense connectivity to UNet paths for 2D PAT artifact removal.
- I. INTRODUCTION: PAT reconstructs an initial pressure distribution from acoustic measurements collected along a detector boundary.Time reversal is considered robust across arbitrary detection geometries and heterogeneous media.
- I. INTRODUCTION: Sparse detector sampling is difficult to scale because each spatial measurement requires a detector, while undersampling produces severe reconstruction artifacts.Iterative methods can reduce artifacts but depend on suitable constraints and repeated forward and adjoint evaluations.
- I. INTRODUCTION: Deep-learning PAT reconstruction commonly uses a post-processing pipeline that applies a CNN to an initially corrupted reconstruction.This approach has achieved image quality comparable to iterative methods in CT, MRI, and PAT.
- I. INTRODUCTION: Model-based learning can outperform post-processing but requires additional computation time, so this work follows the post-processing approach.The proposed FD-UNet is designed specifically for removing artifacts from 2D PAT images reconstructed from sparse data.
- I. INTRODUCTION: FD-UNet incorporates dense connectivity into UNet’s contracting and expanding paths to improve information flow and reduce redundant feature learning.Dense connectivity passes earlier features to later layers through channel-wise concatenation, supporting feature reuse.
A. Related Work
Prior work established UNet-based post-processing for sparse tomographic reconstruction, while FD-UNet extends dense connectivity across both network paths for sparse PAT artifact removal.
- UNet-based post-processing: UNet is widely used for post-processing sparse tomographic images and has performed comparatively well for sparse PAT artifact removal.Its multilevel decomposition and multichannel filtering are suited to artifact removal.
- Alternative reconstruction approaches: Model-based learning can outperform post-processing and iterative reconstruction, but requires additional computation time.It directly incorporates forward and adjoint operators with CNN-learned prior constraints and requires fewer iterations to converge.
- FD-UNet design: FD-UNet applies dense connectivity in both the contracting and expanding paths, unlike DD-Net, which uses it only in the contracting path.The authors state that this extends dense-connectivity benefits throughout the network.
- FD-UNet design: FD-UNet updates the dense-block growth-rate hyperparameter throughout the CNN to improve computational efficiency.DD-Net instead keeps this hyperparameter constant throughout the network.
- Contribution: The paper presents FD-UNet as the first application of UNet with dense connectivity to sparse PAT image artifact removal.The method targets 2D PAT images reconstructed from sparse data using a post-processing approach.
A. Deep Learning Framework
The framework reconstructs an artifact-corrupted image with TR and applies a dense-connectivity CNN to produce an approximately artifact-free output. FD-UNet uses dense blocks throughout U-Net’s contracting and expanding paths, with multiscale processing and feature reuse.
- TR first reconstructs the sparsely sampled pressure data into an artifact-containing image X, which the CNN maps to an approximately artifact-free image Y.
- FD-UNet incorporates dense connectivity into both the contracting and expanding paths of the U-Net architecture.
- The contracting path repeatedly reduces spatial dimensions with max pooling to learn local and global features at multiple spatial scales.
- Dense blocks concatenate outputs from earlier layers into later layers, enabling feature reuse and increasing representational power.
- FD-UNet uses 82 convolution and deconvolution layers, compared with 23 layers in the UNet, while dense connections provide short paths for gradient information.
C. Synthetic Data for Training and Testing
The experiments use simulated PAT data generated from synthetic and anatomically realistic phantoms under varying detector sparsity. The datasets include circles, Shepp-Logan, and vasculature structures, with reconstructions evaluated visually and through reported metrics.
- Synthetic PAT data are generated with k-Wave from 128x128-pixel sources, homogeneous non-absorbing media, and circular detector arrays.
- The datasets include circles, Shepp-Logan, synthetic vasculature, and an anatomically realistic mouse-brain vasculature phantom derived from micro-CT data.
- The circles dataset contains up to five randomly positioned and sized equal-magnitude circles, with 1,000 training and 200 testing images across 10, 15, and 30 detectors.
- Figure 7 compares circle reconstructions at 10, 15, and 30 detectors, highlighting boundary blurring as sampling becomes sparser.
D. Deep Learning Implementation
The CNNs are implemented and trained in TensorFlow on a GPU using mean squared error optimization. Training uses fixed iteration, learning-rate, and mini-batch settings.
- The CNNs are implemented in Python 3.6 with TensorFlow v1.7 and trained and evaluated on an NVIDIA GTX 1080Ti GPU.
- Training runs for 10,000 iterations with mean squared error loss, a 1e-4 learning rate, and mini-batches of three images.
- The implementation uses a fixed mini-batch size of three images during training.
III. EXPERIMENTS AND RESULTS
The experiments compare UNet and FD-UNet for sparse-tomographic artifact removal using PSNR and SSIM as image-quality measures. PSNR captures global quality, while SSIM evaluates similarity of local intensity patterns.
- UNet and FD-UNet are compared across experiments to assess artifact removal and recovered image quality.
- PSNR and SSIM quantify reconstruction quality for the CNN comparisons.
- PSNR measures global image quality, whereas SSIM measures similarity between local patterns of pixel intensities.
A. Circles Dataset
On matched circles data, both CNNs reduce sparse-sampling artifacts, but FD-UNet consistently achieves better reconstruction quality and parameter efficiency than UNet across sparsity and model-complexity tests.
- A. Circles Dataset: Training and testing on matched circles data provides an ideal scenario for comparing the CNNs without data-related limitations.Four-fold cross validation is used because both datasets derive from the same phantom.
- Artifact removal: FD-UNet generally produces near artifact-free images and removes artifacts more consistently than UNet, with no test image favoring UNet.UNet leaves artifacts near the top-left circle boundary and in the background.
- Model efficiency: FD-UNet uses about one-third as many parameters as UNet while achieving higher average PSNR and SSIM for each tested hyperparameter set.Evaluation times are similar because FD-UNet uses smaller convolutional inputs and outputs within dense blocks.
- Model complexity: The most compact FD-UNet configuration, f1 = 8 and k1 = 1, outperforms the more complex UNet configuration with f1 = 64.Increasing model complexity improves both CNNs but with diminishing returns and greater overfitting risk.
- Sampling sparsity: Fewer detectors produce more severe artifacts and lower average PSNR and SSIM, while FD-UNet exceeds UNet on both metrics at every tested sparsity level.At 15 detectors, FD-UNet preserves circle boundaries better; with 10 detectors, neither CNN reliably reconstructs them.
B. Shepp-Logan and Vasculature Phantom Dataset
When trained on circles and tested on other phantoms, both CNNs generalize better to Shepp-Logan than synthetic vasculature; fine-tuning improves both, with FD-UNet retaining the stronger results.
- B. Shepp-Logan and Vasculature Phantom Dataset: Circles-trained CNNs are fairly well matched to Shepp-Logan but significantly mismatched to synthetic vasculature because the latter has different features.The experiment evaluates training and testing under imperfect dataset matching.
- Initial training: Without fine-tuning, both CNNs recover high-quality but blurred Shepp-Logan images, whereas synthetic vasculature reconstructions retain circle-like features and background artifacts.FD-UNet performs slightly better and removes more background artifacts.
- Fine-tuning: Fine-tuning with well-matched data improves both CNNs, especially for synthetic vasculature, removing the circle-like appearance and producing sharper reconstructions.FD-UNet outperforms UNet with higher average PSNR and SSIM for both synthetic phantoms.
C. Mouse Brain Vasculature Dataset
Training on synthetic vasculature enables both CNNs to remove many artifacts from mouse brain vasculature images, while FD-UNet recovers more small vessels across sampling sparsities.
- C. Mouse Brain Vasculature Dataset: The experiment tests whether synthetic phantom training can remove artifacts from anatomically realistic mouse brain vasculature images under multiple sampling sparsities.The datasets are fairly well matched but may differ in anatomically realistic features.
- UNet comparison: UNet removes most background artifacts and appears better than TR reconstruction, but its PSNR and SSIM show no significant quantitative improvement over TR.UNet primarily recovers larger vessels and misses many smaller features.
- FD-UNet comparison: FD-UNet outperforms UNet in average PSNR and SSIM and recovers more small-vessel details, including features highlighted in the 30-detector reconstruction.The figure marks features present in FD-UNet but missing from UNet with arrows.
- Sampling sparsity: As detector count decreases, artifacts become more severe in the TR reconstruction and image quality degrades.The CNNs’ performance remains dependent on the quality of the initial reconstruction.
IV. DISCUSSION
The discussion attributes FD-UNet’s consistent advantage to dense connectivity, while emphasizing that performance depends on training-data match and the quality of the initial reconstruction.
- IV. DISCUSSION: FD-UNet consistently outperforms standard UNet for artifact removal and image enhancement, including with fewer features and a fraction of the parameters.Dense connectivity encourages feature reuse and improves information flow throughout the network.
- Generalization: Dense connectivity may reduce overfitting: FD-UNet and UNet reach similar training PSNR, but FD-UNet performs better on testing data.UNet’s more volatile training loss is consistent with overfitting to previously observed examples.
- Data requirements: Deep-learning artifact removal requires large training datasets, and well-matched training and testing features remain difficult to ensure.Synthetic phantoms can generate arbitrarily large datasets, but poorly matched circles data performs worse on synthetic vasculature until fine-tuned.
- Post-processing limitation: In the post-processing pipeline, features missing or severely obscured in the initial TR reconstruction are likely to remain missing or be reconstructed incorrectly.Direct reconstruction from sensor data may recover some smaller vessels lost during initial reconstruction.
V. CONCLUSION
The paper proposes FD-UNet for artifact removal in sparse-data 2D PAT and compares it with UNet across synthetic and anatomically realistic mouse-brain vasculature datasets.
- FD-UNet removes artifacts from 2D PAT images reconstructed from sparse data.
- The comparison uses synthetic circles, Shepp-Logan, and vasculature phantoms, plus an anatomically realistic mouse brain vasculature dataset.
- FD-UNet is demonstrated to be superior and more compact than UNet for artifact removal and image-quality improvement.