PaperlayerView exact revision on arXiv

Source-linked AI summary

Limited View and Sparse Photoacoustic Tomography for Neuroimaging with Deep Learning

Steven Guan, Amir A. Khan, Siddhartha Sikdar, Parag V. Chitnis

arXiv:1911.04357v2eess.IVcs.CVcs.LGphysics.med-ph

TL;DR

Sparse, limited-view PAT acquisition produces reconstruction artifacts, while iterative correction is computationally expensive. Pixel-DL uses physics-guided pixel-wise interpolation followed by CNN reconstruction; it performed comparably to iterative reconstruction in the mouse-brain vasculature experiment and outperformed other CNN approaches across the reported experiments.

  • Problem

    Sparse spatial sampling and limited sensor views create streaking artifacts that inhibit PAT image interpretation and quantification, while iterative reconstruction is computationally expensive.

    Source: Introduction · p. 2

  • Method

    Pixel-DL applies pixel-wise interpolation guided by photoacoustic wave propagation to sensor data, then uses a CNN to reconstruct the image directly.

    Source: Introduction · p. 3

  • Results

    Pixel-DL consistently outperformed time reversal, Post-DL, and mDirect-DL across the experiments and performed comparably to iterative reconstruction on mouse-brain vasculature despite training on synthetic vasculature.

    Source: Discussion · p. 13

  • Takeaways & Limitations

    Pixel-DL reconstructed images at about 7.9 milliseconds per image, over 1,000 times faster than iterative reconstruction while achieving similar image quality.

    Source: Image Reconstruction Times · p. 13

    At the sparsest sampling level, Post-DL and Pixel-DL could add vessels absent from ground truth, particularly small vessels.

    Source: Comparison between Deep Learning… · p. 14

Abstract

from arXiv · show

Photoacoustic tomography (PAT) is a nonionizing imaging modality capable of acquiring high contrast and resolution images of optical absorption at depths greater than traditional optical imaging techniques. Practical considerations with instrumentation and geometry limit the number of available acoustic sensors and their view of the imaging target, which result in significant image reconstruction artifacts degrading image quality. Iterative reconstruction methods can be used to reduce artifacts but are computationally expensive. In this work, we propose a novel deep learning approach termed pixelwise deep learning (PixelDL) that first employs pixelwise interpolation governed by the physics of photoacoustic wave propagation and then uses a convolution neural network to directly reconstruct an image. Simulated photoacoustic data from synthetic vasculature phantom and mouse-brain vasculature were used for training and testing, respectively. Results demonstrated that PixelDL achieved comparable performance to iterative methods and outperformed other CNN-based approaches for correcting artifacts. PixelDL is a computationally efficient approach that enables for realtime PAT rendering and for improved image quality, quantification, and interpretation.

Abstract

Sparse, limited-view PAT data produce artifacts, while iterative reconstruction can reduce them at substantial computational cost. Pixel-DL transforms sensor data with physics-guided pixel-wise interpolation before CNN reconstruction, addressing limitations of earlier learned approaches.

Introduction

  • Sparse spatial sampling and limited sensor views produce streaking artifacts that inhibit PAT image interpretation and quantification.Fully enclosing the tissue with enough sensors can support high-quality reconstruction, but building such systems is often prohibitively expensive.Source: Introduction · p. 2
  • Iterative reconstruction reduces artifacts using a wave-propagation model and prior constraints, but repeated forward and adjoint evaluations make it computationally expensive.Image quality also depends on the imposed constraints.Source: Introduction · p. 2
  • Post-DL applies a CNN to an artifact-containing initial inversion, so features missing or obscured in that image are unlikely to be recovered.Post-DL has been used for CT, MRI, and PAT reconstruction.Source: Introduction · p. 2
  • Direct-DL reconstructs images from sensor data without an initial inversion, but changing input or output dimensions requires new convolution parameters and architecture modifications.A hybrid using both an initial inversion and sensor data as CNN inputs was reported to improve on Post-DL alone.Source: Introduction · pp. 2–3
  • Model-based learning uses an explicit wave-propagation model with CNN-learned rather than handcrafted prior constraints, but requires more time to train and reconstruct than direct learned approaches.The choice depends on whether an application prioritizes reconstruction speed or quality.Source: Introduction · p. 3
  • Pixel-DL uses wave-propagation-guided pixel-wise interpolation to map relevant sensor information into image space before CNN reconstruction.The interpolated data have dimensions similar to the desired image, simplifying CNN implementation; the approach is compared with conventional and learned methods using vasculature phantoms.Source: Introduction · p. 3

Methods

PAT begins when optical absorption generates an initial photoacoustic pressure, which propagates as acoustic waves. Sensors measure the waves, mapping the initial pressure to time-dependent data.

Photoacoustic Signal Generation

  • A nanosecond laser pulse causes light-absorbing molecules to undergo thermoelastic expansion and generate photoacoustic pressure waves.The initial-pressure formulation assumes negligible thermal diffusion and volume expansion during illumination.Source: Photoacoustic Signal Generation · p. 3
  • The pressure wave is modeled as a wave-equation initial-value problem with initial pressure x and zero initial time derivative.The pressure wave is denoted p(r,t), with c as the speed of sound.Source: Photoacoustic Signal Generation · p. 3
  • Sensors on a measurement surface record time-dependent signals, and the linear operator maps initial pressure x to measured signal y over finite time T.The operator acts on the pressure wave restricted to the computational-domain boundary.Source: Photoacoustic Signal Generation · pp. 3–4

Photoacoustic Image Reconstruction

The section describes time reversal and iterative reconstruction as methods for recovering PAT images from measured signals. Their performance depends on acquisition geometry, acoustic knowledge, and reconstruction constraints.

Photoacoustic Image Reconstruction

  • Time reversal reconstructs an image by running the forward model backward and transmitting measured sensor data into the medium in reverse time order.High-quality reconstruction requires known acoustic properties, enough detectors, and a sensor array that fully encloses the tissue.Source: Photoacoustic Image Reconstruction · p. 4
  • Iterative reconstruction recovers the PAT image from measured signals by minimizing a data-fit term with an isotropic total-variation constraint.The regularization parameter λ is positive; the TV constraint reduces noise and preserves edges.Source: Photoacoustic Image Reconstruction · p. 4
  • The illustrated CNN-based methods aim to reconstruct an essentially artifact-free PAT image from acquired sensor data, including a Pixel-DL input formed by pixel-wise interpolation.The figure depicts 32 sensors in a semi-circle limited-view configuration and contrasts Post-DL, Pixel-DL, and mDirect-DL processing.Source: Figure 1 · p. 5

Deep Learning

The deep-learning approaches transform PAT sensor data before CNN reconstruction, differing chiefly in that initial transformation. The shared FD-UNet architecture has contracting and expanding paths with concatenation connections.

Deep Learning

  • Post-DL first uses time reversal to create an artifact-containing image, whereas Pixel-DL interpolates sensor data pixel-wise into image space before CNN reconstruction.mDirect-DL instead combines linear interpolation and downsampling to match the final image dimensions.Source: Deep Learning · p. 5
  • The three approaches differ in their initial sensor-data processing, while the CNN recovers the final PAT image.Post-DL processes an initial time-reversal reconstruction; Pixel-DL uses pixel-wise interpolation; mDirect-DL uses linear interpolation and downsampling.Source: Deep Learning · p. 5
  • The FD-UNet CNN reconstructs the final PAT image using contracting and expanding paths connected by concatenations.The same architecture is used across approaches with minor modifications; Post-DL uses residual learning, while Pixel-DL inputs contain one feature map per sensor.Source: Figure 2 · p. 6

CNN Architecture: Fully Dense UNet

The reconstruction pipeline uses a Fully Dense UNet and first maps sensor measurements onto the image grid using pixel-wise interpolation based on photoacoustic wave propagation.

CNN Architecture: Fully Dense UNet

  • FD-UNet uses dense connectivity in its contracting and expanding paths to improve information flow, mitigate redundant feature learning, and reduce overfitting.It was reported to outperform UNet for artifact removal and image enhancement in 2D sparse PAT.Source: CNN Architecture: Fully Dense UNet · p. 6

Pixel-Wise Interpolation

  • The interpolation uses a linear wave-propagation model that assumes a homogeneous medium, spherical propagation, and constant sound speed.Under this model, time-of-flight is proportional to the distance between a pixel and sensor.Source: Pixel-Wise Interpolation · pp. 6–7
  • Pixel-wise interpolation maps each sensor’s time-series pressure measurements onto the reconstruction grid, producing a 3D array spanning image space and sensor number.The process is repeated for every sensor in the array.Source: Pixel-Wise Interpolation · pp. 6–7

Deep Learning Implementation

CNNs were trained on augmented vascular images and evaluated on simulated PAT data or held-out vascular datasets. The experiments used standardized image-quality metrics and specified simulation conditions.

Deep Learning Implementation

  • Each CNN was trained separately for its approach, imaging-system configuration, and training dataset, using Adam to minimize mean squared error for 40 epochs.Training used an initial learning rate of 1e-4, batch size three, and took about one hour per CNN on a GTX 1080Ti GPU.Source: Deep Learning Implementation · p. 7

Photoacoustic Data for Training and Testing

  • The synthetic phantom training set contained 500 augmented images, while the mouse-brain test set contained 50 images derived from filtered 3D micro-CT vasculature.Phantom augmentation used scaling, rotation, cropping, shifts, and sums of up to five iterations; mouse-brain images came from maximum-intensity projections of sampled subvolumes.Source: Photoacoustic Data for Training and… · p. 8
  • Fundus and lung datasets each used 500 augmented training images and 50 testing images, with training and testing data completely segregated across experiments.The fundus source contained 45 images, split into 15 training and 30 testing images; the lung source contained 50 CT scans, split into 15 training and 35 testing scans.Source: Photoacoustic Data for Training and… · p. 8
  • PAT data were simulated on a 128x128 grid with 16, 32, or 64 sensors on a semi-circle, and reconstructions were evaluated using PSNR and SSIM.The simulations assumed a homogeneous, non-absorbing medium with sound speed 1500 m/s and density 1000 Kg/m3; time reversal was also used for reconstruction.Source: Photoacoustic Data for Training and… · pp. 8–9

Results

Across the mouse-brain experiments, Pixel-DL produced fewer artifacts and stronger image-quality scores than time reversal, with performance comparable to iterative reconstruction. Its comparisons also reveal trade-offs in false vessels, smoothing, and generalization from synthetic training data.

Mouse Brain Vasculature Experiment

  • The experiment tested whether CNNs trained on synthetic phantom images generalize to non-matched mouse-brain vasculature images.The datasets could differ in features such as vessel connectivity patterns.Source: Mouse Brain Vasculature Experiment · p. 9
  • Pixel-DL consistently exceeded Post-DL in average PSNR and SSIM across sensor sparsity levels and scored similarly to iterative reconstruction.Time reversal had the most artifacts and lowest average PSNR and SSIM; increasing sensor count improved image quality across methods.Source: Figure 4 · p. 10, Table 1 · p. 10, Mouse Brain Vasculature Experiment · p. 10
  • With sparse sampling, especially 16 sensors, Post-DL often added vessels absent from ground truth; Pixel-DL showed similar behavior but typically fewer false vessels.Iterative reconstructions showed less of this issue but appeared overly smoothed compared with deep-learning reconstructions.Source: Mouse Brain Vasculature Experiment · p. 11
  • Pixel-DL outperformed time reversal on both synthetic and mouse-brain vasculature, while mDirect-DL did so only on synthetic images and performed worse than time reversal on mouse-brain images.The authors attribute mDirect-DL’s mouse-brain result to overfitting despite learning a sensor-data-to-image mapping.Source: Figure 5 · p. 11, Mouse Brain Vasculature Experiment · p. 11

Lung and Fundus Vasculature Experiment

For lung and fundus vasculature, Pixel-DL outperformed Post-DL across sparsity levels and matched iterative reconstruction on fundus while outperforming it on lung. Time reversal had the most artifacts and lowest average image-quality scores.

Lung and Fundus Vasculature Experiment

  • The CNNs were trained and tested on lung or fundus datasets, with natural differences between training and testing images due to patient variability and disease status.The testing images came from separate patients’ images than those used for training.Source: Lung and Fundus Vasculature Experiment · p. 12, Figure 6 · p. 12
  • Time reversal produced the most artifacts and lowest average PSNR and SSIM across sparsity levels for both vascular datasets.CNN-based and iterative reconstructions had fewer artifacts and higher average PSNR and SSIM.Source: Figure 6 · p. 12, Lung and Fundus Vasculature Experiment · p. 12
  • Pixel-DL consistently outperformed Post-DL for lung and fundus at every sparsity level, matched iterative reconstruction on fundus, and exceeded it on lung.At 16 sensors, Pixel-DL images appeared sharper and qualitatively superior to iterative images despite similar SSIM and PSNR.Source: Table 2 · p. 12, Lung and Fundus Vasculature Experiment · p. 13

Image Reconstruction Times

Pixel-DL matched iterative reconstruction quality while greatly reducing reconstruction time, and its performance compared favorably with other CNN methods across vasculature experiments. Its results also depended on training and phantom conditions.

Image Reconstruction Times

  • Pixel-DL took ~7.9 milliseconds per image, over 1000 times faster than iterative reconstruction (~491.21 seconds) while producing similar-quality images.Time reversal took ~2.57 seconds per image; Post-DL took ~2.58 seconds with time reversal as its initial inversion.Source: Image Reconstruction Times · p. 13

Discussion

  • Pixel-DL consistently outperformed time reversal, Post-DL, and mDirect-DL, and matched iterative reconstruction on mouse-brain vasculature despite synthetic-phantom-only training.With segregated anatomical data, it matched iterative reconstruction on fundus vasculature and outperformed it on lung vasculature.Source: Discussion · p. 13
  • For lung vasculature, the total variation constraint used by iterative reconstruction was likely suboptimal because the vessels were small and closely grouped.Source: Discussion · p. 13
  • A more varied training dataset may further improve Pixel-DL's generalization and performance.Source: Discussion · p. 13

Comparison between Deep Learning Approaches

The approaches differ chiefly in how they map sensor data into the CNN. Pixel-DL interpolates sensor information into image space, while Post-DL and mDirect-DL use an initial reconstruction or raw sensor data, respectively.

Comparison between Deep Learning Approaches

  • Post-DL feeds a time-reversal image to the CNN, but that initial inversion retains limited-view and sparse-sampling artifacts and removes access to sensor data.The CNN therefore can only use information retained in the image when removing artifacts.Source: Comparison between Deep Learning… · pp. 13–14
  • Pixel-DL replaces initial inversion with pixel-wise interpolation that windows relevant sensor data using a linear acoustic-wave model, giving the CNN a richer reconstruction input without initial-inversion artifacts.Source: Comparison between Deep Learning… · p. 14
  • mDirect-DL gives the CNN full sensor data without initial inversion, but the CNN struggled to generalize and overfit the training data while learning the sensor-to-image mapping.The authors suggest FD-UNet may be unsuitable because it assumes image input; a multidimensional time-series architecture may fit better.Source: Comparison between Deep Learning… · p. 14
  • Post-DL and Pixel-DL can add vessels absent from ground truth, particularly at the sparsest sampling level, where increasing sensor count may address the issue.Penalizing added vessels could instead discourage reconstruction of small vessels.Source: Comparison between Deep Learning… · p. 14

Deep Learning for In Vivo Imaging

Synthetic training data can ease data-collection demands, but its usefulness depends on matching experimental conditions. Pixel-DL is computationally efficient, while its acoustic model and demonstrated scope set boundaries for in-vivo use and transfer to other modalities.

Deep Learning for In Vivo Imaging

  • Synthetic datasets with augmentation can be made arbitrarily large, but CNN reconstruction quality depends on how closely their simulations match experimental conditions.Experimental training data may be infeasible because it is costly, time-consuming, and must be repeated when system configuration or target changes.Source: Deep Learning for In Vivo Imaging · pp. 14–15
  • Pixel-DL's linear acoustic model assumes spherical propagation at constant sound speed; heterogeneous media require another model or training examples of similar heterogeneities.Using homogeneous-medium assumptions for heterogeneous media can add artifacts and potentially affect CNN performance.Source: Deep Learning for In Vivo Imaging · p. 15

Deep Learning for Fast Image Reconstruction

  • Pixel-DL achieved similar or better image quality than iterative reconstruction and was over 1000 times faster, supporting real-time rendering under limited-view and sparse sampling.Source: Deep Learning for Fast Image… · p. 15
  • The authors demonstrated Pixel-DL in silico for vasculature-like PAT targets; they describe ultrasound as a readily adaptable application and other modalities as adaptable when a sensor-to-image model is available.Source: Deep Learning for Fast Image… · p. 15

Related papers

Loading 1911.04357v2…