Source-linked AI summary
DeepPET: A deep encoder-decoder network for directly solving the PET reconstruction inverse problem
Ida Häggström, C. Ross Schmidtlein, Gabriele Campanella, Thomas J. Fuchs
TL;DR
PET reconstruction is slowed by iterative methods and complicated by noisy, ill-posed inverse modeling. DeepPET uses a supervised convolutional encoder–decoder to map sinograms directly to images, achieving 104 times faster reconstruction than standard OSEM while reporting comparable reconstruction quality in the abstract. The study relies on simulated training data and does not compare against regularized iterative techniques.
Problem
PET image reconstruction is computationally slow and requires modeling and regularization choices for noisy, ill-posed sinogram-to-image inversion.
Method
DeepPET uses a supervised deep convolutional encoder–decoder trained on realistic synthetic PET data to directly reconstruct images from sinograms.
Results
104 times faster than standard OSEM, DeepPET reconstructed images in 11 ms per image versus 1104 ms for OSEM, with average rRMSE 0.90 versus 0.50 for OSEM.
Takeaways & Limitations
The results indicate potential for deep learning to support faster PET reconstruction and increased patient throughput, diagnosis, and treatment decisions.
Takeaways & Limitations
The evaluation uses simulated rather than real clinical data, so the trained network may learn simulation-specific assumptions instead of real data properties.
Abstract
from arXiv · showhide
Positron emission tomography (PET) is a cornerstone of modern radiology. The ability to detect cancer and metastases in whole body scans fundamentally changed cancer diagnosis and treatment. One of the main bottlenecks in the clinical application is the time it takes to reconstruct the anatomical image from the deluge of data in PET imaging. State-of-the art methods based on expectation maximization can take hours for a single patient and depend on manual fine-tuning. This results not only in financial burden for hospitals but more importantly leads to less efficient patient handling, evaluation, and ultimately diagnosis and treatment for patients. To overcome this problem we present a novel PET image reconstruction technique based on a deep convolutional encoder-decoder network, that takes PET sinogram data as input and directly outputs full PET images. Using realistic simulated data, we demonstrate that our network is able to reconstruct images >100 times faster, and with comparable image quality (in terms of root mean squared error) relative to conventional iterative reconstruction techniques.
1. Introduction
PET supports sensitive cancer diagnosis and treatment, but producing reliable images from noisy sinogram data is an ill-posed inverse problem. Existing reconstruction methods face artifacts, regularization challenges, and expensive iterative computation, motivating a directly learned solution.
- Clinical motivation: PET’s sensitivity supports earlier and more precise cancer diagnosis, staging, and treatment management, making reliable quantitative image quality essential.PET can detect radiotracer concentrations as low as picomolar.
- PET reconstruction: Sinograms record photon coincidence detections between detector pairs and represent projections of the underlying activity distribution.The sinogram-to-image relationship is established by the projection-slice theorem but is corrupted by random process noise.
- PET reconstruction: PET image reconstruction is an ill-posed inverse problem because noisy sinogram measurements must be mapped back to tracer distributions.The inverse formulation represents explicit models of the physical system’s mean values.
- Existing methods: Standard FBP and maximum-likelihood methods can produce streaks and noise from data/model mismatches, inconsistency, and overfitting.Regularization can reduce fitting noise but requires many iterations before its benefits become apparent.
- Motivation for DeepPET: Deep networks can learn the physical inverse, statistical model, and data-matched regularization directly from examples.The proposed encoder–decoder approach is also motivated by replacing repeated expensive system-matrix operations with a learned inverse operator.
2. Related work
Prior deep-learning work in medical imaging mainly post-processed reconstructed images or assisted iterative reconstruction. This work instead targets direct PET image generation from sinograms using a convolutional encoder–decoder architecture.
- Deep-learning background: Convolutional encoder–decoder models compress image data into latent representations and rebuild them into full datasets.Such architectures have been applied to medical image restoration and synthesis tasks.
- Prior approaches: Earlier imaging studies used deep networks to restore reconstructed images or regress sparse-view CT reconstructions toward full-view images.These approaches generally operate after an image has already been reconstructed.
- Research gap: Deep learning within PET reconstruction itself remained less explored than post-processing reconstructed images.A prior CNN-assisted CT method still resembled conventional regularized iterative least-squares reconstruction.
- This work: The paper aims to design a deep convolutional encoder–decoder that directly and quickly reconstructs high-quality images from PET sinogram data.The authors present it as a first systematic effort of this kind in direct deep-learning-based tomographic reconstruction.
3. Theory
PET reconstruction models measurements with Poisson statistics and solve an optimization problem combining data fidelity with regularization. Deep learning may learn these modeling components from realistic training data, although preprocessing and simulation assumptions constrain the formulation.
- Forward model: PET emission reconstruction uses a Poisson noise model for measured sinogram data generated by a projection operator and additive random and scatter counts.The unknown activity distribution is estimated from these measurements.
- Optimization model: The reconstruction objective minimizes a KL-divergence data residual together with a regularization term weighted by λ.The KL-divergence term accounts for physical aspects such as geometry and statistical distribution.
- Model assumptions: The formulation depends on assumptions about detector geometry, point-spread function, additive counts, Poisson statistics, and the regularizer’s functional form.The optimal regularization function is not generally known and remains an active research problem.
- Learned model: A deep network can learn geometric, statistical, and regularization properties from realistic training examples instead of requiring them to be explicitly specified.Those properties may still be implicitly determined by the model used to generate the training data.
- Precorrection: The study uses precorrected data, which mixes statistical properties and is generally avoided in inverse-problem formulations because PET explicitly uses a Poisson model.The correction is therefore a methodological limitation of the reconstruction setup.
4. Experimental design
The study evaluates DeepPET, a convolutional encoder–decoder that maps simulated PET sinograms directly to images, against conventional reconstruction methods. The experimental design uses realistic patient-derived simulations, compares alternative architectures, and selects models using validation performance before test evaluation.
- DeepPET reconstructs PET images directly from sinogram data using a deep convolutional encoder–decoder rather than traditional iterative reconstruction.
- The dataset used 79 whole-body patient scans, with 128×128 PET slices and corresponding CT images supplying attenuation information for PETSTEP simulations.Each patient had an average of 256 slices, and nine activity-noise levels were simulated per slice.
- The final encoder contracts 288×269×1 sinograms into 1024 feature maps of size 18×17, while the decoder upsamples them into 128×128×1 PET images.The architecture uses strided 3×3 convolutions in the encoder and repeated upsampling, convolution, batch normalization, and ReLU in the decoder.
- Seven alternative CED designs, M1 through M7, were evaluated alongside DeepPET, with validation loss used to identify the chosen model.The models varied in convolutional depth, feature-layer depth, spatial sizes, and optimizer choice.
- Training used MSE loss and stochastic gradient descent, and the validation-selected model was evaluated on a held-out test set against FBP and unregularized OSEM.Image quality was assessed using relative root mean squared error, while reconstruction time was compared across methods.
5. Results
DeepPET training loss decreased as the network learned data features, and its test reconstructions were substantially faster than FBP and OSEM, with lower rRMSE than OSEM. Performance worsened for noisier low-count inputs, occasionally losing true structure.
- Training convergence: Training and validation loss decreased across epochs as the network learned to represent the data features.Figure 3 depicts average MSE against ground-truth simulations for training and validation sets.
- Reconstruction speed: 104 times faster than OSEM and 6 times faster than FBP, DeepPET reconstructed images in 11 ms on average versus 1104 ms and 59 ms.These are average single-image execution times in the test set.
- Reconstruction quality: DeepPET’s average rRMSE was 0.90, compared with 0.86 for FBP and 0.50 for OSEM.OSEM achieved the best rRMSE among the compared methods.
- Qualitative evaluation: Successful test-set reconstructions were shown alongside their PET sinograms, ground truths, and DeepPET outputs.Figure 5 organizes examples in grouped columns, while Figure 6 also compares conventional reconstructions across input noise levels.
- Noise robustness: At very low counts, DeepPET sometimes produced mostly blobs and lost true structure, although this regime was below approximately 100 kcnts generally found in the clinic.The authors suggest this behavior contributed to the relatively large spread in rRMSE.
6. Discussion
DeepPET reconstructs PET images in a single pass, offering substantial speed gains but showing failures at very low counts and several evaluation and generalization limitations.
- Efficiency: DeepPET uses one reconstruction pass, whereas conventional techniques require multiple iterations.The time advantage would likely be larger for regularized iterative reconstruction, which typically uses more iterations.
- Data limitations: The study used simulated rather than real clinical data because simulation provided access to ground-truth images.Simulation simplifications may cause the network to learn artificial properties instead of real data characteristics.
- Low-count behavior: At very low input counts, DeepPET can lose most underlying structure and produce images consisting mainly of one or a few blobs.Conventional methods may retain gross structure despite poor visual and quantitative quality.
- Data limitations: Ground-truth images were reconstructed clinical OSEM images with inherent noise and lower resolution before PET acquisition simulation.The authors suggest sharper ground truth could produce a network better able to generate sharp images.
- Evaluation scope: The evaluation compared DeepPET only with analytical FBP and unregularized OSEM, not regularized iterative techniques.RMSE was used as the final image-quality metric, although smoother images can score favorably even when detail is lost.
- Future scope: Extending the method to 3D would increase network complexity, training time, and memory requirements.The proposed extension replaces 2D convolutions and upsampling steps with 3D versions.
7. Conclusions
The paper presents DeepPET, an encoder–decoder network that directly reconstructs PET images from sinograms. On average, it reduces reconstruction time by more than 100-fold relative to conventional OSEM.
- DeepPET is presented as the first systematic deep-learning study to directly reconstruct PET images from sinogram data.
- DeepPET uses a novel encoder–decoder architecture for PET sinogram data.
- 11 ms average reconstruction time reduces speed relative to conventional OSEM at 1.10 s by a factor of more than 100.