Source-linked AI summary
Synthesis of Positron Emission Tomography (PET) Images via Multi-channel Generative Adversarial Networks (GANs)
Lei Bi, Jinman Kim, Ashnil Kumar, Dagan Feng, Michael Fulham
TL;DR
Scarce annotated medical-image data and limitations of existing synthesis approaches motivate PET image generation for CAD training. M-GAN combines labels and CT images in an adversarial framework, and on 50 lung-cancer PET-CT studies it produced images closer to real PET than conventional GAN variants, with competitive downstream tumor-detection performance against real-image training.
Problem
Medical-image CAD needs large annotated training datasets, but such data are scarce, and existing PET-synthesis approaches face constraints in image variation, spatial or appearance consistency, or tumor synthesis.
Source: §1 · pp. 2–3Method
M-GAN uses adversarial learning with label and CT inputs, using labels to synthesize high-uptake regions and CT to constrain appearance consistency.
Source: §1 · pp. 2–3, §2.1 · p. 4Results
Takeaways & Limitations
The authors conclude that M-GAN synthetic PET images can provide training data for machine-learning methods, with tumor-detection performance competitive to training on real PET images.
Source: §1 · p. 3, §5 · p. 7Tumors occupy a small portion of the image, making the overall evaluation less sensitive to differences between M-GAN and CT-GAN; CT-GAN also cannot reconstruct lesions, especially those adjacent to the mediastinum.
Source: §4 · p. 7
Abstract
from arXiv · showhide
Positron emission tomography (PET) image synthesis plays an important role, which can be used to boost the training data for computer aided diagnosis systems. However, existing image synthesis methods have problems in synthesizing the low resolution PET images. To address these limitations, we propose multi-channel generative adversarial networks (M-GAN) based PET image synthesis method. Different to the existing methods which rely on using low-level features, the proposed M-GAN is capable to represent the features in a high-level of semantic based on the adversarial learning concept. In addition, M-GAN enables to take the input from the annotation (label) to synthesize the high uptake regions e.g., tumors and from the computed tomography (CT) images to constrain the appearance consistency and output the synthetic PET images directly. Our results on 50 lung cancer PET-CT studies indicate that our method was much closer to the real PET images when compared with the existing methods.
1 Introduction
Medical-image CAD depends on large annotated datasets, but annotation is scarce and existing synthetic-image approaches have practical or image-quality limitations. The proposed M-GAN uses labels to synthesize high-uptake regions and CT images to constrain appearance.
1 Introduction
- Annotated medical-image training data are scarce, while phantoms and atlas-based synthesis have limitations in image quality, complexity, processing burden, and parameter tuning.The paper notes that atlas registration can also affect synthetic-image quality.Source: §1 · p. 2
- GAN-based PET synthesis faces complementary constraints: PET-to-PET struggles to create input variation, label-to-PET may lack constraints, and CT-to-PET may miss tumors.The proposed approach combines labels for high-uptake regions with CT for appearance consistency.Source: §1 · pp. 2–3
- M-GAN uses adversarial learning for high-level semantic features and combines label and CT inputs to synthesize PET images directly.The label can specify high-uptake regions on CT and need not be derived from that CT image.Source: §1 · pp. 2–3
2 Methods
M-GAN learns PET synthesis from paired label and CT inputs using an adversarial generator–discriminator framework. The evaluation used 50 lung-cancer patient studies, with tumor-containing slices split by patient between training and testing.
2.1 Multi-channel Generative Adversarial Networks (M-GANs)
- During training, the generator maps label and CT inputs to PET, while the discriminator separates synthetic from real PET images.The optimization seeks a setting that maximizes the discriminator objective while minimizing the generator objective.Source: Figure 1 · p. 3, §2.1 · p. 4
- The generator used a U-net, the discriminator a five-layer convolutional network, and λ was set empirically to 100 to balance the objective terms.The paper describes the general adversarial objective and its M-GAN formulation.Source: §2.1 · p. 4
2.2 Materials and Implementation Details
- The dataset comprised 50 lung-cancer patients, with PET volumes at 200×200 pixels and CT volumes at 512×512 pixels; tumors were identified using thresholding and manually adjusted against clinical reports.The scans came from one hospital and were acquired on a 128-slice Siemens Biograph mCT scanner; slice thickness was 3 mm.Source: §2.2 · pp. 4–5
- Evaluation used 876 tumor-containing trans-axial slices, with patient-disjoint groups of 25 studies alternated as training and test sets.Training took 6 hours over 200 epochs on a 12GB Maxwell Titan X GPU using Torch.Source: §2.2 · p. 5
3 Evaluation
M-GAN outperformed its single-channel variants on PET-image synthesis metrics and produced synthetic PET images that supported tumor detection competitively with real PET images.
3.1 Experimental Results for PET Image Synthesis
- M-GAN had the best performance across MAE and PSNR against the label-only LB-GAN and CT-only CT-GAN variants.Table 1 reports the lowest MAE and highest PSNR for M-GAN.Source: §3.1 · p. 5, Table 1
3.2 Using Synthetic PET Images for Training
- Figure 2 compares synthetic PET made from labels alone or CT alone with M-GAN output made from both inputs.Its panels show the label, CT, and real PET alongside the single-input and dual-input synthetic images.Source: Figure 2 · p. 6
- FCNs trained on M-GAN synthetic PET performed competitively with FCNs trained on real PET for tumor detection.The evaluation counted detections with more than 50% overlap with an annotated tumor as true positives.Source: §3.2 · p. 6, Table 2 · p. 6
4 Discussion
M-GAN had the strongest reported image-synthesis metrics, while single-channel variants showed distinct shortcomings: label-only synthesis lost anatomical detail, and CT-only synthesis often missed tumors.
4 Discussion
- M-GAN was closer to real images than the other GAN variants, with MAE 4.60 and PSNR 28.06.The discussion identifies these as the lowest MAE and highest PSNR scores, respectively.Source: §4 · p. 7
- LB-GAN often produced poor anatomical definitions, including missing lung boundaries and incorrectly synthesized mediastinum regions.The discussion attributes this limitation to the lack of spatial and appearance constraints from CT.Source: §4 · p. 7
- CT-GAN could not reliably reconstruct high-uptake tumors, particularly near the mediastinum, despite competitive MAE and PSNR.Tumor regions occupy a small portion of the image, and CT-GAN’s synthetic images had inconsistent tumor labeling and the lowest FCN detection results.Source: §4 · p. 7
- The detection comparison with real-PET training highlights the combined use of labels for tumors and CT for appearance consistency.The authors point to this integration as an advantage of M-GAN.Source: §4 · p. 7
5 Conclusion
The conclusion presents M-GAN as a multi-channel approach for synthesizing PET images from labels and CT. In 50 lung cancer PET-CT studies, its outputs were closer to real PET than conventional GAN outputs, and supported tumor detection competitively with real PET.
5 Conclusion
- M-GAN embeds multiple input channels in a GAN, using labels to learn high-uptake regions such as tumors and CT data to constrain spatial and appearance consistency.The reported preliminary evaluation used 50 lung cancer PET-CT studies.Source: §5 · p. 7
- M-GAN synthetic PET images were reported as much closer to real PET than images from conventional GAN approaches.Source: §5 · p. 7
- The authors evaluated synthetic images as replacements for original PET and identify optimally combining real and synthetic images as future work.They suggest the framework could potentially boost training data and extend to PET-MRI synthesis.Source: §5 · p. 7