Source-linked AI summary
Cross-Modality Synthesis from CT to PET using FCN and GAN Networks for Improved Automated Lesion Detection
Avi Ben-Cohen, Eyal Klang, Stephen P. Raskin, Shelly Soffer, Simona Ben-Haim, Eli Konen, Michal Marianne Amitai, Hayit Greenspan
TL;DR
Because PET/CT is costly, exposes patients to additional radiation, and is not broadly available, this paper estimates PET-like images from CT using an FCN-cGAN system. Integrated with liver-lesion detection software, the synthesized PET reduced average false positives from 2.9 to 2.1 per case, while the study remained focused on liver-region experiments.
Problem
PET/CT provides clinically useful lesion information but adds radiation and cost compared with CT-only imaging, motivating an automated PET-like alternative.
Method
The system uses an FCN to generate PET-like images from CT scans and a cGAN to refine the synthesized output for malignant liver-lesion analysis.
Results
28% improvement reduced average false positives in existing liver-lesion detection software from 2.9 to 2.1 per case using a pathological SUV threshold of 2.5.
Takeaways & Limitations
Synthesized PET can serve as a false-positive reduction layer for existing automated liver-lesion detection software without manual labeling during training.
Takeaways & Limitations
Experiments focused on the liver, and adapting the method to other tumor types may require organ-specific retraining.
Abstract
from arXiv · showhide
In this work we present a novel system for generation of virtual PET images using CT scans. We combine a fully convolutional network (FCN) with a conditional generative adversarial network (GAN) to generate simulated PET data from given input CT data. The synthesized PET can be used for false-positive reduction in lesion detection solutions. Clinically, such solutions may enable lesion detection and drug treatment evaluation in a CT-only environment, thus reducing the need for the more expensive and radioactive PET/CT scan. Our dataset includes 60 PET/CT scans from Sheba Medical center. We used 23 scans for training and 37 for testing. Different schemes to achieve the synthesized output were qualitatively compared. Quantitative evaluation was conducted using an existing lesion detection software, combining the synthesized PET as a false positive reduction layer for the detection of malignant lesions in the liver. Current results look promising showing a 28% reduction in the average false positive per case from 2.9 to 2.1. The suggested solution is comprehensive and can be expanded to additional body organs, and different modalities.
1. Introduction
PET/CT supports oncology diagnosis, staging, lesion detection, and drug-therapy evaluation, but its cost, radiation exposure, and limited availability motivate PET-like imaging from CT. This work develops an automated FCN-cGAN system focused on malignant liver lesions and builds on prior cross-modality synthesis methods.
- Clinical motivation: PET/CT is used for oncology diagnosis and staging, while increased FDG accumulation can help detect and localize malignant lesions.PET can reveal a malignant liver lesion that is less visible in the corresponding CT image.
- Clinical motivation: PET/CT adds radiation exposure and cost relative to CT-only scans, limiting its availability and motivating less expensive, fast, PET-like alternatives.The paper frames CT-derived PET-like imaging as a potential alternative to standard PET/CT imaging.
- Related work: Prior work explored deep-learning cross-modality synthesis, including CycleGAN and multi-channel GAN approaches for generating CT or PET-like images.A previous PET-synthesis model achieved detection performance comparable to original PET but required manually annotated tumors.
- Contribution: The synthesized PET is intended to improve existing automated lesion detection software, while reconstruction measures compare alternative synthesis methods.The work targets automated detection rather than relying on manual tumor labeling.
- Contribution: The proposed system combines a fully convolutional network with a conditional GAN to synthesize PET images from CT data, emphasizing malignant liver lesions without manual labels.The FCN and cGAN are presented as complementary deep-learning components for virtual PET generation.
- Novelty: Compared with earlier work, the new architecture removes pyramid-based image blending, avoiding threshold definition and reducing processing time while improving reported performance.The dataset was also substantially extended relative to the earlier system.
2. Methods
The framework aligns CT and PET data, normalizes their intensity ranges, and synthesizes PET-like images with an FCN refined by a conditional GAN. Loss weighting emphasizes malignant-lesion-associated high-SUV regions while balancing reconstruction quality across SUV ranges.
- Training Data Preparation: CT and PET scans are aligned using scan offsets and voxel-size ratios before FCN training.An affine transformation with linear interpolation maps PET data to the CT scan.
- Intensity Preprocessing: PET values are clipped and scaled to SUV 0–20, while CT values are windowed to −160 HU to 240 HU.These ranges target malignant-lesion uptake and the liver-parenchyma window used by radiologists.
- Fully Convolutional Network: The FCN converts a VGG-16-based architecture into a fully convolutional model that predicts initial PET-like images from CT input.Fully connected layers become convolutions, and upsampling enables end-to-end learning with pixelwise L2 loss.
- Conditional GAN: The cGAN refines FCN outputs using CT and FCN prediction channels, with generator reconstruction and discriminator classification objectives.The generator receives a simple concatenation of the CT image and corresponding FCN output.
- Loss Weights: SUV-weighted pixel loss prioritizes high-uptake regions, while separate high- and low-SUV reconstruction averages improve cGAN balance.High SUV is defined as > 2.5, and low SUV as ≤2.5; weighting helped the FCN but required modification for cGAN training.
3. Experiments and Results
Experiments evaluated virtual PET reconstruction and its use as a false-positive reduction layer for automated liver lesion detection. The combined FCN–cGAN approach achieved strong reconstruction measurements and reduced false positives while largely preserving detection sensitivity.
- Experimental setup: 60 PET/CT pairs from Sheba Medical Center supported development, validation, and independent testing of liver-region slices.The dataset was acquired from 2014–2015 and included paired contrast-enhanced portal-phase CT and PET scans.
- Reconstruction evaluation: High SUV regions above 2.5 were evaluated separately from low SUV regions because high uptake can indicate malignant lesions.The final reconstruction score averaged the errors for the high- and low-SUV sets.
- Reconstruction evaluation: The combined FCN-4s and cGAN achieved average MAE values of 0.72 and 0.79 and PSNR values of 30.22 and 30.4 using the equation 8 and equation 7 losses, respectively.Equation 8 was preferred because it produced better reconstruction measurements for high SUVs.
- Qualitative results: Virtual PET images showed responses similar to real PET in examples containing malignant lesions, metastases, and benign cysts.Compared with image blending, the proposed method produced fewer marked false-positive blobs in the metastasis example and improved surrounding-tissue appearance in one cyst example.
- Liver lesion detection: The synthesized PET was thresholded at SUV 2.5 and intersected with CT-based lesion candidates to remove some false detections.The layer was added to an existing two-module detector combining FCN global context with localized superpixel sparse-based classification.
- Liver lesion detection: 28%: average FPR decreased from 2.9 to 2.1 per scan with P-value<0.05, while TPR remained similar.Across tested probability thresholds, the FROC comparison reported fewer false positives while preserving TPR; one reported TPR change was from 90.9% to 89.1% and was not significant.
4. Discussion
The FCN–cGAN framework produced realistic PET estimates from CT, improved automated liver-lesion detection, and required no manual labeling for training. Its evaluation showed promising reconstruction and detection results, while the authors identify organ-specific scope and several avenues for refinement.
- Contributions: The FCN–cGAN system synthesizes PET from CT with special attention to malignant lesions, using custom losses and no manually labeled training data.The cGAN refines the FCN output, and the framework was designed for automated use.
- Reconstruction results: The proposed method achieved the best average balance between high- and low-SUV reconstruction regions, although other methods led on individual regional measures.FCN-4s performed best for high-SUV regions, while cGAN and image blending performed best for low-SUV regions.
- Detection results: 28% improvement reduced false positives from an average of 2.9 to 2.1 per case when virtual PET was integrated into existing liver-lesion detection software.The experiment used a pathological SUV threshold of 2.5, and the synthesized PET preserved TPR while reducing false positives across tested thresholds.
- Limitations and future work: The experiments focused on the liver, and adaptation to other tumor types may require retraining the method separately for each organ because CT appearances differ.The authors call for further work on malignancies in other CT regions, preferably without manual annotation.
- Limitations and future work: Virtual PET is blurry, making lesion segmentation difficult; the authors therefore consider detection approaches more relevant for this method.They also describe the current false-positive reduction as based on a relatively naïve thresholding approach.
- Practical significance: The framework’s realistic PET estimation and detection gains were obtained without manual labeling, addressing the difficulty and data burden of annotating medical images.The authors identify the absence of manual labeling as a major strength.