Source-linked AI summary
How to Fool Radiologists with Generative Adversarial Networks? A Visual Turing Test for Lung Cancer Diagnosis
Maria J. M. Chuquicusma, Sarfaraz Hussein, Jeremy Burt, Ulas Bagci
TL;DR
The paper tackles the difficulty of learning discriminative features for malignant-versus-benign lung-nodule diagnosis. It uses DC-GANs to generate realistic nodules and tests them with two radiologists; the generated samples deceived radiologists in the reported experiments, while limitations remain in feature purity, quality, variation, and evaluation scope.
Problem
Distinguishing malignant from benign lung nodules is difficult because nodule appearances vary, creating risks of underdiagnosis, overdiagnosis, unnecessary surgeries, and misdiagnoses.
Method
The study uses unsupervised DC-GANs to generate benign, malignant, and mixed lung nodules, then evaluates generated-versus-real and class-specific distinctions through radiologists’ Visual Turing tests.
Results
Generated samples deceived radiologist 1 in 67% and radiologist 2 in 100% of experiments; mean true-recognition rates were 58.56% and 93.52%, respectively.
Takeaways & Limitations
The experiments indicate that DC-GANs can generate realistic lung-nodule images and provide samples for examining discriminative features, radiologist training, and deep-learning classifiers.
Takeaways & Limitations
Generated samples may contain features from both malignant and benign nodules, and further improvement in image quality and variation is needed; future evaluation should include more experts and 3D generation.
Abstract
from arXiv · showhide
Discriminating lung nodules as malignant or benign is still an underlying challenge. To address this challenge, radiologists need computer aided diagnosis (CAD) systems which can assist in learning discriminative imaging features corresponding to malignant and benign nodules. However, learning highly discriminative imaging features is an open problem. In this paper, our aim is to learn the most discriminative features pertaining to lung nodules by using an adversarial learning methodology. Specifically, we propose to use unsupervised learning with Deep Convolutional-Generative Adversarial Networks (DC-GANs) to generate lung nodule samples realistically. We hypothesize that imaging features of lung nodules will be discriminative if it is hard to differentiate them (fake) from real (true) nodules. To test this hypothesis, we present Visual Turing tests to two radiologists in order to evaluate the quality of the generated (fake) nodules. Extensive comparisons are performed in discerning real, generated, benign, and malignant nodules. This experimental set up allows us to validate the overall quality of the generated nodules, which can then be used to (1) improve diagnostic decisions by mining highly discriminative imaging features, (2) train radiologists for educational purposes, and (3) generate realistic samples to train deep networks with big data.
1. INTRODUCTION
The paper addresses the difficulty of distinguishing malignant from benign lung nodules by using adversarial learning to generate realistic samples and test their perceptual indistinguishability. It evaluates generated nodules through radiologists’ Visual Turing tests and considers class-specific generation for diagnosis, training, and deep-network development.
- Lung nodules vary in shape, size, intensity, location, and other characteristics, making radiographic analysis vulnerable to underdiagnosis, overdiagnosis, unnecessary surgeries, and misdiagnoses.
- Prior CAD research used supervised 2D-CNN, 3D-CNN, and multitask-learning approaches to identify discriminative features associated with malignant and benign nodules.
- GANs provide an unsupervised framework in which a generator creates realistic samples while an adversarial discriminator learns features useful for distinguishing real from generated images.
- The study generates lung nodules with GANs and evaluates their realism using Visual Turing tests administered to radiologists.
- Generated benign, malignant, and mixed nodules are intended to support identification of class-specific features, radiology-resident training, and modeling with complex deep-learning classifiers.
2. MATERIALS AND METHODS
The experiments use annotated LIDC-IDRI nodules and a DC-GAN to generate benign, malignant, and mixed samples. Two radiologists assess generated-versus-real and class-specific distinctions through 18 Visual Turing experiments after image preprocessing.
- Dataset: The final LIDC-IDRI dataset contains 1,145 selected nodules, including 635 benign and 510 malignant nodules, with malignancy-rating-3 nodules excluded.
- Network Architecture: The DC-GAN maps random noise to generated samples, while its discriminator classifies original pdata samples and generated samples as real or fake.
- Network Architecture: The proposed DC-GAN uses convolutional layers, batch normalization, no fully connected generator layers, and ReLU, leaky ReLU, and tanh activations.
- Training: Training produces benign, malignant, and mixed samples with batch size 64; discriminator and generator learning rates are .0001 and .0002, respectively.Training stopped at 114,000 benign, 110,000 malignant, and 99,000 mixed-sample iterations because image quality no longer improved.
- Visual Turing Test: Two radiologists participated in 18 Visual Turing experiments covering all, benign, malignant, and mixed generated-or-real nodule conditions.The CT images came from LIDC-IDRI and were not prepared by the participating radiologists; anisotropic diffusion was applied for visualization.
3. RESULTS
The experiments quantified how often radiologists recognized generated nodules as fake and how consistently they classified nodule types. Generated nodules deceived radiologists in many experiments, while inter-observer agreement remained limited.
- The reported experiments included all-generated or mixed real/generated nodules for FRR, while all-real experiments were excluded from fake-versus-real comparisons.
- FRR measured the percentage of fake nodules recognized as fake, whereas TRR measured the percentage of real nodules recognized as real.
- Radiologist 1 was convinced generated nodules were real in 67% of experiments, compared with 100% for radiologist 2.
- Radiologist 1 had a mean TRR of 58.56%, while radiologist 2 had a mean TRR of 93.52%.
- Both radiologists had mean inter-observer agreement of 58.56% for malignant/benign cases and 44.91% for real/generated cases.
- The authors interpret these statistics as evidence that DC-GAN generated high-quality nodules and that radiologists struggled to distinguish malignant from benign nodules.
4. DISCUSSION & CONCLUSION
The authors conclude that DC-GAN samples can deceive radiologists, but generated images still have limitations in class-specific features, quality, and variation. Future work expands evaluation and moves toward volumetric 3D generation.
- Generated samples contained potentially overlapping malignant and benign features, making visualization difficult for radiologists to interpret.
- The generated samples require further improvement in image quality and variation.
- Future experiments will include more radiologists and experts to expand the evaluation setup.
- The authors plan to explore 3D image generation because CT scans are volumetric and 2D slices can lose volumetric information.