Source-linked AI summary
Understanding Adversarial Attacks on Deep Learning Based Medical Image Analysis Systems
Xingjun Ma, Yuhao Niu, Lin Gu, Yisen Wang, Yitian Zhao, James Bailey, Feng Lu
TL;DR
Medical DNNs are vulnerable to carefully crafted, imperceptible adversarial perturbations, but the extent and detectability of this risk in medical imaging were not fully understood. The paper studies attack generation and detection, finding that medical attacks are easier to craft than natural-image attacks and that simple deep-feature detectors exceed 98% detection AUC. These findings clarify attack behavior and support efforts toward more explainable and robust medical systems.
Problem
The paper addresses limited understanding of adversarial attacks in medical images, including their vulnerability and detectability, despite safety concerns for medical diagnosis.
Method
The paper conducts experiments investigating both the crafting and detection of adversarial attacks on medical images.
Results
Medical attacks are easier to craft than natural-image attacks, while simple deep-feature detectors achieve over 98% detection AUC against all tested attacks across three datasets.
Takeaways & Limitations
The findings provide a basis for understanding medical DNN reliability and for developing more explainable and robust medical deep-learning systems.
Takeaways & Limitations
Whether physical-world medical examples can fool both medical deep-learning systems and medical experts remains unclear.
Abstract
from arXiv · showhide
Deep neural networks (DNNs) have become popular for medical image analysis tasks like cancer diagnosis and lesion detection. However, a recent study demonstrates that medical deep learning systems can be compromised by carefully-engineered adversarial examples/attacks with small imperceptible perturbations. This raises safety concerns about the deployment of these systems in clinical settings. In this paper, we provide a deeper understanding of adversarial examples in the context of medical images. We find that medical DNN models can be more vulnerable to adversarial attacks compared to models for natural images, according to two different viewpoints. Surprisingly, we also find that medical adversarial attacks can be easily detected, i.e., simple detectors can achieve over 98% detection AUC against state-of-the-art attacks, due to fundamental feature differences compared to normal examples. We believe these findings may be a useful basis to approach the design of more explainable and secure medical deep learning systems.
1. Introduction
Medical image DNNs raise unresolved security questions because adversarial attacks can manipulate clinical predictions, while their domain-specific vulnerability and detectability remain insufficiently understood. The paper investigates attack generation and detection, finding greater vulnerability than natural-image models, over 98% detection AUC, and feature differences linked to widespread perturbations outside lesions.
- Motivation: Adversarial attacks can manipulate predictions from medical DNNs with small, imperceptible perturbations, creating safety concerns for clinical deployment.The concern is heightened because medical systems support diagnosis and other safety-critical decisions.
- Motivation: Medical-image adversarial attacks remain insufficiently understood because prior adversarial machine-learning research has focused mainly on natural images.Medical images may contain domain-specific characteristics, including unique biological textures.
- Approach: The paper investigates both the crafting and detection of adversarial attacks across medical-image datasets and DNN systems.Its experiments examine adversarial vulnerability and detectability rather than only attack success.
- Findings: Medical-image attacks can succeed more easily than natural-image attacks because successful attacks require less perturbation.The paper attributes higher vulnerability partly to complex biological textures and, most importantly, overparameterized natural-image architectures with sharp loss landscapes.
- Findings: Over 98% detection AUC was achieved by a simple deep-feature detector against all tested attacks across three medical datasets.The paper presents this as the first reported work on detecting adversarial attacks in the medical-image domain.
- Findings: Adversarial perturbations often spread outside pathological regions, producing deep features recognizably different from normal examples.These feature differences help explain why the attacks are readily detectable.
2. Background of Medical Image Analysis
Medical image analysis uses DNNs across diverse modalities, anatomical areas, and clinical tasks. Common pipelines feed medical images into CNNs that learn intermediate features before producing final outputs.
- Applications: DNNs support medical tasks including diagnosis, organ and lesion segmentation, and image registration.Examples include diabetic-retinopathy staging, lung-disease detection, skin-cancer analysis, vessel and kidney segmentation, and CT–MRI alignment.
- Modalities and areas: Medical-image analysis spans visible-light, hyperspectral, X-ray, and nuclear-magnetic-resonance inputs across anatomical areas such as the brain, chest, skin, and retina.The passage also identifies brain imaging and ophthalmic imaging as important application areas.
- Common pipeline: Most medical-image diagnosis methods use a similar pipeline in which images enter CNNs that learn intermediate medical features before generating final outputs.Representative architectures include AlexNet, VGG, Inception, and ResNet.
3. Preliminaries
The paper introduces adversarial attacks as constrained perturbations of medical images and reviews representative attack and detection methods. It also describes the datasets, models, and experimental setup used for medical image classification.
- Adversarial attacks: Adversarial attacks maximize classification error while keeping xadv within an ϵ-ball of the original image under an Lp-norm constraint.The L∞ norm is commonly used because it is considered consistent with human perception.
- Representative attacks: FGSM applies one perturbation step, whereas BIM and PGD iteratively perturb inputs with smaller steps.PGD projects each intermediate example back onto the ϵ-ball and uses a random start.
- Representative attacks: CW is an optimization-based state-of-the-art attack, with the paper focusing on its L∞ version.The targeted L∞ CW attack can be solved iteratively using PGD.
- Experimental setup: The study focuses on FGSM, BIM, PGD, and CW, applying bounded attacks under a predefined maximum L∞ perturbation ϵ.These attacks are used on AdvTrain and AdvTest subsets according to the experimental pipeline.
- Adversarial detection: Adversarial detection distinguishes perturbed examples from clean examples using features extracted from different DNN layers.The reviewed approaches include Kernel Density and Local Intrinsic Dimensionality.
- Datasets and models: The experiments use public datasets for diabetic retinopathy, thorax disease, and melanoma classification, with Train, Test, AdvTrain, and AdvTest splits.Five datasets are used, and the classification tasks include two-class and multi-class settings.
4. Understanding Adversarial Attacks on Medical Image DNNs
The paper evaluates four white-box adversarial attacks across medical image datasets and finds that medical DNNs are highly vulnerable, especially compared with natural-image models. It attributes this vulnerability to medical-image characteristics and overparameterized networks, while noting that medical adversarial examples can nevertheless be detected effectively.
- Attack Setup: Four L∞-bounded attacks—FGSM, BIM, PGD, and CW—are evaluated on five medical image datasets across varying perturbation sizes.The study reports results on both AdvTrain and AdvTest image subsets.
- Attack Results: BIM, PGD, and CW generally succeed with maximum perturbation ϵ < 1.0/255 on 2-class medical datasets.Their attacks can reduce model accuracy below 1% in the reported experiments.
- Attack Results: Medical attacks require less perturbation than targeted attacks on CIFAR-10 and ImageNet, which often require > 8.0/255 for general success.Attack difficulty is measured by the least maximum perturbation needed for most attacks to succeed.
- Attack Results: Multi-class medical datasets are more vulnerable than 2-class datasets, with attack success increasing as the number of classes rises.At ϵ = 0.3/255, model accuracy decreases as the number of classes increases; BIM, PGD, and CW succeed more than 99% of the time at ϵ = 1.0/255.
- Explanations: The paper links medical models’ vulnerability to larger high-attention regions in medical images and to overparameterized networks learning relatively simple representations for medical tasks.The authors argue that complex biological textures may create vulnerable regions, while networks designed for large-scale natural images may be excessive for medical imaging tasks.
5. Understanding the Detection of Medical Image Attacks
The paper evaluates detectors for medical image attacks and finds that adversarial features differ substantially from normal features, making these attacks highly detectable. Representation and attention analyses suggest that perturbations affect medical images differently from natural images.
- Detection setup: The detection pipeline generates FGSM, BIM, PGD, and CW examples, extracts deep, quantized, KD, and LID features, and trains detectors on AdvTrain before testing on AdvTest.KD and LID use logistic regression, deep features use random forests, and quantized features use SVMs; AUC measures detection performance.
- Detection results: Above 99% detection AUC was achieved by KD-based detectors against all tested attacks across all three medical datasets.The paper contrasts this with less than 80% detection AUC reported for some attacks on natural images.
- Detection results: Above 98% detection AUC was achieved by deep-feature detectors across all testing scenarios, while quantized deep-feature detectors also performed well.These results indicate fundamental differences between adversarial and normal deep features.
- Detection transferability: 100% detection AUC transferred from detectors trained on either FGSM or PGD to other attacks, confirming strong cross-attack detectability.The transferability test trains on source attacks and evaluates against the remaining attacks.
- Feature analysis: t-SNE embeddings show adversarial features are almost linearly separable from normal features, unlike the more similar feature distributions reported for natural images.The comparison uses second-last dense-layer features across datasets and attacks.
- Representation and attention analysis: Medical-image perturbations cause more significant deep-feature distortions and shift attention from lesions toward regions irrelevant to diagnosis.The analysis examines deeper ResNet-50 representations and Grad-CAM attention maps.
6. Discussion and Conclusion
The discussion identifies scope boundaries for current medical-image attacks and outlines remaining defense and physical-world questions. The conclusion reports strong detectability and positions the findings as a basis for more explainable and secure medical systems.
- Discussion: Existing attacks can fool medical DNNs, but their small, imperceptible perturbations have very limited impact on diagnosis when medical experts are involved.Whether physical-world examples can fool both medical systems and experts remains unclear.
- Discussion: More effective medical-image defenses may need to address DNN overparameterization in addition to adapting adversarial training and regularization methods from natural images.The paper identifies effective defense techniques as an imperative for medical adversarial examples.
- Conclusion: Across four attack and detection methods and three benchmark datasets, medical attacks were easier to craft and detect, with deep-feature detectors exceeding 98% detection AUC.Detectors trained on one attack also transferred well to other attacks.
- Conclusion: Attacks often affect widespread areas outside pathological regions, producing deep features that are fundamentally different and easily separable from normal features.This provides the paper’s explanation for the observed detectability of medical adversarial examples.
- Conclusion: The findings may support understanding wrong medical-system decisions and designing more explainable and secure medical deep learning systems.The paper presents this as a basis for future practical defense approaches.