Source-linked AI summary
Adversarial Attack Vulnerability of Medical Image Analysis Systems: Unexplored Factors
Gerda Bortsova, Cristina González-Gonzalo, Suzanne C. Wetstein, Florian Dubost, Ioannis Katramados, Laurens Hogeweg, Bart Liefers, Bram van Ginneken, Josien P. W. Pluim, Mitko Veta, Clara I. Sánchez, Marleen de Bruijne
TL;DR
Medical image analysis systems may be vulnerable to black-box adversarial attacks, but the roles of pre-training and development-data disparity were previously unclear. The paper tests these factors across ophthalmology, radiology, and pathology, finding that pre-training can increase transferability while data and architecture differences reduce attack success. These factors should be considered when designing security-critical MedIA systems.
Problem
The paper addresses limited evidence about how ImageNet pre-training and development-data disparity affect black-box adversarial attack transferability in medical image analysis.
Method
The study varies initialization, development data, and architecture between surrogate and target models across three medical domains using visually subtle, transferability-tuned attacks.
Results
Pre-training may dramatically increase adversarial-example transferability, whereas development-data disparity substantially decreases attack success, with architecture disparity amplifying that decrease.
Takeaways & Limitations
Initialization, development data, and architecture should be considered when designing security-critical MedIA systems intended for clinical deployment.
Takeaways & Limitations
The study leaves open whether adversarial pre-training could improve black-box robustness and notes that perceptibility lacks an accepted SSIM threshold.
Abstract
from arXiv · showhide
Adversarial attacks are considered a potentially serious security threat for machine learning systems. Medical image analysis (MedIA) systems have recently been argued to be vulnerable to adversarial attacks due to strong financial incentives and the associated technological infrastructure. In this paper, we study previously unexplored factors affecting adversarial attack vulnerability of deep learning MedIA systems in three medical domains: ophthalmology, radiology, and pathology. We focus on adversarial black-box settings, in which the attacker does not have full access to the target model and usually uses another model, commonly referred to as surrogate model, to craft adversarial examples. We consider this to be the most realistic scenario for MedIA systems. Firstly, we study the effect of weight initialization (ImageNet vs. random) on the transferability of adversarial attacks from the surrogate model to the target model. Secondly, we study the influence of differences in development data between target and surrogate models. We further study the interaction of weight initialization and data differences with differences in model architecture. All experiments were done with a perturbation degree tuned to ensure maximal transferability at minimal visual perceptibility of the attacks. Our experiments show that pre-training may dramatically increase the transferability of adversarial examples, even when the target and surrogate's architectures are different: the larger the performance gain using pre-training, the larger the transferability. Differences in the development data between target and surrogate models considerably decrease the performance of the attack; this decrease is further amplified by difference in the model architecture. We believe these factors should be considered when developing security-critical MedIA systems planned to be deployed in clinical practice.
1. Introduction
The paper examines adversarial vulnerability in medical image analysis, motivated by healthcare's financial incentives and technical exposure, and focuses on two previously unexplored transferability factors: ImageNet pre-training and development-data disparity.
- Motivation: Adversarial attacks threaten medical image analysis systems that are increasingly used to automate clinical tasks.Healthcare may be especially exposed because attacks could manipulate insurance, clinical, or approval decisions within vulnerable infrastructure.
- Problem setting: Black-box attacks transfer adversarial examples from a surrogate model to a target model without full target-model access.The paper treats this as a realistic setting for medical systems.
- Research gap: The study targets ImageNet pre-training and development-data disparity as previously unexplored factors affecting attack transferability.It also examines their interaction with model-architecture disparity.
- Scope: Experiments cover diabetic-retinopathy detection, chest X-ray pathology classification, and breast-cancer metastasis detection in histological lymph-node sections.These applications span ophthalmology, radiology, and pathology.
2. Related work
Prior work establishes transferability as the basis of non-query black-box attacks, but leaves ImageNet pre-training and development-data disparity insufficiently studied in medical imaging.
- Black-box attacks: Non-query black-box attacks rely on transferring perturbations from a surrogate model to a target model, with effectiveness determined by transferability.Transferability has been linked to similarities between model decision boundaries.
- Pre-training: Existing MedIA studies considered pre-trained or randomly initialized models but did not compare transferability between these initialization regimes.The paper identifies this comparison as an open research gap.
- Development data: Evidence for development-data disparity was limited to earlier experiments on simple fully connected MNIST models, with no focused MedIA study.Prior work reported lower transferability when target and surrogate used different data subsets.
- Architecture disparity: Architecture disparity had been examined incompletely in MedIA, while prior findings suggested that combining it with data disparity further reduces attack transferability.The present study investigates this interaction in medical applications.
3. Methods
The methods define a non-query black-box threat model, generate FGSM and PGD attacks through surrogate networks, and evaluate them across two architectures and three medical datasets.
- Threat model: The threat model assumes untargeted inference-time input manipulation by an attacker who cannot query the target model or access its weights.The attacker may make perturbations that appear subtle or imperceptible to humans.
- Adversarial attacks: FGSM and PGD generate adversarial perturbations, with PGD applying multiple gradient-based steps under an ℓ∞ perturbation bound.FGSM is one-shot, whereas PGD iteratively computes and adds perturbations.
- Adversarial attacks: In the black-box setting, a surrogate network computes the attack, which is then transferred to the target model.The surrogate and target can differ in architecture, initialization, and development data.
- Models and evaluation: Experiments use Inception-v3 and DenseNet-121, training models to convergence and evaluating clean and adversarial performance on independent test sets.Performance is measured for medical image applications using patient-level development/test organization.
- Datasets: The datasets cover diabetic retinopathy fundus images, multi-label chest X-rays, and histopathology patches for lymph-node metastasis detection.Each application uses a separate medical imaging dataset and task.
4. Experimental setup
The experimental design tunes perturbation strength for subtle, transferable attacks and independently varies initialization, development data, and architecture between target and surrogate models.
- Evaluation: Attack effectiveness is measured by target-model AUC or mean AUC for multiclass tasks on each dataset's test set.This provides the common performance measure across experimental setups.
- Perturbation degree: Perturbation degree is selected to maximize transferability while minimizing visual perceptibility before the main experiments.FGSM and PGD are evaluated across ϵ values from 0.01 to 0.06.
- Control condition: Control noise uses spatially shuffled adversarial perturbations to match perturbation magnitude and distinguish adversarial effects from added noise.Perturbation values are randomly permuted across pixels.
- Initialization: Initialization experiments compare pre-trained and randomly initialized target-surrogate pairs under both matching and differing architectures.Four model versions per architecture cover the possible initialization combinations.
- Development data: Data-disparity experiments vary target and surrogate development sets, including equal-sized, halved, and smaller target datasets, while also incorporating pre-training for small-data settings.The design tests both data disparity alone and its interaction with architecture disparity.
5. Results
Attack transferability depended strongly on perturbation choice, pre-training, development-data overlap, and model architecture. Pre-training generally increased transferability when it improved clean performance, whereas data and architecture disparities reduced attack effectiveness, with dataset-specific exceptions.
- 5.1. Perturbation degree: FGSM and PGD performed similarly in radiology and pathology, whereas FGSM reduced ophthalmology target performance more than PGD.Control-noise experiments indicated that image corruption explained only a small part of the performance decrease in most cases.
- 5.1. Perturbation degree: At epsilon = 0.02, perturbations remained visually subtle across applications while providing better transferability than epsilon = 0.01 in most studied settings.Perturbations were most noticeable in radiology, and FGSM produced lower SSIM than PGD across all datasets.
- 5.2. Pre-training on ImageNet: Pre-trained surrogates transferred attacks more effectively to pre-trained targets in ophthalmology and radiology, including across different architectures.The pathology dataset showed the opposite effect, while the transferability pattern broadly tracked the clean-performance gain from pre-training.
- 5.3. Development data disparity: Development-data disparity substantially reduced transferability, and reducing the surrogate training set caused further drops in ophthalmology and radiology.With matched architectures, attacks were less effective when target and surrogate used different data subsets.
- 5.3. Development data disparity: Architecture disparity often affected attack performance more than data disparity in radiology and pathology, while both disparities had similar effects in ophthalmology.When architectures differed, additional data disparity substantially reduced attack performance only in ophthalmology.
- 5.3. Development data disparity: On small datasets, pre-trained models outperformed random-initialized models on clean images but were more vulnerable on adversarial images across all three datasets.In pathology, the transferability pattern under pre-training differed between small and larger training sets; random-initialized small-data models were barely affected by attacks.
6. Discussion
The discussion shows that ImageNet pre-training and development-data parity strongly shape black-box attack transferability in MedIA systems. It also emphasizes perturbation tuning, cautious robustness comparisons, and practical mitigation strategies for clinical deployment.
- 6.1. Perturbation degree: Perturbation degree must be tuned to the image type and attack setting because lower values can underestimate vulnerability while higher values can make attacks visually perceptible.Visual perceptibility differed across modalities, and SSIM does not have an accepted threshold that reliably matches human perception.
- 6.2. Pre-training on ImageNet: 20-50% AUC differences showed substantially higher transferability between pre-trained models than between randomly initialized models in ophthalmology and radiology, including across architectures.The authors caution against generalizing attack effectiveness between differently initialized systems or applications.
- 6.3. Development data disparity: Different development data substantially reduced attack transferability, even for equally sized datasets from the same distribution, with effects comparable to architecture disparity.A 30-40% AUC increase occurred for attacked ophthalmology and pathology models when surrogates used disjoint data.
- 6.2. Pre-training on ImageNet: Pre-training effects tracked clean-performance gains: stronger gains from ImageNet pre-training were associated with greater black-box vulnerability, including for models trained on small datasets.Randomly initialized targets were generally difficult to attack under data disparity, whereas pre-trained models could remain vulnerable.
- 6.3. Development data disparity: Robustness evaluations should model data parity and disparity, including differing development-set sizes, because private-data systems may be less susceptible to external surrogate-based attacks.This conclusion assumes attackers cannot access private development data and other system properties are equal.
- 6.5. Recommendations for developers of MedIA systems: For security-critical clinical systems, the authors recommend layered defenses, limiting public design and data details, and using customized architectures or random initialization only as complementary measures.They recommend secrecy selectively: evaluation methods and dataset descriptions should remain available for safety and performance assessment.
7. Conclusion
The paper identifies pre-training and development-data disparity as previously unexplored factors affecting black-box adversarial attack transferability in medical image analysis.
- Pre-training on ImageNet may dramatically increase adversarial-example transferability in medical image analysis systems.The larger the performance gain from pre-training, the larger the transfer and vulnerability to attacks from pre-trained surrogate models.
- Development-data differences between target and surrogate models substantially reduce black-box attack transferability.
- These factors should be considered when developing medical image analysis systems intended for clinical practice.