An Analysis of Adversarial Attacks and Defenses on Autonomous Driving Models

Yao Deng; Xi Zheng; Tianyi Zhang; Chen Chen; Guannan Lou; Miryung Kim

Source-linked AI summary

An Analysis of Adversarial Attacks and Defenses on Autonomous Driving Models

Yao Deng, Xi Zheng, Tianyi Zhang, Chen Chen, Guannan Lou, Miryung Kim

arXiv:2002.02175v1Feb 6, 2020eess.SPcs.CRcs.LG

TL;DR

The paper addresses uncertainty about adversarial attacks and defenses for regression-based autonomous driving models. It evaluates five attacks and four defenses on three driving models, finding high vulnerability and no single defense that covers all attacks.

Problem
It is unclear how effective adversarial attacks and existing defenses are against regression-based driving models, creating potential autonomous-driving security risks.
Method
The study re-implements five attacks and evaluates them with four defenses on three CNN-based driving models.
Results
The driving models are highly vulnerable in white-box settings, while none of the four defenses effectively protects against all five attacks.
Takeaways & Limitations
Defense components should combine multiple methods, protect model details, and prefer complex driving architectures when computing resources permit.
Takeaways & Limitations
Experiments use three CNN-based models and only the Udacity dataset, excluding more complex architectures and leaving validation on broader models and datasets for future work.

Abstract

from arXiv · show

Nowadays, autonomous driving has attracted much attention from both industry and academia. Convolutional neural network (CNN) is a key component in autonomous driving, which is also increasingly adopted in pervasive computing such as smartphones, wearable devices, and IoT networks. Prior work shows CNN-based classification models are vulnerable to adversarial attacks. However, it is uncertain to what extent regression models such as driving models are vulnerable to adversarial attacks, the effectiveness of existing defense techniques, and the defense implications for system and middleware builders. This paper presents an in-depth analysis of five adversarial attacks and four defense methods on three driving models. Experiments show that, similar to classification models, these models are still highly vulnerable to adversarial attacks. This poses a big security threat to autonomous driving and thus should be taken into account in practice. While these defense methods can effectively defend against different attacks, none of them are able to provide adequate protection against all five attacks. We derive several implications for system and middleware builders: (1) when adding a defense component against adversarial attacks, it is important to deploy multiple defense methods in tandem to achieve a good coverage of various attacks, (2) a blackbox attack is much less effective compared with a white-box attack, implying that it is important to keep model details (e.g., model architecture, hyperparameters) confidential via model obfuscation, and (3) driving models with a complex architecture are preferred if computing resources permit as they are more resilient to adversarial attacks than simple models.

I. INTRODUCTION

Autonomous driving models face adversarial security risks, but the effectiveness of attacks and defenses for regression models remains uncertain. The paper evaluates five attacks and four defenses, finding high white-box vulnerability and incomplete defense coverage.

CNN-based driving models process camera and LiDAR data to predict controls such as steering angle and speed.
Small, human-imperceptible image perturbations can cause CNNs to produce completely wrong decisions, including demonstrated effects on Tesla Autopilot.
The study evaluates five adversarial attacks and four defenses across three driving models.
Except IT-FGSM at 36% attack success rate, the other attacks achieve an average 98% success rate in the white-box setting.
Attack success averages only 4% in the black-box setting, indicating that confidential model architecture and hyperparameters reduce vulnerability.
No defense detects all five attacks effectively, so system builders should deploy multiple defense methods in tandem.

II. BACKGROUND AND RELATED WORK

Autonomous driving models use sensor inputs and neural networks to predict vehicle controls. Adversarial examples exploit small input perturbations, while attacks differ according to the model information they require.

A. Autonomous Driving Model: A deep neural network uses sensor data such as LiDAR and camera inputs to predict vehicle controls including steering angle and speed.
A. Autonomous Driving Model: CNNs are a mainstream autonomous-driving architecture because they offer strong performance with fewer neurons and lower resource consumption.
B. Adversarial Attacks: Adversarial attacks add small perturbations to images to make a target model produce incorrect predictions.
B. Adversarial Attacks: White-box attacks require model details and gradient access, whereas black-box attacks require only arbitrary queries to the model.

C. Adversarial Defenses

Attacks on regression driving models require regression-specific success criteria and adapted perturbation methods. The study selects attack families spanning gradient, optimization, generative, and universal approaches.

A driving-model attack succeeds when the adversarial prediction deviates from the original prediction beyond an acceptable adversarial threshold.
The attack taxonomy includes Fast Gradient Sign, optimization-based, generative-model-based, and universal attacks.
The study re-implements five attacks for regression models, excluding classification-dependent methods such as C&W and DeepFool.
1) Iterative Targeted Fast Gradient Sign Method (IT-FGSM):: IT-FGSM repeatedly applies targeted gradient-sign updates to produce a stronger adversarial example.
2) Optimization-based Approach: The optimization-based method minimizes perturbation magnitude subject to a target prediction, then adapts the target from a class to f(x)+∆ for regression.

3) AdvGAN:

AdvGAN generates adversarial examples through a trained generator, and its universal variant produces one perturbation intended to affect many dataset samples.

AdvGAN combines a prediction-deviation objective with a GAN objective to train a generator that produces image-like adversarial examples.
The implementation constructs a perturbation on one image and incrementally adapts it across the dataset to obtain a universal perturbation.
The universal AdvGAN variant outputs one adversarial perturbation rather than a separate perturbation for each input image.

B. Adversarial Defenses on Autonomous Driving Models

The paper adapts four defense approaches to regression-based autonomous driving models, including proactive training methods and runtime anomaly monitoring. Existing classification-oriented transformations may harm regression predictions, motivating defense designs tailored to driving models.

Many adversarial defenses target image classification and are not directly applicable to regression-based autonomous driving models.Input transformations such as rotation can cause large prediction errors in driving models.
The paper adapts adversarial training and defensive distillation as proactive defenses for autonomous driving models.Adversarial training incorporates generated adversarial examples, while distillation trains a new model using information from the original model.
The reactive defense monitors prediction latency, GPU memory, and GPU utilization for anomalies associated with adversarial computation.The monitoring approach is designed for resource-constrained autonomous vehicles.
The adapted defensive distillation method regularizes training with information from both model outputs f(x) and last-layer tensors z_d.The additional information is intended to improve generalization and robustness against adversarial attacks.

4) Feature squeezing:

The experimental setup evaluates feature squeezing and adversarial attacks using three regression driving models trained on the Udacity dataset. Attack success is defined by exceeding a steering-angle deviation threshold of ∆=0.3.

Feature squeezing: Feature squeezing reduces image color depth or applies median spatial smoothing to make adversarial noise more perceptible or modify pixel values.The supplied setup also specifies attack and defense hyperparameters, including four-bit image-depth reduction.
Dataset and models: The Udacity dataset contains 33805 training frames and 5614 test frames with steering angles normalized to −1 and 1.The images are real-world road scenes collected by a vehicle-mounted front camera.
Dataset and models: The three driving models are adapted VGG16, Epoch, and Nvidia DAVE-2, with test RMSE values of 0.0906, 0.0962, and 0.1055 respectively.These models are reported as fairly accurate on the Udacity test set.
Experimental design: The experiments use 128 ∗128 input images and evaluate attacks under white-box and black-box settings.White-box attackers know the target model, whereas black-box attackers train a proxy driving model offline.
Evaluation metric: Attack success rate measures the fraction of adversarial examples whose steering-angle deviation exceeds the threshold ∆=0.3.The metric is used to assess adversarial attacks and defenses.

A. Research Questions

The research questions examine white-box attack effectiveness, black-box transferability, and defense robustness. Experiments compare five attacks across three driving models using attack success rate and steering-angle deviations.

Research questions: RQ1 evaluates how five adversarial attacks perform on different driving models in the white-box setting.RQ2 examines whether the attacks remain effective when attackers use proxy models in the black-box setting.
Research questions: RQ3 evaluates whether adversarial defense methods improve driving-model robustness against attacks.The experiment section reports results for five attacks and three defenses on three driving models.
White-box attack effectiveness: Over 90% attack success rates are achieved by attacks other than IT-FGSM on all three driving models.IT-FGSM reaches a highest attack success rate of 59.2% on Epoch, whereas Opt uni and AdvGAN reach 100% on Epoch.
White-box attack effectiveness: More complicated driving-model structure is robust to simple adversarial attacks, but the models remain vulnerable to other attacks.IT-FGSM has the lowest success rate on VGG16, while other attacks achieve fairly high success rates across the three models.
White-box attack effectiveness: AdvGAN and AdvGAN Uni generate perturbations resembling lane features, while their attacks produce large steering-angle deviations close to or above the adversarial threshold.In the Epoch example, IT-FGSM causes only a 0.0731 deviation despite a visibly large perturbation.
Overall finding: The overall result is that regression driving models are vulnerable to all five evaluated adversarial attacks.The paper identifies AdvGAN as seemingly the most dangerous because it exploits intrinsic features learned by driving models.

B. RQ2. Effectiveness of Black-box Attacks

Black-box attacks transfer poorly across autonomous driving models compared with white-box attacks. Transferability varies by attack and model complexity, with Opt uni relatively more effective and VGG16 examples transferring best.

Black-box attack success rates drop significantly when adversarial examples are reused across driving models, compared with white-box attacks.The study generates examples on one model and evaluates them on the others.
Opt uni is relatively more effective than other attacks in the black-box setting, achieving 30.0% success on Nvidia from examples generated on VGG16.
Adversarial examples generated on VGG16 transfer better than those generated on simpler models.Opt uni examples generated on VGG16 achieve 30.0% success on Nvidia, versus 5.4% for examples generated on Epoch.
VGG16 is more robust against black-box attacks than Epoch and Nvidia; Nvidia-generated Opt uni examples achieve 9.8% success on Epoch but 2.6% on VGG16.
Black-box attacks perform poorly overall on autonomous driving models, contradicting prior transferability findings for classification models.

C. RQ3. Effectiveness of Adversarial Defenses

The evaluated defenses protect against different subsets of attacks but none provides complete coverage. Feature squeezing detects all five attacks under a favorable threshold, while its false positives and threshold sensitivity limit deployment.

Adversarial training and defensive distillation reduce the success of IT-FGSM and an optimization-based attack but are ineffective against the other attacks.For Epoch, adversarial training reduces IT-FGSM from 59.2% to 39.3%, while distillation reduces it from 60% to near 10% at λ = 0.1.
Defensive distillation smooths neural-network gradients, explaining its effectiveness against IT-FGSM and Opt but not AdvGAN.
Runtime monitoring detects IT-FGSM and optimization attacks, and to some extent AdvGAN, but Opt uni and AdvGAN uni are less likely to be detected.IT-FGSM and Opt create substantial GPU memory and utilization overhead, unlike several other attacks.
At threshold 0.05, feature squeezing detects 78% of attacks with 5% false positives, whereas threshold 0.01 yields 40% false positives.Detection effectiveness decreases significantly above 0.05, and VGG16 examples are more resistant to feature squeezing.
None of the four defense methods effectively detects all five attack types.

VI. DISCUSSION

The discussion recommends combining defenses because each method has gaps, while computationally demanding detection may require edge-based middleware rather than vehicle-side deployment.

Combining multiple defense methods is important because no single method protects effectively against all five investigated attacks.
Feature squeezing detects many attacks but can falsely classify up to 40% of normal images as adversarial examples.
When detection methods require substantial computation, edge computing and adversarial-detection middleware can support faster responses outside the vehicle.

B. Limitations

The evaluation is limited to three CNN-based driving models and the Udacity dataset because existing methods do not apply to sequence-to-sequence video models. Broader architectures, datasets, and regression-oriented tools remain future work.

The experiments cover only three CNN-based driving models and the Udacity dataset.
Top Udacity leaderboard models were excluded because they process driving videos with sequence-to-sequence structures unsupported by the evaluated attacks and defenses.
More complex architectures such as CNN+RNN were not evaluated, so validating the findings across additional models and datasets remains future work.
Existing tools such as CleverHans and Foolbox focus on classification models, leaving adaptation to steering-angle regression as future work.
An optimization-based attack against feature squeezing was omitted because its 20−30-second generation time per MNIST example was considered unrealistic for real-time driving attacks.