Securing Connected & Autonomous Vehicles: Challenges Posed by Adversarial Machine Learning and The Way Forward

Adnan Qayyum; Muhammad Usama; Junaid Qadir; Ala Al-Fuqaha

Source-linked AI summary

Securing Connected & Autonomous Vehicles: Challenges Posed by Adversarial Machine Learning and The Way Forward

Adnan Qayyum, Muhammad Usama, Junaid Qadir, Ala Al-Fuqaha

arXiv:1905.12762v1May 29, 2019cs.LGcs.CRstat.ML

TL;DR

CAVs rely on ML for safety-critical functions, yet adversarial perturbations and broader security vulnerabilities threaten the integrity of their decisions. The paper reviews CAV and ML security literature, formulates the CAV ML pipeline, surveys adversarial attacks and robustification approaches, and identifies open research problems. It concludes that CAV ML systems require security-aware threat modeling, robustness evaluation, interpretability, fairness, and resistance to distribution drift.

Problem
CAVs use ML for tasks from environmental perception to movement prediction, but adversarial perturbations and other security vulnerabilities threaten these safety-critical systems.
Method
The paper synthesizes CAV and ML literature, formulates the CAV ML pipeline, reviews threat models and attacks, and taxonomizes robust ML solutions and evaluation recommendations.
Results
The paper provides a comprehensive analysis of adversarial ML challenges for CAVs and outlines defenses across multiple settings.
Takeaways & Limitations
Secure CAV deployment requires attention to threat modeling, robust ML, interpretability, fairness, accountability, and distribution drift.

Abstract

from arXiv · show

Connected and autonomous vehicles (CAVs) will form the backbone of future next-generation intelligent transportation systems (ITS) providing travel comfort, road safety, along with a number of value-added services. Such a transformation---which will be fuelled by concomitant advances in technologies for machine learning (ML) and wireless communications---will enable a future vehicular ecosystem that is better featured and more efficient. However, there are lurking security problems related to the use of ML in such a critical setting where an incorrect ML decision may not only be a nuisance but can lead to loss of precious lives. In this paper, we present an in-depth overview of the various challenges associated with the application of ML in vehicular networks. In addition, we formulate the ML pipeline of CAVs and present various potential security issues associated with the adoption of ML methods. In particular, we focus on the perspective of adversarial ML attacks on CAVs and outline a solution to defend against adversarial attacks in multiple settings.

I. INTRODUCTION

CAVs combine vehicle connectivity and ML-driven automation, but expanding sensor, communication, and data-processing surfaces create serious security risks. The paper reviews these challenges, organizes adversarial ML threats and defenses, and identifies open research directions.

CAV Foundations: CAVs combine vehicle connectivity with automation, using ML and DL for decision making and perception.Connected vehicles exchange safety and environmental information, while autonomous vehicles add automation capability and cooperation.
Connected Vehicles: Vehicular networks use technologies including DSRC, ITS-G5, LTE, and 5G to support vehicle-to-everything communication.Earlier systems face short-lived connections, non-guaranteed QoS, and unbounded channel access delay.
Security Challenges: CAVs remain vulnerable because sensors, onboard devices, communication links, and data-driven processing expose multiple automotive attack surfaces.Sensor data can support adaptive decisions but may also be used for destructive objectives, making data integrity and security essential.
Security Challenges: ML/DL systems can be fooled by carefully crafted adversarial perturbations, including physical-world attacks against autonomous-vehicle vision systems.These vulnerabilities raise privacy and security concerns for security-critical CAV applications.
Paper Scope: The paper reviews CAV literature, formulates the CAV ML pipeline, categorizes threat models and attack surfaces, surveys attacks and robust ML approaches, and identifies open problems.Its organization covers CAV background, the ML pipeline, adversarial ML threats, robustness solutions, evaluation, and future research.
Autonomous Vehicles and Levels of Automation: SAE’s driving-automation taxonomy contains six levels, ranging from no automation to self-driving automation without human intervention.The paper also introduces the operational design domain and surveys historical naming conventions for autonomous vehicles.

B. Development of Autonomous Vehicles: Historical Overview

Autonomous vehicles evolved through successive research and industrial milestones, alongside increasingly complex connected-vehicle architectures, sensing systems, and software workflows.

Historical milestones: Autonomous-vehicle development progressed from early remote-controlled and experimental vehicles to DARPA challenges and public-road testing.Key milestones include the 1925 American wonder, 1939–1940 Futurama, 1950s General Motors and RCA work, and later DARPA and Google programs.
Historical milestones: 2018 marked Google’s reported achievement of its 10 million-mile self-driving target.
Connected-vehicle architecture: Connected vehicles use onboard units and roadside units to support intervehicle communications and services such as traffic safety, assistance, infotainment, and GPS.The architecture includes vehicle-to-vehicle, infrastructure-to-infrastructure, and infrastructure-to-vehicle communications.
Vehicle systems: Modern vehicles integrate more than 100 electronic control units executing about 100 million expressions of code across interconnected components.This distributed architecture supports functions including braking, lighting, entertainment, and drivetrain or powertrain control.
Sensing and software: Autonomous vehicles combine proximity, short-, medium-, and long-range sensors whose ranges support parking, object detection, collision avoidance, and adaptive cruise control.Examples include ultrasonic sensors at 5m, cameras and short-range radar at 30m, medium-range sensors at 80–160m, and long-range radar at 250m.
Sensing and software: The software workflow receives sensor inputs through interconnected perception, mapping, planning, controller, and system-supervisor modules, producing actuator-control commands.

D. Security-Related Challenges in Developing Robust CAVs

CAV security must address attacks across application functionality and message exchange, including integrity, spoofing, replay, and authenticity threats.

Application-layer attacks: Application-layer attacks target specific vehicular functions such as beaconing and message spoofing, and include integrity and authenticity attacks.
Integrity attacks: Message fabrication lets an adversary alter received content and rebroadcast it, with effects depending on the longitudinal-control-system design.
Integrity attacks: Replay attacks reuse stored messages later, allowing old information to create hazards for vehicular networks and their nodes.
Authenticity attacks: Sybil attacks create multiple fake identities, whereas impersonation attacks exploit legitimate vehicles to gain access and broadcast falsified messages.
Mitigation approaches: Digital signatures can protect message integrity and support data- and entity-level authentication, while timestamp-based nonces can help prevent replay attacks.

2) Network Layer Attacks:

Networked and autonomous vehicles face distributed communication attacks, hardware and software compromise, privacy exposure, and intentional sensor interference.

Network-layer attacks: Vehicular botnets can launch DoS or DDoS attacks that create severe physical congestion on hotspot road segments and increase trip durations.
Network-layer attacks: Network jamming disrupts communications across geographic areas, including single- or multi-channel attacks against IEEE 802.11p configurations.
System-level attacks: System-level attacks modify vehicle hardware or software, potentially making exchanged information inaccurate despite secure communications and deployed security measures.
Privacy breaches: Safety messages expose vehicle identity, location, velocity, and acceleration, enabling eavesdropping and construction of vehicle location profiles.
Privacy breaches: Pseudonymous or anonymous schemes remain vulnerable to privacy breaches when attackers establish connections with roadside units.
Sensor attacks: Current sensors resist environmental noise but cannot resist intentional noise used for attacks such as jamming and spoofing.

2) Heterogeneous and Stringent QoS Requirements:

Vehicular networks combine safety-critical, delay-sensitive communications with dynamic topology, congestion, and real-time requirements, motivating predictive ML and coordinated task pipelines.

Communication requirements: V2V safety communications exchange basic safety messages requiring high reliability and low delay, while V2I links connect vehicles with infrastructure.
ML applications: Historical data-driven prediction can use HMMs, Bayesian methods, RNNs, and LSTMs to address vehicular-network dynamics and time-series data.
Network dynamics: Geographically unbounded vehicular networks face congestion because dense urban traffic, especially during rush hours, increases network load.
Quality-of-service requirements: Vehicular-network applications require hard real-time guarantees for traffic prediction, congestion control, path planning, and timely safety-message delivery.
CAV task pipeline: ML-enabled autonomous driving organizes tasks into perception, prediction, planning, and decision-making and control.

D. Applications of ML for the Decision Making and Control

Machine learning supports perception, decision making, control, and connected-vehicle coordination, but every stage of the CAV ML pipeline can be exposed to adversarial manipulation.

ML-based CAV applications include end-to-end driving, lane keeping, traffic-flow adaptation, and resource orchestration.These applications use learning for vehicle control, infrastructure coordination, and networking, caching, and computing decisions.
CAVs combine autonomy and interactivity through complex sensory equipment and communication among vehicles and infrastructure.Autonomous vehicular networks monitor, plan, and control themselves while remaining interactive with their infrastructure.
The CAV ML pipeline collects heterogeneous sensor and intervehicle data, preprocesses it, extracts features, trains models, and produces decisions or actions.Inputs include cameras, RADAR, GPS, video, network, and traffic information; outputs include stopping at signs and predicting traffic flow.
Adversaries can target data collection and processing, the trained model, or its outputs across the CAV ML pipeline.The paper generalizes attack surfaces from generic ML systems to autonomous and connected vehicles.
The paper presents taxonomies of adversarial examples, perturbation methods, benchmarks, and threat models for ML-based autonomous systems.These taxonomies organize the adversarial-ML landscape relevant to CAVs.

1) Adversarial Attacks:

Adversarial attacks manipulate learning inputs or processes to induce incorrect model behavior, using different perturbation scopes, constraints, norms, and evaluation benchmarks.

Poisoning attacks manipulate training data, whereas evasion attacks manipulate test data or real-time inputs to produce false results.Adversarial examples generally refer to samples used to fool ML/DL systems during inference.
Adversarial perturbation crafting uses local search, combinatorial optimization, or convex relaxation to solve the attack objective.FGSM exemplifies local search, while combinatorial optimization seeks an exact solution to the stated objective.
Attack scopes range from perturbing each legitimate input to applying universal perturbations across a complete dataset.Most studies discussed in the passage use the per-input scope.
Perturbation magnitude is commonly constrained or measured with L2, L∞, and L0 norms.L2 limits Euclidean distance, L∞ limits each pixel’s maximum change, and L0 minimizes the number of changed pixels.
Adversarial inputs may be indistinguishable, content-preserving, content-constrained, payload-constrained, or unconstrained.The constraints determine whether human perception, original content, or an intended input function must be preserved.
Adversarial-ML evaluations commonly use MNIST, CIFAR-10, and ImageNet with victim models including LeNet, AlexNet, VGG, GoogLeNet, CaffeNet, and ResNet.The paper notes that researchers use an inconsistent range of datasets and victim models despite these recurring benchmarks.

B. Threat Models for Adversarial ML Attacks on CAVs

Threat modeling frames adversarial ML security by identifying system stakeholders, goals, adversities, invariants, attacker capabilities, and attack dimensions relevant to CAVs.

Threat modeling asks what the system is, who its stakeholders are, what it intends to do, and what adverse events may occur.These questions establish the system principals, goals, and adversities from a hypothetical attacker’s perspective.
System invariants specify what must remain true despite adverse events, supporting security goals, vulnerability identification, and countermeasure development.The process requires careful reasoning, expertise, and time.
The paper restricts its threat-model discussion to adversarial ML threats in CAVs and aggregates the relevant issues in a taxonomy.The scope is deliberately limited to vulnerabilities arising from ML techniques in CAVs.
Learning-system attacks are categorized by influence, specificity, and security violation, covering training-data control, targeted intentions, integrity, and availability.Influence distinguishes causative from exploratory attacks, while specificity distinguishes targeted from indiscriminate attacks.
White-box, gray-box, and black-box attacks differ according to the adversary’s knowledge of the model and training data.Black-box attacks treat the adversary as a normal user inferring behavior from system outputs and may exploit transferability.
Adversarial capabilities describe what attacks an adversary can launch, against which attack surfaces, and during inference or training.Capabilities depend on the attacker’s knowledge and access to the system.

4) Adversarial Specificity:

Adversarial attacks on CAV ML systems vary by target specificity, operational strength, and security objective, with documented relevance to autonomous-vehicle behavior and safety.

Targeted attacks induce a specific misclassification, such as making one road sign consistently map to a predetermined wrong class.Non-targeted attacks instead degrade model performance regardless of the data class.
False-positive attacks make negative samples appear positive, while false-negative attacks produce the opposite type of classification error.In autonomous-vehicle image classification, a false positive can assign high confidence to an incorrect class.
Iterative adversarial attacks are stronger than single-step attacks but require repeated model queries, more time, and more computational resources.Their generation depends on frequent interactions with the ML system.
Adversarial goals traditionally target confidentiality, integrity, or availability, with privacy identified as an additional dimension.These goals constitute the classical CIA model plus privacy.
Prior work shows that DNNs can be fooled by minimally perturbed inputs, geometric transformations, and model backdoors tested on autonomous vehicles.A trojan trigger caused a tested car to misbehave when it encountered a specific roadside billboard.
Adversarial ML threatens self-driving vehicles because ML/DL support onboard decisions as well as V2I, V2V, and V2X communications.The paper situates adversarial perturbations alongside other traditional security risks in self-driving vehicles.

1) Autonomous Vehicles Accidents Due to Unintended Adversarial Conditions:

Autonomous vehicles can fail under unintended adversarial conditions, while adversarial ML research has developed defenses that modify data, add auxiliary models, or modify models.

1) Autonomous Vehicles Accidents Due to Unintended Adversarial Conditions:: Reported autonomous-vehicle malfunctions include sensor-related crashes and a 2018 Uber pedestrian fatality involving a DNN-based system.Table V summarizes accidents caused by malfunctions in different self-driving-vehicle components.
1) Autonomous Vehicles Accidents Due to Unintended Adversarial Conditions:: Adversarial traffic signs, logos, and vehicle camouflage have been used to evade or fool DNN-based perception systems in autonomous driving.These attacks target traffic-sign detection and object detection under physical-world conditions.
Robust Machine Learning Methods:: Adversarial robustness is defined as the ability of an ML model to restrain adversarial examples, motivating defenses against carefully crafted perturbations.The paper distinguishes reactive detection after training from proactive robustness before attacks.
Robust Machine Learning Methods:: Robust ML defenses are categorized as modifying data, adding auxiliary models, or modifying models.Data modification includes adversarial retraining and data preprocessing.
Modifying Data:: Adversarial training augments training sets with adversarial examples to improve DNN robustness and may also provide better regularization.Reported evaluations include MNIST with 10 classes and ImageNet with 1000 classes.
Modifying Data:: Input reconstruction and feature squeezing reduce the effect of adversarial perturbations by cleaning inputs or reducing the adversary’s available feature space.Feature squeezing combines heterogeneous feature vectors into a single space, while reconstruction projects inputs toward learned data manifolds.

4) Features Masking:

Feature-masking defenses seek to suppress input features or learned representations that are especially vulnerable to adversarial perturbations, although defensive distillation was later shown to fail against one proposed attack.

4) Features Masking:: A masking layer before softmax forces weights for sensitive input features toward zero to reduce their exposure to adversarial perturbations.The masking layer is added to the classifier’s classification pathway.
4) Features Masking:: Spectral methods use natural spectral geometry in datasets to generate adversarially robust features for robust model development.The approach was empirically demonstrated as effective for generating robust features.
4) Features Masking:: Generative models can clean adversarial perturbations before inputs reach an otherwise non-modified ML model.One study reported that adversarial examples lie in low-probability regions of the training-data distribution regardless of attack type and targeted model.
4) Features Masking:: Model-modification defenses alter parameters or learned features, including defensive distillation, network verification, and input-gradient regularization.Network verification uses an SMT solver and was reported as NP-complete for the stated ReLU-based problem.
4) Features Masking:: Defensive distillation increased resilience to very small perturbations in its proposed formulation, but Carlini et al. showed it did not withstand their attack.The method transfers a probability distribution vector from one model into the original DNN model.
4) Features Masking:: Other defenses include uncertainty-aware hybrid GP-DNN models, interpretable-neuron detection, and classifier masking that adds logit noise against low-distortion attacks.These methods respectively modify model design, exploit task-relevant neurons, or mask the deep model’s output.

6) Masking ML Model:

Masking and auxiliary-model defenses extend adversarial protection beyond direct model modification, while evaluation recommendations emphasize adversary-aware and worst-case testing.

6) Masking ML Model:: Classifier masking frames adversarial ML as a learning-and-masking problem and introduces noise into DNN logits to defend against low-distortion attacks.The method is presented as a secure-learning approach.
6) Masking ML Model:: Auxiliary-model defenses use additional ML models to enhance the robustness of a main model, including generative models for adversarial detection.A binary detector can classify inputs as legitimate or adversarial, while an outlier class can identify adversarial examples.
6) Masking ML Model:: Ensemble defenses can combine detectors and input reconstructors, but an ensemble of weak defenses may still fail to provide strong protection.PixelDefend integrates an adversarial detector with an input reconstructor.
Adversarial Defense Evaluation:: The paper presents a taxonomy of adversarial-defense evaluation methods and recommendations.Figure 13 depicts the evaluation taxonomy.
Adversarial Defense Evaluation:: Adversarial-defense evaluation serves security, worst-case robustness, and measurement of ML progress toward human-level abilities.The paper treats adversarial robustness as a measure orthogonal to ordinary performance.

2) Common Evaluation Recommendations:

Reliable adversarial-defense evaluation requires explicit attack coverage, ablations, diverse settings, broader domains, randomness-aware testing, transfer attacks, and adaptive upper-bound analysis.

2) Common Evaluation Recommendations:: Evaluations should use both targeted and untargeted attacks and explicitly state which attack types were considered.The paper notes that untargeted attacks are theoretically easier, while practical performance can differ.
2) Common Evaluation Recommendations:: Ablation analysis should remove defense components and test a similar undefended model to assess each component and their combination.This clarifies the goals and effectiveness of combined defense strategies.
2) Common Evaluation Recommendations:: Diverse testing should cover random noise, broader threat models, tuned and converged attack hyperparameters, and hard-label and gradient-based attacks.Attack sensitivity to hyperparameters should also be investigated.
2) Common Evaluation Recommendations:: Defenses should be evaluated beyond images, with domain-specific limitations stated explicitly when a method only handles image perturbations.The paper identifies imaging as the dominant evaluation domain in adversarial ML research.
2) Common Evaluation Recommendations:: Randomized defenses require attacks ensembled over inference randomness, fixed-randomness checks, and an explicit threat model describing adversarial knowledge.Standard attacks may otherwise be hindered by defense-induced stochasticity.
2) Common Evaluation Recommendations:: Transferability and adaptive attacks should test defenses using substitute models and full access to the defense under the strongest relevant threat model.Adaptive attacks provide an upper bound on robustness and should be evaluated across multiple settings.
2) Common Evaluation Recommendations:: Robustness testing includes structural-coverage methods and must account for a reported trade-off between adversarial robustness and standard accuracy.MC/DC-based criteria and DeepGauge target undesired DNN behaviors, while a computer-vision study demonstrated the trade-off in a simple setting.
2) Common Evaluation Recommendations:: Autonomous-vehicle testing tools should explore perception, decision-making, and control under realistic road conditions to detect potentially fatal erroneous behavior.DeepTest automatically generates cases by varying weather and lighting and found erroneous behaviors in three leading Udacity challenge DNNs.

VI. OPEN RESEARCH ISSUES

Open research issues span adapting ML to distributed and evolving vehicular data, improving interpretability, robustness, privacy, fairness, accountability, and security evaluation. The paper also identifies limits of current defenses and practical challenges for deep reinforcement learning in autonomous vehicles.

ML/DL methods cannot be naively applied to vehicular networks because their distinctive characteristics require adapted learning approaches.The paper identifies adaptation of ML methods to vehicular-network features as a promising research direction.
Distributed data generation and storage require learning methods that can acquire data scalably from multiple connected-vehicle units.Centralized data-access assumptions do not fit the distributed vehicular ecosystem.
Lack of interpretability enables adversaries to construct adversarial examples, motivating secure, explainable, and interpretable ML for CAVs.Interpretability is also linked to understanding failures involving adversarial examples.
Existing adversarial defenses often work only against specific attack types and may fail against stronger or unseen attacks, leaving robust ML an open problem.Most defenses target computer-vision tasks, while adversarial ML is expanding to other application domains and safety-critical vehicle communication.
Privacy, fairness, accountability, and security-centric metrics remain open needs for CAV ML systems involving human subjects and potentially biased predictions.The paper highlights unresolved formalization of perturbation properties and difficulty measuring perceptual similarity.
Deep reinforcement learning for autonomous-vehicle perception remains impractical in realistic settings because road and vehicular state-action spaces are continuous and highly complex.The paper states that fine control is required for efficacy, although work on deep RL-based autonomous vehicles is developing.