PaperlayerView exact revision on arXiv

Source-linked AI summary

Adversarial Sensor Attack on LiDAR-based Perception in Autonomous Driving

Yulong Cao, Chaowei Xiao, Benjamin Cyr, Yimeng Zhou, Won Park, Sara Rampazzi, Qi Alfred Chen, Kevin Fu, Z. Morley Mao

arXiv:1907.06826v2cs.CRcs.CVeess.SPstat.ML

TL;DR

LiDAR-based perception security in autonomous vehicles remains insufficiently explored, especially for spoofing front-near obstacles. This paper models strategically controlled spoofing as an optimization problem and combines optimization with global sampling, achieving attack success rates around 75%.

  • Problem

    Security of LiDAR-based perception in autonomous vehicles lacks thorough exploration despite LiDAR’s importance for driving safety and the need for object detection to process spoofed inputs.

  • Method

    The paper models strategically controlled LiDAR spoofing as an optimization problem for fooling the machine-learning object detector.

  • Results

    Around 75% average attack success rates are achieved by combining optimization with global sampling.

  • Takeaways & Limitations

    The study targets fake obstacles around 5 meters in front of a victim vehicle because they have the highest potential to trigger immediate erroneous driving decisions.

  • Takeaways & Limitations

    Real-world attack performance and practicality are not directly demonstrated because dynamically aiming the attack device at a victim vehicle’s LiDAR requires high precision and road testing.

Abstract

from arXiv · show

In Autonomous Vehicles (AVs), one fundamental pillar is perception, which leverages sensors like cameras and LiDARs (Light Detection and Ranging) to understand the driving environment. Due to its direct impact on road safety, multiple prior efforts have been made to study its the security of perception systems. In contrast to prior work that concentrates on camera-based perception, in this work we perform the first security study of LiDAR-based perception in AV settings, which is highly important but unexplored. We consider LiDAR spoofing attacks as the threat model and set the attack goal as spoofing obstacles close to the front of a victim AV. We find that blindly applying LiDAR spoofing is insufficient to achieve this goal due to the machine learning-based object detection process. Thus, we then explore the possibility of strategically controlling the spoofed attack to fool the machine learning model. We formulate this task as an optimization problem and design modeling methods for the input perturbation function and the objective function. We also identify the inherent limitations of directly solving the problem using optimization and design an algorithm that combines optimization and global sampling, which improves the attack success rates to around 75%. As a case study to understand the attack impact at the AV driving decision level, we construct and evaluate two attack scenarios that may damage road safety and mobility. We also discuss defense directions at the AV system, sensor, and machine learning model levels.

1 INTRODUCTION

This work studies the unexplored security of LiDAR-based AV perception and develops strategically controlled spoofing attacks that target front-near obstacles and driving decisions.

  • Motivation and gap: LiDAR-based perception had not been thoroughly explored for AV security despite LiDAR’s safety relevance and widespread use.LiDAR provides 360-degree views and 3D road representations, while prior security research focused mainly on camera-based perception.
  • Threat model and goal: The study targets spoofed obstacles close to a victim AV’s front using injected LiDAR points against Baidu Apollo’s perception system.The threat model is laser-based LiDAR spoofing, and the attack goal is to alter driving decisions through front-near obstacles.
  • Blind spoofing limitation: Blindly applying existing spoofing patterns failed to create road obstacles after Apollo’s machine-learning object-detection step.The experiments attribute this difficulty partly to an approximately 8° horizontal spoofing angle, insufficient for generating a nearby road-obstacle point cloud.
  • Strategic attack design: The authors formulate strategically controlling spoofed points as an optimization problem and design input-perturbation and objective-function models around Apollo’s processing logic.The objective function incorporates post-processing because model outputs are transformed before becoming perceived obstacles.
  • Optimization and results: Around 75% average attack success was achieved by combining global sampling with optimization after direct optimization became trapped by local extrema.Direct optimization reached only 30% average success, whereas the combined algorithm improved the rate to around 75%.
  • Driving-decision impact: Two simulator scenarios successfully triggered attacker-desired decisions: emergency braking and AV freezing at an intersection.The scenarios were evaluated using real-world Baidu Apollo driving-data traces and represent risks to road safety and traffic mobility.
  • Defenses: The paper proposes defenses at AV-system, sensor, and machine-learning-model levels.One system-level example is filtering LiDAR points originating from ground reflection.

2 BACKGROUND

Baidu Apollo processes LiDAR point clouds through preprocessing, DNN-based object detection, and post-processing before prediction and planning use perceived obstacles.

  • Preprocessing: Apollo’s LiDAR pipeline begins with raw n × 4 point clouds, coordinate transformation, ROI filtering, and generation of an 8 × 512 × 512 feature matrix.The ROI filter removes unrelated regions using HDMap information, and points within a default 60-meter range are mapped to grid cells.
  • DNN-based object detection: A DNN receives the feature matrix and produces per-cell output metrics, including the probability that a cell belongs to an obstacle.These metrics provide the inputs used by subsequent post-processing stages.
  • Post-processing: Post-processing clusters cells above objectness and positiveness thresholds, then reconstructs candidate obstacle bounding boxes.The default thresholds described are 0.5 for objectness and 0.1 for average positiveness.
  • Downstream driving decisions: Perceived obstacle information is passed to prediction and planning, which support decisions such as stopping, lane changing, and yielding.Prediction estimates future obstacle trajectories before planning selects future driving routes and decisions.
  • LiDAR spoofing: LiDAR spoofing injects seemingly legitimate measurements through the sensor’s physical channel without physical contact or digital-measurement interference.Prior work showed fake points could be created at distances both farther from and closer than the spoofer’s position.
  • Adversarial machine learning: The paper frames adversarial example generation as finding valid input perturbations that make a model produce a targeted output.The cited optimization formulation minimizes perturbation size subject to the target model output and valid-input constraints.

3 ATTACK GOAL AND THREAT MODEL

The attack seeks to make LiDAR perception report fake obstacles about 5 meters ahead, using practical laser spoofing under specified attacker-access assumptions.

  • Attack goal: The attack goal is to make LiDAR perception detect front-near fake obstacles that can maliciously alter an AV’s driving decisions.Front-near obstacles are defined as approximately 5 meters in front of the victim AV because they have high potential to trigger immediate erroneous decisions.
  • Threat model: The threat model uses LiDAR spoofing by roadside devices or nearby attack vehicles that aim malicious laser pulses at the victim’s LiDAR.The paper describes the attack as stealthy because the pulses are invisible and the devices are relatively small.
  • Attacker knowledge: The first security analysis assumes white-box access to the machine-learning model and perception system.The authors consider this obtainable through additional engineering effort to reverse-engineer the software.

4 LIMITATION OF BLIND SENSOR SPOOFING

Blindly replaying LiDAR spoofing traces does not generate front-near obstacles in Baidu Apollo’s perception pipeline. Limited spoofing coverage produces point clouds too sparse and narrow to mimic important road obstacles or fool object detection.

  • Spoofing attack setup: The reproduced attack uses a photodiode, delay component, and infrared laser to synchronize with the victim LiDAR and inject crafted echo pulses.The consistent firing sequence lets the attacker select which fake points appear in the point cloud.
  • Spoofing attack setup: Around 100 dots can be spoofed across the tested viewing region, but only 60 points across 8–10 central vertical lines are stable with high intensity.The setup uses a VLP-16 LiDAR with 16 vertical viewing angles and an 8° horizontal angle of coverage.
  • Blind spoofing experiments: None of the tested blind traces, sampled angles, or random point patterns generated spoofed obstacles in Baidu Apollo’s LiDAR perception pipeline.The random-pattern experiment used 100 patterns with 160 points at distances of 4–6 meters from the victim AV.
  • Why blind spoofing fails: The blind attack’s limitations arise from narrow horizontal coverage and restricted vertical coverage imposed by the spoofing laser’s firing frequency.These constraints limit the number of spoofed points, especially when targeting LiDARs with more vertical lines.
  • Blind spoofing experiments: The attack traces cover a much narrower angle and contain far fewer points than a real vehicle reflection, making them difficult for the machine-learning detector to recognize as obstacles.A real vehicle point cloud can span a wider angle and include many more points than the reproduced attack traces.

5 IMPROVED METHODOLOGY: ADV-LIDAR

Adv-LiDAR treats strategically controlled LiDAR spoofing as an adversarial machine-learning problem. It models physically feasible point-cloud perturbations, designs an objective aligned with Apollo’s post-processing, and evaluates the resulting attacks at the driving-decision level.

  • Methodology overview: Adv-LiDAR addresses three challenges: modeling constrained 3D point-cloud perturbations, overcoming optimization limits, and aligning the objective with obstacle post-processing.These challenges arise because sensor attacks constrain inputs, optimization may have low success, and model outputs are post-processed before perceived obstacles are produced.
  • Methodology overview: The attack is formulated as an optimization problem that searches for adversarial spoofed point clouds constrained by physically achievable LiDAR attacks.The formulation maps pristine 3D point clouds and spoofed point clouds through the perception pre-processing function into model input feature matrices.
  • Input perturbation analysis: Input perturbation analysis models the spoofing capability set and the merging function that combines attack traces with pristine point clouds after pre-processing.The analysis quantifies what 3D point-cloud changes the sensor attack can produce and how those changes enter the machine-learning input.
  • Objective function design and model analysis: Model analysis studies Apollo’s post-processing logic and incorporates the strategies that convert model outputs into perceived obstacles into the objective function.This makes the objective reflect the front-near-obstacle attack goal rather than only changes in raw model output.
  • Driving decision case study: The study evaluates two attack scenarios using generated adversarial 3D point clouds to assess effects at the AV driving-decision level.The scenarios examine the downstream impact of spoofed obstacles after LiDAR-based perception.

6 INPUT PERTURBATION ANALYSIS

The paper models LiDAR spoofing capability and translates it into constrained perturbations of the detector’s input feature matrix. The perturbation pipeline combines attack-observed point placement changes with differentiable preprocessing and global spatial transformations.

  • Point placement: Distance changes move spoofed points along the sensor-point axis by Δr, while altitude changes shift their height by Δh.Distance is controlled through laser-pulse delay; altitude is controlled through delay intervals of 2.304 µs.
  • Point placement: Azimuth changes rotate spoofed points around the LiDAR on the horizontal plane by Δθ, with the spoofer movable around the sensor.The LiDAR spoofer can reach any horizontal angle by changing its location.
  • Attack capability: The attack capability A models spoofed point clouds with 20, 40, or 60 points and applies changes to distance, altitude, and azimuth.Around 60 spoofed points were considered the highest reliably achievable number in the experiments.
  • Feature-matrix merging: The merging function ⊕ is modeled as a differentiable operation because preprocessing prevents directly adding spoofed feature matrix t′ to the original matrix x.The preprocessing pipeline includes coordinate transformation, ROI filtering, and input feature extraction.
  • Feature-matrix perturbation: The spoofing capability Φ(A) is represented with constrained global spatial transformations applied to spoofed feature matrices mapped from corresponding spoofed 3D point clouds.The transformations include rotation, translation, and scaling; height features additionally receive a global scale scalar sh.
  • Parameterization: The transformation parameters are simplified by aligning the spoofed input feature matrix with the x axis, allowing optimization of τx alone for translation.Because τx/τy has fixed ratio tanθ, alignment sets θ=0 and τy=0; the process is treated as scaling and ϵ is removed.

7 GENERATING ADVERSARIAL EXAMPLES

The paper generates adversarial examples by minimizing a detector-aware loss over constrained spoofing transformations. Because vanilla optimization can stall on noisy or flat loss surfaces and bad local minima, the proposed process combines global sampling with local optimization.

  • Adversarial objective: The adversarial loss Ladv is designed from post-processing to favor front-near obstacle detections at an attacker-selected target position.It weights model positiveness and objectness across input-feature cells using a Gaussian mask centered on the mapped target position.
  • Optimization: Vanilla optimization uses Adam to minimize the objective over transformation parameters θ, τx, and sh, producing t′ and x′=x⊕t′.The paper calls this direct solution vanilla optimization.
  • Optimization limitations: Small optimization steps can become trapped near local minima, whereas large steps are less effective because noisy local loss points in the wrong direction.The loss surface is noisy at small scales and relatively flat at large scales under the hard perturbation constraints.
  • Sampling-based improvement: The improved algorithm samples transformation parameters at a larger scale before optimizing at a smaller scale.It computes feasible parameter ranges for placing the transformed spoofed point cloud in the target area, uniformly samples rotation and translation parameters, and then optimizes selected initializations.
  • Adversarial point-cloud generation: The optimized transformation parameters generate an adversarial spoofed 3D point cloud T′ through a dual transformation function GT.The resulting point cloud satisfies t′=Φ(T′) and is constructed from T using θ, τx, and sh.
  • Optimization limitations: Figure 9 compares loss over rotation θ and translation τx, showing why step-size selection is difficult for vanilla optimization.The green trajectory represents a small step size and the red trajectory a large step size.

8 EVALUATION AND RESULTS

The evaluation measures attack effectiveness and robustness under variations in the input point cloud and spoofed points. Global sampling improves optimization-based spoofing, reaching around 75% success with 60 spoofed points and remaining robust across subsequent frames and imprecise spoofed point traces.

  • Experiment Setup: The evaluation uses 300 real-world LiDAR point-cloud frames, targeting obstacles 2–8 meters in front of the victim AV.The trace was collected with a Velodyne HDL-64E S3 on local roads in Sunnyvale, California.
  • Attack Effectiveness: 43.3% average success rate is achieved with global sampling, compared with 18.9% for vanilla optimization, a 2.65× improvement.The improvement addresses optimization becoming trapped in local minima.
  • Attack Effectiveness: Around 75% success rate is achieved with 60 spoofed points using the improved optimization method.Success rates increase as the attacker can inject more spoofed points.
  • Attack Effectiveness: All successful spoofed obstacles are classified as vehicles, although vehicle-type spoofing was not an explicit attack objective.
  • Robustness Analysis: With 60 spoofed points, attack success remains above 75% on average across 15 subsequent frames, showing robustness to changes in the evaluated point cloud.The 15 frames cover approximately 1.5 seconds after the frame used to generate the adversarial point cloud.
  • Robustness Analysis: More than 82% success is achieved across all three tested attack capabilities when spoofed point traces vary, reaching 90% with 60 points.The result indicates that precise reproduction of the spoofed point pattern is not always required for high success rates.

9 DRIVING DECISION CASE STUDY

The case study evaluates how successful adversarial point clouds affect Baidu Apollo’s driving decisions in simulation. Spoofed front-near obstacles trigger emergency braking and can keep a vehicle stopped after a traffic light turns green.

  • Case Study Setup: Two attack scenarios are evaluated at the driving-decision level using Baidu Apollo’s Sim-control replay environment.The experiments use synthesized continuous frames of successful adversarial point clouds on Baidu Apollo 3.0.
  • Emergency Brake Attack: 43 km/h to 0 km/h within 1 second occurs in the emergency brake attack after a spoofed front-near obstacle is detected.The resulting hard brake may hurt passengers or cause rear-end collisions.
  • Emergency Brake Attack: The victim AV stops instead of overtaking because Apollo requires a minimum distance for overtaking based on the obstacle’s relative speed.The spoofed obstacle is positioned near the vehicle, preventing the overtaking decision.
  • AV Freezing Attack: The AV freezing attack prevents a stationary vehicle from moving after the traffic signal turns green.The attack may be exploited to cause traffic jams.
  • AV Freezing Attack: The victim AV stops instead of deviating or side-passing because Apollo requires a 15-meter minimum distance for those decisions.

10 DISCUSSION

The discussion covers limitations, generality, attack consequences, and defenses. The study’s real-world practicality remains unvalidated, while proposed defenses span AV processing, sensor design, sensor fusion, randomization, and adversarial retraining.

  • Limitations and Future Work: Real-world attack performance and practicality are not directly demonstrated because dynamically aiming the attack device at a vehicle LiDAR requires high precision.The authors identify physical-world road testing as future work.
  • Limitations and Future Work: The analysis uses only a subset of spoofing capability and does not comprehensively evaluate modules beyond perception.The objective function could therefore be improved to target specific abnormal driving decisions more directly.
  • Generality: The methodology decomposes LiDAR-based AV perception attacks into spoofing-capability formulation, adversarial-example generation, and driving-decision evaluation.Although the construction is specific to Baidu Apollo, the methodology is described as generalizable to other LiDAR-based perception systems.
  • AV-System-Level Defenses: The attack can inject at most 60 points, while a detected real vehicle may contain up to a thousand points; ground reflections help form spoofed obstacles after preprocessing.Mapping the 3D point cloud into a 2D matrix loses height information, allowing ground-reflection points to cluster into obstacles.
  • AV-System-Level Defenses: AV-system defenses include filtering ground reflections and retaining or adding features that reduce information loss during preprocessing.
  • Sensor-Level Defenses: Sensor fusion can increase attacker effort, but it requires the majority of sensors to function correctly and is not a perfect defense.
  • Sensor-Level Defenses: Reducing LiDAR receiving angles or filtering unwanted spectra can reduce spoofing susceptibility, but also limits sensing range and sensitivity.
  • Sensor-Level Defenses: Randomizing LiDAR pulse timing, grouping, or waveform can make expected reflections harder for an attacker to predict.

11 RELATED WORK

Related work covers security vulnerabilities in vehicle systems and sensors, attacks on traditional vehicle sensing, and adversarial examples primarily studied in image-based machine learning. This paper focuses on LiDAR-based perception in autonomous driving.

  • Vehicle Systems Security: Prior vehicle-security studies examine in-vehicle networks, infotainment systems, and connected-vehicle systems, whereas this work targets autonomous-driving perception.
  • Vehicle Systems Security: LiDAR-based AV perception is presented as an attack surface absent from traditional vehicle systems designed for human drivers.
  • Vehicle-Related Sensor Attacks: Earlier sensor attacks targeted tire-pressure sensors, magnetic wheel-speed sensors, and other sensors used in traditional or autonomous vehicles.
  • Adversarial Example Generation: Adversarial-example research has extensively addressed image classification, segmentation, object detection, pose estimation, and physical-world attacks.The paper extends this research direction to LiDAR-based AV perception.

12 CONCLUSION

The paper presents the first security study of LiDAR-based perception in autonomous vehicles, showing that controlled spoofing can target front-near obstacles despite the failure of blind spoofing. It combines optimization with global sampling, reports a 2.65× average increase in attack success rates, and evaluates safety and mobility impacts.

  • 12 CONCLUSION: 2.65× average attack-success increase is achieved by combining optimization with global sampling to address limitations of direct optimization.The method models input perturbations and the objective function before applying the combined algorithm.
  • 12 CONCLUSION: Blindly applying reproduced LiDAR spoofing is insufficient because machine-learning-based object detection prevents the intended front-near obstacle attack.The attack goal is to spoof obstacles close to the front of a victim AV.
  • 12 CONCLUSION: The study constructs and evaluates two attack scenarios that may compromise autonomous-vehicle safety and mobility.The paper also discusses defenses at the AV-system, sensor, and machine-learning-model levels.

APPENDIX

The appendix documents the adversarial-example generation procedure, its optimization inputs and outputs, and an example of the resulting spoofed sensor traces.

  • APPENDIX: The experiment uses Adam with learning rate 1e-4 and TensorFlow as the backbone.Lt is set to 12.5, while Lθ is the angle generating a 2-meter distance from the target position.
  • APPENDIX: Figure 14 displays collected reproduced-attack traces, with points inside the yellow circle identified as spoofed by the sensor attack.The figure provides a visual trace of the reproduced sensor attack.
  • APPENDIX: Algorithm 1 generates an adversarial 3D point cloud from a target model, original point cloud, spoofed point cloud, optimizer, and iteration limit.The stated output is the adversarial 3D point cloud X′.
  • APPENDIX: The procedure initializes transformation parameters, computes adversarial loss, and updates parameters using the optimizer.The parameters include θ, τx, and sh; the update is based on ladv.
Loading 1907.06826v2…