PaperlayerView exact revision on arXiv

Source-linked AI summary

Towards Robust LiDAR-based Perception in Autonomous Driving: General Black-box Adversarial Sensor Attack and Countermeasures

Jiachen Sun, Yulong Cao, Qi Alfred Chen, Z. Morley Mao

arXiv:2006.16974v1cs.CRcs.CVcs.LG

TL;DR

LiDAR-based perception is vulnerable because current architectures ignore physically meaningful occlusion patterns, limiting the generality and effectiveness of existing attacks and defenses. The paper identifies this vulnerability, builds a black-box attack, and proposes CARLO and SVF; mean attack success rates are about 80%, 5.5%, and 2.3%, respectively.

  • Problem

    Existing LiDAR spoofing attacks have effectiveness and generality limitations, while current perception architectures may ignore occlusion information needed for secure detection.

  • Method

    The paper studies three LiDAR perception designs, exploits ignored occlusion patterns for a black-box attack, and uses them in CARLO and SVF defenses.

  • Results

    Mean attack success rates are around 80% for the black-box attack, 5.5% after CARLO, and 2.3% after SVF.

  • Takeaways & Limitations

    Occlusion patterns serve as physical invariants for detecting spoofed data and for learning robust LiDAR perception architectures.

  • Takeaways & Limitations

    Existing attacks are constrained by white-box access requirements or limited generality across perception models.

Abstract

from arXiv · show

Perception plays a pivotal role in autonomous driving systems, which utilizes onboard sensors like cameras and LiDARs (Light Detection and Ranging) to assess surroundings. Recent studies have demonstrated that LiDAR-based perception is vulnerable to spoofing attacks, in which adversaries spoof a fake vehicle in front of a victim self-driving car by strategically transmitting laser signals to the victim's LiDAR sensor. However, existing attacks suffer from effectiveness and generality limitations. In this work, we perform the first study to explore the general vulnerability of current LiDAR-based perception architectures and discover that the ignored occlusion patterns in LiDAR point clouds make self-driving cars vulnerable to spoofing attacks. We construct the first black-box spoofing attack based on our identified vulnerability, which universally achieves around 80% mean success rates on all target models. We perform the first defense study, proposing CARLO to mitigate LiDAR spoofing attacks. CARLO detects spoofed data by treating ignored occlusion patterns as invariant physical features, which reduces the mean attack success rate to 5.5%. Meanwhile, we take the first step towards exploring a general architecture for robust LiDAR-based perception, and propose SVF that embeds the neglected physical features into end-to-end learning. SVF further reduces the mean attack success rate to around 2.3%.

1 Introduction

The paper identifies ignored LiDAR occlusion patterns as a general vulnerability across perception architectures and develops black-box attack and defense approaches. Its attack reaches around 80% mean success rates, while CARLO and SVF reduce attack success to 5.5% and 2.3%.

  • Vulnerability: LiDAR perception models across three state-of-the-art 3D detection designs generally ignore occlusion patterns that encode physical invariants.A vehicle behind another can legitimately produce fewer points, but the same sparse pattern should not indicate an unobstructed front-near vehicle.
  • Vulnerability: Attackers can exploit this gap by spoofing almost two magnitudes fewer points while still inducing a fake front-near vehicle detection.The attack abuses models’ failure to distinguish occluded from unobstructed point-cloud patterns.
  • Attack: Over 80% success rates are achieved across all three model designs using a black-box attack with 60 spoofed points.The attack enumerates occlusion patterns of a 3D vehicle mesh and renders traces with ray-casting over roughly 15,000 KITTI samples.
  • Defense: CARLO detects spoofed vehicles using occlusion-related physical invariants without changing existing perception models.It uses free space inside detected bounding boxes and point locations within their corresponding frustums.
  • Robust architecture: SVF sequentially fuses LiDAR front-view and 3D representations so end-to-end learning uses neglected physical features.SVF reduces attack success to 2.3% without sacrificing original performance and is reported resilient to white-box and adaptive attacks.

2 Background

LiDAR-based perception uses point clouds for 3D object detection, with major architectures organized around bird’s-eye view, voxels, or direct point-wise processing. LiDAR measures distances through laser pulses, and spoofing attacks inject deceptive physical signals into the sensor.

  • BEV-based detection: Bird’s-eye view-based models project point clouds into a top-down 2D structure and use convolutional neural networks for detection.Apollo 5.0 is described as an industry-level BEV-based model with hard-coded BEV feature maps and grid-level confidence scores.
  • Voxel-based detection: Voxel-based models divide point clouds into voxels or pillars, extract learnable local features, and apply 2D convolutional detection layers.PointPillars is an industry-adopted voxel-based architecture used by Autoware.
  • Point-wise detection: Point-wise models operate directly on point clouds, commonly using two-stage architectures with 3D region proposals followed by box regression and classification.These models avoid transforming point clouds into regular 2D structures or voxels for feature extraction.
  • LiDAR sensing: A LiDAR measures distances by timing reflected laser pulses fired across predetermined vertical and horizontal angles.The resulting points form digital 3D representations of surrounding objects and spaces.
  • Spoofing attacks: Spoofing attacks inject deceptive physical signals into a victim sensor, and prior work has demonstrated fake-point injection against LiDAR.Adv-LiDAR additionally uses adversarial machine learning to deceive LiDAR-based perception models.

3 Threat Model

The threat model assumes sensor-level attackers can control spoofed points within measured capabilities without access to the target model, while defenders strengthen software rather than AV hardware. The experiments use a Velodyne VLP-16 PUCK LiDAR and settings based on prior work.

  • Attack setting: The sensor-level experiments target a Velodyne VLP-16 PUCK LiDAR using the setup and devices of prior work.The paper adopts the same attack setup as Cao et al. and details the devices in an appendix.
  • Attack capability: The attack capability includes the number of spoofed points, with hardware improvements enabling stable spoofing of at most 200 points.The improvements include a tuned comparator circuit and a wider-azimuth commercial lens.
  • Attacker capabilities: Attackers are assumed to control spoofed points within the observed sensor attack capability without accessing the machine-learning model or perception system.The threat model is based on demonstrated sensor attack settings and relaxes white-box assumptions.
  • Defense setting: The defense setting grants adversaries white-box access while restricting defenders to software-level changes because AV hardware cannot be modified for cost reasons.This setting is intended to cover strong spoofing attacks without requiring sensor hardware changes.

4 Limitations of Existing Attacks

Existing LiDAR spoofing attacks struggle to achieve their goals across different perception models because they lack either effectiveness, model access, or generality. Adv-LiDAR is limited by white-box requirements and poor transferability.

  • Blind sensor-level spoofing lacks control strategies and generally fails to fool the evaluated models except Apollo 5.0.The reproduced sensor attack traces were ineffective on the other target models.
  • Adv-LiDAR requires access to model parameters and pre- and post-processing modules, creating a white-box limitation for real-world deployment.The paper notes that few AV companies publicly release their perception systems.
  • Adv-LiDAR targets Apollo 2.5 with model-specific differentiable processing functions that are not directly applicable to other models.Constructing equivalent differentiable functions for other architectures is non-trivial.
  • None of 20 Adv-LiDAR traces optimized for Apollo 5.0 successfully attacked PointPillars or PointRCNN.The result demonstrates limited transferability across perception models.
  • Existing spoofing attacks cannot easily achieve the attack goal on all target models.The section evaluates existing attacks on three target model designs and discusses their limitations.

5 A General Design-level Vulnerability

The paper identifies a general design-level vulnerability: current LiDAR detectors overlook physically meaningful occlusion and sparsity patterns. Experiments show that translated occluded or distant vehicle points remain detectable, enabling low-point spoofing attacks.

  • 5.1 Behind the Scenes of Adv-LiDAR: LiDAR occlusion makes partially visible vehicles legitimate, while increasing distance reduces point-cloud density for otherwise similar vehicles.These properties motivate the inter-occlusion and sparsity-based false-positive hypotheses.
  • 5.2 Experimental Validation: 99.5% of picked occluded vehicles showed below 10% confidence-score fluctuations after their points were moved to front-near locations, validating FP1.The experiment used 100 KITTI validation samples across three target models.
  • 5.2 Experimental Validation: 99.5% of picked distant vehicles showed below 7.5% confidence-score fluctuations after translation to front-near locations, validating FP2.The distant vehicles were selected beyond 30 meters from the AV and evaluated on three target models.
  • 5.3 Vulnerability Identification: The experiments indicate that detectors are insensitive to object location and cannot distinguish sparse injected points from reflected points of solid objects.These behaviors correspond to the overlooked sparsity and intra-occlusion patterns.
  • 5.3 Vulnerability Identification: A translated occluded vehicle can be spoofed with 38 points occupying 4.92° horizontally at 6 meters, far fewer than a fully exposed vehicle requires.The attack exploits occlusion and sparsity patterns compatible with the sensor’s attack capability.
  • 5 A General Design-level Vulnerability: Current LiDAR detectors generally ignore occlusion patterns, leaving a gap between model capacity and LiDAR’s physical structure.The neglected patterns are treated as physical invariants that existing architectures fail to learn.

6 Black-box Spoofing Attack

The paper introduces a general black-box LiDAR spoofing attack that exploits models’ neglect of occlusion-related physical features and evaluates its effectiveness and robustness across target models.

  • Attack Construction: The attack constructs realistic traces from occluded or distant vehicles and customized ray-cast renderings, then injects them at target locations in point clouds.Both digital and preliminary physical spoofing settings are considered.
  • Attack Effectiveness: More than 80% ASR is achieved universally on all target models when more than 60 points are spoofed.ASR stabilizes around 85% with more than 80 spoofed points.
  • Attack Effectiveness: Rendered attack traces achieve comparable ASR to real-world traces across target models, enabling efficient customized trace generation.The rendered traces can be programmed into hardware for physical spoofing attacks.
  • Robustness Analysis: ASR remains around 70% at an l-2 distance of about 10 cm between original and perturbed traces.PointRCNN’s ASR decreases faster than PointPillars’ and Apollo 5.0’s as trace variation increases.
  • Robustness Analysis: ASR increases with higher original AP, suggesting that better-performing models can be more vulnerable to the ignored occlusion dimension.The authors attribute this vulnerability to models overfitting because they do not notice the hidden dimension.
  • Robustness Analysis: Feature squeezing and ME-Net fail to defend the black-box attack without harming original AP.These image-based defenses are evaluated on Apollo 5.0 because of its similarity to image-model pipelines.

7 Physics-Informed Anomaly Detection

The paper proposes CARLO, a physics-informed anomaly detector that uses LiDAR occlusion invariants through free-space and laser-penetration analysis. CARLO substantially lowers spoofing success while preserving detection performance, although its components involve efficiency and error trade-offs.

  • Defense Design: CARLO treats occlusion patterns as invariant physical features to detect spoofed vehicles without changing existing perception models.It combines free-space detection with laser-penetration detection.
  • Free Space Detection: Free-space detection distinguishes valid and spoofed vehicles using the ratio of free-space volume within detected bounding boxes.The valid and spoofed ratio distributions are reported as clearly separated, including after accounting for model error.
  • Free Space Detection: Free-space detection requires around 100 ms per vehicle because ray-casting every detected bounding box is too time-consuming for real-time use.The timing is measured in a C++ implementation on a commodity Intel i7-6700K CPU.
  • Laser Penetration Detection: Laser-penetration detection uses the arrangement of points across the free-space, bounding-box, and behind-box regions for greater efficiency.Its error-considered distributions overlap because of noise from ground-plane points, causing erroneous anomaly detections.
  • CARLO Design: CARLO hierarchically combines laser-penetration filtering with free-space checking, achieving around 8.5 ms mean processing time per vehicle.The first stage filters clearly fake or valid boxes, while uncertain boxes undergo free-space checking.
  • Evaluation: CARLO reduces mean ASR to around 5.5% and keeps recall near 95% across three guarded models.At maximum attack capability, ASR falls from more than 95% to below 9.5%.
  • Evaluation: Anomaly-detection precision reaches at least 99.5% for all target models, with manually verified misdetections occurring only for vehicles at least 40 meters away.The reported misdetections therefore do not affect immediate driving behavior.
  • Evaluation: CARLO reduces Adv-LiDAR ASR from more than 95% to below 5% and also withstands the adaptive attack.The adaptive attack cannot break CARLO because its holistic point modification barely changes bounding-box size.

8 Physics-Embedded Perception Architecture

SVF embeds front-view LiDAR features into end-to-end perception through sequential fusion, addressing the vulnerability of existing fusion architectures to spoofing attacks. It reduces attack success while largely preserving detection performance.

  • Why front-view features help: Front-view projection preserves LiDAR range and occlusion information, exposing neighboring occluder-occludee structure and abnormal spoofed-point sparsity.Valid vehicle points cluster in the front view, whereas spoofed points scatter.
  • Why existing fusion designs fail: Existing view-fusion architectures remain vulnerable because BEV or 3D features dominate decisions, leaving front-view features noncritical.Reproduced MV3D and MVF models achieved attack success rates as high as the original PointPillars model.
  • SVF architecture: SVF sequentially combines front-view semantic segmentation, view fusion, and 3D object detection.Front-view point-wise vehicle-confidence scores augment the 3D representation before bounding-box prediction.
  • SVF architecture: Semantic segmentation is used because it aggregates front-view information without requiring object-level prediction and has stronger established performance than front-view 3D detection.The segmentation module is agnostic to the specific implementation.
  • Evaluation: SVF models achieve comparable AP to original models, although two-stage training causes marginal degradation and a small number of false negatives.The reported false negatives do not influence driving behaviors, and the degradation may be compensated by improved training strategies.
  • Evaluation: 2.3% mean ASR is achieved by SVF, compared with 80% for the attacks, while reducing maximum-capability ASR from more than 95% to less than 4.5%.The reported robustness improvement over CARLO-guarded models is around 2.2×.

9 Discussion and Future Work

The discussion contrasts attack practicality, attack completeness, and the trade-offs between CARLO and SVF. The paper identifies physical deployment constraints and the lack of guaranteed robustness as important boundaries.

  • Attack practicality and completeness: The physical attack demonstration is limited because the implementation controls spoofed points only at 10cm-level precision.Only two fine-controlled physical attack traces were constructed for proof of concept.
  • Attack practicality and completeness: Accurate laser aiming at a target LiDAR remains challenging without real-world road tests and precision instruments.The paper leaves real-world testing as future work.
  • Attack practicality and completeness: The identified vulnerability is not complete, so other vulnerabilities may remain hidden in autonomous-driving systems.The authors propose model verification and broader empirical studies as future research.
  • Defense discussion: CARLO avoids retraining as a practical postdetection module, whereas SVF requires retraining but achieves better defense performance.Both are model-agnostic and maintain comparable AP with the original model.
  • Limitations: The defenses lack guarantees as sensor attack capability increases, and stronger attacks may eventually break them.The paper gives approximately 1500 injected points as an example where model-level distinction may become impossible.

10 Related Work

Related work spans vehicular-system security, 3D adversarial machine learning, and LiDAR-based perception security. This paper focuses specifically on robustness against spoofing in autonomous-vehicle LiDAR perception.

  • Vehicular system security: Prior vehicular-security studies address in-vehicle networks, cache side channels, connected-vehicle systems, and secured communications or transactions.The paper distinguishes its focus on emerging autonomous-vehicle systems.
  • 3D adversarial machine learning: Prior 3D adversarial-machine-learning work studies point-cloud classification, 3D mesh classification, and heuristic detection of adversarial examples.The present work instead targets LiDAR-based perception.

11 Conclusion

The paper identifies a general vulnerability in LiDAR perception, develops a black-box spoofing attack and CARLO defense, and proposes SVF for stronger robustness. Reported mean spoofing ASR falls from 80% to 5.5% with CARLO and 2.3% with SVF.

  • Conclusion: 80% mean success rate is achieved by the black-box spoofing attack across target models.The attack is based on the identified general vulnerability in LiDAR-based perception architectures.
  • Conclusion: 5.5% mean spoofing attack success rate is reported after applying CARLO.CARLO is presented as the paper’s model-agnostic spoofing-attack detector.
  • Conclusion: 2.3% mean spoofing attack success rate is reported with SVF.SVF is presented as a general architecture for robust LiDAR-based perception.

Appendices

The physical spoofing attack uses a synchronized laser-based device and is illustrated in laboratory environments. Its hardware and control design support targeting different locations and angles.

  • Appendices: The attack combines a photodiode, delay component, and infrared laser.The photodiode synchronizes with victim LiDAR signals, while the delay component triggers the laser in later firing cycles.
  • Appendices: The photodiode triggers the delay component when it captures laser signals from the victim LiDAR sensor.
  • Appendices: The attack can be programmatically controlled to target different locations and angles.
  • Appendices: Figure 22 reports average attack success rates for the proposed black-box attack on PointPillars and PointRCNN.
  • Appendices: Figure 21 depicts physical spoofing in in-lab environments.

B Supplementary Attack Evaluation

The supplementary evaluation defines A2SR to characterize attack performance across recall levels and examines attack behavior, defenses, and robustness across LiDAR perception architectures. Results compare PointPillars, PointRCNN, Apollo 5.0, fusion designs, and SVF models using figures and ablations.

  • B Supplementary Attack Evaluation: A2SR averages attack success rate across 11 recall intervals to characterize attacks beyond default-threshold ASR.The recall evaluation follows the Moderate KITTI category, and A2SR is tested on PointPillars and PointRCNN.
  • B Supplementary Attack Evaluation: PointPillars generally has higher A2SR than PointRCNN, indicating higher confidence for spoofed points in PointPillars.The passage attributes this difference to point-wise features containing more detailed information than voxel-based features.
  • B Supplementary Attack Evaluation: Figure 23 compares original, matrix-estimation-reconstructed, and feature-squeezed point-cloud feature maps.
  • B Supplementary Attack Evaluation: ME-Net eliminates fake vehicles but introduces new false negatives, while feature squeezing does not effectively eliminate them on Apollo 5.0.The evaluation uses 100 KITTI validation samples, median smoothing for feature squeezing, and settings from prior work for matrix estimation.
  • B Supplementary Attack Evaluation: Figures 24 and 25 report APs for weakened view-fusion models and weakened SVF models, respectively.Figure 25 distinguishes PointPillars and PointRCNN with PP and PR labels.
  • B Supplementary Attack Evaluation: CARLO processes detected boxes, LiDAR ray directions, and point clouds to return valid and adversarial bounding-box sets.Its algorithm initializes these sets and evaluates each box using frustum, free-space, and laser-intersection computations with FSD and LPD thresholds.
  • B Supplementary Attack Evaluation: SVF models rely more on front-view features than augmented confidence-score features in the reported weakening experiments.This conclusion is based on zeroing out each feature group and comparing the weakened models’ performance in Figure 25.
Loading 2006.16974v1…