AdvSim: Generating Safety-Critical Scenarios for Self-Driving Vehicles

1. Introduction

• Prior searches often test ground-truth planning alone [15] [12]; image-based tests were limited or simplified [33] [32], while other methods use coarse or unrealizable behavior changes [15].Ground-truth planning tests can miss occlusion-related perception and forecasting failures.Source: §1 · pp. 1–2
• AdvSim frames worst-case scenario generation as a black-box attack on any LiDAR-based autonomy system, using physically feasible changes to actor behavior.This focuses on challenging driving situations rather than perturbing only appearance or texture.Source: §1 · p. 2
• AdvSim generates over 4000 adversarial scenarios for modern autonomy systems, and using them in training further improves system safety.The scenarios target a wide range of autonomy systems.Source: §1 · p. 2
• AdvSim optimizes actors’ trajectories in real traffic scenes, updates LiDAR data with a high-fidelity simulator that accounts for occlusion, and evaluates safety, traffic-rule, and comfort outcomes.The autonomy system processes the modified sensor data before its planned trajectory is scored.Source: §1 · p. 2

Self-Driving System

• Autonomy research spans modular perception–prediction–planning pipelines, end-to-end control, and interpretable neural motion planners; AdvSim evaluates a wide range, including modular and interpretable systems.Examples of interpretable neural planners include NMP, DSDNet, and P3; the passage also traces end-to-end control methods to ALVINN.Source: §2 · p. 2

Safety-Critical Scenario Generation

• Prior scenario-generation work varies actor-action parameterizations and black-box searches, but often tests ground-truth planners or image-based systems on few scenes, unlike AdvSim’s full-stack, dataset-scale approach.AdvSim represents actors with kinematic bicycle-model trajectories for physical feasibility and fine-grained control, and benchmarks black-box search algorithms. Table 1 summarizes the comparison with prior works.Source: §2 · pp. 2–3, Table 1 · p. 3

Physically Realizable Adversarial Examples

• Prior physical adversarial work targets images, meshes, photorealistic properties, LiDAR detectors or points, and vehicle textures, whereas AdvSim generates realistic adversarial maneuvers by other agents.Image-space perturbations include constraints on view angles, distances, and color distortion; LiDAR-based work creates invisible or detrimental objects or spoofs points with laser devices.Source: §2 · p. 3

3.1. Problem Setup

• The attack perturbs actors in existing traffic scenes and maximizes a cost on the SDV’s planned trajectory after simulating LiDAR and evaluating the full system on sensor inputs.The sensor inputs include raw LiDAR, high-definition maps, and other information, so the formulation targets the full autonomy system rather than only planning.Source: §3.1 · p. 4

3.2. Modeling Adversarial Behaviors

• Actor trajectories use a kinematic bicycle model with bounded curvature-change and acceleration controls, optional bounded initial-state changes, and rejection sampling plus L2 projection to enforce collision-free feasibility.Feasibility excludes collisions with other actors or the original expert SDV trajectory.Source: §3.2 · p. 4

3.3. Realistic LiDAR Simulation

• LiDARsim [30] updates only modified sweep regions, removing original actors and filling their LiDAR shadows, then adding perturbed actors while modeling occlusion in merged range images.The simulated background is cached while the SDV trajectory remains fixed, speeding up the query function.Source: §3.3 · pp. 4–5

3.4. Adversarial Scenario Search

• The adversarial loss combines deviation from the recorded human path, cumulative collision cost, and traffic-rule and comfort costs for lane violations, high acceleration, and jerk.Using multiple costs lets AdvSim identify failures such as unnatural trajectories, collisions, and hard braking.Source: §3.4 · p. 5
• A query-based black-box loop maximizes Ladv using BO [43], GA [2], RS [18], NES [20], or Bandit-TD [21], updating actor trajectories, LiDAR, and evaluations.The loop uses past query pairs to select perturbations and evaluates the full autonomy system on each generated scenario.Source: §3.4 · pp. 5–6

4.1. Experimental Setup

• Evaluation used UrbanScenarios logs and open-loop 5-second plans to assess collision, imitation, comfort, perception, and forecasting performance across IL, PLT, NMP, and P3.The dataset contains 5,000 25-second logs; curation yielded 3,953 training and 409 validation scenarios. Candidate scenarios were selected for SDV interactions and high trajectory collision rates.Source: §4.1 · pp. 6–7

4.2. Experimental Results

• 32.8% average collision rate—over 200% above the original set—was achieved across modern autonomy systems when perturbing one vehicle and using Bayesian optimization.AdvSim also caused collisions with non-perturbed actors, including in qualitative examples for several systems.Source: §4.2 · p. 7, Figure 4 · p. 7
• Curriculum learning improved performance on original and adversarial scenarios, while AdvSim robust training further improved most planning metrics using the original expert trajectories as ground truth.Perturbation constraints keep the original expert trajectory a valid collision-avoiding planning solution; the paper also evaluates generalization on scenarios generated for other systems, but reports no outcome for that test here.Source: §4.2 · p. 7
• Scenarios generated for the same target system usually performed best, while scenarios generated for the more robust P3 system transferred more strongly across systems.Transferability is measured by cumulative collision rate up to 5 seconds.Source: §4.2 · pp. 7–8, Table 4 · p. 7
• As the number of perturbed actors increased, planning and prediction metrics worsened, but collision rate decreased for m ≥ 3; Bayesian optimization was the most efficient tested search algorithm.The authors attribute the collision-rate decrease to finite trajectory sampling and the greater difficulty of finding physically plausible candidates when attacking multiple actors.Source: §4.2 · p. 8, Table 5 · p. 8, Table 6 · p. 8
• The hybrid adversarial objective generated worst-case scenarios across multiple metrics, and the proposed design outperformed closest-distance and training-cost baselines on all planning metrics.The authors use M3 by default because collisions are key for evaluation, while allowing a subset of costs when a particular planning metric is the focus.Source: §4.2 · p. 8, Table 8 · p. 8

• AdvSim generates physically plausible failure scenarios that expose risks to the full autonomy stack by simulating sensor data from perturbed actor behaviors.Source: §5 · p. 8
• AdvSim generates failure cases at scale for a wide range of autonomy systems.Source: §5 · p. 8
• Training with AdvSim-generated scenarios can further improve autonomy-system robustness and safety.Source: §5 · p. 8

A.1. Data Curation

• Each log contributes its highest-interaction 6-second segment; curation yields 3,953 training and 409 validation scenarios from 5,000 logs.Every 2 seconds, the authors sample 100 trajectories per SDV behavior and filter collisions with the vehicle directly ahead or static vehicles before selecting the segment.Source: §A.1 · p. 11

A.2. Generating Physical Plausible Trajectories

• For each perturbed actor, 2e5 trajectories are sampled and filtered for collisions, off-road driving, and actors with fewer than 100 feasible candidates.Actors are selected from the region of interest, prioritizing those closest to the ego-car; feasible trajectories avoid other actors and the SDV expert trajectory over a 6-second horizon.Source: §A.2 · p. 11
• Perturbations use realistic kinematic-bicycle bounds and are normalized to an ℓ∞ ball with magnitude ϵ = 1.0.The physical constraints are specified in Table 1.Source: §A.2 · p. 11, Table 1 · p. 12

A.3. Black box Search Algorithms

• Bayesian optimization [43] uses a Gaussian-process surrogate with UCB (β = 3), a 75-query budget, and 20 initialization queries.The authors omit additive GP models and auto-subspace search because the attack space is low-dimensional.Source: §A.3 · p. 12
• The genetic algorithm [29, 2] evolves candidate trajectories by fitness-weighted sampling, preserves the best candidate, and uses adaptive parameter scaling.Adaptive scaling is used to reduce sensitivity to hyperparameter values.Source: §A.3 · p. 12
• Random search [18] [3] iteratively tests Cartesian-basis perturbations, accepting improvements or applying inverse perturbations otherwise, for 100 iterations at ϵ(k) = 0.25.The method does not use dimension-reduction techniques because the perturbation space is low-dimensional.Source: §A.3 · p. 13
• NES [20] and Bandit-TD [21] estimate gradients for PGD optimization; Bandit-TD additionally uses time-dependent priors to improve query efficiency.The authors use official PyTorch implementations and refer readers to [21] for further algorithm details.Source: §A.3 · p. 13

A.4. Robust Training

• AdvSim generated safety-critical scenarios on both the training and validation sets for robust training.The P3 model [39] was first trained for 80k iterations until convergence.Source: §A.4 · p. 13
• The converged P3 model was retrained for 10k iterations using either selected challenging scenarios (CL baseline) or generated adversarial scenarios (robust train).Retraining used a learning rate of 2e−4 and early stopping.Source: §A.4 · p. 13

B.1. Generalization of Robust Training

• Robustly trained PLT and P3 consistently outperform their standard counterparts on adversarial scenarios generated for different autonomy systems.The comparison is reported in Table 4.Source: §B.1 · p. 13, Table 4 · p. 14
• The evaluation tests robustly trained PLT and P3 on safety-critical scenarios generated with respect to other autonomy systems.Source: §B.1 · p. 13, Table 4 · p. 14
• The reported evaluation metric is cumulative collision rate up to 5s.Source: §B.1 · p. 13

B.2. Supplementary Metrics for Table 3

• Generating scenarios against the same target autonomy system usually yields the best transferability, measured by L2 human prediction error at 5 seconds.Table 5 reports the transferability comparison, but the supplied passages do not include its cell values.Source: Table 5 · p. 14, §B.2 · p. 14

C. Qualitative Examples

• The qualitative examples cover simulated LiDAR sweeps, AdvSim scenarios and motion-forecasting predictions, challenging non-collision cases, and perturbations of multiple vehicles.The section presents these examples in Figures 1–4.Source: C. Qualitative Examples · p. 14

C.1. Realistic LiDAR Simulation for AdvSim

• The simulated LiDAR scenes are described as realistic and as matching the intended actor-motion perturbations.The visualizations distinguish simulated points in red from original LiDAR points in blue and show the perturbed actor’s bounding boxes.Source: §C.1 · p. 14

C.2. More Visualizations of AdvSim Scenarios

• In additional examples, the P3 system’s plans collide with other actors, and occupancy predictions are visualized over the planning horizon.The displayed occupancy predictions use blue-purple for pedestrians and yellow-red for vehicles.Source: §C.2 · p. 14, Figure 2 · p. 16

C.3. Non-collision Scenarios Generated by AdvSim

• Some generated scenarios do not cause P3 collisions, but can still challenge the SDV by requiring plans with larger costs.These cases occur when AdvSim cannot generate a perturbation that causes a collision, and the paper presents them as useful for testing and analysis.Source: §C.3 · p. 14

C.4. Perturbing Multiple Actors with AdvSim

• AdvSim’s multi-agent perturbations produce physically plausible behaviors for multiple actors and scenarios that are difficult for self-driving systems.The examples are described as qualitative illustrations of multi-agent perturbations evaluated in Table 4.Source: §C.4 · p. 15, Figure 4 · p. 17