Source-linked AI summary
Learning to Collide: An Adaptive Safety-Critical Scenarios Generating Method
Wenhao Ding, Baiming Chen, Minjun Xu, Ding Zhao
TL;DR
Rare and long-tail events make real-world evaluation of autonomous-driving algorithms difficult, especially in safety-critical settings. The paper uses autoregressive scenario building blocks and a task-guided generative model to seek risky scenarios for a given driving algorithm. Experiments show more efficient generation than grid search or human design, with adaptability to routes and parameters.
Problem
Rare events are difficult to collect and neglected in large data streams, creating a challenge for evaluating autonomous-driving algorithms in safety-critical domains.
Method
The framework samples traffic scenarios from autoregressive building blocks and uses the task algorithm as an environment to guide generation toward risky-event distributions.
Results
The framework generates risky scenarios more efficiently than grid search and human design, while providing stronger adaptability and diversity across task-conditioned routes and parameters.
Takeaways & Limitations
Task-conditioned generation can produce safety-critical scenarios that are aligned with daily experience while adapting to different routes and parameters.
Abstract
from arXiv · showhide
Long-tail and rare event problems become crucial when autonomous driving algorithms are applied in the real world. For the purpose of evaluating systems in challenging settings, we propose a generative framework to create safety-critical scenarios for evaluating specific task algorithms. We first represent the traffic scenarios with a series of autoregressive building blocks and generate diverse scenarios by sampling from the joint distribution of these blocks. We then train the generative model as an agent (or a generator) to investigate the risky distribution parameters for a given driving algorithm being evaluated. We regard the task algorithm as an environment (or a discriminator) that returns a reward to the agent when a risky scenario is generated. Through the experiments conducted on several scenarios in the simulation, we demonstrate that the proposed framework generates safety-critical scenarios more efficiently than grid search or human design methods. Another advantage of this method is its adaptiveness to the routes and parameters.
I. INTRODUCTION
The paper addresses rare-event and long-tail challenges in autonomous-driving evaluation by generating realistic, safety-critical scenarios. It combines a structured scenario representation with task-guided generation to produce diverse scenarios for evaluating driving algorithms.
- Motivation: Rare events are difficult to collect and often neglected in large data streams, challenging autonomous-driving systems in safety-critical domains.
- Motivation: Grid search is time- and labor-intensive, while worst-case evaluation can produce extremely risky scenarios that are unlikely to occur in real-world deployment.
- Approach: The paper represents traffic scenarios with factorized conditional probabilities, called reusable building blocks, and samples from their dependency structure to generate new scenarios.
- Approach: The generative model is treated as an agent and a specific task algorithm as its environment to guide safety-critical scenario generation.
- Contributions: The framework generates traffic scenarios by sampling joint distributions of autoregressive building blocks and is designed to evaluate driving-algorithm safety.
II. RELATED WORK
Prior work addresses rare and safety-critical scenarios through long-tail learning, adversarial evaluation, importance sampling, and generative models, but rare-event sampling and scenario safety remain unresolved. The paper therefore uses task guidance to focus generation on risky-event distributions.
- Long-tail data imbalance makes rare events difficult to collect and challenges safety-critical autonomous-driving applications.
- Grid search for risky scenarios is time and labor intensive, while worst-case evaluation may produce extreme scenarios unlikely to occur in the real world.
- Generative models can create diverse or open-world scenarios, but prevalent models lack an explicit process for generating rare events.
- Existing generative approaches still face the problems of sampling low-probability rare events and verifying that generated scenarios are safe-critical and reasonable for autonomous vehicles.
- Task-guided methods use evaluation-task feedback to improve sampling efficiency, motivating the use of a task algorithm to guide safety-critical scenario generation.
III. METHOD
The method represents traffic scenarios with human-informed directed graphical models whose nodes are conditional building blocks, then samples and trains a generator using task feedback. It focuses on compact critical scenarios while acknowledging difficulty scaling to scenes with many participants.
- A. Scenario representation: Traffic scenarios are represented as graphical-model nodes corresponding to building blocks such as location, orientation, and velocity.
- A. Scenario representation: Human knowledge converts an undirected graph into a directed graph, avoiding an often-intractable partition function and difficult-to-define potential functions.
- A. Scenario representation: The directed representation factorizes scenarios into independent and autoregressive conditional building blocks that can be sampled according to dependency relationships.
- A. Scenario representation: In the cyclist example, nodes encode spawn position, orientation, trigger distance, and input states such as the autonomous vehicle route and target speed.
- C. Task-guided data generation: The training procedure builds the factorized model, samples states and actions, simulates scenarios, computes rewards, and updates the policy parameters.
- A. Scenario representation: The method focuses on scenarios with few participants, while scenes involving hundreds of vehicles and pedestrians remain challenging and require future study.
- A. Scenario representation: The implementation models continuous blocks with Gaussian distributions, discrete blocks with multinomial distributions, and rescales sampled actions into real-world scenario parameters.
B. Scenario generation framework
The framework models scenario generation as an agent interacting with an AV environment, using rewards to produce safety-critical scenarios while preserving structured dependencies and diversity.
- The scenario generator is treated as an agent, while the evaluated full-stack AV algorithm serves as its environment.
- The environment state combines the AV route and lane map with task-algorithm parameters such as target speed.These inputs let the generator adjust obstacle parameters for the evaluated driving task.
- The reward combines distance-based risk, a collision bonus, and a penalty for obstacles spawned too close to the route.The penalty uses a threshold to exclude unreasonable scenarios.
- A probabilistic graphic model represents scenario elements as building blocks whose dependencies support structured scenario generation.The framework uses position and orientation distributions as part of this conditional representation.
C. Optimization process
The optimization process applies REINFORCE to update the scenario policy from rewards, while entropy regularization promotes policy diversity and autoregressive Gaussian structure enables joint-probability calculation.
- REINFORCE updates the generative policy parameters using sampled actions and their associated rewards.The gradient is based on the expected reward objective under the policy distribution.
- An entropy term is added to the objective to encourage diversity in the generated policy.
- The autoregressive Gaussian policy permits joint-probability calculation through the chain rule.The joint distribution remains Gaussian, making entropy calculation tractable.
- The optimization results compare collision rate and iterations required to reach stability, with iteration comparisons limited to methods that require iteration.
A. Experimental settings
The experiments implement the framework in PyTorch with Adam and use Carla and its Scenario Runner Library to generate and evaluate traffic scenarios.
- The implementation uses PyTorch and Adam for model training and optimization.Experiment hyperparameters are listed in Table I.
- Carla provides the simulation platform, while the Carla Scenario Runner Library generates the traffic scenarios.The modified Scenario04 serves as the test-bed.
- The evaluated AV algorithm is a trajectory-following model controlled by a PID controller.Its state and action modules use single fully connected layers.
B. Verification experiment
The verification experiment tests whether the framework produces risky scenarios across routes and target speeds, and examines whether its autoregressive structure captures policy dependencies.
- Training uses 10 routes and target speeds from 20km/h to 50km/h, followed by testing on 4 different routes.
- The model outputs different policies when routes and target speeds change, indicating adaptation to the supplied conditions.
- High-probability regions correspond to dangerous intersection blind spots that human drivers tend to overlook.
- Different sampled initial positions produce different orientation distributions, supporting learned dependencies between policies.
1) Grid Search:
The section compares grid search, random sampling, independent policies, and the proposed method for generating risky scenarios. It evaluates collision rate, convergence time, and adaptability across routes and target speeds.
- Grid Search: Grid search discretizes policies and evaluates all combinations, but its search space grows exponentially with scenario-representation dimension.The tested discretization uses steps for [X, Y, Θ, D].
- Evaluation: The comparison uses collision rate after stable training and iterations required to reach stability across 30 experiments per method.Experiments vary routes and target speeds.
- Independent Policy: The independent-policy method has slightly lower collision rate than the proposed method because decoupling actions can lose constraints and reduce generation efficiency.Its policies are modeled as independent Gaussian blocks conditioned only on state S.
- Time Cost: The independent-policy method requires more time to stabilize because variance in each policy can influence the others and destabilize training.Grid search also remains time-consuming even with large discretization steps that produce few collision scenarios.
- Adaptability: The proposed method is more adaptive across routes and target speeds, whereas grid search must restart from scratch for each setting.Lower variance indicates greater adaptability in the reported comparison.
D. Experiments on other scenarios
The framework is tested on three additional intersection scenarios involving red-light crossing, left turns, and right turns. It finds risky distributions for all three settings.
- Scenario Settings: The three scenarios involve straight travel against a red-light-running vehicle, a left turn facing an approaching vehicle, and a right turn facing a vehicle from the left.Each scenario uses position X, position Y, and speed V as the action dimensions, with fixed orientation where specified.
- Results: The framework finds risky distributions for all building blocks in each of the three new scenarios.The generated risky scenarios are presented in Fig. 8.
E. Exploration of solution space
The paper explores multiple stable solutions and identifies a limitation of Gaussian policies: they may represent only a sub-space of a multimodal solution space. Future work targets richer mappings and more complex scenarios.
- Solution Diversity: Six different stable solutions are found, indicating that the risky-parameter solution space can contain multiple feasible modes.The solutions are initialized with different values.
- Solution Space: Gaussian policies optimized with REINFORCE may capture only a sub-space when the full risky-scenario solution space is non-convex or multimodal.The paper gives feasible solutions on both sides of a route as an example.
- Future Work: Future work proposes mapping action space to scenario-representation space with implicit or complex distributions, including VAE- and flow-based models.The proposed direction aims to model multimodal safety-critical scenario distributions.
- Framework: The framework combines reusable scenario building blocks with task-algorithm guidance to generate risky scenarios from probability distributions.The representation uses dependencies informed by human knowledge, while the task algorithm guides the generative model.
- Conclusion: The reported advantages are higher efficiency than grid search and human design, plus adaptability and diversity conditioned on task algorithms.These conclusions are based on simulation experiments.