PaperlayerView exact revision on arXiv

Source-linked AI summary

Collision Snapshot Guided Time-Reversed Safety-Critical Scenario Generation

Taehyung Kim, Jongeun Choi

arXiv:2609.06433v1cs.ROcs.AI

TL;DR

Safety-critical events are rare, while existing scenario-generation methods can produce implausible or insufficiently diverse interactions and struggle to choose where and when to insert adversarial vehicles. COSTER uses learned traffic priors to construct collision snapshots and a CVAE to roll the inserted vehicle backward in time, outperforming existing methods in plausibility, diversity, and data efficiency while improving downstream driving safety.

  • Problem

    Safety-critical scenario generation is challenged by rare events and by prior methods' limited plausibility and diversity when perturbing existing agents or selecting adversarial-vehicle insertion conditions.

  • Method

    COSTER uses learned traffic priors to determine scenario-specific collision configurations and times, then performs CVAE-based time-reversed rollout from a collision snapshot.

  • Results

    COSTER outperforms existing methods in plausibility, diversity, and data efficiency, while agents trained on its scenarios reduce collision rates by 31% on Waymo Open Motion Dataset safety-critical scenarios and improve ego task completion.

  • Takeaways & Limitations

    Traffic-prior-based collision snapshots followed by generative rollout provide a way to synthesize safety-critical scenarios with improved scenario quality, diversity, data efficiency, and downstream driving safety.

  • Takeaways & Limitations

    COSTER is currently limited to vehicle-to-vehicle collisions and may overlook hazardous low-entropy scenarios.

Abstract

from arXiv · show

The generation of safety-critical traffic scenarios is essential for training and evaluating autonomous vehicles. Prior approaches typically perturb the trajectories of existing agents in a traffic scenario using simplified adversarial objectives to induce safety-critical interactions, which can limit the plausibility and diversity of the generated scenarios. Although inserting new adversarial vehicles can alleviate this limitation, determining when and where to introduce them in a scenario-specific manner remains challenging. In this work, we introduce \underline{CO}llision \underline{S}napshot guided \underline{T}im\underline{E}-\underline{R}eversed safety-critical scenario generation (COSTER), a framework that leverages learned traffic priors to determine plausible collision times and locations. COSTER first constructs a collision snapshot by inserting a new vehicle in contact with the target vehicle at the identified collision state within a traffic scenario. Starting from this collision snapshot, a conditional variational autoencoder is used to perform a time-reversed rollout, reconstructing the trajectory of the inserted vehicle backward toward earlier timesteps. Experiments show that COSTER outperforms existing methods in plausibility, diversity, and data efficiency. Moreover, agents trained on COSTER-generated scenarios reduce collision rates by 31\% on safety-critical scenarios from the Waymo Open Motion Dataset while also improving ego task completion. The project website is available at https://anonym-121.github.io/COSTER/.

I. INTRODUCTION

Safety-critical events are rare, motivating simulation-based generation of diverse and plausible scenarios. COSTER addresses limitations of prior methods by using traffic priors to choose collision configurations and times, then generating scenarios through time-reversed rollout.

  • Motivation: Safety-critical events occur infrequently because they arise from complex combinations of diverse environmental factors.This scarcity challenges data-driven modeling and generalization of safety-critical events.
  • Motivation: Simulation can generate large volumes of safety-critical scenarios for autonomous-driving training and validation.
  • Limitations of prior methods: Perturbing existing vehicles requires hand-crafted adversarial objectives and depends strongly on the original scenario configuration.When the perturbed agent is not sufficiently close, behavior may become unrealistically aggressive or fail to produce a safety-critical scenario.
  • COSTER: COSTER uses learned traffic priors to construct scenario-specific collision configurations and select collision times according to evolving traffic context.This is intended to improve plausibility, diversity, and avoid out-of-distribution initialization.
  • COSTER: COSTER combines a velocity-entropy-based collision-time strategy with a CVAE-based time-reversed rollout that updates surrounding-agent information at each timestep.
  • Results: COSTER outperforms state-of-the-art methods in scenario quality and diversity, while trained agents achieve improved downstream driving safety.

II. RELATED WORK

Prior scenario-generation methods use simplified objectives, manual interaction designs, pretrained components, or modifications constrained by the base scene. COSTER instead uses traffic priors for plausible collision snapshots and entropy-based collision-time selection.

  • Existing methods: Trajectory-level methods often depend on simplified adversarial objectives, including distance minimization or marginal objectives.
  • Existing methods: CaDRE broadens collision configurations through quality-diversity optimization, while GOOSE and SEAL improve realism using goal-conditioned and skill-based hierarchical reinforcement learning.
  • Limitations: CaDRE, GOOSE, and SEAL retain limitations involving predefined measure spaces, manual interaction-mode selection, pretrained CAT dependence, or base-scenario configuration.
  • Adv-BMT: Adv-BMT inserts a new vehicle and generates its trajectory backward, but randomly initializes the vehicle relative to the target without explicitly modeling scenario-specific traffic context.Random initialization can place the generative model under out-of-distribution initial conditions and produce implausible trajectories.
  • COSTER: COSTER estimates contextually plausible adversarial states in contact with the target, enabling time-reversed rollout to start from an in-distribution initial state.This avoids map-inconsistent or physically infeasible initialization during trajectory generation.
  • COSTER: COSTER selects collision time using velocity entropy to improve data efficiency while promoting diverse and contextually plausible collision scenarios.

III. PROPOSED METHOD

COSTER represents traffic scenarios as maps and vehicle trajectories, then constructs a collision scenario in two stages: selecting a collision snapshot and sampling the inserted vehicle's preceding trajectory backward in time.

  • Problem formulation: A traffic scenario consists of a vectorized map and trajectories for N vehicles across T timesteps.
  • Problem formulation: Each vehicle trajectory is represented by vehicle states across time, and COSTER aims to generate a collision scenario containing an inserted adversarial vehicle.
  • Two-stage construction: COSTER first selects a collision snapshot χ containing the collision time and adversarial-vehicle state at that time.
  • Two-stage construction: It then trains a time-reversed trajectory distribution and samples the adversarial vehicle's earlier trajectory conditioned on its collision-time state and the scenario history.

B. Local Region-Based Vehicle State Modeling

COSTER represents traffic context through local regions aligned to vectorized lane geometry, then predicts whether regions contain agents and their position, velocity, and size. These predictions support scenario-specific adversarial vehicle placement.

  • Region construction: Lane-line samples define rectangular local regions with coordinates aligned to the lane direction.Each region uses adjacent sampled points to establish its origin and orientation, with width along x and lane-segment length along y.
  • Region representation: Each region represents lane type, traffic-light state, agent position, velocity, size, and occupancy.Occupancy is binary, while position, velocity, and size are represented as two-dimensional quantities.
  • Context encoding: Multi-context gating refines regional features using shared global context while reducing the computational cost of cross-attention.The resulting context-aware features condition the prediction heads for each region.
  • State prediction: Independent decoder heads predict occupancy with a Bernoulli distribution and state attributes with discretized categorical distributions.Continuous values are recovered from predicted bins by bin-center decoding at inference.
  • Training objective: The model is trained end-to-end with binary cross-entropy for occupancy and categorical cross-entropy for position, velocity, and size.The loss aggregates predictions across regions and state variables.

C. Collision Snapshot Determination

COSTER selects a collision time from traffic uncertainty around the target vehicle, then constructs a collision snapshot using the most probable nearby region and sampled adversarial vehicle attributes. The vehicle is translated into contact at a high-probability placement.

  • Collision-time selection: Velocity entropy is used to select the collision time because it is empirically correlated with collision risk.This enables explicit collision-time selection rather than determining it indirectly through an adversarial objective.
  • Collision-time selection: At each timestep, COSTER models velocity distributions in local regions within an elliptical neighborhood centered on the target vehicle.The regional traffic model is applied independently at each timestep.
  • Collision-time selection: The timestep with the largest velocity entropy is selected as t_col.Entropy is computed from the normalized velocity distribution obtained by averaging regional distributions.
  • Vehicle placement: At t_col, the region with the highest predicted occupancy probability is selected around the target vehicle.The adversarial vehicle’s heading, velocity, and size are sampled from the selected region’s predicted categorical distributions.
  • Vehicle placement: The sampled adversarial vehicle is translated toward the target while maintaining its relative heading until contact, with the final position chosen by predicted position probability.This produces a collision snapshot consistent with the modeled regional traffic context.

D. Adversarial Trajectory Generation

COSTER reconstructs the inserted vehicle’s trajectory backward from the collision snapshot using a CVAE with dynamic surrounding attention. It samples multimodal trajectories and refines representative behaviors toward physically feasible motion tokens.

  • Time-reversed rollout: A CVAE reconstructs the adversarial vehicle trajectory backward from the collision time toward earlier timesteps.The model conditions on the collision snapshot, map, target trajectory, and a latent variable.
  • Dynamic surrounding attention: Dynamic surrounding attention updates neighboring-agent context at every reverse-decoding timestep.An RNN and cross-attention module use relative position and velocity differences to condition the predicted reverse displacement.
  • CVAE training: During training, posterior trajectory information helps the CVAE capture long-horizon strategic behaviors.The decoder predicts displacement from its input representation and a sampled latent variable, while the posterior uses an encoded ground-truth trajectory segment.
  • CVAE training: During inference, the decoder samples latent variables from the prior while the ELBO combines reconstruction likelihood with posterior regularization.The second term regularizes the posterior toward the prior through KL divergence.
  • Trajectory refinement: Generated trajectories are clustered by lateral displacement and arc length, then projected onto nearby motion-vocabulary tokens for physical feasibility.These two refinement stages summarize multimodal behavior and reduce sensitivity to outliers.

A. Experiment Setting

Experiments use WOMD scenarios to assess COSTER's vehicle-state inference and downstream components against established baselines. The evaluation includes state-distribution modeling with MMD and past-trajectory prediction accuracy.

  • Models are trained on WOMD v1.1, containing 530,000 nine-second scenes recorded at 10 Hz and downsampled to 2 Hz.
  • State inference is evaluated on an identical set of 1,000 WOMD scenarios for each method.
  • COSTER, TrafficGen, LCTGen, and a Mean Value baseline are compared for vehicle-state inference using maximum mean discrepancy, where lower MMD indicates closer distributional matching.
  • COSTER represents each state variable as a discretized categorical distribution, while TrafficGen and LCTGen use Gaussian-mixture modeling.

C. Time-Reversed Trajectory Generation Assessment

COSTER is assessed through time-reversed trajectory reconstruction and full safety-critical scenario generation. Results indicate that context-aware collision initialization and dynamic rollout improve accuracy, plausibility, and diversity over comparison methods.

  • Time-Reversed Trajectory Generation: COSTER with dynamic surrounding attention achieves the lowest displacement errors across all reported past-trajectory metrics.The evaluation reports minADE, minFDE, avgADE, avgFDE, and inference time per trajectory.
  • Time-Reversed Trajectory Generation: Dynamic surrounding attention reduces long-horizon drift by updating surrounding-agent context at each decoding step.
  • Safety-Critical Scenario Generation: COSTER achieves the highest scenario realism and collision rates while also attaining the highest diversity in collision placement and dynamics.The reported metrics include SUR, CR, CPD, CSD, ORR, and ADD; the passage specifically states highest SUR, matching best CR, and highest CPD and CSD.
  • Safety-Critical Scenario Generation: Random collision initialization can produce implausible or out-of-distribution starting conditions, whereas context-aware snapshot initialization alleviates these failures.
  • Safety-Critical Scenario Generation: Qualitatively, COSTER generates multimodal, plausible scenarios that account for surrounding traffic, unlike baselines that can leave the map, fail to induce collisions, or ignore nearby vehicles.

E. Analysis on Velocity Entropy-based Selection

The velocity-entropy strategy selects collision times in regions with complex traffic interactions. These regions offer richer candidate adversarial states and broader collision variation.

  • Velocity entropy selects collision times by identifying regions of complex traffic interaction.
  • A richer candidate pool yields broader collision variation in both impact location and relative speed.

F. Downstream Evaluation on Policy Safety

COSTER-generated scenarios improve policy performance in both normal and safety-critical WOMD settings. The results support training with diverse variants generated from a single raw scenario.

  • COSTER is evaluated against CAT, SEAL, and Adv-BMT using RL-based traffic agents trained on 1,000 generated scenarios per method.
  • The evaluation uses WOMD-Normal for generalization and WOMD-Hard for high-risk scenarios selected by SafeShift scoring.
  • COSTER consistently outperforms prior methods across both evaluation sets, achieving the highest route completion rate while maintaining low collision and off-road rates.
  • On WOMD-Hard, COSTER substantially improves route completion and achieves the lowest collision rate among the compared methods.
  • Diverse safety-critical variants derived from a single raw scenario facilitate effective training and more efficient learning.
Loading 2609.06433v1…