Source-linked AI summary
One Diffusion Model, Two Roles: Guided Trajectory Planning and Safety-Critical Scenario Generation in Closed-Loop Simulation
Arka Pal, Rajesh Kumar, Hannes Eriksson, Rémi Lacombe, Arvid Laveno Ling, Ankit Gupta, Maciej Wozniak
TL;DR
Autonomous-driving planners need both multimodal interactive trajectory generation and evaluation against rare safety-critical interactions. The paper uses one pretrained diffusion traffic model with SSDS planning, DAPSE guidance, and controllable adversarial generation; stronger nominal planning performance nevertheless degrades more under generated long-tail scenarios.
Problem
Learning-based planners must generate safe, human-like interactive trajectories while standard logged-data benchmarks provide limited evidence about robustness to rare safety-critical interactions.
Method
A single pretrained diffusion model is used for SSDS-based ego planning, DAPSE clean-sample energy guidance, and controllable closed-loop generation of adversarial driving behaviors.
Results
The framework improves nominal interactive planning and generates realistic guided scenarios that expose planner weaknesses; SSDS-DP degrades more than DP under adversarial scenarios.
Takeaways & Limitations
A shared learned traffic prior can support both motion planning and systematic closed-loop robustness evaluation.
Takeaways & Limitations
In the evaluated scenarios, both planners rely on longitudinal braking rather than lateral evasive maneuvers, limiting their ability to resolve imminent intersection conflicts.
Abstract
from arXiv · showhide
Diffusion probabilistic models can capture the multi-modal, interaction-rich distribution of joint future trajectories in driving scenes. We show that a single pretrained diffusion traffic model can serve two complementary roles in the autonomous driving development loop: as an ego motion planner, and as a controllable generator of safety-critical scenarios for stress-testing the planners. On the planning side, we introduce a Single-Stream Dual-Stream (SSDS) diffusion-transformer decoder that fuses scene context via joint attention rather than late cross-attention, improving closed-loop performance on nuPlan. We further propose Decoupled Annealing Posterior Sampling with Energy (DAPSE), a training-free guidance scheme that injects arbitrary energy functions at the clean-sample level, avoiding the first-order approximation errors while requiring no auxiliary networks. Beyond planning, we leverage the same diffusion model as a controllable scenario generator to create realistic long-tail driving interactions for closed-loop evaluation. Through inference-time guidance, selected agents are steered toward safety-critical behaviors, including aggressive cut-ins, lead-vehicle braking, and combined longitudinal-lateral interactions, while preserving realistic traffic behaviors. Evaluated in closed-loop nuPlan simulations with independent black-box planners, the generated scenarios expose failure modes that remain hidden under standard benchmarks. Although the SSDS-based planner achieves stronger nominal performance, it experiences larger degradation under these challenging scenarios, demonstrating that benchmark superiority does not necessarily translate to robustness. These results demonstrate that a single learned traffic prior can simultaneously improve motion planning and provide a realistic framework for systematic planner robustness evaluation.
1 Introduction
The paper addresses the paired challenge of generating safe, human-like trajectories and evaluating planners against rare safety-critical interactions. It uses one diffusion traffic model for both guided ego planning and controllable adversarial scenario generation.
- Motivation: Learning-based planners must handle multimodal interactive traffic while being tested beyond nominal logged-driving distributions.Regression objectives can average plausible behaviors, while standard benchmarks are dominated by nominal driving.
- Motivation: Diffusion models learn joint ego-neighbor trajectory distributions, enabling multimodality and inference-time steering with arbitrary energy functions.Low-energy guidance supports planning, while high-risk guidance supports adversarial scenario generation.
- Framework: The study builds both roles on a DiT traffic model trained on nuPlan’s 1500 hours of human driving data.The framework is evaluated in the autonomous-driving planning benchmark’s closed-loop setting.
- Contributions: SSDS-DiT, DAPSE, and controllable closed-loop scenario generation form the paper’s three contributions.SSDS-DiT fuses trajectory and context streams, DAPSE provides training-free constraint guidance, and the generator creates targeted cut-ins and braking events.
- Framework: The shared framework independently samples ego and adversarial trajectories while feeding controller-tracked histories back at each timestep.The adversarial decoder is steered by scenario-specific energies such as ETTC, Elc, and Ebrake.
2 Related Work
Prior work advances imitation learning, diffusion-based joint trajectory generation, and controllable scenario synthesis, but this paper combines an expressive joint planner with guided adversarial evaluation. Its emphasis is realistic, closed-loop highway interactions for robustness testing.
- Data-Driven Motion Planning: Imitation learning scales motion planning through expert data, while standard regression can struggle with multimodal future behavior.Related approaches include end-to-end and object-level representations, followed by generative models for diverse intents.
- Joint Prediction and Planning: Joint trajectory models unify prediction and planning by generating interactive futures for ego and neighboring agents.Gen-Drive uses diffusion for scene generation, while Large Trajectory Models use data-intensive transformers.
- Safety-Critical Scenario Generation: Scenario-generation methods synthesize diverse traffic, but robustness evaluation requires interactions beyond nominal distributions and may sacrifice realism.Adversarial methods optimize behaviors toward planner failures, creating a realism concern identified in prior work.
- Safety-Critical Scenario Generation: SafeSim enables guided closed-loop safety-critical generation but focuses mainly on intersections and trajectory proposals.This paper instead targets high-speed highway interactions such as aggressive cut-ins and sudden braking.
- Positioning: The proposed framework combines a more expressive joint planner with energy-guided adversarial behaviors in closed-loop simulation.The same diffusion framework evaluates the planner under controllable long-tail scenarios.
3 Preliminaries: Diffusion over Joint Trajectories
The diffusion model represents ego and neighboring-agent futures as a joint trajectory tensor conditioned on scene context. Its forward process corrupts clean trajectories to Gaussian noise, while the model predicts clean states from noisy inputs.
- Trajectory Representation: The scene condition collects agent states, lane and traffic-light features, static objects, and the ego navigation route.These inputs condition the joint trajectory-generation process.
- Trajectory Representation: The trajectory tensor x(t) has shape R^(M+1)×τ×4 across agents, horizon steps, and four state components.The indexed state x_i,k represents agent i at trajectory timestep k.
- Trajectory Representation: Agent i=0 denotes the ego vehicle, while i>0 denotes one of its M nearest neighbors.This indexing makes the representation jointly cover ego planning and neighbor prediction.
- Trajectory Representation: The clean state at each trajectory step contains planar position and heading encoded by cosine and sine.The displayed state components include p_x, p_y, cos θ, and sin θ.
- Diffusion Process: A variance-preserving linear-noise process corrupts x(0) to Gaussian noise x(1), and μθ predicts the clean sample from noisy data.The model is trained over diffusion time t∈[0,1].
4 Planner: SSDS Diffusion Planning with DAPSE Guidance
The planner fuses trajectory and scene tokens early through SSDS, then uses DAPSE to guide clean-sample estimates with arbitrary energies during diffusion. This provides training-free steering without auxiliary networks or intermediate-energy approximations.
- Scene Encoding: The scene encoder processes multimodal traffic context for downstream trajectory planning.The context includes agent histories, lane features, traffic-light information, routes, and static objects.
- SSDS Decoder: SSDS treats trajectory and context as separate streams whose tokens interact through joint attention before a unified single-stream fusion stage.This enables early symmetric interaction while preserving modality-specific processing.
- SSDS Decoder: Unlike standard DiT cross-attention, SSDS performs early symmetric cross-stream interaction followed by deep fusion.The design targets stronger integration of scene context during trajectory generation.
- DAPSE: DAPSE uses decoupled noise levels to apply arbitrary clean-sample energy guidance without auxiliary networks or first-order intermediate-energy approximations.It extends DAPS from measurement likelihoods to arbitrary, non-analytical energy functions.
- DAPSE: The Langevin update combines reconstruction toward the reverse ODE estimate, energy guidance, and injected noise.The clean-sample update is performed over inner MCMC iterations.
- DAPSE: DAPSE generalizes DAPS and recovers the original DAPS update when E_0=-log q_0(y|x(0)).The decoupling permits correction of global errors committed at early diffusion steps.
5 Scenario Generator: Adversarial Scenario Generation
The pretrained diffusion framework is repurposed for closed-loop adversarial scenario generation, steering agents toward realistic safety-critical interactions through inference-time guidance. Guidance objectives cover collision risk, lane changes, braking, road compliance, and composed cut-in–braking behaviors.
- Scenario generation: The pretrained DiT decoder generates adversarial scenarios for planner evaluation through guided sampling in closed-loop simulation.DPS is used for diverse, composable adversarial energy functions without DAPSE’s annealing-schedule overhead.
- Collision-risk guidance: TTC guidance incorporates relative positions and velocities to target future collision risk rather than instantaneous proximity.A temporal lead shifts the interaction target to the ego vehicle’s anticipated future position, producing more temporally consistent interceptions.
- Lane-change guidance: Lane-change guidance drives an adversary toward the lane centerline, producing lateral maneuvers that follow the underlying road structure.The objective uses the adversary’s lateral position relative to the corresponding lane-centerline position in the ego-anchor frame.
- Braking and compliance: Braking guidance penalizes progress beyond a desired profile, encouraging gradual deceleration while maintaining lane consistency.A one-shot trigger activates the objective based on target speed, and the target velocity is progressively updated using a predefined deceleration rate.
- Braking and compliance: Drivable-area guidance penalizes trajectories outside road geometry, while sequential objectives compose braking and cut-ins into coordinated longitudinal-lateral interactions.The order can be braking followed by cut-in or the reverse, with transitions controlled by a distance threshold.
6 Experiments
Closed-loop experiments evaluate SSDS-DP against DP across training scales, DAPSE guidance, and generated adversarial scenarios. SSDS-DP improves interactive nominal planning, while guided scenarios expose substantial robustness gaps despite realistic behaviors.
- Planning: SSDS-DP improves most in reactive hard scenarios, reaching a +14.4-point gain at 650K training scenarios.Its advantage is strongest when inter-agent interaction modeling matters, while the gap narrows in non-reactive settings.
- Planning: At 1M training scenarios, DP improves significantly and outperforms SSDS-DP in some evaluation splits.The reported results indicate no clear winner at the largest training scale.
- Guidance: DAPSE shrinks an initially long, high-speed ego trajectory after energy-based updating, although the intermediate trajectory becomes more jittery.Noise addition and reverse diffusion then produce a shorter trajectory than the original unconditional sample.
- Scenario generation: The generated adversarial set covers cut-ins, lead-agent braking, combined cut-in and braking, and intersection interactions through different guidance combinations.These combinations target lateral merging, longitudinal conflicts, and safety-critical interactions while preserving road-compliance constraints where specified.
- Scenario generation: Under combined cut-in and braking, both planners degrade substantially, with SSDS-DP showing the larger drop and increased ego-at-fault collisions.The generated behaviors retain realism deviations comparable to SafeSim while reducing TTC-in-bound and comfort scores.
- Scenario generation: Across qualitative adversarial scenarios, DP reacts earlier and brakes more decisively than SSDS-DP, while neither planner attempts lateral evasive maneuvers.Both planners struggle in intersection scenarios because braking alone cannot resolve an imminent crossing conflict.
- Overall findings: The unified diffusion framework supports both stronger interactive planning and controllable long-tail scenario generation for systematic planner stress-testing.Generated scenarios expose failure modes in planners that perform strongly under nominal benchmarks while maintaining realistic agent behaviors.
7 Conclusion
The paper presents a single pretrained diffusion model as both an interactive planner and a controllable generator of realistic long-tail scenarios for closed-loop evaluation. It finds that strong nominal benchmark performance can coexist with difficulty handling complex multi-agent interactions.
- Planning: The SSDS-DP improves interactive planning, while DAPSE provides training-free, zero-shot safety steering.Both components operate within the unified diffusion framework.
- Scenario generation: The framework generates realistic closed-loop long-tail scenarios that expose critical planner failure modes.These scenarios support evaluation of complex multi-agent interactions beyond nominal benchmark behavior.
- Robustness: Learned planners can struggle with complex multi-agent interactions despite strong nominal benchmark performance.The analysis attributes this struggle to reliance primarily on reactive braking rather than proactive evasive maneuvers.
- Conclusion: A single learned traffic prior can serve as both an effective planner and a rigorous evaluator in the autonomous-vehicle development loop.The paper identifies this dual use as the central outcome of its unified diffusion framework.