PaperlayerView exact revision on arXiv

Source-linked AI summary

TrafficGen: Learning to Generate Diverse and Realistic Traffic Scenarios

Lan Feng, Quanyi Li, Zhenghao Peng, Shuhan Tan, Bolei Zhou

arXiv:2210.06609v2cs.RO

TL;DR

Autonomous-driving simulation needs diverse and realistic scenarios, but real-world trajectories are costly to collect and often fragmented. TrafficGen learns traffic context from real-world data with an autoregressive encoder-decoder model that generates vehicle placement and long trajectories, and it can augment existing scenes. The resulting scenarios improve vehicle-placement and trajectory generation over baselines and improve the safety of reinforcement-learned driving policies in simulation.

  • Problem

    Autonomous-driving safety evaluation is limited by insufficient realistic scenarios and by costly, fragmented real-world trajectory data.

  • Method

    TrafficGen uses an autoregressive encoder-decoder architecture to encode traffic context, generate vehicle initial states, and roll out long trajectories, including for scenario augmentation.

  • Results

    TrafficGen improves vehicle-placement and trajectory-generation performance over baselines and produces generated simulation environments that improve driving-policy safety.

  • Takeaways & Limitations

    TrafficGen provides a data-driven way to enlarge scenario coverage and train safer driving policies with more complex and denser simulated traffic.

Abstract

from arXiv · show

Diverse and realistic traffic scenarios are crucial for evaluating the AI safety of autonomous driving systems in simulation. This work introduces a data-driven method called TrafficGen for traffic scenario generation. It learns from the fragmented human driving data collected in the real world and then can generate realistic traffic scenarios. TrafficGen is an autoregressive generative model with an encoder-decoder architecture. In each autoregressive iteration, it first encodes the current traffic context with the attention mechanism and then decodes a vehicle's initial state followed by generating its long trajectory. We evaluate the trained model in terms of vehicle placement and trajectories and show substantial improvements over baselines. TrafficGen can be also used to augment existing traffic scenarios, by adding new vehicles and extending the fragmented trajectories. We further demonstrate that importing the generated scenarios into a simulator as interactive training environments improves the performance and the safety of driving policy learned from reinforcement learning. More project resource is available at https://metadriverse.github.io/trafficgen

I. INTRODUCTION

TrafficGen addresses the lack of realistic, complete, and diverse traffic scenarios for autonomous-driving simulation by learning from fragmented real-world trajectories. It generates and edits traffic scenes and long vehicle trajectories, extending scenario coverage beyond replayed data and hand-crafted or heuristic methods.

  • Motivation: Only 30% of Waymo Motion Dataset trajectories last more than 10 seconds, and only 12% cover the whole scenario.Fragmentation results from occlusion of traffic participants by the moving data-collection vehicle.
  • TrafficGen: TrafficGen learns from fragmented and noisy real-world trajectories to generate vehicle initial states and long, complete trajectories.Its encoder-decoder architecture represents HD maps and vehicle states, then produces initial-state distributions and long-term multimodal trajectories.
  • TrafficGen: TrafficGen can generate new traffic on a map, add vehicles and trajectories, and inpaint fragmented trajectory segments into longer ones.It supports both empty maps for novel scenarios and partially populated maps for scenario augmentation.
  • Application: Generated scenarios substantially improve driving-agent safety when used as higher-complexity, higher-density reinforcement-learning environments.The scenarios are imported into a simulator to train driving policies with reinforcement learning.
  • Motivation: Existing simulators rely largely on recorded traffic or hand-crafted rules, limiting scenario diversity and realism.Real-world replay preserves fidelity but requires costly data collection, while rule-based generation struggles to reflect real traffic complexity.
  • Related work: Unlike methods that separately model traffic snapshots or trajectories, TrafficGen integrates both capabilities and can edit existing scenarios.This integration supports producing new scenarios while augmenting existing ones.

III. METHOD

TrafficGen represents map regions and vehicle states as vectors, fuses their context with attention-based encoding, and uses the resulting scene representation to generate scenarios. The method supports empty or partially populated maps and models scenarios as map-plus-vehicle-state sequences.

  • Problem formulation: A traffic scenario consists of an HD map and vehicle state series over T time steps, and TrafficGen generates a similar-distribution scenario with different states and potentially longer duration.With an existing traffic fragment, the model can augment or inpaint the scenario.
  • Neural architecture: The architecture splits each HD map into regions, encodes all regions, and decodes both vehicle initial-state distributions and multi-modal long-term trajectories.The figure depicts one encoder stack for simplicity, although the implementation uses five stacked encoding blocks.
  • Vector-based traffic representation: TrafficGen vectorizes each lane into small regions, assigning each region a local coordinate system and vehicle attributes such as position, heading, speed, and bounding-box dimensions.Each region is represented by a vector that combines lane and traffic-control information with local vehicle state.
  • Context representation: The encoder processes the unordered region set with context fusion, while multi-context gating reduces complexity when maps contain up to 1000 vectors.The fused region embeddings and pooled context form the global scene representation used by the decoder.

B. Decoder for Traffic Scenario Generation

The decoder generates traffic scenarios autoregressively: it first places vehicles by sampling map-region and vehicle-attribute distributions, then repeatedly rolls out multi-modal trajectories. Re-encoding the updated context allows subsequent vehicle generation and scenario editing.

  • Decoder overview: The decoder first places vehicles on the map and then generates their long-term trajectories from fused region features and a global scene feature.Vehicle placement and trajectory generation are separate stages within the autoregressive decoder.
  • Vehicle placement: Vehicle placement samples a region from a categorical spatial distribution, then samples local position, heading, speed, and size from modeled attribute distributions.Local position is represented with a mixture of K bivariate normal distributions, and the other attributes are modeled analogously.
  • Autoregressive sampling: After each vehicle is added, TrafficGen re-encodes the traffic context and generates the next vehicle until a predefined vehicle count is reached.Random masking during training helps the model capture context and edit existing traffic scenarios.
  • Trajectory generation: The trajectory decoder replaces past-trajectory inputs with the global context feature and is trained using the minimum MSE among predicted trajectories and the ground truth.This design supports real-time trajectory sampling while addressing distributional shift in long-horizon generation.
  • Trajectory generation: For each vehicle, the decoder samples one trajectory from K predicted probabilities and uses only the first l steps before updating the state, with l ≤ L controlling update frequency.Repeated decoder rollouts concatenate these partial segments into long trajectories and reduce long-term cumulative error.

A. Experiment setting.

TrafficGen is trained and evaluated on filtered Waymo Open Dataset scenarios, using fixed-duration trajectories and spatially cropped scenes. The vehicle-placement evaluation also illustrates an autoregressive sampling process that constructs traffic snapshots incrementally.

  • Dataset: The source data contains about 70,000 Waymo scenarios with 20-second trajectories, filtered to retain scenes containing at least 8 agents.Scenes are cropped to a 120m-by-120m rectangle centered on the ego vehicle.
  • Dataset: The filtered cases are split into 50,000 training scenarios and 1,000 non-overlapping test scenarios.The trained models are benchmarked on the held-out test set.
  • Vehicle-placement sampling: At each autoregressive iteration, the model samples a vehicle's region from the spatial distribution and adds it to the scene, producing a traffic snapshot when the target vehicle count is reached.The iteration-varying heatmap visualizes how the spatial distribution changes during sampling.

B. Quantitative Results of TrafficGen

TrafficGen is evaluated for realistic vehicle placement and trajectory generation using distributional, collision, and reconstruction measures. The experiments compare its placement fidelity and assess how sampling frequency affects interaction-aware trajectories.

  • Vehicle placement: MMD measures the distribution distance between synthesized and real vehicle attributes using position, heading, and speed statistics.Scores are averaged across test scenarios for the reported placement comparison.
  • Vehicle placement: TrafficGen produces vehicle placements closer to real-scene attribute distributions than SceneGen.MMD is computed for position, heading, and speed after replacing vehicles in test scenarios.
  • Trajectory generation: Trajectory quality is evaluated with scenario collision rate, average distance error, and final distance error.These metrics assess behavioral consistency and similarity to ground-truth trajectories.
  • Trajectory generation: Higher sampling frequency significantly reduces collision rate, indicating that real-time sampling supports more interaction-aware trajectory generation.The 9-second sampling interval corresponds to a traditional motion forecasting setting.

C. Ablation study

The ablation study tests traffic-context masking and vector-based traffic representation. Both existing vehicles and traffic lights contribute to performance, while vector-based representation improves generation for SceneGen and TrafficGen.

  • Context masking: Masking existing vehicles or traffic lights worsens MMD performance, showing that both context sources contribute to TrafficGen.The two context components are masked separately during sampling.
  • Traffic representation: Vector-based traffic representation improves scenario-generation performance for both SceneGen and TrafficGen.It provides a prior that supports relative position and heading prediction to a reference region.

D. Application for Improving RL Safety

TrafficGen-generated scenarios are converted into interactive MetaDrive environments for PPO training and evaluation on held-out real-world scenarios. Agents trained with generated data achieve lower safety violations than those trained on heuristic scenarios, while generated and original Waymo data provide better success rates than heuristic data.

  • Training datasets: TrafficGen-generated data, original Waymo data, augmented data, and heuristic data define the compared training conditions.Augmented data increases vehicle-generation number N, whereas heuristic data uses procedural rules with IDM and lane-changing control.
  • Results: Agents trained in generated scenarios show lower safety violations than agents trained in procedurally generated scenarios.Safety violation is measured as average episodic collision over 100 test environments.
  • Results: Agents trained with TrafficGen-generated data and original Waymo data achieve better success rates than agents trained in heuristic scenarios.Success rate is the fraction of test episodes in which the agent reaches its destination.

V. CONCLUSION

TrafficGen learns from fragmented and noisy real-world trajectories to generate and edit traffic scenarios. Its autoregressive encoder-decoder design supports vehicle placement, long trajectories, scenario augmentation, and improved reinforcement-learning driving performance and safety.

  • Method: TrafficGen learns from fragmented and noisy real-world trajectories using an encoder-decoder architecture.It encodes a scene snapshot with vector-based context and autoregressively adds vehicles before generating long trajectories.
  • Capabilities: TrafficGen accepts empty maps or maps with partial traffic to create novel scenarios or augment existing ones.Augmentation includes adding or removing vehicles and inpainting or extending fragmented trajectories.
  • Results: TrafficGen outperforms baselines in vehicle placement and long-trajectory generation while improving reinforcement-learning driving-agent performance and safety.The synthesized scenarios are imported into a driving simulator for training.

APPENDIX

The appendix describes how generated road networks and traffic flows are imported into MetaDrive, then used to evaluate reinforcement-learning driving agents. It also specifies the agents’ observations, rewards, termination conditions, and evaluation metrics.

  • Scenario importing: MetaDrive imports HD maps as lane-based road structures and uses connectivity information to support vehicle localization and route searching.Lane fragments provide Frenet coordinates, while a road graph supports algorithms such as BFS for routing.
  • Procedural generation: Procedural generation combines road-structure building blocks with randomized parameters and hand-crafted traffic rules to create varied simulated maps and traffic flows.Different random seeds produce distinct road shapes, while IDM controls initialized vehicles toward assigned destinations.
  • Driving task: The RL agent observes lidar-like point clouds, its vehicle state, and navigation information toward route checkpoints.The lidar representation has 240 dimensions and a 50m maximum detecting distance; checkpoints are spaced 50m apart on average.
  • Reward and cost scheme: The reward combines displacement, speed, and terminal components, with displacement providing dense feedback for forward motion.The speed reward uses current velocity relative to a maximum velocity of 80 km/h, while the terminal reward is assigned only at the final time step.
  • Evaluation: Agents are evaluated over 100 held-out environments using destination success rate and average episodic cost, with results averaged across five training seeds.Episodes terminate when the agent leaves the drivable area, reaches its destination, or exceeds the 1000-step horizon; collisions do not terminate episodes.

IX. IMPLEMENTATION DETAILS

This section records implementation settings and data-filtering assumptions for TrafficGen and PPO experiments. It also references result tables for TrafficGen and PPO.

  • Results tables: Tables V and VI report results for TrafficGen and PPO, respectively.The supplied passages identify the tables but do not provide their contents.
  • Hardware and training: The experiments use servers with 8 Nvidia 1080ti GPUs and 256 G memory, while PPO training uses 8 Nvidia A100 GPUs with concurrent trials.PPO trials use 6 CPUs and 10 parallel rollout workers, with approximately 100 G memory per trial.
  • TrafficGen settings: TrafficGen uses feature dimension 1024, three-layer MLPs with hidden dimensions [2048, 1024, 256], and a learning rate of 3e-4 for 30 epochs.The stated architecture settings apply to attribute modeling and both networks are trained with the same learning rate.
  • Data assumptions: TrafficGen filters vehicles using a maximum 5-meter distance from the nearest road center lane and a heading difference limited to ±90°.The paper presents these assumptions as prior information that helps identify valid vehicles and support the model.
Loading 2210.06609v2…