PaperlayerView exact revision on arXiv

Source-linked AI summary

TrafficSim: Learning to Simulate Realistic Multi-Agent Behaviors

Simon Suo, Sebastian Regalado, Sergio Casas, Raquel Urtasun

arXiv:2101.06557v1cs.ROcs.AIcs.CVcs.LG

TL;DR

Self-driving evaluation needs scalable simulation with realistic, interactive traffic behavior, but existing approaches either encode rigid rules or fail under long-horizon distribution shift. TRAFFICSIM learns a joint latent-variable policy from demonstrations, trains it through differentiable closed-loop simulation with common-sense losses, and produces realistic diverse scenarios that also improve motion-planner training.

  • Problem

    Existing traffic simulators use heuristic rules that miss irregular maneuvers and complex interactions, while learned models can be brittle under distributional shift during long-horizon simulation.

  • Method

    TRAFFICSIM uses an implicit latent-variable joint actor policy, unrolls it through fully differentiable simulation, and combines imitation with time-adaptive common-sense objectives.

  • Results

    TRAFFICSIM achieves the lowest scenario reconstruction error among diverse baselines while producing realistic long-horizon scenarios with minimal collisions and traffic-rule violations.

  • Takeaways & Limitations

    TRAFFICSIM-generated trajectories provide effective data augmentation for training motion planners, improving open-loop planning metrics against real scenarios.

  • Takeaways & Limitations

    Motion-forecasting-style learned behavior models remain brittle to distributional shift and compounding error when used directly for long-horizon simulation.

Abstract

from arXiv · show

Simulation has the potential to massively scale evaluation of self-driving systems enabling rapid development as well as safe deployment. To close the gap between simulation and the real world, we need to simulate realistic multi-agent behaviors. Existing simulation environments rely on heuristic-based models that directly encode traffic rules, which cannot capture irregular maneuvers (e.g., nudging, U-turns) and complex interactions (e.g., yielding, merging). In contrast, we leverage real-world data to learn directly from human demonstration and thus capture a more diverse set of actor behaviors. To this end, we propose TrafficSim, a multi-agent behavior model for realistic traffic simulation. In particular, we leverage an implicit latent variable model to parameterize a joint actor policy that generates socially-consistent plans for all actors in the scene jointly. To learn a robust policy amenable for long horizon simulation, we unroll the policy in training and optimize through the fully differentiable simulation across time. Our learning objective incorporates both human demonstrations as well as common sense. We show TrafficSim generates significantly more realistic and diverse traffic scenarios as compared to a diverse set of baselines. Notably, we can exploit trajectories generated by TrafficSim as effective data augmentation for training better motion planner.

1. Introduction

Simulation offers scalable, safe, controllable evaluation for self-driving systems, but realistic multi-agent behavior remains difficult. TRAFFICSIM addresses this gap by learning socially consistent behaviors from human driving data while incorporating common sense for long-horizon simulation.

  • Motivation: Simulation can evaluate self-driving systems across diverse, dynamic scenarios more safely, controllably, and cost-efficiently than physical testing or immutable replay.Physical testing is expensive and limited, while replay cannot react to divergent SDV plans.
  • Problem: Automatically generating realistic multi-agent behavior is the central simulation challenge addressed in this work.The paper focuses on simulating dynamic agents rather than scene layout or rendering.
  • Problem: Heuristic traffic models encode rules effectively but lack the diversity and nuance of human behavior, including irregular maneuvers and complex interactions.Examples include U-turns, nudging past stopped vehicles, and negotiations at unprotected left turns.
  • Problem: Learning-based approaches capture more diverse behaviors but often lack common sense, are brittle under distributional shift, and can be computationally expensive.These limitations complicate their use for robust traffic simulation.
  • Approach: TRAFFICSIM learns a joint multi-agent policy from real-world demonstrations, unrolls it through differentiable simulation, and balances imitation with common sense.The resulting model is reported to remain realistic over long horizons with minimal collisions and traffic-rule violations.

2. Related Work

Prior traffic simulators rely mainly on rigid heuristics, while motion-forecasting and imitation-learning models offer richer behavior but struggle when deployed in long-horizon closed-loop simulation. TRAFFICSIM combines learned interaction modeling with simulation-oriented robustness.

  • Simulation Environments: Microscopic traffic simulators encode rules such as staying in lane and avoiding collisions, but remain unrealistic at street level under rigid assumptions.They cannot reproduce irregular maneuvers or complex interactions such as nudging, U-turns, yielding, and merging.
  • Motion Forecasting: Motion-forecasting models capture increasingly complex behavior and interaction, making them relevant to virtual traffic simulation.Their usefulness follows from modeling actor behavior beyond simple pairwise dynamics.
  • TRAFFICSIM: TRAFFICSIM targets the gap between rule-based realism and learned behavior by modeling multi-agent interactions for traffic simulation.Its approach is positioned against heuristic, forecasting, and imitation-learning baselines.
  • Motion Forecasting: Motion-forecasting models are brittle to distributional shift and compounding error, preventing direct long-horizon simulation.The limitation arises when predicted states diverge from the training distribution during rollout.
  • Imitation Learning: Imitation learning addresses train-test mismatch through closed-loop optimization, while back-propagation through simulation enables direct optimization over rollout behavior.The cited discussion connects this strategy to compounding-error mitigation.

3. Learning Multi-Agent Traffic Behaviors

TRAFFICSIM builds a differentiable, joint policy that observes map and motion context, samples socially consistent multi-agent plans, and simulates them sequentially. Training unrolls the policy and combines demonstration learning with common-sense objectives over time.

  • Scene Context: The model represents each timestep as a collection of actor bounding-box states and extracts map, traffic-control, and motion context for policy input.Actor states include position, dimensions, and heading; the observation pipeline combines map features with encoded past trajectories.
  • Joint Policy: A joint actor policy models future states for all actors together, enabling parallel samples of socially consistent plans over a planning horizon.The policy explicitly represents uncertainty through an implicit latent variable model.
  • Joint Policy: The graph-based interaction module propagates messages among actors so the latent representation captures individual goals, style, and multi-agent interactions.The same interaction design parameterizes prior, posterior, and decoder networks.
  • Simulation: Traffic scenarios are generated sequentially by extracting context, sampling a joint plan at each timestep, and advancing the simulation with one or multiple planned steps.Using multiple steps can accelerate simulation with minimal loss in quality.
  • Learning: Training unrolls the policy in closed loop and back-propagates losses through the fully differentiable simulation across time.This directly addresses compounding error between open-loop training and closed-loop execution.
  • Learning: A time-adaptive multi-task objective combines imitation from demonstrations with common-sense constraints such as collision avoidance.Common-sense supervision is increasingly favored over imitation across the simulation horizon, with collision loss applied to prior samples.

4. Experimental Evaluation

The evaluation measures realism, interaction consistency, rule compliance, reconstruction, diversity, and simulation efficiency on ATG4D. TRAFFICSIM produces more realistic and diverse scenarios than baselines, supports effective planner data augmentation, and enables faster simulation with limited quality loss.

  • Evaluation Setup: The evaluation uses multiple metrics because no single metric fully captures generated-traffic quality.The metrics target diversity, realism, and coverage of real-world scenarios.
  • Evaluation Setup: Scenario collision rate measures the percentage of actors colliding in each sampled scenario, with lower values preferred.Actors count as colliding when their bounding boxes overlap beyond a small IOU threshold at any timestep.
  • Evaluation Setup: Traffic rule violation rate measures whether actors stay within drivable areas and obey traffic-light signals.The reported ground-truth violation rate is 1.26% because human drivers sometimes behave non-compliantly.
  • Evaluation Setup: Scenario reconstruction evaluates closeness to ground truth using minSADE/minSFDE for the best sample and meanSADE/meanSFDE across samples.The metric is intended to capture irregular maneuvers and complex interactions present in real traffic.
  • Experimental Results: TRAFFICSIM achieves the best scenario reconstruction and interaction results while matching IDM-like traffic-rule violation performance without directly encoding rules.Heuristic models follow rules but miss naturalistic behavior, whereas forecasting and imitation baselines show unrealistic interactions or worse violations when unrolled.
  • Experimental Results: Synthetic TRAFFICSIM data significantly improves open-loop planner metrics, especially planning L2 error, while matching collision rate and progress against equal-sized real-data training.This supports using generated scenarios as data augmentation with a small observed behavior gap.
  • Ablation Study: Closed-loop training with back-propagation through simulation is the most important component for robust policy learning, while longer-horizon plans improve interaction reasoning.Adding common sense further reduces collision and traffic-rule violation rates.
  • Efficiency: Multi-step updates provide a 4x simulation speedup with minimal quality degradation when the update interval does not exceed the modeled planning horizon.The speed-quality tradeoff can be adjusted at simulation time without retraining.

5. Conclusion

TRAFFICSIM generates diverse, realistic traffic by jointly producing socially consistent plans for all actors through differentiable simulation and real-world imitation with common sense. The authors identify applications in data augmentation, interactive scenario design, and safety evaluation.

  • TRAFFICSIM generates diverse and realistic traffic scenarios with socially consistent plans for all actors jointly.The model is trained by back-propagating through a fully differentiable simulation while imitating real-world self-driving trajectories and incorporating common sense.
  • The method uses real-world trajectory observations and common sense within fully differentiable simulation training.
  • TRAFFICSIM enables data augmentation, interactive scenario design, and safety evaluation.

Appendix

The supplementary material adds methodological, implementation, baseline, metric, and qualitative-result details. Its contents are organized across three sections.

  • Section A provides additional details of the TRAFFICSIM method.
  • Section B provides implementation details for baselines and metrics.
  • Section C provides additional qualitative results.

A. Additional TRAFFICSIM Details

The appendix describes TRAFFICSIM’s inputs, differentiable modules, joint interaction policy, simulation procedure, and time-adaptive multi-task training objective. These components combine map and motion context with scene-level interaction and collision-aware supervision.

  • Input Parameterization: TRAFFICSIM uses a rasterized map with 13 channels and initializes scenarios with 3 seconds of past actor states.Each actor history contains seven bounding boxes sampled 0.5 seconds apart.
  • Global Map Module: The global map module extracts near- and long-range topology using a four-block multi-scale convolutional backbone.Blocks use convolutional layers with [8, 16, 32, 64] channels and max pooling between resolutions.
  • Local Observation Module: The local observation module is lightweight and differentiable, producing a 192-dimensional actor context from map and motion features.Local features are processed by a three-layer CNN and max-pooled across spatial dimensions.
  • Scene Interaction Module: A graph neural network scene interaction module parameterizes the joint actor policy across the Prior, Posterior, and Decoder networks.The interaction computation uses graph message passing over actor nodes and edges.
  • Simulating Traffic Scenarios: Traffic scenarios are generated sequentially by extracting scene context, sampling joint actor plans, and simulating actors forward from an initial history.Multiple scenarios and arbitrary actor counts can be processed in parallel by batching samples and actors.
  • Time-Adaptive Multi-Task Loss: The time-adaptive multi-task loss balances imitation and common-sense supervision while decreasing imitation weight over simulation time.The imitation weight reaches zero at the label horizon, when labels are no longer available; collision loss is reweighted by 0.01.

B. Additional Experiment Details

This section provides additional details about the experimental setup, including baselines, metrics, and evaluation configuration.

  • The section describes additional baselines used for comparison.
  • The section describes additional evaluation metrics.
  • The section describes the experimental setup.

B.1. Baselines

The baselines include heuristic and learned behavior models, while TRAFFICSIM simulates actor states iteratively from map and initial-state inputs. The comparison also includes data augmentation and adversarial imitation-learning setups.

  • Heuristic baseline: IDM is a heuristic car-following baseline that models reactive keep-lane behavior by following a designated headway vehicle.Its actors are parameterized against lane centerlines and use specified visibility, reaction, acceleration, and desired-speed settings.
  • TRAFFICSIM: TRAFFICSIM takes a rasterized high-definition map and initial actor states as input and outputs simulated actor states over T timesteps.The simulation loop extracts actor-specific local context at each timestep and updates the environment using decoded actor plans.
  • TRAFFICSIM: The TRAFFICSIM algorithm caches global map features, extracts local actor context, samples a scene latent, and decodes actor plans through the simulation module.The environment is advanced by taking the first κ steps of the generated plans before continuing the rollout.
  • Learned baselines: MTP predicts mixtures of Gaussian trajectory modes but assumes conditional independence across actors and does not explicitly reason about interactions during future unrolling.The implementation uses 16 Gaussian modes and selects the closest matching mode for loss computation.
  • Learned baselines: ESP models multi-agent interaction autoregressively, while ILVM is adapted by replacing voxelized LiDAR input with rasterized actor bounding boxes.The ILVM adaptation reduces backbone capacity to improve inference speed without observed performance degradation.
  • Additional baselines: The evaluation also includes data augmentation and adversarial imitation-learning baselines configured to factor out model-architecture effects.The adversarial baseline uses differentiable observation and interaction modules, backpropagation through simulation, and periodic behavior-cloning loss for stability.

B.2. Metrics

The experiments evaluate scenario reconstruction, interaction safety, traffic-rule compliance, and diversity. Reconstruction is limited to 8 seconds because ground-truth trajectories are available only over that horizon.

  • Scenario Reconstruction: Scenario reconstruction metrics are computed over K traffic scenarios sampled for each simulation environment.An environment is defined by its map, traffic control, and initial actor states.
  • Scenario Reconstruction: Scenario reconstruction is evaluated only through Tlabel = 8s because ground-truth actor trajectories are available only over that duration.This limits the comparison to the portion of each rollout with corresponding labeled trajectories.
  • Interaction Reasoning: The scenario-level collision-rate metric identifies interactions between actors using bounding-box overlap.Two actors count as colliding when their bounding-box IoU exceeds 0.1, with at most one collision counted per actor.
  • Traffic Rule Compliance: Traffic-rule compliance uses high-definition lane-graph annotations to derive reachable drivable lane segments under traffic-control constraints.Connections influenced by traffic control, such as a red traffic light, are removed during reachability analysis.
  • Diversity: The map-aware diversity metric filters rule-violating samples and measures the average temporal distance between each actor’s two most distinct trajectories.It reuses the drivable-area raster from the traffic-rule compliance evaluation.

B.3. Experimental Setup

The experimental setup uses synthetic traffic scenarios to train an imitation planner and applies bounded latent-space optimization for collision-aware simulation-time constraints.

  • TRAFFICSIM for Data Augmentation: TRAFFICSIM generates approximately 15k synthetic examples from initializations drawn from training scenarios, using the same amount of real data for comparison.The imitation planner predicts a 5s horizon represented by 10 waypoints from rasterized maps and actor bounding-box history.
  • TRAFFICSIM for Data Augmentation: The imitation planner is trained with supervision from all actor trajectories in the synthetic scenarios.Its inputs are rasterized map and actor bounding-box history, and it directly regresses the future plan.
  • Incorporating Constraints at Simulation-Time: At simulation time, the method resamples up to 10 times for collision-free plans and otherwise returns generated samples sorted by collision loss.This bounds runtime while selecting minimum-cost plans when enough collision-free samples are unavailable.
  • Incorporating Constraints at Simulation-Time: The model remains frozen while gradients optimize latent samples using a collision loss evaluated on the first 5 timesteps.Latent-space optimization influences actor plans while remaining within the model distribution and uses five steps at learning rate 1e-2.

C. Additional Qualitative Results

Additional qualitative results show traffic scenarios sampled from TRAFFICSIM and a six-second comparison between baseline simulations and the proposed model, with collisions and rule violations highlighted.

  • TRAFFICSIM samples: Figures 10 and 11 show additional traffic scenarios sampled from TRAFFICSIM, with colored triangles indicating heading and tracking instances over time.The visualizations present sampled multi-agent trajectories as evolving traffic scenes.
  • Baseline comparison: Figure 12 compares baseline-generated traffic scenarios with TRAFFICSIM using snapshots taken 6s after simulation begins.Red circles mark collisions and traffic-rule violations in the comparison.
Loading 2101.06557v1…