PaperlayerView exact revision on arXiv

Source-linked AI summary

GameFormer: Game-theoretic Modeling and Learning of Transformer-based Interactive Prediction and Planning for Autonomous Driving

Zhiyu Huang, Haochen Liu, Chen Lv

arXiv:2303.05760v2cs.RO

TL;DR

Autonomous-driving systems need to predict future mutual behavior among traffic participants so the AV can make interactive decisions. GameFormer uses hierarchical level-k game theory with Transformer encoding and iterative interaction decoding, and reports state-of-the-art Waymo interaction prediction, stronger open- and closed-loop planning, and leading nuPlan performance.

  • Problem

    Existing prediction models often fail to explicitly model future interactions among agents and with the AV, despite the need for proactive coordination in critical driving situations.

  • Method

    GameFormer combines a Transformer encoder with hierarchical level-k decoders that iteratively condition current agent behavior on preceding-level predictions and shared scene context.

  • Results

    GameFormer achieves state-of-the-art Waymo interaction-prediction accuracy, outperforms baselines in open- and closed-loop planning, and delivers leading performance on nuPlan.

  • Takeaways & Limitations

    The framework jointly reasons about the AV motion plan and multiple agents’ behaviors across interaction prediction and planning evaluations.

Abstract

from arXiv · show

Autonomous vehicles operating in complex real-world environments require accurate predictions of interactive behaviors between traffic participants. This paper tackles the interaction prediction problem by formulating it with hierarchical game theory and proposing the GameFormer model for its implementation. The model incorporates a Transformer encoder, which effectively models the relationships between scene elements, alongside a novel hierarchical Transformer decoder structure. At each decoding level, the decoder utilizes the prediction outcomes from the previous level, in addition to the shared environmental context, to iteratively refine the interaction process. Moreover, we propose a learning process that regulates an agent's behavior at the current level to respond to other agents' behaviors from the preceding level. Through comprehensive experiments on large-scale real-world driving datasets, we demonstrate the state-of-the-art accuracy of our model on the Waymo interaction prediction task. Additionally, we validate the model's capacity to jointly reason about the motion plan of the ego agent and the behaviors of multiple agents in both open-loop and closed-loop planning tests, outperforming various baseline methods. Furthermore, we evaluate the efficacy of our model on the nuPlan planning benchmark, where it achieves leading performance.

1. Introduction

Existing motion-prediction models often do not explicitly model future mutual interactions between agents and the AV, limiting proactive coordination. GameFormer addresses this with hierarchical game-theoretic reasoning and reports strong prediction and planning performance.

  • Motivation: Existing models often represent interactions from past trajectories but fail to explicitly model future interactions with the AV.This can leave planning modules reacting passively to predictions.
  • Motivation: Conditional prediction incorporates AV plans but still models only one-way influence, neglecting dynamic mutual effects between the AV and other road users.The paper frames these approaches as leader-follower games with limited interaction levels.
  • Approach: GameFormer applies level-k game theory, where level-0 agents act independently and higher-level agents respond to lower-level agents’ predicted actions.The hierarchy iteratively represents increasing reasoning depths among interacting agents.
  • Contributions: The framework uses hierarchical decoding to capture interactions, iteratively refine predictions, and train agent behavior against preceding-level predictions.Its contributions include interactive prediction and planning, Waymo interaction-prediction evaluation, and open- and closed-loop planning tests.
  • Contributions: Experiments report state-of-the-art interaction prediction on the Waymo benchmark and leading planning performance on nuPlan.The paper also evaluates planning in open-loop driving scenes and closed-loop simulations.

2. Related Work

Prior work progresses from recurrent, convolutional, and graph-based scene modeling toward Transformer architectures and conditional prediction. GameFormer extends this line by combining Transformer scene encoding with hierarchical interaction decoding.

  • Motion Prediction: Earlier motion-prediction systems used LSTMs for agent histories, CNNs for rasterized scenes, and GNNs for agent interactions.These approaches represented temporal, spatial, and relational information with separate neural architectures.
  • Motion Prediction: Transformer encoder-decoder models became popular because of compact descriptions and strong performance in motion prediction.They are suited to heterogeneous driving-scene information and relationships among scene elements.
  • Prediction for Planning: Joint multi-agent prediction frameworks provide efficient and consistent multimodal trajectories but often ignore how AV actions influence other agents.This limits their suitability for downstream planning.
  • Prediction for Planning: Conditional prediction models address AV influence by incorporating planning information into forecasts of other agents.The supplied overview places GameFormer’s decoder in this broader prediction-and-planning context.
  • Decision-making: Imitation learning commonly trains driving policies offline from large-scale driving datasets, while offline reinforcement learning has also attracted interest.These methods target decision-making policies for planning modules.

3. GameFormer

GameFormer formulates multi-agent prediction and AV planning as hierarchical level-k reasoning over multimodal trajectories. A Transformer encoder builds shared scene context, while decoders iteratively condition each level on preceding-level agent futures and interaction-aware learning.

  • Problem Formulation: The task considers an AV and neighboring agents with historical states and map information, jointly predicting multimodal futures for the AV and other agents.The AV’s predicted future is treated as a planned trajectory, and multimodality represents uncertainty.
  • Game-theoretic Formulation: Level-k policies are optimized against other agents’ level-(k−1) policies, while level-0 policies act independently without considering other agents’ likely actions.Here, each policy is a multimodal predicted trajectory modeled as a Gaussian mixture, and the loss is computed at trajectory level.
  • Scene Encoding: Historical agent states and vectorized map polylines are encoded into a shared scene representation using LSTMs, an MLP with pooling, and a Transformer encoder.The resulting scene context captures relationships among scene elements for subsequent decoding.
  • Future Decoding with Level-k Reasoning: The level-0 decoder combines learnable modality queries with agent-history features and scene context to produce multimodal trajectory components and scores.The modality embedding initializes possible futures, while MLP heads decode Gaussian-mixture components and their scores.
  • Future Decoding with Level-k Reasoning: At each level k≥1, the decoder encodes preceding-level trajectories, pools modalities using their scores, models agent-future interactions with self-attention, and appends them to scene context.Cross-attention then uses the updated context to decode current-level trajectories, with masking preventing an agent from accessing its own preceding-level future.
  • Learning Process: The interaction loss applies a repulsive term only to interaction pairs whose future-state distances fall below a safety threshold.The distance is the L2 distance between future (x,y) positions across agent modes and reasoning levels.

4. Experiments

Experiments evaluate GameFormer for interaction prediction and planning across WOMD and nuPlan settings. Results cover joint and marginal prediction, open- and closed-loop planning, qualitative behavior, and ablations of reasoning depth, future modeling, and decoder structure.

  • Experimental setup: The prediction-oriented model jointly predicts two interacting agents 8 seconds into the future on WOMD, while the planning variant predicts multiple neighboring agents around the AV on WOMD and nuPlan.The prediction model uses 20 neighboring agents as background and generates trajectories only for the two labeled agents.
  • Interaction prediction: GameFormer (J, M=6) outperforms existing methods on WOMD interaction prediction in position errors averaged across object types and evaluation times.The reported averages cover vehicles, pedestrians, and cyclists at 3, 5, and 8 seconds.
  • Interaction prediction: The marginal model with EM aggregation significantly improves mAP, while the joint model offers better prediction accuracy and computational efficiency for planning tests.The marginal model is comparable overall to the ensemble and MTR, but marginal ensembles impose substantial computational burden.
  • Open-loop planning: Decoding level 4 achieves the lowest miss rate and collision rate in the open-loop planning analysis, despite only slight ADE decreases with additional layers.The optimal iteration depth is reported empirically for this test.
  • Open-loop planning: GameFormer outperforms vanilla IL, DIM, MultiPath++, MTR-e2e, and DIPP on reported open-loop planning comparisons, especially collision rate against DIPP.The authors attribute the advantage to explicitly considering all agents’ future behaviors and iteratively refining interactions.
  • Open-loop planning: Qualitative results show plausible AV trajectories, diverse interaction handling, and more interpretable planning through predictions of surrounding agents.Interaction examples include yielding, accelerating for precedence, creeping before a turn, and stopping for a pedestrian.
  • Closed-loop planning: Closed-loop WOMD tests execute the planned trajectory while other agents follow logged trajectories, making the resulting success rate a lower bound for safety assessment.Planning-based methods are evaluated over three training runs, with means and standard deviations reported.
  • Ablation study: Ablations show that previous-level agent futures, self-attention over those futures, interaction loss, independent decoder layers, and intermediate trajectories each support planning performance.Removing interaction loss significantly increases collision rate, while simplifying decoder sharing or omitting intermediate outputs degrades performance.

5. Conclusions

GameFormer uses hierarchical game theory and level-k interaction decoders to iteratively refine predicted agent trajectories. It achieves state-of-the-art interaction prediction and leading planning performance across Waymo and nuPlan evaluations.

  • 5. Conclusions: GameFormer applies hierarchical game theory through level-k interaction decoders that iteratively refine interacting agents’ future trajectories.A learning process regulates current-level predicted behaviors using results from the previous level.
  • 5. Conclusions: GameFormer achieves state-of-the-art accuracy in interaction prediction on the Waymo open motion dataset.
  • 5. Conclusions: GameFormer outperforms baseline methods in both open-loop and closed-loop planning tests.
  • 5. Conclusions: GameFormer delivers leading performance on the nuPlan planning benchmark.

A. Experiment Details

The experiments define separate prediction and planning inputs, architectures, training procedures, and evaluation protocols. Prediction models use agent histories and local map context, while planning evaluates joint AV prediction and trajectory quality in open-loop scenarios.

  • A.1. Prediction-oriented Model: Prediction scenes use one focal agent, 10 nearest surrounding agents per target, and one second of historical states sampled at 10Hz.Two target agents are predicted, with up to 20 distinct background agents in a scene.
  • A.1. Prediction-oriented Model: Local map inputs include up to 6 drivable lanes, 4 nearby crosswalks, and waypoint features describing geometry, speed limits, and traffic controls.
  • A.1. Prediction-oriented Model: The encoder combines separate LSTM histories for target and background agents with encoded local-map context to form scene representations.Lane and crosswalk features are pooled and concatenated into the local map tensor.
  • A.1. Prediction-oriented Model: The level-0 decoder uses target history and modality embeddings as queries, while encoded scene context supplies keys and values for multimodal trajectory prediction.Joint prediction uses 6 modalities, whereas marginal prediction uses 64 fixed intention-point modalities.
  • A.1. Prediction-oriented Model: Training predicts six joint futures for two agents and applies imitation loss at each level, selecting joint trajectories closest to ground truth for backpropagation.
  • A.2. Planning-oriented Model: Planning uses the AV and 10 surrounding agents, two seconds of history, local map features, and a 100-meter, 1000-waypoint reference route.Open-loop testing checks collisions against other agents’ logged futures and counts endpoint misses beyond 4.5 meters.

A.3. Baseline Methods

The evaluation compares GameFormer with imitation, probabilistic, offline reinforcement-learning, prediction, and differentiable planning baselines. The refinement planner converts trajectories to controls and optimizes weighted costs that include safety and reference-route adherence.

  • A.3. Baseline Methods: Vanilla Imitation Learning directly outputs a single-modal AV trajectory without explicitly reasoning about other agents’ future trajectories.
  • A.3. Baseline Methods: DIM generates expert-like future trajectories probabilistically and uses a gradient-based planner guided by a specified goal and goal likelihood.
  • A.3. Baseline Methods: RIP extends DIM with epistemic uncertainty awareness by selecting the worst-likelihood model among 6 trained DIM models for robust planning.
  • A.3. Baseline Methods: CQL learns offline decisions from the same state inputs as DIM and outputs the next-step target pose, with reward based on distance traveled and goal reaching.Because the dataset contains perfect driving data, the reward excludes a collision penalty.
  • A.3. Baseline Methods: MultiPath++ and MTR-e2e provide prediction-based comparisons using six possible trajectories, with MTR-e2e modified for end-to-end open-loop planning.
  • A.4. Refinement Planner: The refinement planner converts planned trajectories into control actions and solves a weighted nonlinear least-squares objective over control-cost residuals.Costs include speed, control effort, comfort, route alignment, traffic-light violations, and collision avoidance using predicted states.
  • A.4. Refinement Planner: The nuPlan framework processes observations and maps, selects a route path, and uses the prediction and planning components for realistic driving scenarios.

B.1. Interaction Prediction

On the WOMD interaction prediction benchmark, GameFormer’s joint model performs strongly across object categories. It has the lowest minFDE across categories, surpasses MTR for cyclists, and is comparable to MTR elsewhere with a simpler structure.

  • B.1. Interaction Prediction: GameFormer’s joint prediction model achieves the lowest minFDE across all object categories in Table S1.
  • B.1. Interaction Prediction: GameFormer surpasses MTR in the cyclist category and achieves comparable performance in other categories.The comparison is reported for the WOMD interaction prediction benchmark.
  • B.1. Interaction Prediction: The reported interaction-prediction performance is attributed to GameFormer’s joint training of interaction patterns despite its simpler structure than MTR.

B.3. Abalation Study

Closed-loop planning success plateaus at decoding level 2, while inference time continues increasing, indicating a performance–efficiency trade-off.

  • At decoding level 2, closed-loop planning success reaches a plateau while computation time continues to increase.Success rate without collision is the main metric, and inference time is measured without the refine motion planner.
  • Using two reasoning levels may provide a favorable balance between closed-loop planning performance and efficiency.

C. Additional Qualitative Results

Additional qualitative results show GameFormer handling complex interaction scenarios, refining joint trajectories across reasoning levels, and jointly planning the AV with neighboring-agent predictions.

  • C.1. Interaction Prediction: GameFormer handles varied interaction pairs and complex urban driving scenarios in additional interaction-prediction examples.Figure S1 marks interacting agents in red and background neighboring agents in magenta, with six joint trajectories predicted for the interacting pair.
  • C.2. Level-k Prediction: Iterative reasoning refines initially independent target-agent predictions into consistent and human-like higher-level trajectories.At level 0, predicted trajectories may collide; higher interaction levels produce more consistent joint trajectories.
  • C.3. Open-loop Planning: Open-loop results demonstrate joint planning of the AV trajectory and prediction of neighboring-agent behaviors.Figure S3 uses a red box and trajectory for the AV and magenta boxes with blue predicted trajectories for neighboring agents.
  • C.4. Closed-loop Planning: Videos visualize closed-loop performance in interactive urban driving scenarios from both WOMD and nuPlan.
Loading 2303.05760v2…