Source-linked AI summary
Goal-GAN: Multimodal Trajectory Prediction Based on Goal Position Estimation
Patrick Dendorfer, Aljoša Ošep, Laura Leal-Taixé
TL;DR
Goal-GAN addresses the challenge of predicting pedestrian trajectories when multiple destinations and physically feasible paths are possible. It estimates a multimodal goal distribution from past motion and scene context, then routes trajectories toward sampled goals. The method achieves state-of-the-art results on ETH, UCY, and SDD while generating multimodal, diverse, and feasible trajectories.
Problem
Pedestrian trajectory prediction must represent multiple plausible modes and diverse paths while respecting physical scene constraints.
Method
Goal-GAN uses a two-stage architecture that estimates goal positions from motion and visual context, then routes trajectories toward sampled goals with differentiable stochastic training.
Results
Goal-GAN achieves state-of-the-art results on ETH, UCY, and SDD while generating multimodal, diverse, and feasible trajectories.
Takeaways & Limitations
Explicit goal estimation makes trajectory generation interpretable while supporting realistic paths across alternative scene modes.
Abstract
from arXiv · showhide
In this paper, we present Goal-GAN, an interpretable and end-to-end trainable model for human trajectory prediction. Inspired by human navigation, we model the task of trajectory prediction as an intuitive two-stage process: (i) goal estimation, which predicts the most likely target positions of the agent, followed by a (ii) routing module which estimates a set of plausible trajectories that route towards the estimated goal. We leverage information about the past trajectory and visual context of the scene to estimate a multi-modal probability distribution over the possible goal positions, which is used to sample a potential goal during the inference. The routing is governed by a recurrent neural network that reacts to physical constraints in the nearby surroundings and generates feasible paths that route towards the sampled goal. Our extensive experimental evaluation shows that our method establishes a new state-of-the-art on several benchmarks while being able to generate a realistic and diverse set of trajectories that conform to physical constraints.
1 Introduction
Human trajectory prediction must capture multiple plausible destinations while producing diverse paths that remain feasible in the scene. Goal-GAN addresses this with explicit goal estimation followed by routing toward sampled goals.
- Pedestrian futures are inherently ambiguous because destinations and directions cannot be inferred uniquely from past observations.
- Effective predictors should cover distinct scene modes and preserve diversity within each mode, including variation in walking style, velocity, and obstacle avoidance.
- Generative methods can produce diverse outputs but may neglect environmental structure, yielding infeasible trajectories and incomplete directional coverage.
- Goal-GAN separates prediction into posterior goal estimation from motion and visual context, then routes trajectories toward sampled goals while reacting to local obstacles.
- Goal-GAN combines an interpretable goal-position distribution, differentiable stochastic sampling, and adversarial training to generate realistic multimodal trajectories.
2 Related Work
Prior trajectory-prediction work uses interaction modeling and generative sampling to represent one-to-many futures, but diverse predictions may not respect scene structure or separate feasible destinations clearly.
- Trajectory predictors model human-human interactions, human-space interactions, or both, alongside earlier physics-based and data-driven approaches.
- Scene-aware methods use bird-view images or occupancy grids to predict trajectories that respect structural constraints.
- Generative approaches improve diversity through sampling or latent-noise training, yet high-variance outputs may remain unrealistic and fail to distinguish feasible destinations.
3 Problem Definition
The task predicts multiple future pedestrian paths from past motion and a bird-view scene, despite observing only one realized future trajectory during training. Outputs should cover feasible modes while respecting scene constraints.
- The model predicts future pedestrian positions in the 2D ground plane from past trajectories and bird-view visual information.
- Because datasets provide only one observed future path, the task requires generating multiple plausible future samples for the same past trajectory.
- Generated samples should cover all feasible modes and remain compliant with the physical constraints of the scene.
- The architecture frames prediction around sampled goals and routes future trajectories toward those goals.
4 Goal-GAN
Goal-GAN uses motion and scene information to estimate a distribution over pedestrian goals, samples a goal differentiably, and routes trajectories toward it while attending to local surroundings. Adversarial and distance-based losses train outputs for realism, feasibility, and goal consistency.
- Goal-GAN contains a Motion Encoder, Goal Module, and Routing Module that respectively encode dynamics, estimate goals, and generate goal-directed paths.
- Motion Encoder: The Motion Encoder embeds relative displacements and uses an LSTM to capture trajectory speed and direction for downstream prediction.
- Goal Module (GM): The Goal Module combines scene features with motion features to output a multimodal goal distribution representing alternative destination directions.
- Goal Module (GM): Gumbel-Softmax samples discrete goal locations while preserving backpropagation through the stochastic sampling process.
- Routing Module (RM): The Routing Module conditions recurrent prediction on dynamics and the sampled goal, using local visual attention to react to obstacles and nearby structures.
- Training: GAN training distinguishes feasible from infeasible trajectories, while best-of-many distance, adversarial, goal-achievement, and cross-entropy losses supervise trajectory and goal predictions.
5 Experimental Evaluation
Goal-GAN is evaluated on standard pedestrian benchmarks and a synthetic dataset designed to test multimodality and feasibility. It achieves strong benchmark performance while producing diverse, goal-directed trajectories that better respect scene constraints.
- 5.1 Benchmark Results: Nearly 15% lower overall error than S-BiGAT and CGNS, with average FDE reduced by 0.12m versus the current state-of-the-art on ETH and UCY.The authors attribute this improvement to covering plausible modes through the Goal Module.
- 5.1 Benchmark Results: Goal-GAN achieves state-of-the-art ADE and FDE on SDD, reducing error by 25% versus SoPhie and exceeding other goal-conditioned methods.It is reported as being on par with P2TIRL, which was unpublished when the work was prepared.
- 5.2 Assessing Multimodality of Predictions on Synthetic Dataset: On the synthetic dataset, adding the goal module increases mode coverage by 7.36 and feasibility by 10.26 over the visual-attention vanilla GAN baseline.The synthetic setup controls multimodality and feasibility, enabling these dimensions to be evaluated directly.
- 5.2 Assessing Multimodality of Predictions on Synthetic Dataset: Adding L2 loss reduces ADE by 1.47 and increases feasibility by 8.26, while omitting the Gumbel-Softmax Trick causes a drastic performance drop.The results support the role of L2 loss for accuracy and GST for stable back-propagation through stochastic goal sampling.
- 5.3 Qualitative Evaluation: Goal-GAN generates diverse trajectories routed toward estimated modes, whereas the vanilla GAN spreads trajectories broadly and produces infeasible paths.The goal probability map provides an interpretable visualization of the estimated modes.
- 5.3 Qualitative Evaluation: On real scenes, the model assigns probability to plausible open areas and suppresses goals on streets, parked cars, trees, and other infeasible regions.Generated paths also vary in direction and speed when the scene permits multiple plausible outcomes.
6 Conclusion
Goal-GAN is a two-stage network for pedestrian trajectory prediction that produces multimodal, diverse, and feasible trajectories. It achieves state-of-the-art results on the ETH, UCY, and SDD datasets.
- Goal-GAN achieves state-of-the-art results on the ETH, UCY, and SDD datasets while generating multimodal, diverse, and feasible trajectories.