Source-linked AI summary
It Is Not the Journey but the Destination: Endpoint Conditioned Trajectory Prediction
Karttikeya Mangalam, Harshayu Girase, Shreyas Agarwal, Kuan-Hui Lee, Ehsan Adeli, Jitendra Malik, Adrien Gaidon
TL;DR
Human trajectory forecasting must handle multiple plausible futures, long-term goals, and interactions among pedestrians. PECNet infers stochastic endpoints with a VAE, conditions joint trajectory prediction on sampled endpoints, and uses non-local social pooling plus a truncation trick. It achieves state-of-the-art performance across Stanford Drone, ETH, and UCY benchmarks, with reported improvements of 20.9% and 40.8%.
Problem
Human motion prediction is difficult because pedestrians pursue latent goals, adjust around obstacles, and interact socially, while multiple future paths may remain plausible.
Method
PECNet infers stochastic pedestrian endpoints with a variational auto-encoder, then predicts socially compliant trajectories conditioned on sampled endpoints and non-local social pooling.
Results
PECNet achieves state-of-the-art performance across Stanford Drone, ETH, and UCY, improving reported performance by 20.9% on SDD and 40.8% on ETH and UCY.
Takeaways & Limitations
Endpoint conditioning produces stable, plausible intermediate goals and supports rich, diverse, socially compliant trajectories across varied scenes.
Takeaways & Limitations
Using ground-truth endpoints during training decouples endpoint and social-pooling optimization and performs worse empirically.
Abstract
from arXiv · showhide
Human trajectory forecasting with multiple socially interacting agents is of critical importance for autonomous navigation in human environments, e.g., for self-driving cars and social robots. In this work, we present Predicted Endpoint Conditioned Network (PECNet) for flexible human trajectory prediction. PECNet infers distant trajectory endpoints to assist in long-range multi-modal trajectory prediction. A novel non-local social pooling layer enables PECNet to infer diverse yet socially compliant trajectories. Additionally, we present a simple "truncation-trick" for improving few-shot multi-modal trajectory prediction performance. We show that PECNet improves state-of-the-art performance on the Stanford Drone trajectory prediction benchmark by ~20.9% and on the ETH/UCY benchmark by ~40.8%. Project homepage: https://karttikeya.github.io/publication/htf/
1 Introduction
PECNet frames human trajectory prediction as endpoint inference followed by socially compliant trajectory planning. It addresses multimodality and interaction by modeling stochastic intermediate goals and conditioning joint predictions on sampled endpoints.
- Human trajectory prediction matters for autonomous navigation, social robots, self-driving cars, and human-robot interaction.
- Pedestrian motion is challenging because people pursue goals, adjust paths around obstacles, and respond to other agents and social behavior.
- PECNet models intermediate stochastic goals, called endpoints, to represent possible local destinations and disentangle future trajectories.
- The approach first estimates and samples endpoint distributions, then predicts socially compliant trajectories conditioned on histories and all agents’ estimated endpoints.
- PECNet combines endpoint-conditioned variational inference, self-attention social pooling, and benchmark improvements of 20.9% on SDD and 40.8% on ETH and UCY.
2 Related work
Prior trajectory-prediction research progressed from statistical motion models toward interaction-aware and multimodal forecasting. Goal-conditioned methods predict intent or destinations before generating trajectories, motivating PECNet’s endpoint formulation.
- Earlier methods learned statistical behavioral patterns from observed motion trajectories for future prediction.
- Subsequent approaches incorporated scene semantics and interactions among agents using models including MDPs, Bayesian networks, and recurrent networks.
- Social modeling has used social forces and LSTM-based neighbor connections to represent pedestrian interactions.
- Multimodal forecasting methods model multiple plausible future paths from the same pedestrian history, including CVAE-based DESIRE and conditional flow VAE approaches.
- Goal-conditioned approaches predict an agent’s final intent or goal before forecasting the full trajectory, while PECNet uses destination position for pedestrian prediction.
3 Proposed Method
PECNet decomposes forecasting into endpoint inference and endpoint-conditioned trajectory prediction. It combines a VAE, non-local social pooling, and a sample-dependent truncation trick for multimodal prediction.
- Proposed Method: PECNet predicts each pedestrian’s future positions jointly from past trajectories while respecting social norms.
- Endpoint VAE: The endpoint is the pedestrian’s last observed future-sequence point, modeled as a stochastic sub-goal representing the desired destination.
- Endpoint VAE: The Endpoint VAE encodes past histories and ground-truth endpoints, samples latent variables, and decodes possible future endpoints.
- Truncation Trick: The truncation trick changes endpoint-sampling variance according to sample count, trading diversity for performance without retraining.
- Endpoint conditioned Trajectory Prediction: Sampled endpoint representations are combined with past-history encodings and passed through social pooling and a predictor to generate future paths.
- Endpoint conditioned Trajectory Prediction: Using ground-truth endpoints during training would decouple endpoint and social-pooling optimization and produced inferior empirical performance.
- Endpoint conditioned Trajectory Prediction: Non-local attention social pooling aggregates relevant neighbor information using a permutation-invariant mechanism and a spatio-temporal social mask.
- Training: Training uses KL divergence, average endpoint loss, and average trajectory loss to optimize the endpoint VAE and full prediction module jointly.
4 Experiments
Experiments evaluate PECNet on Stanford Drone and ETH/UCY benchmarks, compare it with published baselines, and examine endpoint conditioning, social pooling, truncation, sample count, and multimodality.
- Quantitative Results: PECNet achieves 20.9% better performance than previous state of the art on Stanford Drone using both ADE and FDE.Even without social pooling and truncation, the ablation achieves 10.56 ADE, supporting the contribution of endpoint conditioning.
- Quantitative Results: The social pooling module improves performance by 0.33 ADE, approximately 2.1%, compared with the corresponding ablation.Reported results are averaged over 100 trials.
- Conditioned Way-point Positions and Oracles: Conditioning on later future positions lowers ADE and FDE, with a sharper FDE decrease after the seventh future position in both proposed and oracle models.The oracle receives ground-truth conditioned positions, separating endpoint-inference error from trajectory-prediction error.
- Effect of Number of Samples: At K = 10 samples, PECNet matches the previous state-of-the-art ADE of 12.58 achieved with K = 20.The reported trend is decreasing ADE and FDE as the number of samples increases.
- Effect of Number of Samples: The truncation trick improves K = 1 performance from 22.85 ADE and 48.8 FDE to 17.29 ADE and 35.12 FDE, a gain of approximately 24.7%.The largest improvements occur at small sample counts, using an empirically chosen c ≈1.2.
5 Conclusion
PECNet combines endpoint conditioning, social pooling, and a truncation trick to produce diverse, socially compliant trajectory predictions. It achieves state-of-the-art performance across Stanford Drone, ETH, and UCY benchmarks.
- PECNet predicts rich and diverse multimodal, socially compliant trajectories across varied scenes.
- The model uses endpoint conditioning to represent intermediate goals in pedestrian trajectory prediction.
- PECNet introduces a truncation trick that adjusts trajectory-prediction diversity without retraining.
- PECNet achieves state-of-the-art performance on Stanford Drone, ETH, and UCY benchmarks.