Source-linked AI summary
Social Ways: Learning Multi-Modal Distributions of Pedestrian Trajectories with GANs
Javad Amirian, Jean-Bernard Hayet, Julien Pettre
TL;DR
Pedestrian trajectory predictors must represent uncertainty in neighbors’ future motion and intentions, including multimodal outcomes. This paper uses a GAN-based trajectory sampler with InfoGAN and attention pooling informed by hand-designed interaction features, improving accuracy partly on high-variance datasets and preserving multimodality best on a dedicated benchmark.
Problem
Recent neural motion-prediction systems do not explicitly model the uncertain and multimodal nature of neighbors’ future trajectories.
Method
The approach uses an InfoGAN-based trajectory sampler and attention pooling that combines learned interaction assessment with hand-designed geometric features.
Results
The approach partly improves prediction accuracy on datasets with larger predictive-distribution variances and achieves the best multimodality preservation among evaluated GAN variants.
Takeaways & Limitations
The method provides plausible trajectory samples that retain predictive-distribution uncertainty and multimodality for higher-level decision-making.
Takeaways & Limitations
The formulation assumes access to consecutive observed samples and includes an evaluation setting where interactions between agents are not considered.
Abstract
from arXiv · showhide
This paper proposes a novel approach for predicting the motion of pedestrians interacting with others. It uses a Generative Adversarial Network (GAN) to sample plausible predictions for any agent in the scene. As GANs are very susceptible to mode collapsing and dropping, we show that the recently proposed Info-GAN allows dramatic improvements in multi-modal pedestrian trajectory prediction to avoid these issues. We also left out L2-loss in training the generator, unlike some previous works, because it causes serious mode collapsing though faster convergence. We show through experiments on real and synthetic data that the proposed method leads to generate more diverse samples and to preserve the modes of the predictive distribution. In particular, to prove this claim, we have designed a toy example dataset of trajectories that can be used to assess the performance of different methods in preserving the predictive distribution modes.
1. Introduction
Pedestrian trajectory prediction supports time-critical applications but must handle complex, interacting motions and multiple plausible futures. Social Ways addresses this with an InfoGAN-based sampler, learned attention pooling, and a synthetic benchmark for evaluating multimodality.
- Online pedestrian-motion prediction can support early alarms, preventive actions, and collision avoidance in applications such as autonomous driving.
- Pedestrian trajectories depend on obstacles, terrain, goals, personal factors, and interactions including avoidance, meeting, and following.
- Social Ways uses a GAN-based sampler to propose plausible future trajectories conditioned on past motion from all pedestrians, while representing uncertainty and potential multimodality.
- The method trains an InfoGAN trajectory-prediction architecture without L2 loss to better preserve the multimodal predictive distribution than previous works.
- Its attention-based pooling combines hand-designed interaction features with learned weighting, and its synthetic dataset targets evaluation of multimodality preservation.
2. Related work
Prior work spans closed-form, Bayesian, data-driven, recurrent, graph-based, and generative approaches to pedestrian motion prediction. A central challenge is modeling interactions and the inherently multimodal outcomes of pedestrian motion.
- Closed-form and Bayesian approaches use geometric optimization, simple motion models, or parameterized social-interaction assumptions, which limit flexibility across contexts.
- Data-driven models learn motion and interaction features from pedestrian or visual data, including Gaussian-process dynamical models and learned avoidance cues.
- Recurrent architectures such as Social-LSTM encode individual trajectories and aggregate neighboring hidden states to predict future positions.
- Other neural approaches represent interactions with spatio-temporal graphs, influence-weighted trajectory combinations, distance histograms, or occupancy grids.
- Generative methods address multimodality by sampling potential outcomes, because recurrent predictors may average distinct possible trajectories.
- Social-aware recurrent models incorporate training-data priors and layered Gaussian processes to account for motion variability.
- Earlier GAN-based trajectory sampling pooled a random vector with hidden representations of other pedestrians to model interactions.
3. Problem statement and system overview
The paper formulates pedestrian trajectory prediction from recent observations of all agents, then uses a conditional GAN to generate plausible future samples while modeling multimodal neighbor behavior. Its generator encodes trajectory histories, noise, and attention-weighted social information; training omits L2 loss and adds Info-GAN mutual-information optimization to preserve diversity.
- Problem formulation: The task is to predict each pedestrian’s trajectory over the next T time steps from τ + 1 observed samples of all pedestrians.The notation distinguishes the target agent from the observed trajectories of the other agents.
- Problem formulation: Recent-neighbor observations alone do not explicitly represent uncertainty and multimodality in neighbors’ future trajectories.The paper argues that learning this uncertainty implicitly may be too demanding for standard neural motion-prediction systems.
- System overview: Social Ways GAN conditions independent random trajectory samples on observed tracklets and noise to generate plausible future trajectories for a selected agent.The model is intended to capture both the agent’s whereabouts and the surrounding crowd’s impact on its trajectory.
- Generator network: The generator encodes an agent’s past trajectory with LSTM-E, then decodes samples with LSTM-D using encoded history, noise, and neighboring-agent information.The decoder input combines the target agent’s encoded history, the noise vector, and representations of neighboring trajectories.
- Attention pooling: Attention pooling weights neighboring agents using distance, bearing angle, and distance of closest approach embedded into interaction features.The weights are obtained through scalar products and softmax operations involving hidden trajectory histories and interaction embeddings.
- GAN training: Training omits the L2 term because it negatively affects sample diversity and adds Info-GAN mutual-information maximization to reduce mode collapse.Info-GAN introduces a latent code c and a Q sub-network that estimates its likelihood from generated trajectories.
4. Experimental results
Experiments evaluate prediction accuracy and predictive-distribution quality on real and synthetic trajectories. S-Ways improves errors in higher-variance scenes and Info-GAN best preserves multimodality in the toy benchmark.
- Experimental setup: The system is implemented with fully connected and LSTM components, using observed and predicted trajectories to generate and discriminate future motion.The Generator uses observed positions and velocities, latent codes, noise, and pooled vectors; the Discriminator processes observed and future trajectories.
- Experimental setup: Evaluation uses ETH and UCY real-world trajectory datasets, with four subsets for training and validation and one held out for testing.The experiments include ETH, Hotel, ZARA01, ZARA02, and Univ.
- Evaluation metrics: ADE averages Euclidean prediction error across time steps, while FDE measures Euclidean error at the final predicted position.Stochastic models are evaluated using the closest of K = 20 generated samples to ground truth, with τ = 8 observed frames and T = 12 predicted frames.
- Real-data accuracy: S-Ways yields significantly lower ADE and FDE on ETH and Hotel, but not on ZARA, where narrower paths produce less trajectory variance.The authors associate stronger performance with more complex scenes and nonlinear trajectories.
- Predictive distributions: Qualitative samples cover ground-truth trajectories and plausible alternatives in scenarios involving nonlinear actions, collision avoidance, and group behaviors.The experiment generated 128 samples, shown as magenta points, alongside observed, ground-truth, and constant-velocity trajectories.
- Predictive distributions: On the toy dataset, Info-GAN with Unrolled-GAN preserves almost all modes after 90,000 iterations, with a slight advantage for Info-GAN.Vanilla-GAN, L2-GAN, and S-GAN-V20 fail to preserve multimodality; L2 loss converges faster but produces worse distribution preservation.
- Predictive distributions: On real Stanford Drone trajectories, S-Ways displays separate predictive modes, whereas Vanilla-GAN does not produce varied paths.The comparison uses subsets from the Hyang-6 and Gates-2 scenes.
5. Conclusions and Future Works
The paper presents an InfoGAN-based approach for sampling individual pedestrian trajectories and evaluates its ability to preserve multimodal predictive distributions. Results indicate clearer mode separation than Vanilla-GAN and identify remaining opportunities to improve generative models and decision-making use.
- Approach: InfoGAN produces samples from individual pedestrians’ predictive trajectory distributions while incorporating hand-designed interaction features as an attention-pooling prior.The features are inspired by neuroscience and biomechanics literature.
- Multimodality evaluation: InfoGAN achieves the best results among evaluated variants for preserving multimodality on the purpose-built trajectory benchmark.The benchmark was designed specifically to assess preservation of modes in predictive distributions.
- Prediction accuracy: The approach partly improves state-of-the-art prediction accuracy on datasets where predictive distributions have the largest variances.The conclusion attributes this finding to extensive evaluations on commonly used datasets.
- Future work: The authors identify further room to improve generative models for pedestrian motion prediction and to exploit them in decision making.These are stated as future directions in the conclusion.