Source-linked AI summary
Stochastic Trajectory Prediction via Motion Indeterminacy Diffusion
Tianpei Gu, Guangyi Chen, Junlong Li, Chunze Lin, Yongming Rao, Jie Zhou, Jiwen Lu
TL;DR
Pedestrian trajectory prediction must represent the indeterminacy and multi-modality of future human motion. MID reverses a conditioned diffusion process to progressively discard indeterminacy, and it achieves state-of-the-art performance on the Stanford Drone and ETH/UCY benchmarks.
Problem
Human trajectories are indeterminate and multimodal, leaving many plausible future paths after an observed history.
Method
MID learns a parameterized Markov chain conditioned on observed trajectories to reverse motion indeterminacy diffusion, using a Transformer-based architecture for temporal dependencies.
Results
MID achieves state-of-the-art performance on the Stanford Drone and ETH/UCY human trajectory prediction benchmarks.
Takeaways & Limitations
Adjusting the diffusion-chain length provides a potential trade-off between prediction diversity and determinacy.
Takeaways & Limitations
Reverse diffusion can be expensive because sampling requires multiple steps; with 100 steps, MID took 17.368s versus 0.443s for Trajectron++ on 512 ZARA1 trajectories.
Abstract
from arXiv · showhide
Human behavior has the nature of indeterminacy, which requires the pedestrian trajectory prediction system to model the multi-modality of future motion states. Unlike existing stochastic trajectory prediction methods which usually use a latent variable to represent multi-modality, we explicitly simulate the process of human motion variation from indeterminate to determinate. In this paper, we present a new framework to formulate the trajectory prediction task as a reverse process of motion indeterminacy diffusion (MID), in which we progressively discard indeterminacy from all the walkable areas until reaching the desired trajectory. This process is learned with a parameterized Markov chain conditioned by the observed trajectories. We can adjust the length of the chain to control the degree of indeterminacy and balance the diversity and determinacy of the predictions. Specifically, we encode the history behavior information and the social interactions as a state embedding and devise a Transformer-based diffusion model to capture the temporal dependencies of trajectories. Extensive experiments on the human trajectory prediction benchmarks including the Stanford Drone and ETH/UCY datasets demonstrate the superiority of our method. Code is available at https://github.com/gutianpei/MID.
1. Introduction
Human trajectory prediction must address the multi-modality and indeterminacy of future pedestrian motion. MID models this variation through reverse diffusion, using a conditioned Markov chain and Transformer-based architecture to balance diversity and determinacy.
- Future pedestrian motion is indeterminate, so many plausible paths can follow the same observed history.
- Prior stochastic methods commonly represent future-motion multi-modality with latent variables in GANs or conditional variational auto-encoders.
- MID explicitly models motion indeterminacy by reversing a diffusion process that progressively converts ambiguous walkable regions into a desired trajectory.
- The framework conditions a parameterized Markov chain on observed trajectories and encodes history behavior and social interactions as a state embedding.
- A Transformer-based architecture captures temporal dependencies in trajectories.
- MID achieves state-of-the-art performance on widely used human trajectory prediction benchmarks while providing a potential diversity–accuracy trade-off.
2. Related Work
Related work covers conventional trajectory forecasting, stochastic models for multimodal futures, and diffusion-based generative models. MID introduces diffusion modeling and a Transformer architecture to trajectory prediction.
- Pedestrian Trajectory Prediction: Trajectory forecasting methods estimate future positions from observed paths, often emphasizing complex social interactions.
- Stochastic Prediction Model: Stochastic prediction methods model multimodal future motion with GANs, CVAEs, grid-based location encoders, or pedestrian goals.
- Denoising Diffusion Probabilistic Models: Denoising diffusion probabilistic models learn parameterized Markov chains that gradually denoise a common distribution into a specific data distribution.
- Denoising Diffusion Probabilistic Models: MID applies diffusion modeling to trajectory indeterminacy and uses a Transformer-based architecture for temporal dependencies.
3. Proposed Approach
MID formulates stochastic trajectory prediction as a reverse diffusion process that progressively reduces indeterminacy from noisy walkable regions to plausible trajectories. It conditions Gaussian Markov transitions on observed behavior and social interactions, using a Transformer to model temporal dependencies.
- Motion Indeterminacy Diffusion: MID treats trajectory prediction as reversing diffusion, progressively reducing indeterminacy from ambiguous walkable areas to desired trajectories.The forward process adds noise until trajectories become corrupted, while the learned reverse process denoises them toward predictions.
- Motion Indeterminacy Diffusion: The diffusion and reverse-diffusion trajectories form a Gaussian Markov chain with K as the maximum number of diffusion steps.Fixed variance schedulers control injected noise, and sufficiently large K yields an approximately standard Gaussian endpoint.
- Reverse Process: The reverse process is conditioned on a state feature learned from observed trajectories and modeled with shared parameterized Gaussian transitions.The state feature is produced by a temporal-social encoder, while the diffusion model parameters and encoder parameters are trained with trajectory data.
- Training Objective: Training maximizes a variational lower bound because the exact reverse-process log-likelihood is intractable, using Gaussian posterior transitions and a simplified reparameterized loss.The loss matches the learned reverse transition with the tractable forward posterior, including mean-based KL calculations.
- Inference: At inference, MID starts from yK ∼ N(0, I) and progressively generates trajectories toward y0 using the trained network, previous prediction, state embedding, and step index.The random variable z is standard Gaussian, and the network predicts transitions during reverse denoising.
- Network Architecture: MID encodes history behavior and social interactions as a condition and uses a Transformer-based decoder to model temporal dependencies and Gaussian Markov transitions.The decoder receives noisy trajectories, noise, the encoder feature, and a time embedding; the framework is encoder-agnostic.
4. Experiments
Experiments evaluate MID on Stanford Drone and ETH/UCY benchmarks, compare it with existing methods, and test its diffusion and Transformer components. Results also examine how reverse-diffusion steps affect the diversity–determinacy trade-off.
- Comparison with state-of-the-art methods: MID achieves 7.61/14.30 average ADE/FDE on Stanford Drone, the best performance among compared methods under standard 20-sample evaluation.The comparison includes trajectory-only and trajectory-plus-image methods; lower ADE/FDE is better.
- Comparison with state-of-the-art methods: MID achieves 0.21 ADE and 0.38 FDE on ETH/UCY using only trajectory input with 20 samples.The paper reports comparable performance on this benchmark and greater benefit on the larger Stanford Drone dataset.
- Ablation Studies: Removing the diffusion model from the same encoder–decoder framework significantly degrades results, while replacing only the decoder with Transformer does not improve the CVAE-based framework.These ablations distinguish the contribution of the diffusion formulation from simply changing the decoder architecture.
- Ablation Studies: Transformer decoders outperform Linear and LSTM alternatives for MID, while increasing the Transformer beyond 512 dimensions does not improve performance.The ablation attributes the architecture’s benefit to modeling temporal dependencies in trajectories.
- Analysis of Reverse Diffusion Process: Fewer reverse-diffusion steps produce more diverse trajectories, whereas additional steps reduce diversity and increase determinacy.Adjusting the step count therefore provides a flexible diversity–determinacy trade-off.
- Analysis of Reverse Diffusion Process: Trajectory contours are diverse early in reverse diffusion, then progressively concentrate and fit the ground-truth path.The visualization depicts the process at ten-step intervals.
5. Conclusion & Discussion
MID formulates trajectory prediction as a conditioned reverse diffusion process that gradually discards indeterminacy to obtain acceptable trajectories. Its chain length provides a diversity–determinacy trade-off, while experiments report state-of-the-art performance on Stanford Drone and ETH/UCY benchmarks.
- MID learns a parameterized Markov chain conditioned on observed trajectories to gradually discard indeterminacy from ambiguous areas.
- In ETH/UCY visualizations, MID predictions are reported as closer to ground truth than Trajectron++ across five scenes.
- The chain length can be adjusted to trade off prediction diversity and determinacy.
- MID uses a Transformer-based architecture to model complex temporal dependencies in trajectories.
- MID achieves state-of-the-art performance on the Stanford Drone and ETH/UCY benchmarks.
- Reverse diffusion can be expensive: MID takes 17.368s versus 0.443s for Trajectron++ on 512 ZARA1 trajectories with 100 diffusion steps.
A.1. Derivations of Loss Function
The derivation obtains the loss function by simplifying the variational objective and removing a term without learnable parameters. The resulting formulation is used as the model’s training loss.
- The appendix derives the loss function for the MID training objective.
- The last term is ignored because it has no learnable parameters.
- After this simplification, the remaining expression defines the loss function used for optimization.
A.2. Derivations of Reparameterization
The derivation explains how MID trains a noise-prediction network for reverse diffusion and samples trajectories by iteratively denoising Gaussian noise. The sampling procedure starts from noise and proceeds backward through the chain.
- A.2. Derivations of Reparameterization: The reverse-transition loss matches the model transition with the ground-truth posterior, both modeled as Gaussian distributions.
- A.2. Derivations of Reparameterization: During sampling, MID initializes the future trajectory from standard Gaussian noise and iterates from step K down to 1.
- A.2. Derivations of Reparameterization: The sampling-number ablation evaluates the trade-off between min 3/min 5 and diversity.
- A.2. Derivations of Reparameterization: The model predicts noise from the noisy trajectory, diffusion step, and conditioning features.
- A.2. Derivations of Reparameterization: The trained noise-prediction network is used to obtain the mean of the Gaussian reverse transition.
- A.2. Derivations of Reparameterization: The next trajectory step is sampled using Gaussian noise, with zero additional noise at the final step.
B. Implementation Details
MID uses a diffusion model with a Transformer decoder and an encoder for conditioning features, implemented with specified variance scheduling and MLP upsample–downsample layers. Training and inference procedures are provided algorithmically and in code.
- The implementation section covers training hyperparameters, network architecture, training and inference algorithms, and code.
- The diffusion model uses a variance schedule from β1 = 0.0001 to βK = 0.05 and a three-layer Transformer with 512-dimensional representations, 1024-dimensional feedforward layers, and four attention heads.
- MLP sub-networks upsample raw trajectories from 2d to 512d and downsample Transformer outputs through 512d-256d-2d.
- The sub-network conditions its MLP layers on step-number and state embeddings concatenated as c = [k, sin(k), cos(k), f].
- Algorithms 1 and 2 provide pseudocode for MID training and inference, and the implementation code is available online.
C. Additional Experiments
Additional experiments examine how reverse diffusion steps and sampling choices affect trajectory-prediction performance, highlighting trade-offs between diversity, determinacy, accuracy, and computation cost.
- Reverse diffusion steps: Reducing diversity improves ADE and FDE predictions with fewer samples, demonstrating a contradiction between diversity and determinacy in the few-sample setting.Figure 7 reports min 3/min 5 ADE and FDE curves across reverse diffusion steps from 0 to 100.
- Sampling tricks: Increasing the number of samplings significantly improves performance, following a sampling strategy used by Expert.Sampling tricks may also include post-processing, such as clustering in YNet or choosing the best sample in Expert.
- Sampling tricks: More samplings increase computational cost, so the authors do not encourage using them despite their performance benefit.