Source-linked AI summary
MultiPath: Multiple Probabilistic Anchor Trajectory Hypotheses for Behavior Prediction
Yuning Chai, Benjamin Sapp, Mayank Bansal, Dragomir Anguelov
TL;DR
Future behavior prediction for planning must represent uncertain, multi-modal outcomes rather than only a single trajectory. MultiPath uses fixed trajectory anchors with discrete intent probabilities and Gaussian control refinements, producing compact parametric distributions efficiently. Across synthetic and real-world datasets, it reports better prediction quality than unimodal and sampling-based alternatives, while using one inference pass.
Problem
Future-agent prediction requires representing stochastic, multi-modal outcomes and supporting both likely discrete trajectories and tractable probabilistic queries for planning.
Method
MultiPath fixes trajectory anchors as intent modes, predicts their probabilities, and regresses Gaussian-distributed waypoint offsets and uncertainties conditioned on each anchor.
Results
MultiPath improves likelihood and trajectory-set metrics over previous models across synthetic and real-world datasets, requiring only one feed-forward inference pass.
Takeaways & Limitations
The parametric anchor mixture provides compact weighted trajectories together with direct likelihood evaluation for probabilistic planning queries.
Takeaways & Limitations
Time-step distributions are conditionally independent given an anchor, although recurrent next-time-step dependencies can be added.
Abstract
from arXiv · showhide
Predicting human behavior is a difficult and crucial task required for motion planning. It is challenging in large part due to the highly uncertain and multi-modal set of possible outcomes in real-world domains such as autonomous driving. Beyond single MAP trajectory prediction, obtaining an accurate probability distribution of the future is an area of active interest. We present MultiPath, which leverages a fixed set of future state-sequence anchors that correspond to modes of the trajectory distribution. At inference, our model predicts a discrete distribution over the anchors and, for each anchor, regresses offsets from anchor waypoints along with uncertainties, yielding a Gaussian mixture at each time step. Our model is efficient, requiring only one forward inference pass to obtain multi-modal future distributions, and the output is parametric, allowing compact communication and analytical probabilistic queries. We show on several datasets that our model achieves more accurate predictions, and compared to sampling baselines, does so with an order of magnitude fewer trajectories.
1 Introduction
MultiPath addresses stochastic, multi-modal future-agent prediction for motion planning by combining fixed trajectory anchors with probabilistic refinements. It produces compact, weighted trajectory hypotheses and improves likelihood and sample-set prediction metrics over relevant baselines.
- Motivation: Future-agent prediction matters for safe, comfortable, and efficient robot operation in real-world environments such as self-driving.The task requires scene context including road semantics, traffic lights, and past observations of other agents.
- Motivation: Agents’ unknown motivations make future behavior inherently stochastic, requiring multiple possible outcomes and their likelihoods.
- Motivation: MultiPath seeks weighted discrete trajectories covering likely outcomes while retaining closed-form likelihood evaluation for planning and probabilistic queries.These capabilities support discrete trajectory reactions and queries such as expected collision risk in a space-time region.
- Approach: Fixed trajectory anchors factor uncertainty into intent uncertainty over anchors and normally distributed control uncertainty conditioned on intent.The anchors are modes found from training data and can correspond to coarse future templates such as changing lanes or slowing down.
- Approach: The model predicts anchor probabilities, waypoint offsets, and uncertainties, yielding a Gaussian mixture at each time step from one scene-processing pipeline.Anchor-specific MAP trajectories provide a compact weighted set of future hypotheses.
- Results: MultiPath achieves higher likelihood than unimodal parametric distributions and describes futures better than sampling methods with far fewer samples.
2 Related work
Prior trajectory-prediction methods are commonly deterministic or stochastic, while direct distribution models trade flexibility against compactness and sampling convenience. MultiPath’s anchor-based approach is positioned against these limitations and draws on established anchor methods for multi-modal prediction.
- Deterministic models: Deterministic approaches predict a single most-likely trajectory per agent, typically through supervised regression.
- Stochastic models: Stochastic approaches use random sampling during training and inference to capture future non-determinism, including policy, CVAE, GAN, and diversity-optimized methods.
- Distribution models: Direct probability models include parametric distributions and probabilistic occupancy grids, with occupancy grids requiring dense state-space storage and offering less obvious trajectory extraction.
- Anchor methods: Predefined anchors have prior use in semi-parametric models, object detection, and human-pose estimation for handling multi-modal problems.
3 Method
MultiPath represents future trajectories with fixed anchor modes, context-dependent offsets, and Gaussian control uncertainty, using scene-wide and agent-centric neural processing. Its factorized GMM supports efficient joint prediction, likelihood evaluation, and weighted trajectory extraction.
- Model objective: Given past trajectories and contextual information, MultiPath outputs a parametric future-trajectory distribution and a compact weighted set of explicit trajectories.
- Uncertainty factorization: Intent uncertainty is modeled as a softmax distribution over K anchor trajectories, each represented as a sequence of states.
- Uncertainty factorization: Conditioned on intent, control uncertainty is assumed unimodal and represented by a Gaussian distribution for each anchor waypoint.
- Trajectory refinement: The model predicts scene-specific offsets from anchor states, refining static trajectories using road geometry, traffic lights, and interactions with other agents.
- Inference: Conditionally independent time-step distributions allow all future steps to be predicted jointly with one inference pass, while recurrent dependencies remain an optional extension.
- Distribution: Marginalizing intent yields a Gaussian mixture with time-fixed mixture weights, closed-form normalization, compact representation, and efficient occupancy-grid evaluation.
- Anchor construction: Anchors are estimated before parameter learning, using k-means or uniformly sampled trajectory space when clustering produces redundant modes.
- Training: The network is trained by imitation learning to maximize recorded-trajectory log-likelihood, with hard assignment to the closest anchor avoiding direct GMM fitting and expectation-maximization.
4 Experiments
The experiments compare MultiPath with single-trajectory, Min-of-K, CVAE, and linear baselines across synthetic and real-world datasets using likelihood and trajectory-distance metrics. MultiPath outperforms the baselines on the autonomous-driving benchmark and performs at or above state of the art on single-trajectory distance metrics.
- Experimental setup: MultiPath is evaluated against regression, Min-of-K, CVAE, and linear baselines using likelihood, distance, and trajectory-set metrics.The comparisons use matched input representations and comparable architectures for the main learned baselines.
- Synthetic data: MultiPath with K = 3 anchors correctly learns the three-way intersection’s intent and uncertainty distributions, whereas single-trajectory regression averages across diverging paths.The synthetic paths use intent probabilities {0.3, 0.5, 0.2} and sine-wave control variation.
- Metrics: Log-likelihood evaluates p(ŝ|x), while ADE and minADE_M measure displacement against a groundtruth or the closest trajectory in a predicted set.Log-likelihood is scaled by 2 × T for two-dimensional trajectories; minADE_M avoids penalizing plausible trajectories that differ from the logged outcome.
- Autonomous driving: MultiPath represents uncertain driving behavior with up to five weighted trajectories and uncertainty ellipses across speed variation, intersection intent, and lane changes.The regression baseline predicts only a single intent in the illustrated examples.
- Stanford Drone: On the Stanford Drone Dataset, MultiPath performs at or above state of the art on best single-trajectory distance metrics, although CAR-Net outperforms its comparable regression model.The authors also report better performance for their single-trajectory model when it predicts uncertainty.
5 Conclusion
The paper concludes that MultiPath predicts parametric future-trajectory distributions for real-world agents. Across synthetic and real-world datasets, it improves likelihood and trajectory-set metrics over single-trajectory and stochastic models using one feed-forward inference pass.
- MultiPath predicts parametric distributions of future trajectories for agents in real-world settings.
- Across synthetic and real-world datasets, MultiPath improves likelihood and trajectory-set metrics over previous single-trajectory and stochastic models.
- MultiPath requires only one feed-forward inference pass.
A In-depth analysis
The in-depth analysis presents alternative representations of the same Tab. 1 experiments on the autonomous-driving dataset.
- The analysis reports strictly different representations of the same autonomous-driving experiments from Table 1.
A.1 Group by groundtruth final waypoint
The evaluation groups predictions by final-waypoint behavior and examines how performance varies across trajectory types, time horizons, anchor counts, and backbone choices.
- Groundtruth final waypoint: MultiPath decisively outperforms the regression baseline on faster trajectories, while the baseline performs reasonably well for stationary and slow agents.The comparison uses log-likelihood and groups samples by groundtruth final waypoint.
- Time step: Distance-based prediction errors grow slightly super-linearly with prediction time up to 6 seconds.The authors attribute this pattern most likely to smooth acceleration in heading-normalized trajectories.
- Number of anchors K: The number of anchors K is a key hyperparameter evaluated across multiple MultiPath configurations.Experiments vary K for both MultiPath µ, Σ and MultiPath µ models.
B.3 Importance of heading normalization
Heading normalization improves MultiPath across all three evaluated metrics, reflecting the importance of representing vehicle motion relative to agent orientation.
- Importance of heading normalization: Agent-specific feature maps are rotated into an agent-centric coordinate system so each vehicle’s heading points in the same direction.The crop is centered on the agent and uses differentiable bilinear warping.
- Importance of heading normalization: Heading-normalized models outperform their counterparts on all three metrics.The comparison evaluates models with and without compensating heading during agent-centric feature cropping.
B.4 Road semantic information and traffic lights
The ablation evaluates semantic road and traffic-light context, while the anchor visualization reflects dataset-specific trajectory structure.
- Road semantic information and traffic lights: Including semantic road information and traffic-light context significantly improves all distance-based metrics.The gain is less apparent in log-likelihood, which the authors associate with context mainly preventing false-positive trajectories.
- Anchor visualization: The autonomous-driving anchor set reflects North American traffic patterns, including greater curvature for left turns and predominantly left-sided u-turns.The visualization shows anchors for different values of K.
D Stanford Drone Dataset experimental details
The Stanford Drone Dataset uses a top-down video representation and a hand-designed anchor set because the dataset is small and lacks heading information.
- Input representation: The input combines oriented-box histories with the current RGB video frame in a resized, padded 800 px × 800 px representation.The representation maintains the original aspect ratio.
- Anchor construction: The anchor set contains 64 straight trajectories from 16 orientations and four final-waypoint distances, plus one stationary anchor.Enumeration replaces K-means because the dataset is small and lacks heading information.
E CARLA experimental details
The CARLA dataset includes training, validation, and test scenes from Town01 and Town02, with multimodal trajectory information for the autopilot and other agents. Experiments follow a prior training-data setup using cropped top-down images and two LIDAR channels.
- The dataset contains 60,701 training, 7,586 validation, and 7,567 Town01 test scenes, plus 16,960 Town02 test scenes.
- Each scene provides the autopilot’s LIDAR observation, two seconds of past positions, and four seconds of future positions for five agents at 5Hz.
- The experiments use a 100 × 100 pixel top-down-image crop and only the two LIDAR channels representing the specified comparison setup.