What the Constant Velocity Model Can Teach Us About Pedestrian Motion Prediction

Christoph Schöller; Vincent Aravantinos; Florian Lay; Alois Knoll

Source-linked AI summary

What the Constant Velocity Model Can Teach Us About Pedestrian Motion Prediction

Christoph Schöller, Vincent Aravantinos, Florian Lay, Alois Knoll

arXiv:1903.07933v3Mar 19, 2019cs.CVcs.LGcs.RO

TL;DR

Pedestrian motion prediction is essential for safe autonomous robots and vehicles, but it is unclear whether complex neural models benefit from the additional information they receive. The paper evaluates a simple Constant Velocity Model and analyzes neural-network inputs, finding that environmental priors harm generalization while longer histories and interactions provide limited value.

Problem
Safe autonomy requires accurate pedestrian future-motion prediction, while the usefulness of neural networks’ additional input information remains uncertain.
Method
The paper extensively evaluates a Constant Velocity Model against neural models and analyzes how neural networks process environmental, historical, and interaction information.
Results
The Constant Velocity Model can outperform state-of-the-art pedestrian motion-prediction models, while neural networks learn environmental priors that negatively affect generalization.
Takeaways & Limitations
The CVM should be included as a standard baseline, and stronger focus on pedestrian environments and more diverse datasets is a promising direction.

Abstract

from arXiv · show

Pedestrian motion prediction is a fundamental task for autonomous robots and vehicles to operate safely. In recent years many complex approaches based on neural networks have been proposed to address this problem. In this work we show that - surprisingly - a simple Constant Velocity Model can outperform even state-of-the-art neural models. This indicates that either neural networks are not able to make use of the additional information they are provided with, or that this information is not as relevant as commonly believed. Therefore, we analyze how neural networks process their input and how it impacts their predictions. Our analysis reveals pitfalls in training neural networks for pedestrian motion prediction and clarifies false assumptions about the problem itself. In particular, neural networks implicitly learn environmental priors that negatively impact their generalization capability, the motion history of pedestrians is irrelevant and interactions are too complex to predict. Our work shows how neural networks for pedestrian motion prediction can be thoroughly evaluated and our results indicate which research directions for neural motion prediction are promising in future.

I. INTRODUCTION

The paper argues that pedestrian motion prediction is safety-critical, yet a simple Constant Velocity Model can match state-of-the-art neural performance. It analyzes environmental priors, motion history, and interactions to identify why additional neural-network inputs may not help.

Accurate future-motion prediction is essential for autonomous robots and vehicles to operate safely without endangering humans.
The Constant Velocity Model achieves state-of-the-art performance despite using only the pedestrian’s latest relative motion.
Neural networks implicitly learn environmental priors that strongly reduce their generalization to new scenes.
Longer motion histories are mostly redundant, and removing them does not degrade neural-network prediction accuracy.
Neighbor interactions are too complex for neural networks to predict reliably and usually have little influence on pedestrian trajectories.

II. RELATED WORK

Prior work addresses pedestrian motion prediction through tracking, long-term trajectory models, interaction modeling, and environmental information. This paper focuses on long-term prediction with a Constant Velocity Model and examines what its success implies for neural approaches.

Earlier tracking methods use short-term motion models, filters, Bayesian networks, and Constant Velocity formulations for people in images.
Long-term prediction methods include recurrent models and approaches that explicitly represent pedestrian interactions.
Other approaches incorporate environmental information through navigation maps, transferable scene models, occupancy grids, or trajectory hypothesis scoring.
This work studies long-term pedestrian prediction with a CVM and analyzes implications of its success for neural networks and motion prediction.
The prediction problem estimates a pedestrian’s future trajectory from personal history, optionally incorporating neighboring pedestrians’ histories.
Models commonly observe only past information and predict relative displacements, which can reduce error margins versus predicting absolute positions.

IV. CONSTANT VELOCITY MODEL

The Constant Velocity Model assumes the latest relative motion best predicts the future and repeats that motion across the prediction horizon. Its simplicity comes with sensitivity to measurement noise.

The model uses the most recent relative motion as the principal predictor of future trajectory.
The CVM assumes pedestrians continue with the velocity and direction observed over their latest two timesteps.
Because it applies no filtering, the CVM is sensitive to measurement noise.
The CVM predicts each future displacement by repeating the latest relative displacement for every prediction step.

V. EXPERIMENTS

The experiments compare pedestrian motion predictors on ETH and UCY data under a standard long-term forecasting setup. Leave-one-out evaluation tests generalization across scenes using ADE and FDE in meters.

The evaluation uses ETH and UCY datasets containing 1950 unique pedestrians across five scenes with different walkway layouts and obstacles.
Each experiment observes the last 8 positions and predicts the next 12 timesteps, covering 3.2 seconds of observation and 4.8 seconds of prediction.
Leave-one-out cross-validation trains on four scenes and evaluates on the remaining scene to measure generalization to new scenarios.
Models are evaluated in meters using Average Displacement Error and Final Displacement Error.

A. Training

The evaluation compares the CVM with common baselines and four state-of-the-art neural models, using specified training procedures for trainable models and sampling for comparability.

Trainable models used Adam with learning rate 0.0004, batch size 64, 35 epochs, and mean squared error loss.Training scenes were randomly split, with 10% used for validation; models converged without overfitting.
The Constant Acceleration baseline uses the last three positions and assumes continued acceleration.
Linear Regression predicts each trajectory component independently, with each predicted variable depending on the full motion history.
The Feed Forward network flattens eight motion-history timesteps and predicts 12 future timesteps through two hidden layers and a linear output layer.
The evaluation includes RED, SR-LSTM, S-GAN, and SoPhie alongside the CVM and baseline models.For comparability with generative models, OUR-S adds angular noise sampled from N(0, σ2) with σ = 25°; S-GAN used a pre-trained model and RED was re-implemented.

C. Results

The CVM and its sampled variant perform strongly across quantitative and qualitative evaluations, including comparisons with state-of-the-art models and scenes with nonlinear trajectories.

Quantitative: OUR-S achieved the best average ADE and FDE and outperformed the state-of-the-art generative models S-GAN and SoPhie.The comparison considers the best predicted samples for the sampled models.
Qualitative: The CVM produced close approximations even when pedestrian trajectories were nonlinear.In the sampled comparison, OUR-S, S-GAN, and RED were evaluated using sampled or predicted trajectories as described in the evaluation procedure.
Quantitative: OUR outperformed all other models without sampling, including state-of-the-art RED and SR-LSTM.Its advantage was especially strong for the Hotel scene.
Quantitative: ConstAcc performed worst, indicating that continual acceleration or deceleration is detrimental over long prediction horizons.
Qualitative: Sharp or changing trajectory turns were difficult to foresee, while the CVM’s linear predictions were overall good approximations of future trajectories.

VI. ANALYSIS OF NEURAL NETWORK BEHAVIOR

The analysis examines why a simple CVM performs strongly by testing how neural networks use additional information, focusing on environmental priors, motion history, and interactions.

The study analyzes environmental priors, motion history, and pedestrian interactions as influences on neural-network performance.
Hotel trajectories are mostly vertical, while trajectories in the other scenes are mostly horizontal.
This scene imbalance causes neural networks to learn an environmental prior that contradicts Hotel at test time.The learned prior negatively affects generalization to new scenes.
The analysis uses FF and RED because their simple training dynamics help disentangle experimental effects from training noise.FF is a basic neural network, whereas RED is a state-of-the-art model with good performance.

A. Environmental Priors

Environmental constraints and scene semantics bias pedestrian motion, and neural networks can learn these priors implicitly. This learned bias harms generalization, while relative inputs and rotations improve performance.

A. Environmental Priors: Physical constraints and scene semantics bias pedestrians toward particular motion patterns.Examples include impassable buildings and walking toward or away from a shopping center.
A. Environmental Priors: Neural networks can associate coordinate ranges with scene-specific motion patterns, even without explicit environment inputs.They may also learn motion patterns typical of an entire scene.
A. Environmental Priors: Relative inputs and additional rotations strongly improved model performance, especially for the Hotel scene.These modifications were designed to dampen the effects of learning environmental priors.
A. Environmental Priors: Data augmentation helps prevent neural networks from learning environmental priors.The authors connect this mechanism to previously reported benefits of data augmentation and hypothesize it explains poor LSTM performance under absolute positions without augmentation.

B. Motion History

Long motion histories provide mostly redundant information for pedestrian prediction. Neural networks consequently rely primarily on the latest timestep, and removing older history does not degrade predictions.

B. Motion History: Long motion histories are less relevant than commonly believed for pedestrian motion prediction.The Constant Velocity Model uses only the last two timesteps while achieving strong performance.
B. Motion History: The latest relative motion contributes 68.2% for FF and 80.3% for RED to the predicted trajectory.For FF, timestep t −1 contributes 8.1%, while the other five timesteps contribute 3.8%–6% each with fluctuating influence.
B. Motion History: All history timesteps are highly correlated, with correlation coefficients ranging from 0.91 to 1.0.Closer timesteps are more correlated, indicating mostly shared and redundant information.
B. Motion History: For the evaluated networks, redundant history information likely acts as noise rather than signal, explaining their reliance on the latest timestep.The gradient analysis measures each timestep’s average influence on the output trajectory.
B. Motion History: Depriving networks of longer motion histories does not degrade prediction performance, confirming that long history is not predictive for this task.Networks were trained with relative histories of sizes 7 through 1 while predicting the next 12 timesteps.

C. Pedestrian Interactions

Neighbor information does not reliably improve pedestrian motion prediction and can worsen it. The analysis indicates that interactions are either difficult to predict or usually have limited influence on trajectories.

C. Pedestrian Interactions: Interaction-aware prediction requires anticipating neighbors’ future trajectories to avoid potential collisions.A model must therefore implicitly and simultaneously predict neighboring pedestrians’ futures.
C. Pedestrian Interactions: Both FF and RED performed worst when given neighbors’ motion histories.The authors attribute this likely to interaction complexity and the difficulty of finding robust solutions while predicting multiple pedestrians.
C. Pedestrian Interactions: Providing neighbors’ true future trajectories still did not enable the models to exploit interaction information reliably.This suggests interactions are either not predictable or have small average impact on trajectories and prediction error.
C. Pedestrian Interactions: A model given neighbors’ future trajectories predicted a path that would collide with standing neighbors.The failure occurred despite fully observing those neighbors.
C. Pedestrian Interactions: Interaction-aware predictions were infrequent, and most true trajectories lacked obvious interaction-aware behavior.The authors therefore indicate that interactions are less relevant than commonly believed and too complex to predict from neighbors’ motion histories alone.

VII. CONCLUSION

The paper finds that a simple Constant Velocity Model can outperform state-of-the-art neural models and uses this result to analyze why additional inputs often fail to help. It identifies environmental priors, redundant history, and complex interactions as central issues, while highlighting stronger environmental modeling as a future direction.

VII. CONCLUSION: Extensive experiments show that the Constant Velocity Model can outperform state-of-the-art pedestrian motion prediction models.The conclusion presents this as the basis for analyzing neural networks’ use of additional information.
VII. CONCLUSION: Neural networks learn environmental priors that negatively affect generalization, while data augmentation can alleviate this problem.The paper also finds that long motion histories are mostly redundant and ignored by the networks.
VII. CONCLUSION: The paper recommends including the Constant Velocity Model as a standard baseline because of its simplicity and strong benchmark performance.Strong baselines are presented as important amid a trend toward increasingly complex models.
VII. CONCLUSION: A stronger focus on pedestrians’ environments is identified as a promising direction for improving generalization and exploiting environmental information.The authors suggest datasets with more diverse environments and testing whether these insights transfer to other motion prediction domains.