Source-linked AI summary
Context-Aware Trajectory Prediction
Federico Bartoli, Giuseppe Lisanti, Lamberto Ballan, Alberto Del Bimbo
TL;DR
The paper addresses human trajectory prediction in crowded spaces, where motion reflects both nearby people and static surroundings. It proposes a context-aware LSTM with pooling for these interactions, evaluates it on UCY and MuseumVisits, and reports lower prediction errors than prior state-of-the-art methods.
Problem
Predicting human motion in crowded spaces requires accounting for both human-human and human-space interactions, which existing models may treat incompletely.
Method
The paper extends Social-LSTM with context-aware pooling in an LSTM model to encode interactions with neighboring people and manually identified static scene objects.
Results
The approach obtains lower prediction errors than other state-of-the-art solutions on a new museum dataset and a subset of UCY.
Takeaways & Limitations
Considering both human-human and human-space interactions is fundamental for trajectory prediction in the evaluated crowded scenarios.
Takeaways & Limitations
The experiments omit fine-tuning with simulated trajectories, although prior work reported that it may reduce sequence prediction errors by about 50%.
Abstract
from arXiv · showhide
Human motion and behaviour in crowded spaces is influenced by several factors, such as the dynamics of other moving agents in the scene, as well as the static elements that might be perceived as points of attraction or obstacles. In this work, we present a new model for human trajectory prediction which is able to take advantage of both human-human and human-space interactions. The future trajectory of humans, are generated by observing their past positions and interactions with the surroundings. To this end, we propose a "context-aware" recurrent neural network LSTM model, which can learn and predict human motion in crowded spaces such as a sidewalk, a museum or a shopping mall. We evaluate our model on a public pedestrian datasets, and we contribute a new challenging dataset that collects videos of humans that navigate in a (real) crowded space such as a big museum. Results show that our approach can predict human trajectories better when compared to previous state-of-the-art forecasting models.
1 Introduction
Predicting human trajectories in crowded spaces requires modeling both interactions among moving people and influences from the surrounding environment. The paper introduces a context-aware LSTM approach and evaluates it against prior methods on UCY and a new MuseumVisits dataset.
- Predicting pedestrian motion is challenging but important for applications including robotics, smart spaces, and automotive systems.
- Existing trajectory-prediction methods model either human-space interactions or human-human interactions, motivating a combined treatment.
- The proposed context-aware recurrent model extends Social-LSTM with pooling that incorporates static scene objects alongside neighboring people.
- The model observes past human positions and surrounding interactions to predict near-future trajectories.
- The paper evaluates the approach on the UCY dataset and introduces MuseumVisits, a challenging dataset collected in a real crowded museum.
2 Approach
The approach models each person’s trajectory with an LSTM and extends Social-LSTM-style pooling to represent both human-human and human-space interactions. It uses neighbor occupancy, distances to manually identified static objects, and observed trajectories to predict future positions.
- Context-Aware LSTM: Each person’s trajectory is represented with an LSTM whose input includes spatial coordinates and recurrent hidden states.The model predicts the next position using a bivariate Gaussian distribution whose parameters are obtained from the LSTM output state.
- Human-Human Interactions: Human-human interactions are encoded by pooling neighboring trajectories into an m × n occupancy grid.The occupancy matrix represents the presence or absence of neighbors around each person at each time step.
- Human-Human and Human-Space Interactions: The model adds context-aware pooling to represent static scene elements alongside human-human interactions.Static objects can include entry or exit points and objects such as museum artworks, which are manually identified in the scene.
- Human-Human and Human-Space Interactions: Distances to static objects are used to model how strongly each element can influence a person’s path instead of weighting all objects equally.This distance-based representation differs from the naive formulation, in which each neighboring static element contributes equally.
- Trajectory Prediction: At test time, the model uses observed paths, neighboring-person trajectories, and distances to static objects to estimate near-future trajectories.Training assumes that trajectories for all persons in a scene are available beforehand, and optimization uses negative log-likelihood.
3 Experiments
Experiments evaluate the context-aware trajectory-prediction models on UCY ZARA sequences and the new MuseumVisits dataset using cross-validation and prediction errors in meters. Results compare interaction-aware configurations and qualitative trajectory behavior against baseline methods.
- Datasets: MuseumVisits contains rich human-human and human-space interactions, while UCY ZARA provides moderately crowded scenes with people entering and leaving continuously.MuseumVisits includes groups stopping near artworks; ZARA sequences were acquired from a bird’s-eye view.
- Evaluation protocol: The protocol observes trajectories for 3.2 seconds and predicts the following 4.8 seconds, corresponding to 8 observed and 12 predicted frames.Results are reported using Average displacement error, described as mean squared error in meters between predicted and ground-truth points.
- Evaluation setup: The evaluation uses five-fold cross-validation on five MuseumVisits sequences and two-fold cross-validation on the UCY ZARA sequences.
- Experimental results: For most MuseumVisits sequences, context-aware O-LSTM has the lowest errors, whereas context-aware S-LSTM performs best on the ZARA sequences.The reported pattern is associated with museum movement between points of interest and ZARA pedestrians adjusting paths around other people.
- Experimental results: A pooling variant that weights neighboring static objects equally produces higher prediction errors than the proposed context-aware representation.The authors interpret this as evidence that static objects influence trajectories differently and should not be treated identically.
- Qualitative results: Qualitative examples show the context-aware model accounting for static artworks and nearby pedestrian trajectories, including stopping behavior and interactions with both types of context.The examples also include cases where the model is less accurate or produces a noisy trajectory because of an incorrectly estimated motion model.
4 Conclusion
The approach models both interactions between people and the semantic elements of the surrounding space. It introduces a context-aware pooling mechanism and a challenging museum dataset, with experiments reporting lower prediction errors than prior state-of-the-art methods.
- The model captures interactions between persons and the rich semantic elements characterizing the space.
- Qualitative MuseumVisits examples compare LSTM, Social-LSTM, Context-aware O-LSTM, and ground-truth trajectories while showing nearby-person paths and observed artworks.
- Context-aware pooling estimates how much scene elements influence a person’s trajectory.
- The authors introduce a challenging trajectory dataset collected in the crowded hall of a large art museum.
- Experimental results on the new dataset and a UCY subset report lower prediction errors than other state-of-the-art solutions.