How would surround vehicles move? A Unified Framework for Maneuver Classification and Motion Prediction

Nachiket Deo; Akshay Rangesh; Mohan M. Trivedi

Source-linked AI summary

How would surround vehicles move? A Unified Framework for Maneuver Classification and Motion Prediction

Nachiket Deo, Akshay Rangesh, Mohan M. Trivedi

arXiv:1801.06523v1Jan 19, 2018cs.CV

TL;DR

Autonomous vehicles need perception-only forecasts of surrounding vehicles for safe, smooth freeway path planning. The paper proposes a unified framework combining motion, maneuver patterns, and inter-vehicle interaction, and evaluates it on real vehicle-mounted-sensor data. The framework outperforms an IMM trajectory-prediction baseline and runs in real time at about 6 frames per second.

Problem
Reliable prediction of surrounding-vehicle motion is needed for autonomous-vehicle path planning, but prior work lacks a unified approach combining motion, traffic patterns, and inter-vehicle interaction.
Method
The framework combines HMM maneuver recognition, IMM and maneuver-specific VGMM trajectory prediction, and a vehicle-interaction module using global scene context.
Results
The framework outperforms an IMM-based trajectory-prediction baseline and runs in real time at about 6 frames per second.
Takeaways & Limitations
Probabilistic modeling, maneuver-based trajectory subcategorization, and interaction-aware simultaneous prediction each provide better predictions within the evaluated setting.

Abstract

from arXiv · show

Reliable prediction of surround vehicle motion is a critical requirement for path planning for autonomous vehicles. In this paper we propose a unified framework for surround vehicle maneuver classification and motion prediction that exploits multiple cues, namely, the estimated motion of vehicles, an understanding of typical motion patterns of freeway traffic and inter-vehicle interaction. We report our results in terms of maneuver classification accuracy and mean and median absolute error of predicted trajectories against the ground truth for real traffic data collected using vehicle mounted sensors on freeways. An ablative analysis is performed to analyze the relative importance of each cue for trajectory prediction. Additionally, an analysis of execution time for the components of the framework is presented. Finally, we present multiple case studies analyzing the outputs of our model for complex traffic scenarios

I. INTRODUCTION

The paper addresses perception-only prediction of surrounding freeway vehicles by unifying instantaneous motion, learned traffic patterns, and inter-vehicle interaction. It evaluates this framework on real vehicle-mounted-sensor data and analyzes the relative importance of these cues.

Reliable surround-vehicle prediction is needed for autonomous-vehicle path planning over horizons of up to 10 s.
Prior approaches separately emphasize motion models, probabilistic traffic patterns, maneuver classes, or inter-vehicle interaction, with limitations for long-term, safety-critical, or underrepresented behaviors.
Existing work often restricts motion, interaction, or evaluation settings and lacks a unified analysis combining all three prediction cues.
The proposed framework combines HMM maneuver recognition, IMM and maneuver-specific VGMM trajectory prediction, and an interaction module using global vehicle context.
The study evaluates the framework on real Californian freeway data captured using vehicle-mounted cameras and performs ablations of the three cues.

III. OVERVIEW

The overview defines a perception-only pipeline that predicts future positions and uncertainty for all nearby vehicles by combining motion, maneuver-specific probabilistic models, and scene-level interaction.

The system uses eight cameras, tracks vehicles within 40 m longitudinally, and projects their histories into the ego vehicle’s ground-plane frame.
The model predicts future positions and associated uncertainty for all tracked vehicles from recent track-history snippets.
Its trajectory module combines an IMM motion model with a probabilistic model learned from freeway trajectory patterns.
The framework bins freeway motion into 10 maneuver classes and learns a separate VGMM for each class to represent intra-maneuver variability.
The vehicle interaction module uses relative scene configuration to resolve maneuver feasibility and improve predictions in interacting traffic.

A. Maneuver classes

The paper organizes freeway trajectories into interpretable maneuver classes and uses HMMs to recognize partial maneuvers before maneuver-specific trajectory prediction.

Maneuver classes: Freeway surround-vehicle motion is represented using 10 maneuver classes in the ego-vehicle frame.
Maneuver classes: The classes comprise four lane passes, two overtakes, two cut-ins, and two drifts into the ego vehicle’s lane.
Maneuver classes: Overtakes and cut-ins are relatively rare, safety-critical, and challenging for trajectory prediction.
Maneuver recognition: HMMs recognize maneuvers from short, partially observed trajectory snippets rather than only after maneuvers are complete.
Maneuver recognition: Separate left-right HMMs use ground-plane coordinates and instantaneous velocities as features, with Gaussian-mixture emissions trained by Baum-Welch.
Trajectory prediction: The prediction module outputs future x and y locations plus a 2×2 covariance matrix by averaging motion-model and probabilistic predictions.

A. Motion Models

The motion component uses an interacting multiple model framework to combine Bayesian filters representing common freeway motion patterns.

The IMM framework combines Bayesian filters by weighting models according to transition probabilities and fit to observed motion.
The motion-model ensemble includes constant velocity, constant acceleration, and constant turn-rate-and-velocity models.
The constant-velocity model captures a majority of freeway vehicle motion.
The constant-turn-rate-and-velocity model can help represent motion during lane changes.

B. Probabilistic Trajectory Prediction

The framework estimates conditional distributions over future vehicle velocities from trajectory history and maneuver class, using maneuver-specific VGMMs. Predicted locations and uncertainty are derived from the velocity distribution.

The model estimates the conditional distribution of future vehicle velocities given past positions, velocities, and maneuver class.
Predicted future locations are obtained by cumulatively summing the predicted velocities through an accumulator matrix.
Prediction uncertainty is represented by the covariance of the predicted velocities and propagated to locations using the corresponding expression.
For each maneuver class, a VGMM estimates the joint distribution of past and future trajectory coefficients, enabling the conditional distribution of future coefficients.
VGMM parameters use Dirichlet and Gauss-Wishart conjugate priors, with posterior parameters estimated by Variational Bayesian Expectation Maximization.

VI. VEHICLE INTERACTION MODULE

The vehicle interaction module jointly assigns maneuver labels by balancing HMM confidence with ego-vehicle and pairwise trajectory-clearance terms. The resulting optimization is reformulated for integer linear programming and its assignments drive trajectory prediction.

The module assigns discrete maneuvers to all surround vehicles using HMM confidence and the feasibility of their joint future trajectories.
The objective combines individual HMM energies with ego-vehicle and pairwise interaction energies.
HMM individual energy is the negative log likelihood, so higher maneuver confidence produces lower energy.
Ego-vehicle energy uses the reciprocal closest approach over the prediction horizon, while pairwise energy uses reciprocal minimum distance between predicted trajectories.
Pairwise energy penalizes assignments in which two vehicles become very close during the prediction horizon, with weighting λ selected by cross-validation.
After reformulation, integer linear programming produces optimal maneuver assignments that the trajectory prediction module uses for future predictions.

A. Dataset

The evaluation uses synchronized, multimodal vehicle-mounted sensing from freeway drives, with four human-ground-truthed sequences reserved for testing. Tracks are labeled across ten maneuver classes and evaluated using leave-one-sequence-out cross-validation.

The dataset contains 52 video sequences from approximately 45 minutes of freeway driving under varied lighting, vehicle, traffic-density, and traffic-behavior conditions.
The test vehicle synchronously captures its surroundings with 8 RGB cameras, LIDARs, and RADARs at 15 fps.
Four longest sequences, each about 3 minutes, were human-ground-truthed for evaluation; three represent free-flowing traffic and one heavy or stop-and-go traffic.
Evaluation tracks are projected onto the ground plane and assigned labels from 10 maneuver classes, with maneuver boundaries marked when tracks contain multiple maneuvers.
Leave-one-sequence-out cross-validation trains HMMs and VGMMs on the other three evaluation sequences plus 48 training sequences.
Trajectory augmentation uses lateral inversion for opposite maneuver instances and longitudinal shifts of ±2, 4, and 6 m for additional same-class instances.

B. Evaluation Measures and Experimental Settings

The experiments predict five seconds of future trajectory from three seconds of track history and compare motion-only, monolithic probabilistic, maneuver-specific, and interaction-aware systems. Ablations show progressively better errors from probabilistic modeling, maneuver-specific VGMMs, and vehicle interaction, especially for overtakes, cut-ins, and stop-and-go traffic.

Evaluation Measures: 5 seconds of future trajectory are predicted from each 3-second track-history snippet, using mean and median absolute error as evaluation measures.Errors are reported at prediction instants from 1 to 5 seconds into the future.
Compared Systems: The IMM trajectory predictor serves as the motion-model baseline, while M-VGMM removes maneuver recognition by using one VGMM for all maneuvers.The monolithic model makes predictions irrespective of the maneuver being performed.
Ablative Analysis: C-VGMMs further improve mean and median absolute deviations by using separate VGMMs for maneuver classes.This system selects the VGMM associated with the maneuver having the highest HMM log likelihood.
Ablative Analysis: Probabilistic trajectory models outperform the IMM baseline, with M-VGMM achieving lower mean and median absolute errors.The reported comparison indicates fewer and less drastic average errors for the probabilistic model.
Challenging Maneuvers: C-VGMM significantly outperforms CV and M-VGMM for overtakes and cut-ins, with the difference becoming more pronounced at longer prediction horizons.The analysis attributes these gains to addressing non-uniform motion and underrepresented rare maneuver classes.
Vehicle Interaction: The VIM corrects some HMM maneuver assignments and improves trajectory prediction, with a stronger effect in dense stop-and-go traffic.Dense traffic causes more vehicles to affect one another, increasing the proportion of labels reassigned by the VIM.

D. Analysis of execution time

The framework’s maneuver-aware components improve prediction in several challenging traffic situations, while the complete system remains suitable for real-time deployment.

C-VGMM runs faster than M-VGMM despite adding HMM maneuver recognition, because each maneuver-specific model is substantially leaner.
The vehicle interaction module nearly doubles C-VGMM runtime, but the most complex framework configuration still operates at almost 6 fps.
M-VGMM and C-VGMM predict nonlinear lateral motion in overtakes that the IMM predicts incorrectly.
C-VGMM detects early overtakes and predicts adjacent-lane movement with longitudinal acceleration despite lacking corresponding instantaneous-motion cues.
C-VGMM correctly predicts deceleration in drift-into-ego-lane cases, whereas IMM and M-VGMM make poor predictions in the initial case.
Both probabilistic models correctly predict lane merging in lane-pass-right-back examples, suggesting implicit encoding of lane information.

F. Vehicle Interaction Model Case Studies

The vehicle interaction module revises independently recognized maneuvers using the scene-wide configuration of surrounding vehicles. In the case studies, these revisions align predictions more closely with feasible traffic behavior and ground truth.

The VIM changes vehicle 3’s predicted lane pass to a cut-in because vehicles 1 and 5 block its forward path.
The VIM changes vehicle 18’s predicted right overtake to drift into the ego lane, correctly predicting tail-gating when the right lane is occupied.
The VIM changes vehicle 1’s predicted left overtake to a right overtake, making the prediction closely match ground truth when the left lane is occupied.
The framework outperforms an IMM trajectory-prediction baseline and runs in real time at about 6 frames per second.
Ablation results indicate that probabilistic modeling, maneuver-specific trajectory models, and interaction-aware simultaneous prediction improve trajectory prediction over motion-only or independent prediction.
The framework is presented as general, with possible improvements to each of its three interacting modules.