PaperlayerView exact revision on arXiv

Source-linked AI summary

Conditional Generative Neural System for Probabilistic Trajectory Prediction

Jiachen Li, Hengbo Ma, Masayoshi Tomizuka

arXiv:1905.01631v2cs.CVcs.AIcs.LGcs.ROstat.ML

TL;DR

Trajectory prediction in complex, interactive environments must represent uncertainty while remaining realistic and feasible. The paper proposes CGNS, combining conditional latent-space learning, variational divergence minimization, attention to context and interactions, and differentiable barrier functions. Across pedestrian benchmarks and a roundabout driving dataset, the authors report better prediction accuracy than various baselines.

  • Problem

    Accurate trajectory prediction must handle uncertain futures, multiple-agent interactions, static context, and realistic feasibility requirements.

  • Method

    CGNS combines conditional latent-space learning, variational divergence minimization, soft attention, and differentiable barrier functions to generate feasible trajectory samples.

  • Results

    The model achieves better prediction accuracy than various baselines on pedestrian benchmarks and a roundabout driving dataset.

  • Takeaways & Limitations

    CGNS produces realistic, feasible, and diverse future trajectory hypotheses for highly interactive scenarios.

Abstract

from arXiv · show

Effective understanding of the environment and accurate trajectory prediction of surrounding dynamic obstacles are critical for intelligent systems such as autonomous vehicles and wheeled mobile robotics navigating in complex scenarios to achieve safe and high-quality decision making, motion planning and control. Due to the uncertain nature of the future, it is desired to make inference from a probability perspective instead of deterministic prediction. In this paper, we propose a conditional generative neural system (CGNS) for probabilistic trajectory prediction to approximate the data distribution, with which realistic, feasible and diverse future trajectory hypotheses can be sampled. The system combines the strengths of conditional latent space learning and variational divergence minimization, and leverages both static context and interaction information with soft attention mechanisms. We also propose a regularization method for incorporating soft constraints into deep neural networks with differentiable barrier functions, which can regulate and push the generated samples into the feasible regions. The proposed system is evaluated on several public benchmark datasets for pedestrian trajectory prediction and a roundabout naturalistic driving dataset collected by ourselves. The experimental results demonstrate that our model achieves better performance than various baseline approaches in terms of prediction accuracy.

I. INTRODUCTION

The paper proposes CGNS to predict diverse, realistic, and feasible trajectories for multiple interactive agents while incorporating context, interactions, and soft feasibility constraints.

  • The system combines latent space learning with variational divergence minimization to model trajectory distributions and generate diverse motion hypotheses.
  • CGNS jointly predicts future trajectories for multiple highly interactive agents using static context, entity interactions, and feasibility constraints.
  • Block attention extracts critical features from historical trajectories, while Gaussian mixture attention focuses on relevant portions of scene image sequences.
  • Soft constraint incorporation is presented as a method for regulating generated trajectories within feasible regions.
  • The proposed system is evaluated on pedestrian forecasting benchmarks and a roundabout vehicle-motion anticipation task.

II. RELATED WORK

Prior trajectory prediction methods range from classical time-series models to conditional generative models, while attention mechanisms help identify informative scene features. This paper combines conditional latent-space learning and variational divergence minimization for trajectory forecasting conditioned on historical trajectories and context images.

  • Trajectory and Sequence Prediction: Classical Kalman-filter, time-series, and autoregressive methods generally suit short-term prediction when interactions can be ignored.
  • Attention Mechanism: Attention mechanisms focus neural models on informative obstacles or entity features, motivating the paper’s trajectory and Gaussian mixture attention mechanisms.
  • Generative Models: Generative models approximate data distributions to produce samples with appropriate variance, including explicit and implicit density approaches.
  • Generative Models: The proposed framework combines conditional VAE-style latent-space learning with adversarial variational divergence minimization.
  • Problem Formulation: The forecasting objective conditions future trajectories on historical trajectories and context images for multiple agents.

IV. METHODOLOGY

CGNS combines environment-aware feature extraction with a generative neural sampler to produce feasible, realistic, and diverse trajectories for interactive agents.

  • IV. METHODOLOGY: CGNS contains a deep feature extractor with environment attention and a generative neural sampler.The feature extractor processes historical context images and trajectories, while the sampler generates future trajectories.
  • IV. METHODOLOGY: The feature extractor extracts static and dynamic obstacle information from historical context images and trajectories of multiple interactive agents.Its environment attention mechanism identifies areas and entities requiring greater attention for predicting a target entity.
  • IV. METHODOLOGY: All CGNS components are implemented as deep neural networks and trained end-to-end.The system architecture includes feature extraction, conditional encoding, trajectory generation, and discrimination of predictions from ground truth.
  • Environment-Aware Deep Feature Extraction: Soft block attention is applied to trajectories, while Gaussian mixture attention is applied to context images to identify crucial forecasting features.These mechanisms target informative portions of historical trajectories and scene images before feature fusion.
  • Environment-Aware Deep Feature Extraction: The trajectory attention mask is learned from trajectory matrices, while context attention masks are obtained from Gaussian mixture parameters and applied before VGG-19 feature extraction.Interaction-aware and context-aware features are concatenated and processed by recurrent and fully connected layers.

C. Deep Generative Sampling

The deep generative sampler combines conditional latent-space learning with adversarial variational divergence minimization to sample realistic future trajectories from learned conditional representations.

  • C. Deep Generative Sampling: The generative neural sampler comprises an encoder that learns a lower-dimensional conditional latent distribution and a generator that produces realistic trajectories.An auxiliary discriminator distinguishes generated trajectories from ground truth.
  • Variational Divergence Minimization: The encoder, generator, and discriminator are optimized jointly through conditional latent-space learning and variational divergence minimization.Adversarial learning implements the divergence-minimization objective with deep generator and discriminator networks.
  • Conditional Latent Space Learning: Conditional latent space learning models the conditional distribution p(Y |C) using an encoder E and generator G.The encoder obtains Q(z|C, Y), while the generator models p(Y |z, C).
  • Conditional Latent Space Learning: The latent-space objective maximizes a variational lower bound using reconstruction error and KL divergence.The conditional prior is p(z|C) = N(0, I).
  • Variational Divergence Minimization: Variational divergence minimization compares the real trajectory distribution Pdata(Y |C) with its GNS approximation PGNS(Y |C).The method uses an f-divergence framework and variational lower bounds.

Y pGNS(Y |C)f

The VDM component formulates adversarial optimization to approximate the conditional trajectory distribution, using a chosen divergence and jointly trained generator, discriminator, and latent-space terms.

  • Y pGNS(Y |C)f: A variational lower bound is minimized through a minimax game between the generated conditional distribution and a mapping T(Y |C).The generator and mapping are parameterized by θ and φ, respectively.
  • Y pGNS(Y |C)f: The method minimizes the Pearson-χ2 divergence between Pdata + PGNS and 2PGNS.This specifies the divergence selected for the variational divergence minimization objective.
  • Y pGNS(Y |C)f: Adversarial learning makes the divergence objective tractable using generator and discriminator networks.The corresponding adversarial loss functions are derived for deep-network optimization.
  • Y pGNS(Y |C)f: Additional latent-space learning terms are included to distinguish the effects of latent-space learning within the overall objective.The VDM inputs include latent variables sampled from the encoded latent distribution.

D. Soft Constraint Incorporation

CGNS incorporates vehicle-kinematic feasibility through a differentiable barrier loss that pushes generated trajectories toward regions satisfying acceleration and curvature bounds.

  • D. Soft Constraint Incorporation: A differentiable barrier function incorporates soft vehicle-kinematic constraints into the neural-network loss.The barrier pushes predicted trajectories toward feasible regions rather than enforcing a nondifferentiable hard constraint.
  • D. Soft Constraint Incorporation: The feasibility loss constrains the absolute values of predicted accelerations and path curvatures using empirical upper bounds amax and κmax.The bounds are defined for the prediction horizon k+1:k+Tf.
  • D. Soft Constraint Incorporation: Acceleration and curvature values in the feasibility term are calculated from predicted waypoints.The sign function sgn(·) is used in the loss formulation.
  • D. Soft Constraint Incorporation: The feasibility loss is applied to vehicle trajectory prediction but not to human trajectory prediction.This marks the scope boundary of the proposed kinematic constraint term.

E. Conditional Generative Neural System (CGNS)

The CGNS combines conditional latent space learning with variational divergence minimization in an end-to-end trainable generative system. Training compensates for generator–discriminator imbalance by updating the discriminator multiple times per iteration.

  • CLSL and VDM provide complementary strengths within the proposed system's unified objective.
  • The complete objective is formulated so the system can be trained end-to-end.
  • Multiple discriminator updates compensate for faster generator improvement caused by the reconstruction loss.

V. EXPERIMENTS

The experiments evaluate CGNS on three public trajectory-prediction benchmarks and a roundabout vehicle-behavior prediction task, comparing performance with state-of-the-art baselines.

  • CGNS is validated on three publicly available benchmark datasets for trajectory prediction.
  • The evaluation includes probabilistic behavior prediction for multiple interactive on-road vehicles in a roundabout scenario.
  • Model performance is compared with several state-of-the-art baseline approaches.

A. Datasets

The evaluation uses pedestrian datasets, a vehicle roundabout dataset, ADE/FDE metrics, and established trajectory-prediction baselines. The datasets cover varied pedestrian and vehicle environments.

  • Datasets: ETH, UCY, and Stanford Drone Dataset provide pedestrian trajectories from outdoor, indoor, and university-campus scenarios.
  • Evaluation Metrics: ADE measures average trajectory distance from ground truth, while FDE measures distance at the final predicted time step.
  • Evaluation Metrics: The evaluation predicts 12 future time steps (4.8s) from 8 observed time steps (3.2s) for ETH and UCY.
  • Evaluation Results: Tables I–III report ADE/FDE comparisons for ETH/UCY pedestrians, SDD pedestrians, and RD vehicles.
  • Baselines: The comparison includes CVM, LR, P-LSTM, S-LSTM, S-GAN, S-GAN-P, CAR-Net, SoPhie, and DESIRE.

C. Implementation Details

The implementation uses differentiable neural components trained end-to-end, with specified architecture dimensions and optimization settings. Experiments also examine context extraction and attention-mask complexity.

  • Implementation Details: The system consists of differentiable deep-neural functions and is trained end-to-end efficiently.
  • Implementation Details: The encoded latent space has dimension two, while recurrent components use 128 hidden units.
  • Implementation Details: Adam optimization uses learning rate 0.002 with λ1 = 5.0, λ2 = λ3 = λ4 = λ5 = 1.0, and α1 = α2 = 1000.
  • Implementation Details: A single Gaussian replaces the Gaussian-mixture context mask because it did not visibly improve prediction accuracy or diversity in this task.
  • Pedestrian Experiments: On ETH and UCY, CGNS is evaluated without context features to match prior methods' historical-trajectory input.
  • Quantitative Analysis: CGNS achieves the best prediction-error performance on SDD and the smallest error among baselines in most roundabout cases, especially for long-term prediction.

E. Qualitative Analysis

Qualitative analyses show that CGNS generates smooth, feasible, realistic vehicle trajectories and benefits increasingly from static context as the forecasting horizon grows. Ablation results indicate that combining CLSL and VDM improves prediction error, while adding interaction information and context has horizon-dependent effects.

  • Qualitative prediction results: CGNS generates smooth, feasible, and realistic vehicle trajectories that follow road curves, with groundtruth near the highest-density predictions.The generated trajectory distribution is approximated using kernel density estimation.
  • Ablative analysis: Combining T + CLSL + VDM produces a notable prediction-error improvement over using either T + CLSL or T + VDM alone.The ablation compares settings using ADE and FDE on the RD dataset.
  • Ablative analysis: The complete T + I + CLSL + VDM system offers little obvious short-term improvement over partial systems, but its superiority becomes more remarkable at longer horizons.Static context has limited short-term influence because short trajectory segments can be approximated linearly.
  • Ablative analysis: As the forecasting horizon increases, road geometry increasingly constrains vehicle motion, making context-image information more beneficial.The reported gain is attributed to the growing importance of road curvature for long-term prediction.
Loading 1905.01631v2…