PaperlayerView exact revision on arXiv

Source-linked AI summary

GOHOME: Graph-Oriented Heatmap Output for future Motion Estimation

Thomas Gilles, Stefano Sabatini, Dzmitry Tsishkou, Bogdan Stanciulescu, Fabien Moutarde

arXiv:2109.01827v4cs.CVcs.RO

TL;DR

GOHOME addresses limitations of existing trajectory-prediction representations by using graph operations and heatmap outputs to model future-position uncertainty and multimodality. It achieves state-of-the-art performance across three datasets, including 2nd place on Argoverse MR6 and a more than 15% MR6 decrease with its best ensemble.

  • Problem

    Existing trajectory-prediction methods use distributions, coordinates, anchors, or maps that may be poorly adapted to road context, collapse modes, or restrict possible futures.

  • Method

    GOHOME represents the HD map as a lane-level graph, ranks likely lanelets, and projects sparse lane representations into a global future-position heatmap without image-based convolutions.

  • Results

    GOHOME reaches state-of-the-art performance on Argoverse, NuScenes, and Interaction, placing 2nd on Argoverse MR6 and achieving a more than 15% MR6 decrease with its best ensemble.

  • Takeaways & Limitations

    Heatmap outputs provide multimodality and uncertainty representation, while graph operations reduce computation by focusing on the most useful lanelets.

Abstract

from arXiv · show

In this paper, we propose GOHOME, a method leveraging graph representations of the High Definition Map and sparse projections to generate a heatmap output representing the future position probability distribution for a given agent in a traffic scene. This heatmap output yields an unconstrained 2D grid representation of agent future possible locations, allowing inherent multimodality and a measure of the uncertainty of the prediction. Our graph-oriented model avoids the high computation burden of representing the surrounding context as squared images and processing it with classical CNNs, but focuses instead only on the most probable lanes where the agent could end up in the immediate future. GOHOME reaches 2$nd$ on Argoverse Motion Forecasting Benchmark on the MissRate$_6$ metric while achieving significant speed-up and memory burden diminution compared to Argoverse 1$^{st}$ place method HOME. We also highlight that heatmap output enables multimodal ensembling and improve 1$^{st}$ place MissRate$_6$ by more than 15$\%$ with our best ensemble on Argoverse. Finally, we evaluate and reach state-of-the-art performance on the other trajectory prediction datasets nuScenes and Interaction, demonstrating the generalizability of our method.

I. INTRODUCTION

Trajectory prediction must represent both natural control variability and uncertainty about an agent’s future maneuver, while avoiding rigid or computationally wasteful representations. GOHOME addresses these issues with an unconstrained heatmap generated from graph-encoded map structure and sparse lane projections.

  • Trajectory prediction involves aleatoric uncertainty from control noise and epistemic uncertainty about the agent’s future choice.Aleatoric uncertainty spreads possible positions through variations in acceleration and curvature, whereas epistemic uncertainty concerns unknown destinations or maneuvers.
  • Gaussian predictions impose a predefined 2D shape, coordinate regression can collapse modes, and anchor- or map-based methods restrict possibilities to predefined sets.
  • Heatmap outputs represent future position distributions without restricting uncertainty to a parametric form and support multimodal prediction.
  • Full-image convolutional processing is costly because networks operate over square images while roads and drivable areas occupy a sparse portion of the space.
  • GOHOME replaces HOME’s fully convolutional, restricted-image approach with graph operations over vectorized HD maps and highlights heatmap-based ensembling without mode collapse.

II. RELATED WORK

Prior trajectory-forecasting methods use recurrent, convolutional, graph, anchor-based, generative, and heatmap representations to encode context and multimodality. GOHOME combines graph processing with heatmap prediction without using convolutional networks.

  • Recurrent trajectory models are often combined with CNNs over top-view images to incorporate local map and context information.
  • GNN-based approaches process compact map encodings, including global interaction graphs and connected lane graphs, to capture map connectivity.
  • Anchor, proposal, and candidate-based methods identify or score discrete trajectory modes using trajectory priors, map lanes, goals, or lane nodes.
  • Generative approaches can produce multimodal predictions but require multiple forward passes, do not guarantee sample diversity, and introduce randomness undesirable in production systems.
  • Heatmap methods provide natural multimodality and avoid mode collapse, but most prior approaches use full CNN architectures.
  • GOHOME is presented as the first method combining a GNN architecture with heatmap output without using CNNs.

III. METHOD

GOHOME represents the HD map as a connected lanelet graph, integrates agent trajectories and interactions with graph-based encoders and attention, and predicts heatmaps from the most probable lanes. The lanelet-level design provides an efficient, high-level representation while retaining connectivity.

  • III. METHOD: GOHOME predicts a future-position heatmap, samples endpoints from it, and reconstructs a trajectory for each endpoint.
  • III. METHOD: The local HD map is encoded as a graph of lanelets, each representing a 10 to 20 meter road segment with geometric and predecessor, successor, left, and right connectivity.
  • III. METHOD: The model scores lanelets, generates partial heatmaps for the top-ranked lanes, projects them into a global heatmap, and samples endpoints from the result.
  • A. Graph neural network for HD-Map input: GraphEncoder1 spreads connectivity information using four graph convolutions with relation-specific learned weights and fixed HD-Map adjacency matrices.
  • A. Graph neural network for HD-Map input: Agent trajectories are encoded separately, updated with lane information through cross-attention, and integrated across agents through self-attention before final graph encoding.
  • A. Graph neural network for HD-Map input: Unlike methods applying graph operations to lane nodes or using global attention over polylines, GOHOME applies graph convolutions to lanelets for efficient high-level connectivity propagation.

B. Heatmap generation through Lane-level rasters

GOHOME constructs heatmaps through sparse lane-level rasters in curvilinear coordinates, projects selected rasters into Cartesian space, and trains the result against a Gaussian target with focal loss. Lane ranking limits computation to likely lanes while preserving performance according to the supplied method description.

  • a) Lane raster generation: Each lane raster spans 20m longitudinally and 4m transversely in a discretized Frenet-Serret reference frame.
  • a) Lane raster generation: Longitudinal and lateral probability components are broadcast into an (h, w, 8) feature volume with complexity (h+w)×8 instead of h×w×8.
  • a) Lane raster generation: Raster features are combined with Cartesian coordinates, heading, lane occupancy, and curvature before a sigmoid produces lane-level probabilities.
  • c) Cartesian image connection: Lane-level rasters are projected into the full Cartesian heatmap, with overlapping projected pixels averaged.
  • The final target heatmap is a Gaussian centered on the ground-truth position, and training uses the pixel-wise focal loss from HOME and CenterNet.
  • b) Lane ranking: Only the top k lanelets selected by classification score receive lane-level raster computation and projection, reducing unnecessary computation.
  • b) Lane ranking: Lanelet classification uses a sigmoid output with binary cross-entropy weighted by 1e−2 alongside the pixel-wise loss.
  • c) Cartesian image connection: A preliminary projection of raster features into Cartesian coordinates helps the model account for overlaps before final probability estimation.

C. Sparse sampling for Miss Rate Optimization and Full trajectory generation

GOHOME samples diverse end points from the heatmap by repeatedly selecting locally concentrated probability and suppressing nearby locations, then reconstructs full trajectories from history and each sampled end point.

  • Sparse sampling: The MissRate optimization algorithm iteratively selects the grid point with maximum surrounding probability within radius r.After each selection, probabilities in that local neighborhood are set to zero to prevent duplicate locations.
  • Full trajectory generation: A two-hidden-layer fully connected model infers full trajectories from sampled end points, using the agent history and end point as inputs.Each hidden layer has 64 features, and training uses the ground-truth end points.

D. Model ensembling

Heatmaps provide a shared spatial representation that allows predictions from different models to be averaged without deciding which trajectory modalities correspond.

  • Model ensembling: Multimodal trajectory predictions are difficult to ensemble because corresponding modalities cannot be reliably identified for averaging.Even shortest-distance matching does not guarantee that predictions represent the same decision.
  • Model ensembling: Probability heatmaps can combine information from different models in a common reference system without assumptions or mode collapse.This representation supports averaging predictions spatially rather than matching discrete trajectory modes.

IV. EXPERIMENTAL RESULTS

The experiments evaluate GOHOME on Argoverse, NuScenes, and Interaction using leaderboard displacement and miss-rate metrics, with dataset-specific sampling radii and shared implementation settings.

  • Datasets: GOHOME is evaluated on Argoverse, NuScenes, and Interaction datasets, all of which provide local HD maps as lanelet graphs.The datasets use different history and future horizons and sampling frequencies.
  • Metrics: The evaluation reports MRk, minFDEk, and minADEk for dataset-specific values of k following the respective leaderboards.Argoverse uses k=1,5; NuScenes uses k=1,5,10; Interaction uses k=1,6.
  • Implementation details: All models use batch size 32, Adam optimization, scheduled learning-rate reductions, 64-channel layers, normalization, ReLU activations, and upsampled lanelets.Lanelets are upsampled to an average length of 10m.
  • Sampling configuration: The architecture is shared across datasets, while sampling radius r is tuned to uncertainty spread and the metric being optimized.Argoverse uses a default radius of 1.8m; NuScenes and Interaction use different radii for their uncertainty characteristics and objectives.
  • Results: GOHOME ranks first in multiple metrics on both NuScenes and Interaction.The reported leaderboard and validation results cover the NuScenes and Interaction evaluations.

B. Comparison with State-of-the-art

GOHOME achieves strong benchmark performance while using a lighter and faster model than HOME, and its ensemble results further improve Argoverse MissRate performance.

  • Benchmark comparison: GOHOME reaches 2nd place on Argoverse MR6 with a lighter and faster model than HOME.The comparison is reported on the online Argoverse test set against published methods.
  • Efficiency comparison: Inference time is evaluated as a function of output range in the reported comparison.
  • Ensemble comparison: More than 15% MR6 decrease is achieved by the best reported ensemble relative to the existing state of the art.The ensemble results are presented as improving the state of the art by a significant margin.

C. Ablation studies

The ablations examine computational efficiency from graph encoding, lane-raster projections, and selecting only the highest-ranked lanes. These choices reduce scaling costs while preserving or improving performance.

  • Computational efficiency: GOHOME replaces convolution operations with graph operations to reduce computation burden.The study compares HOME, GNN-HOME with a GNN encoder and CNN decoder, and GOHOME.
  • Computational efficiency: Graph encoding and lane rasters are evaluated against full CNN encoding and decoding using FLOPs, parameters, and frames per second.The comparison uses an average of 140 lanelets and 10 agents per sample, with inference measurements reported on the Argoverse validation set.
  • Lane ranking: Selecting the top k lanes does not decrease performance and can improve metrics while providing effective speed-up.The experiments fix an input range of 128, an output range of 192, and 0.5m x 0.5m pixel resolution.

3) Image size and resolution scaling:

GOHOME scales more efficiently than image-based HOME as the predicted range and resolution increase. Top-20 lane rasters produce less-than-linear range scaling and more efficient resolution scaling than CNN decoders.

  • Image size and resolution scaling: A 192m image range may be insufficient for datasets requiring 6- or 8-second predictions.The paper notes that longer straight trajectories can extend far distances without necessarily increasing input size.
  • Image size and resolution scaling: Less-than-linear scaling is achieved by combining lane rasters with top-20 lane ranking, whereas HOME and GNN-HOME CNN decoders scale quadratically.The comparison fixes the input range at 128 meters and the resolution at 0.5m per pixel.
  • Image size and resolution scaling: More efficient scaling becomes clear beyond 2 pixels per meter, especially for Lane Rasters outputs.Lane-raster resolution still incurs quadratic complexity, but with a much smaller coefficient that supports realistic training and inference times at finer resolutions.

D. Qualitative results

GOHOME’s qualitative outputs assign distinct heatmap modes to different lane possibilities and align those modes with the corresponding lane curvature. The lane classification and heatmap spread represent different uncertainty sources.

  • D. Qualitative results: Different heatmap modes are assigned to each lane possibility and align with the corresponding lane along its curvilinear direction.The qualitative examples show that each mode has a spread following the lane curvature.
  • D. Qualitative results: Lane prediction represents epistemic uncertainty, while heatmap-mode spread represents aleatoric uncertainty in trajectory controls.The lane choice reflects uncertainty about the driver’s destination, whereas spread reflects variations from control noise.
  • D. Qualitative results: GOHOME generates a global heatmap probability distribution without image-based convolution and reaches state-of-the-art performance on three datasets.The conclusion attributes the framework to graph operations, ranking, and projections, with scaling across predicted range and resolution.
Loading 2109.01827v4…