PaperlayerView exact revision on arXiv

Source-linked AI summary

GoRela: Go Relative for Viewpoint-Invariant Motion Forecasting

Alexander Cui, Sergio Casas, Kelvin Wong, Simon Suo, Raquel Urtasun

arXiv:2211.02545v2cs.ROcs.AIcs.CVcs.LGcs.MA

TL;DR

Motion forecasting for self-driving vehicles must support safe maneuvering, but per-agent viewpoint encoding is computationally costly and fixed shared viewpoints can be data inefficient and vulnerable to uncommon poses. GORELA uses pairwise relative geometry for shared viewpoint-invariant encoding and lane-graph goal-based prediction. The authors report improved forecasting across urban and highway settings, with HighwaySim gains over baselines across its metrics.

  • Problem

    Per-target encoding is computationally expensive for multi-agent prediction, while shared fixed-viewpoint encoding can be sample inefficient and vulnerable to uncommon SDV poses.

    Source: §I · p. 1

  • Method

    GORELA uses pairwise relative positional encodings in a shared viewpoint-invariant scene representation and predicts trajectories from lane-graph-anchored goals.

    Source: §I · p. 1, §IV · pp. 3, 5

  • Results

    GORELA predicts more accurate multi-agent multimodal trajectories on urban and highway roads, and largely outperforms HighwaySim baselines on all reported metrics.

    Source: §VI · p. 6, §V · p. 6

  • Takeaways & Limitations

    Viewpoint invariance supports shared scene computation and offline map-embedding caching, while the model’s reported performance spans urban and highway settings.

    Source: §I · p. 1, §VI · p. 6

Abstract

from arXiv · show

The task of motion forecasting is critical for self-driving vehicles (SDVs) to be able to plan a safe maneuver. Towards this goal, modern approaches reason about the map, the agents' past trajectories and their interactions in order to produce accurate forecasts. The predominant approach has been to encode the map and other agents in the reference frame of each target agent. However, this approach is computationally expensive for multi-agent prediction as inference needs to be run for each agent. To tackle the scaling challenge, the solution thus far has been to encode all agents and the map in a shared coordinate frame (e.g., the SDV frame). However, this is sample inefficient and vulnerable to domain shift (e.g., when the SDV visits uncommon states). In contrast, in this paper, we propose an efficient shared encoding for all agents and the map without sacrificing accuracy or generalization. Towards this goal, we leverage pair-wise relative positional encodings to represent geometric relationships between the agents and the map elements in a heterogeneous spatial graph. This parameterization allows us to be invariant to scene viewpoint, and save online computation by re-using map embeddings computed offline. Our decoder is also viewpoint agnostic, predicting agent goals on the lane graph to enable diverse and context-aware multimodal prediction. We demonstrate the effectiveness of our approach on the urban Argoverse 2 benchmark as well as a novel highway dataset.

I. INTRODUCTION

Motion forecasting for self-driving vehicles must be accurate, generalize across scenarios, and arrive in time for safe maneuvering. GORELA uses pairwise relative geometry for shared, viewpoint-invariant scene encoding, addressing the accuracy, generalization, and computation trade-offs of prior approaches.

I. INTRODUCTION

  • Per-target scene processing prioritizes accuracy and generalization but scales poorly, while shared fixed-viewpoint encoding reduces latency but can lose generalization to rare SDV poses.The latter includes uncommon states such as a U-turn.Source: §I · p. 1
  • GORELA represents pairwise relationships between agents and map entities to create a shared, viewpoint-invariant scene encoding.It predicts goals anchored to lane-graph nodes and generates trajectories conditioned on those goals.Source: §I · p. 1, Figure 1 · p. 1
  • The authors report that GORELA is more effective, generalizes better to novel viewpoints, and is less data hungry than competing methods across urban and highway datasets.The proposed encoding also permits offline map-embedding computation and avoids viewpoint translation and rotation augmentation during training.Source: §I · p. 1

II. RELATED WORK IN MOTION FORECASTING

Prior motion-forecasting work has addressed trajectory history, map encoding, agent-map fusion, and future decoding with different trade-offs in invariance, compute, and prediction quality. GORELA combines shared viewpoint-invariant scene encoding with pairwise relative geometry and lane-graph goal prediction.

Agent history encoding

  • Vectorized history encoders transform past poses to a common coordinate frame and process them with 1D CNNs, RNNs, or Transformers.Earlier approaches rasterized trajectories for 2D CNN processing [3], [7].Source: §II · pp. 1–2

Map encoding

  • Map encoders progressed from BEV rasters to lane graphs, while agent-centered RoIs or repeated per-agent encoding increase inference cost and shared global frames sacrifice viewpoint invariance.GORELA’s lane-graph encoder aims to provide shared, viewpoint-invariant scene encoding.Source: §II · p. 2

Heterogeneous fusion

  • Prior agent-map fusion ranges from ordered late fusion to early trajectory fusion and cross-attention; GORELA instead uses pairwise relative positional edge attributes in a heterogeneous graph.Against HDGT, the authors cite bounded encodings, a goal-based decoder, and late-fusion map caching as distinctions, and report superior Argoverse 2 performance.Source: §II · p. 2

Future trajectory decoding

  • Goal-based decoding separates endpoint prediction from trajectory completion, while GORELA anchors goals to lane-graph nodes and uses a greedy sampler without ensemble clustering.The passage notes that ensemble clustering can be prohibitive when reaction time matters.Source: §II · p. 2

III. PAIR-WISE RELATIVE HETEROGENEOUS GNN

The proposed heterogeneous spatial GNN represents nodes with semantic features and poses, and models directed relationships through pairwise relative positional encodings and heterogeneous message passing. Its relative geometry is designed to be bounded and viewpoint-invariant.

III. PAIR-WISE RELATIVE HETEROGENEOUS GNN

  • The GNN models interactions in a heterogeneous spatial graph whose nodes have general attributes, semantic categories, and 3-DOF poses.Directed relationships use continuous edge attributes and discrete edge classes.Source: §III · p. 2

Pair-wise relative positional encoding

  • Pairwise relative encoding starts from centroid displacement and heading difference, but replaces frame-dependent displacement with distance and the displacement angle relative to node j’s heading.This parameterization is described as viewpoint-invariant.Source: §III · pp. 2–3
  • Centroid distances are mapped to sine and cosine features at multiple frequencies to bound their representation across the distance range of interest.The stated range extends from a handful of meters to hundreds of meters.Source: §III · p. 3

Heterogeneous message passing

  • The HMP layer updates a node from incoming neighbors by projecting source features and edge attributes, aggregating messages, then fusing them with projected node features.Projection weights depend on edge class; the update function can retain or forget prior node information, and the authors note that GAT does not use edge embeddings directly to update node embeddings.Source: §III · p. 3

IV. VIEWPOINT-INVARIANT MOTION FORECASTING

GORELA forecasts multiple agents by encoding relative geometry between agents and map elements in a shared heterogeneous graph, then predicting goals and completing trajectories.

IV. VIEWPOINT-INVARIANT MOTION FORECASTING

  • GORELA separately computes agent and map features, fuses them on one heterogeneous graph, then predicts each agent’s goal and full trajectory.Map features can be computed offline, while the scene encoder is shared across agents.Source: §IV · p. 3, Figure 3 · p. 4
  • Past waypoints are encoded as pair-wise relative positional encodings to the current pose, then processed by a 1D CNN and GRU for each agent.The CNN extracts local temporal patterns, and the GRU aggregates them into an agent trajectory embedding; computation can be batched across agents.Source: §IV · pp. 3–4
  • The lane graph samples map centerlines into lane-segment nodes and adds crosswalk nodes, lane relations, dilated successor and predecessor edges, and conflict edges.A viewpoint-agnostic encoder computes map embeddings offline for large map tiles, which can be served onboard from cache.Source: §IV · p. 4
  • Heterogeneous message-passing layers fuse agents and map nodes using agent-to-agent, agent-to-map, and map-to-agent edges alongside map relations.Agents connect to the nearest lane node at the current time; agent pairs connect when less than 100 meters apart.Source: §IV · p. 4
  • The decoder classifies goals over lane-graph nodes, regresses continuous goal offsets, and greedily samples K goals while balancing nearby and distinct modes.It removes goals within γ meters and downweights those within ν meters, where ν > γ.Source: §IV · pp. 4–5
  • For each goal, a shared MLP predicts future waypoints in agent-relative coordinates, using a common trajectory prior across vehicles, pedestrians, and cyclists.The goal is supplied as ground truth to trajectory completion during training.Source: §IV · p. 5
  • Training combines goal classification, goal regression, and trajectory-completion losses; trajectory completion uses teacher forcing with ground-truth goals.Goal classification uses focal loss, while regression and waypoint coordinates use Huber losses.Source: §IV · p. 5

V. EXPERIMENTAL EVALUATION

The evaluation reports GORELA results on Argoverse 2 and HighwaySim, examines component ablations, and includes qualitative forecasts. The reported comparisons favor GORELA over the tested baselines, while ablations show higher BrierMinFDE when individual components are replaced or removed.

Comparison against state-of-the-art

  • GORELA outperforms all published Argoverse 2 leaderboard methods and largely outperforms HighwaySim baselines across metrics.On Argoverse 2, the paper also reports large validation-set gains and better-calibrated mode probabilities.Source: §V · pp. 5–6

Ablation study

  • Replacing the goal-based decoder with an MLP increases BrierMinFDE@K=6 by 51%, while bypassing the scene encoder increases it by 6%.The ablation caption identifies these as Argoverse 2 validation experiments.Source: Table III · p. 5, §V · p. 6
  • Replacing PairPose with a global-coordinate GNN without edge attributes increases BrierMinFDE@K=6 by 30%; replacing the greedy sampler with top-k increases it by 14%.The top-k sampler is the approach used by.Source: Table III · p. 5, §V · p. 6

Qualitative results

  • Qualitative Argoverse 2 forecasts include accurate focal-agent modes in mined hard cases, but also lane-change and corner-cutting failures.The paper describes the displayed forecasts as 6-second multimodal predictions for focal and scored agents, with ground truth in gray.Source: §V · p. 6, Figure 4 · p. 6

VI. CONCLUSION

GORELA forecasts multimodal trajectories for agents on urban and highway roads. Its contributions include viewpoint-invariant pairwise relative positional encoding, heterogeneous graph reasoning over agents and map nodes, and a probabilistic goal-based decoder with a greedy diversity sampler.

Conclusion

  • GORELA predicts more accurate multi-agent, multimodal trajectories on urban and highway roads using viewpoint-invariant relative encoding, heterogeneous graph reasoning, and a goal-based decoder with greedy sampling.The stated contributions also include multi-edge adjacency matrices connecting lane-lane, agent-lane, and agent-agent nodes.Source: §VI · p. 6

VII. APPENDIX

The appendix adds experiments on robustness to rare SDV states, sample efficiency, and runtime, plus urban and highway qualitative comparisons. It also provides implementation details for reproducibility.

Appendix

  • Supplementary experiments address rare-SDV-state robustness, sample efficiency, runtime, and qualitative urban and highway comparisons, alongside implementation details for reproducibility.Source: §VII · p. 9

A. ADDITIONAL EXPERIMENTS

The additional experiments report gains in sample efficiency, rotation robustness, and encoder runtime scaling, alongside graph-layer ablations and qualitative forecasts in urban and highway scenes.

A. ADDITIONAL EXPERIMENTS

B. IMPLEMENTATION DETAILS

Implementation combines relative-pose and map-node features with multi-task training and sampling choices. Supplementary figures visualize robustness, data-scale, runtime, and forecast comparisons.

B. IMPLEMENTATION DETAILS

  • Agent inputs include PairPose, its finite difference, velocity, bounding-box size, and observation status; map-node inputs include geometry, edge degrees, intersection and crosswalk flags, and lane-boundary attributes.Source: B. IMPLEMENTATION DETAILS · p. 10
  • Training equally weights goal classification, goal regression, and trajectory completion, using teacher forcing when ground-truth goals are available.Source: B. IMPLEMENTATION DETAILS · p. 10
  • Loss excludes agents leaving 10 m of lane-graph boundaries; training uses scale augmentation from 0.8 to 1.2 and ran for 17 epochs.Source: B. IMPLEMENTATION DETAILS · p. 10
  • The greedy goal sampler uses hard-threshold radius γ=2, downweight radius ν=4, and downweight factor τ=10.Source: B. IMPLEMENTATION DETAILS · p. 10
  • Supplementary figures show rotated-versus-original predictions, data-scale performance, encoder runtime scaling by actors and nodes, and qualitative forecasts in Argoverse 2 and HighwaySim.Source: Figure 6 · p. 11, Figure 7 · p. 11, Figure 8 · p. 11

Related papers

Loading 2211.02545v2…