PaperlayerView exact revision on arXiv

Source-linked AI summary

LAformer: Trajectory Prediction for Autonomous Driving with Lane-Aware Scene Constraints

Mengmeng Liu, Hao Cheng, Lin Chen, Hellward Broszio, Jiangtao Li, Runjiang Zhao, Monika Sester, Michael Ying Yang

arXiv:2302.13933v1cs.CV

TL;DR

Trajectory prediction must account for stochastic agent motion and scene constraints, but existing approaches can struggle with complex settings such as intersections. LAformer selects highly potential lane segments over time and refines first-stage predictions using observed and predicted trajectories. It outperforms other models on Argoverse 1 and nuScenes, with superior generalized performance reported for multimodal forecasting.

  • Problem

    Trajectory prediction must model stochastic agent motion and interactions while respecting scene constraints, and existing methods can struggle with complex scenes such as intersections.

    Source: §1 · pp. 1–2

  • Method

    LAformer selects top potential lane segments at each future time step to align motion with scene information, then refines first-stage anchor trajectories using observed and predicted trajectories.

    Source: §1 · p. 2

  • Results

    LAformer outperforms other models on Argoverse 1 and nuScenes and demonstrates superior generalized performance for multimodal motion prediction.

    Source: §1 · p. 2, §5 · p. 8

  • Takeaways & Limitations

    Ablation and sensitivity studies verify the efficacy of both the lane-aware estimation and motion refinement modules.

    Source: §5 · p. 8

    The approach assumes high-quality 2D trajectory and HD-map data from road-agent tracking and environmental perception.

    Source: §3.1 · p. 3

Abstract

from arXiv · show

Trajectory prediction for autonomous driving must continuously reason the motion stochasticity of road agents and comply with scene constraints. Existing methods typically rely on one-stage trajectory prediction models, which condition future trajectories on observed trajectories combined with fused scene information. However, they often struggle with complex scene constraints, such as those encountered at intersections. To this end, we present a novel method, called LAformer. It uses a temporally dense lane-aware estimation module to select only the top highly potential lane segments in an HD map, which effectively and continuously aligns motion dynamics with scene information, reducing the representation requirements for the subsequent attention-based decoder by filtering out irrelevant lane segments. Additionally, unlike one-stage prediction models, LAformer utilizes predictions from the first stage as anchor trajectories and adds a second-stage motion refinement module to further explore temporal consistency across the complete time horizon. Extensive experiments on Argoverse 1 and nuScenes demonstrate that LAformer achieves excellent performance for multimodal trajectory prediction.

1. Introduction

LAformer addresses trajectory prediction under stochastic motion and complex scene constraints with temporally dense lane selection and a second-stage motion refinement module. The paper reports strong performance on Argoverse 1 and nuScenes, including superior generalized multimodal prediction performance.

1. Introduction

  • Trajectory prediction remains challenging because agent behavior is stochastic, agents influence one another, and scene constraints vary across traffic contexts.Existing approaches may miss early spatial-temporal use of scene information, while rasterized maps can be costly and imprecise around complex intersections.Source: §1 · p. 1
  • Spatially dense lane methods can compound errors from inaccurate early predictions, while temporally dense alignment must handle variable lanes and uncertain motion.The introduction notes that distance-only alignment can be insufficient at intersections, during lane changes, or on turns.Source: §1 · p. 2
  • LAformer estimates lane alignment at each future time step and selects top-k potential lane segments to guide a lane-conditioned decoder.The method uses motion and lane information for step-wise selection, filtering irrelevant segments to lessen decoder representation requirements.Source: §1 · p. 2
  • A second-stage refinement module uses first-stage predictions as anchor trajectories and considers observed and predicted trajectories to exploit temporal consistency across past and future.The refinement module further reduces prediction offsets.Source: §1 · p. 2
  • LAformer achieves excellent performance on Argoverse 1 and nuScenes and shows superior generalized performance for multimodal motion prediction.The introduction identifies both as benchmark datasets.Source: §1 · p. 2

2. Related Work

Related work addresses agent interactions, multimodal trajectory generation, scene-context extraction, and lane-guided prediction. LAformer contrasts with spatially dense lane methods by estimating motion-to-lane alignment at each time step and selecting promising lane information.

2. Related Work

  • Interaction models commonly aggregate agent hidden states through pooling, graph message passing, or attention, and this work uses attention mechanisms for interaction modeling.Figure 2 shows agent-motion and scene encodings fused by an attention-based global interaction graph.Source: §2 · p. 2, Figure 2 · p. 3
  • Generative models produce multiple predictions but do not directly estimate mode likelihoods; MDNs can model mode distributions yet may collapse to one mode under single-ground-truth supervision.The paper explores a Laplacian MDN with winner-takes-all training; it also avoids time-consuming ensembling used to reduce excessive predictions.Source: §2 · pp. 2–3
  • CNN-based scene-context extraction can miss fine-grained lane geometry and traffic rules, while unified vectorization represents trajectories and HD-map scenes in a common form.The paper adopts this vectorized representation for scene-aware prediction.Source: §2 · p. 3
  • Prior lane-guided methods use maneuver proposals, feasible goals, fixed trajectory anchors, or historical trajectories, but are categorized as spatially dense and do not fully model temporal uncertainty.Such methods may leave the prediction module to filter irrelevant scene information, a challenge in complex intersections.Source: §2 · p. 3
  • LAformer estimates the likelihood of lane alignment at each time step and selects highly potential lane information rather than relying on a decoder to infer motion–lane relations implicitly.Its framework feeds aligned motion and candidate-lane information to the decoder, which predicts multimodal trajectories.Source: §2 · p. 3, Figure 2 · p. 3

3. Method

LAformer predicts future trajectories from observed agent histories and an HD map, combining agent–scene encoding, temporally dense lane selection, multimodal decoding, and second-stage motion refinement.

3.1. Problem formulation

  • The task predicts target-agent future trajectories from observed agent histories and an HD map, representing trajectories and lane centerlines as vectors.Source: §3.1 · pp. 3–4

3.2. Agent motion and scene encoding

  • GIG encodes trajectory and lane vectors with MLP and GRU layers, then cross-attends between agent and lane encodings and applies self-attention among agents.Source: §3.2 · p. 4

3.3. Temporally dense lane-aware estimation

  • At each future step, lane scoring estimates lane probabilities; top-k segments and their scores inform cross-attention, with the nearest-to-truth lane as the binary-classification positive.Source: §3.3 · p. 4

3.4. Multimodal conditional decoder

  • A latent normal vector conditions a Laplacian mixture decoder that predicts multiple trajectories, trained with Winner-Takes-All regression and mode-classification losses.Source: §3.4 · pp. 4–5

3.5. Motion refinement

  • The second stage encodes the complete trajectory and predicts offsets from the first-stage predictions, using offset and turning-angle losses optimized with Winner-Takes-All.Source: §3.5 · p. 5

4. Experiments

Experiments on Argoverse 1 and nuScenes report strong benchmark performance, while ablations, sensitivity tests, computational measurements, and qualitative examples examine LAformer’s components and operating characteristics.

4.1. Experimental setup

  • Evaluation uses nuScenes six-second and Argoverse 1 three-second prediction horizons, with official data partitions and online test servers.The target agent’s observed history is two seconds in both datasets; sampling is 2 Hz for nuScenes and 10 Hz for Argoverse 1.Source: §4.1 · p. 5

4.2. Quantitative results and comparison

  • LAformer achieves state-of-the-art ADE and FDE on the Argoverse 1 validation set and performs on par with the runner-up HiVT on the test set.Table 1 covers the validation and online test sets with K = 6.Source: §4.2 · p. 5, Table 1 · p. 6
  • On nuScenes, LAformer is slightly behind FRM in ADE but outperforms other lane-based models and shows stronger generalization across the two benchmarks.The authors attribute the possible performance difference from FRM to its relationship reasoning versus LAformer’s attention-based interaction modeling and focus on scene constraints.Source: Table 2 · p. 6, §4.2 · p. 6

4.3. Ablation study

  • Ablations show gains from second-stage refinement and temporally dense lane estimation over spatially dense estimation.The refinement module yields about 3% lower FDE, while temporally dense estimation reduces ADE by about 4% and FDE by about 8% versus spatially dense estimation.Source: §4.3 · p. 6, Table 3 · p. 6
  • Including both second-stage angle and offset losses improves ADE and FDE by about 2%, while adding latent variable z improves performance by less than 0.5 cm.The latent-variable ablation samples z from a multivariate normal distribution with dimension 2.Source: §4.3 · p. 7, Table 4 · p. 7, Table 5 · p. 7

4.4. Sensitivity analysis of the hyper-parameters

  • The best lane-segment count varies by stage, and larger k can reduce performance by including irrelevant segments; changing loss weights produces only marginal differences.The first-stage results improve up to k = 3 before declining, while k = 2 works better than 3 in the second stage; ADE6 fluctuates within 1 cm across tested loss weights.Source: §4.4 · p. 7, Table 6 · p. 7, Table 7 · p. 7

4.5. Computational performance

  • LAformer has 2,645K parameters and takes about 115 ms per scenario with an average of 12 agents, making its speed comparable to LTP and faster than DenseTNT and PGP.The authors describe it as close to real-time use cases at 10 Hz, though inference speed is not a main strength relative to HiVT.Source: §4.5 · p. 7, Table 8 · p. 7

4.6. Qualitative results

  • In intersection examples, LAformer produces more accurate right-turn and acceleration predictions, while the complete model preserves diversity and accuracy when the lane-aware module is ablated.On nuScenes, its multimodal predictions also cover straightforward driving and complex intersections, with modes aligned to lane segments.Source: §4.6 · pp. 7–8, Figure 3 · p. 8, Figure 4 · p. 8

5. Conclusion

LAformer combines temporally dense lane awareness with second-stage motion refinement for multimodal trajectory prediction. It outperforms other models on Argoverse 1 and nuScenes, and ablations and sensitivity studies verify both modules’ efficacy.

5. Conclusion

  • LAformer outperforms other models on the Argoverse 1 and nuScenes motion forecasting benchmarks.Source: §5 · p. 8
  • Extensive ablation and sensitivity studies verify the efficacy of LAformer’s lane-aware and motion refinement modules.Source: §5 · p. 8
  • LAformer takes observed trajectories and an HD map as input and outputs multimodal predicted trajectories using a Transformer-based temporally dense lane-aware module and second-stage motion refinement.Source: §5 · p. 8

Related papers

Loading 2302.13933v1…