PaperlayerView exact revision on arXiv

Source-linked AI summary

LaPred: Lane-Aware Prediction of Multi-Modal Future Trajectories of Dynamic Agents

ByeoungDo Kim, Seong Hyeon Park, Seokhwan Lee, Elbek Khoshimjonov, Dongsuk Kum, Junsoo Kim, Jeong Soo Kim, Jun Won Choi

arXiv:2104.00249v1cs.CVcs.LG

TL;DR

LaPred tackles multi-modal future-trajectory prediction by explicitly incorporating lane-level static context and nearby-agent interactions. It extracts lane-conditioned features, learns self-supervised lane attention, and generates multiple trajectories, achieving state-of-the-art performance on some nuScenes and Argoverse metrics.

  • Problem

    Future motion is multi-modal and depends on agent intention, static surroundings, and interactions, while existing models offer limited explicit control over lane-conditioned predictions.

  • Method

    LaPred extracts instance-level per-lane joint features and uses self-supervised reference-lane identification to attention-weight them before generating multiple trajectories.

  • Results

    LaPred achieves state-of-the-art performance on some nuScenes and Argoverse benchmark metrics and produces reasonable predictions in challenging traffic scenarios.

  • Takeaways & Limitations

    Lane information provides a structured basis for generating diverse trajectories that reflect lane constraints and important traffic modes.

  • Takeaways & Limitations

    The formulation assumes a unique reference lane always exists among the candidates and that conditioned future motion depends solely on that lane and its associated nearby agent.

Abstract

from arXiv · show

In this paper, we address the problem of predicting the future motion of a dynamic agent (called a target agent) given its current and past states as well as the information on its environment. It is paramount to develop a prediction model that can exploit the contextual information in both static and dynamic environments surrounding the target agent and generate diverse trajectory samples that are meaningful in a traffic context. We propose a novel prediction model, referred to as the lane-aware prediction (LaPred) network, which uses the instance-level lane entities extracted from a semantic map to predict the multi-modal future trajectories. For each lane candidate found in the neighborhood of the target agent, LaPred extracts the joint features relating the lane and the trajectories of the neighboring agents. Then, the features for all lane candidates are fused with the attention weights learned through a self-supervised learning task that identifies the lane candidate likely to be followed by the target agent. Using the instance-level lane information, LaPred can produce the trajectories compliant with the surroundings better than 2D raster image-based methods and generate the diverse future trajectories given multiple lane candidates. The experiments conducted on the public nuScenes dataset and Argoverse dataset demonstrate that the proposed LaPred method significantly outperforms the existing prediction models, achieving state-of-the-art performance in the benchmarks.

1. Introduction

LaPred addresses multi-modal trajectory prediction by explicitly modeling lane-level environmental and interaction context. It uses self-supervised lane attention and instance-level lane features, achieving strong results on nuScenes and Argoverse.

  • 1. Introduction: LaPred addresses the need for multiple plausible trajectory samples because agent motion depends on intentions, static surroundings, and interactions with other agents.Different maneuvers and lane changes can produce a multi-modal future distribution.
  • 1. Introduction: LaPred predicts diverse future trajectories using instance-level lane entities and per-lane features relating lanes, target motion, and nearby-agent interactions.The method is designed to represent distinct future-trajectory modes associated with different lanes.
  • 1. Introduction: The model is evaluated on public Argoverse and nuScenes datasets and reports significant gains over existing methods.The experiments target complex traffic scenarios with semantic-map information.
  • 1. Introduction: Self-supervised reference-lane identification guides attention toward lanes likely to be followed by the target agent.The auxiliary task supplies lane-aware attention for prediction.
  • 1. Introduction: LaPred achieves state-of-the-art performance for some categories of the Argoverse and nuScenes benchmarks.This is the paper’s stated benchmark-level outcome.

2. Related Works

Prior work models interactions, scene context, and trajectory multimodality using grids, global pooling, graphs, rasterized maps, and generative models. LaPred instead jointly represents each lane with its associated agents and uses auxiliary lane identification to focus prediction.

  • 2. Related Works: Existing interaction models include grid-based pooling, global pooling, and spatiotemporal graph methods, each imposing different spatial or processing assumptions.Grid methods require hand-crafted coordinate systems and tensor resolutions, whereas global methods can handle arbitrary agent counts.
  • 2. Related Works: Scene-context approaches use sensor-derived images, rasterized semantic maps, or instance-level representations to model static surroundings.These approaches differ in how they preserve spatial relationships between trajectories and the scene.
  • 2. Related Works: Rasterized semantic-map methods encode map information as image channels but make trajectory–lane relationships difficult to reason about.Instance-level scene representations address this relationship more directly.
  • 2. Related Works: LaPred jointly represents each lane and its associated agents, yielding a simpler architecture while achieving empirical prediction gains.The method also adds an auxiliary task to attend to lanes the target agent tries to follow.
  • 2. Related Works: Generative trajectory methods often require large sample sets to cover low-probability but traffic-relevant modes, motivating more structured diversity mechanisms.The paper connects lane information with meaningful modes of the trajectory distribution.

3. Proposed Lane-Aware Multi-Modal Trajectory Prediction Method

LaPred formulates future motion as a mixture over candidate reference lanes, using lane-conditioned trajectory features and attention-weighted aggregation. An encoder–decoder then generates multiple future trajectories, under assumptions about lane identity and conditional interactions.

  • 3. Proposed Lane-Aware Multi-Modal Trajectory Prediction Method: The task is to predict a target agent’s future trajectory from its past trajectory, candidate lanes, and selected nearby-agent trajectories.Each lane candidate is represented by equally spaced coordinates of equal length.
  • 3. Proposed Lane-Aware Multi-Modal Trajectory Prediction Method: LaPred treats the future distribution as a mixture over mutually exclusive candidate reference lanes and aggregates lane-conditioned predictions using their posterior probabilities.The reference lane is assumed to exist among the candidates.
  • 3. Proposed Lane-Aware Multi-Modal Trajectory Prediction Method: The formulation assumes that exactly one candidate is the reference lane and that, conditioned on that lane, future motion depends solely on its lane and associated nearby agent.These are explicit modeling assumptions in the problem formulation.
  • 3. Proposed Lane-Aware Multi-Modal Trajectory Prediction Method: An encoder represents each lane-conditioned observation, and a decoder models the predictive distribution from which multiple trajectory samples are generated.The pipeline includes feature extraction, reference-lane identification, weighted aggregation, and trajectory decoding.

3.2. Structure of LaPred Network

LaPred extracts shared trajectory–lane features for candidate lanes, attends to them with learned weights, and decodes multiple future trajectories. Preprocessing selects connected lanes and influential nearby agents before this aggregation.

  • 3.2. Structure of LaPred Network: Lane candidates are selected near the target and extended along connected preceding and succeeding segments until a predefined length is reached.When more than N connected segments exist, only N lane instances are retained.
  • 3.2. Structure of LaPred Network: Preprocessing selects the nearest nearby agent in front of the target for each lane candidate as the most influential agent.Agents are searched within a fixed range around each lane candidate’s center point.
  • 3.2. Structure of LaPred Network: TFE separately encodes target history, lane geometry, and the selected nearby-agent trajectory with CNN–LSTM networks, then concatenates them into joint features.TFE parameters are shared across lane candidates, while the three encoders use different weights.
  • 3.2. Structure of LaPred Network: LA computes candidate attention weights and forms the joint representation by weighted feature combination, with hard selection evaluated as an alternative to soft selection.Missing candidates are represented with zero-vector inputs when fewer than N lanes are found.
  • 3.2. Structure of LaPred Network: MTP combines the LA output with target-history features and generates K hypotheses using partially shared generator networks to reduce overfitting.Only one non-shared branch is updated for a given input, while shared layers are updated across inputs.

3.3. Training Details

LaPred trains with prediction and reference-lane selection objectives, adding a lane-offset loss that encourages trajectories to remain close to the selected reference lane.

  • The total training objective combines trajectory prediction and cross-entropy reference-lane classification losses.The prediction loss is mean absolute error, while the classification loss selects the reference lane from candidates.
  • The position loss uses smooth L1 distance between predicted and ground-truth future positions.
  • LaPred adds a lane-offset loss because target agents tend to remain close to their reference lane.
  • The lane-offset loss penalizes predictions that are farther from the reference lane than the ground-truth trajectory.It uses the distance from a point to a lane and applies the penalty when the predicted distance exceeds the ground-truth distance.

4. Experiments

LaPred is evaluated on nuScenes and Argoverse using ablations, benchmark comparisons, and qualitative trajectory examples. Results indicate that lane information, nearby-agent trajectories, lane-selection loss, and soft feature aggregation improve prediction accuracy and lane-compliant multimodal forecasts.

  • Dataset and Metrics: The evaluation uses nuScenes and Argoverse HD-map datasets with ADE and FDE metrics for future-trajectory prediction.nuScenes predicts six seconds from two seconds of history, while Argoverse predicts three seconds from two seconds of history.
  • Ablation Study: Adding lane information produces the largest ablation improvement, while nearby-agent trajectories, lane-offset loss, and soft feature aggregation provide additional gains.Method E uses soft selection, and the ablation compares Methods A through E as successive additions to the baseline.
  • Ablation Study: Soft feature combining performs better than hard feature combining because considering diverse lane candidates is more helpful than selecting a single best lane candidate.The authors report a non-negligible gain for Method E over hard feature combining.
  • Quantitative Results: LaPred outperforms competing methods on most nuScenes metrics and improves over competitors on several Argoverse metrics, with comparable performance on the remaining reported measures.On nuScenes, it is slightly worse than MHA-JAM for FDE10, ADE15, and FDE15; on Argoverse, gains are reported for ADE5, FDE5, ADE6, FDE6, and ADE12.
  • Prediction Examples: LaPred follows provided lane structures more closely than the baseline, particularly improving prediction accuracy in the lateral direction.The baseline uses only the target agent’s past trajectory and does not use lane information.
  • Prediction Examples: With K = 5, LaPred generates multiple trajectories associated with different lane candidates, producing diverse hypotheses meaningful in a traffic context.Both methods generate five hypotheses, but LaPred associates them with different lane candidates while the baseline produces inadequate predictions without lane information.

5. Conclusions

LaPred uses instance-level lane information and self-supervised reference-lane identification to generate lane-compliant multi-modal trajectories. Experiments on nuScenes and Argoverse show state-of-the-art performance on some metrics.

  • 5. Conclusions: LaPred fuses per-lane trajectory features using weights learned from a self-supervised reference-lane identification task, then decodes multi-modal trajectories compliant with lane structure.The encoder represents surrounding lanes and agent trajectories, while the decoder generates future trajectories.
  • 5. Conclusions: State-of-the-art performance was achieved on some metrics in experiments conducted on the nuScenes and Argoverse datasets.The experiments also produced reasonable predictions in challenging traffic scenarios.

A. System Setup and Training Details

The setup specifies past and future trajectory inputs, lane-candidate configuration, and Adam optimization with validation-based learning-rate decay.

  • A. System Setup and Training Details: Training uses the time steps of past and future trajectories together with the specified lane-candidate settings.
  • A. System Setup and Training Details: Lane candidates use either 130m coverage with 0.5m spacing and M = 260 points, or 80m coverage with 1.0m spacing and M = 80 points.The 130m setting extends 100m forward and 30m backward; the 80m setting extends 50m forward and 30m backward.
  • A. System Setup and Training Details: The model is optimized with Adam at an initial learning rate of 0.0003, halved when validation loss plateaus for more than three epochs.

B. Network Architecture

The network architecture is organized into TFE, LA, and MTP blocks, whose detailed layer configurations are provided in Tables 4–6.

  • B. Network Architecture: Tables 4–6 provide the detailed architectures of the TFE, LA, and MTP blocks, respectively.Table 4 describes TFE layers using units, kernel size, stride, and padding notation.

C. Single-agent vs. Multi-agent features for employing nearby agents.

The experiment compares nearby-agent aggregation strategies for lane-aware features on the nuScenes validation set. LaPredSL treats one agent per lane, while LaPredML and LaPredM aggregate multiple agents with max-pooling.

  • C. Single-agent vs. Multi-agent features for employing nearby agents.: The nuScenes validation comparison evaluates LaPredSL against LaPredML and LaPredM for aggregating nearby agents in the TFE block.LaPredSL considers a single agent per lane; LaPredML and LaPredM aggregate multiple nearby agents per lane using max-pooling.
Loading 2104.00249v1…