Learning Lane Graph Representations for Motion Forecasting

• HD-map geometry and semantics inform actor behavior, making effective map use essential for plausible and accurate motion forecasts.Source: §1 · p. 1
• Heuristic map-based paths can miss rare safety-critical non-compliant behavior, while raster-CNN approaches encode topology implicitly and lose information [14] [3].Source: §1 · pp. 1–2
• The proposed LaneGCN constructs a lane graph from vectorized maps to avoid information loss and capture complex topology and long-range lane dependencies.The approach uses raw map data to build the lane graph and extracts its map features with LaneGCN.Source: §1 · p. 2, Figure 1 · p. 2
• The model represents actors and lanes as graph nodes and uses spatial attention and LaneGCN to model actor-to-lane, lane-to-lane, lane-to-actor, and actor-to-actor interactions.Actor features come from a 1D CNN, while lane features come from LaneGCN.Source: §1 · p. 2
• Experiments on the large-scale Argoverse motion forecasting benchmark show significant improvements over the state of the art.Source: §1 · p. 2

Map Representations

• Prior HD-map work parameterizes lane boundaries as polylines, lane graphs as DAGs, lane types graphically, or road topology with undirected graphs.Source: §2 · pp. 2–3

Learning Map Representations for Autonomy

• Autonomy work rasterizes map layers or bird’s-eye-view semantics [14] [3], while vector methods encode centerline offsets or lane features with 1D CNNs and LSTMs [34].Source: §2 · p. 3
• Unlike VectorNet [16], LaneGCN uses a sparse map-topology lane graph, task-specific multi-type and dilated operators, and polyline-segment rather than polyline-level map nodes.Nodes in different polylines can interact through dilated connections.Source: §2 · p. 3

Graph Convolutional Networks

• GCNs extend grid convolutions to graph-defined neighbors via adjacency matrices; LaneGCN adds multiple adjacency matrices and multi-scale dilated convolutions to capture lane-graph topology and long-range dependencies.Source: §2 · p. 3

3 Lane Graph Representations for Motion Forecasting

• The four-module architecture uses ActorNet and MapNet to encode actors and lanes, four interaction blocks to fuse their features, and a prediction header to produce multimodal trajectories.Fusion proceeds through actor-to-lane, lane-to-lane, lane-to-actor, and actor-to-actor interactions; LaneGCN handles lane-to-lane propagation, while spatial attention layers handle the other interactions.Source: §3 · pp. 3–4, Figure 2 · p. 4

3.1 ActorNet: Extracting Traffic Participant Representations

• ActorNet represents each observed BEV trajectory as 2D displacements, zero-pads shorter sequences, and concatenates a padding mask to form a 3 × T input.The mask indicates whether each time step is padded; the temporal feature at t = 0 is used as the actor feature.Source: §3.1 · p. 4
• ActorNet uses a 1D CNN with three convolution groups at different scales, followed by feature-pyramid fusion and a residual block to produce actor features.Each group has two residual blocks, the first block has stride 2, and layers use kernel size 3, 128 output channels, layer normalization, and ReLU.Source: §3.1 · pp. 4–5

3.2 MapNet: Extracting Structured Map Representation

• MapNet builds a lane graph from vectorized map data and applies LaneGCN to compute map features, rather than encoding maps as raster images with 2D convolutions.Source: §3.2 · p. 5
• The lane graph represents each lane by an ordered sequence of 2D BEV centerline points and records predecessor, successor, left-neighbor, and right-neighbor connections.Fig. 3 illustrates the centerlines and corresponding lane nodes with these relationships.Source: Map Data · p. 5, Figure 3 · p. 5
• Lane centerlines and connectivity provide geometric and semantic map information relevant to motion forecasting because vehicles generally plan routes by reference to them.Source: Map Data · p. 5

Map Data:

• Instead of rasterizing maps, the model builds a fine-resolution lane graph whose nearby nodes can provide accurate map information for any actor location.Source: Lane Graph Construction · pp. 5–6

Lane Graph Construction:

• Each lane node is a centerline segment between consecutive points, with predecessor, successor, left-neighbour, and right-neighbour connections represented by separate adjacency matrices.Predecessor and successor encode travel direction; lateral neighbours are the spatially closest nodes on adjacent lanes.Source: Lane Graph Construction · p. 6
• Vanilla graph convolution is unsuitable because node features are unclear, one Laplacian loses connection direction, and long-range dependencies are difficult to model.These limitations motivate the LaneConv operator.Source: LaneConv Operator · p. 6

LaneConv Operator:

• Lane-node features encode each segment’s shape, including size and orientation, as well as its center location.Source: LaneConv Operator · p. 6
• LaneConv uses connection-specific adjacency and weight matrices to propagate information along forward, backward, and cross-lane neighbours, improving over vanilla graph convolution.Source: LaneConv Operator · p. 7
• For long-range lane dependencies, dilated LaneConv propagates information k steps through predecessor and successor connections; these sparse powers can be computed efficiently.Dilation addresses the distances high-speed actors may travel over forecasting horizons of several seconds, and is applied only along lanes.Source: LaneConv Operator · p. 7
• Combining LaneConv with multiple dilation sizes yields a multi-scale operator used to build LaneGCN.Source: LaneGCN · p. 7

LaneGCN:

• LaneGCN stacks four residual blocks, each combining LaneConv(1,2,4,8,16,32), a linear layer, and a residual connection.The architecture is shown in Fig. 4.Source: LaneGCN · pp. 7–8, Figure 4 · p. 8
• All LaneGCN layers have 128 feature channels.Source: Figure 4 · p. 8, LaneGCN · p. 8
• Layer normalization [2] and ReLU follow each LaneConv and linear layer.Source: LaneGCN · p. 8

3.3 FusionNet

• FusionNet stacks actor-to-lane (A2L), lane-to-lane (L2L), lane-to-actor (L2A), and actor-to-actor (A2A) modules to capture information flows between actors and lane nodes.The resulting actor features are used by the prediction header for motion forecasting.Source: §3.3 · p. 8
• A2L adds real-time traffic information to lane nodes, L2L propagates it over the lane graph, L2A returns updated map features to actors, and A2A models actor interactions.Source: §3.3 · p. 8
• L2L uses a LaneGCN with the same architecture as MapNet, while spatial attention is used for A2L, L2A, and A2A.The attention layer operates in the same way across those three modules.Source: §3.3 · p. 8
• The A2L, L2A, and A2A context-distance thresholds are 7, 6, and 100 meters, respectively; each module uses two residual blocks and 128 output feature channels.Context nodes are selected by their distance from the node being updated.Source: §3.3 · p. 9

3.4 Prediction Header

• For each actor, the header predicts K future trajectories and confidence scores using separate regression and classification branches.The regression branch applies a residual block and linear layer to predict K sequences of BEV coordinates.Source: §3.4 · p. 9
• For each mode, the classification branch derives a distance embedding, combines it with the actor feature, and outputs a confidence score.An MLP produces the distance embeddings; a residual block and linear layer produce the K confidence scores.Source: §3.4 · p. 9

3.5 Learning

• The differentiable model is trained end-to-end using the sum of classification and regression losses, with α = 1.0.Source: §3.5 · p. 9
• For each actor, the positive trajectory is the prediction with the minimum final displacement error against ground truth.Source: §3.5 · p. 9
• Classification uses a max-margin loss, while regression applies smooth ℓ1 loss to all predicted time steps.Source: §3.5 · p. 9

Dataset:

• Argoverse contains over 30K Pittsburgh and Miami scenarios, and the task predicts agents’ locations over a 3-second horizon from 2 seconds of input.The train, validation, and test splits contain 205942, 39472, and 78143 sequences, respectively, with no geographical overlap; the model uses actor trajectories and lane centerlines with connectivity, without other preprocessing or rasterized drivable-area and ground-height maps.Source: Dataset · p. 10

Metrics:

• The evaluation uses minADE and minFDE at K=1 and K=6, with Miss Rate also reported for leaderboard comparisons; the server ranks by minFDE at K=6.Miss Rate counts best-mode predictions whose final location is more than 2.0 meters from ground truth.Source: Metrics · p. 10, Comparison with the state-of-the-art · p. 11

Comparison with the state-of-the-art:

• LaneGCN significantly outperforms four top leaderboard entries and two official baselines across all reported metrics on the Argoverse test set.The comparison was submitted at the time of ECCV submission (2020/03/15); the leaderboard ranks methods by minFDE at K=6.Source: Table 1 · p. 11, Comparison with the state-of-the-art · p. 11

Importance of each module:

• Adding the model’s modules improves performance; actor-to-lane and lane-to-lane flow adds useful traffic information, while map-mediated actor interactions reduce the incremental gain from actor-to-actor fusion.The authors attribute the smaller gain from adding A2A after A2L, L2L, and L2A to information from different actors propagating over the lane graph.Source: Importance of each module · pp. 11–12, Table 2 · p. 12
• Residual blocks, multi-type connections, and dilation each improve lane-graph ablations over vanilla graph convolution, with multi-type connections and dilation significantly boosting performance.The residual block adds about 7% parameters and facilitates training; the final LaneConv combines the evaluated components.Source: Lane Graph Operators · p. 12, Table 3 · p. 12

Qualitative Results:

• Across four hard test cases, LaneGCN follows lane topology in right-turn and unprotected-left-turn examples and predicts greater deceleration at a stop, but no method handles extreme acceleration well.Ground truth is not shown because test labels are unavailable; the authors suggest adding traffic lights and signs to the lane graph as future work.Source: Qualitative Results · pp. 12–13

5 Conclusion

• The model significantly outperforms the state of the art on the large-scale Argoverse motion forecasting benchmark.The conclusion reports no numerical comparison.Source: §5 · p. 13
• The model replaces rasterized map input with a lane graph built from vectorized map data, using LaneGCN to extract map topology features.Source: §5 · p. 13
• Spatial attention and LaneGCN fuse actor and lane information through a complete set of actor-map interactions.Source: §5 · p. 13
• Future work will explore incorporating other map data.Source: §5 · p. 13

Appendix

• ActorNet uses a 1D CNN and Feature Pyramid Network to extract temporal features and merge multi-scale features, while MapNet uses a Lane Graph Network built from LaneConv operators.MapNet stacks four multi-scale LaneConv residual blocks to extract lane topology features.Source: Appendix · pp. 13–14
• The Actor-Map Fusion Cycle stacks four fusion networks—A2L, L2L, L2A, and A2A—where A2L, L2A, and A2A use two attention residual blocks and L2L is another LGN.The updated actor features feed the Prediction Header, which produces multimodal trajectories and their confidence scores.Source: Appendix · p. 14