PaperlayerView exact revision on arXiv

Source-linked AI summary

Generation of Vectorized Maps Beyond Vehicle View

Clara Gomez, Alberto Jaenal, Antonio Artuñedo, Jorge Godoy, Jorge Villagra

arXiv:2609.07511v1cs.ROcs.AIcs.LG

TL;DR

Online vectorized maps are limited to the sensor-observed region, leaving beyond-view road structure unrepresented. The paper introduces BeyondFormer to predict vectorized continuations from in-view maps. Experiments report feasibility in simplified road configurations, alongside remaining limitations in map accuracy and scope.

  • Problem

    Onboard-sensor vectorized mapping does not provide road information beyond the observable region, leaving beyond-view vectorized map generation unexplored.

    Source: §I · p. 1

  • Method

    BeyondFormer encodes in-view maps and their endpoints, then uses a Transformer decoder with geometry and topology heads to generate map continuations.

    Source: C. System Overview · p. 3, F. Multi-head Map Decoder · p. 4

  • Results

    Experiments show beyond-view map generation is feasible on simplified road configurations; BeyondFormer outperforms the classical baseline with an overall average error decrease of 3.14m.

    Source: B. Beyond-view map generation… · p. 5, §V · p. 6

  • Takeaways & Limitations

    BeyondFormer provides a reproducible starting point for studying the task and a foundation for research toward more complex driving scenarios.

    Source: §V · p. 6

    Evaluation is limited to controlled scenarios with simple road layouts and assumes accurate in-view maps; complex intersections, roundabouts, and robustness remain for future work.

    Source: A. Limitations and Future Work · p. 6

Abstract

from arXiv · show

Autonomous driving relies on High Definition (HD) maps for safe navigation. Traditional HD maps construction is costly in hardware, data and human resources, which together with its update limitations hinders scalability. Recent works have proposed online alternatives for HD vectorized mapping from onboard sensors. However, sensor field of view is limited, and the range of the reconstructed maps ahead of the vehicle is insufficient for safe planning. This paper aims to address this limitation by proposing the novel beyond-view vectorized map generation problem: given vectorized maps of the area sensed by the vehicle (in-view), to generate plausible map continuations. To experimentally assess its feasibility, we propose BeyondFormer, which, to the best of out knowledge, is the first work designed towards beyond-view map generation. Given the novelty of the problem, we generate the first dataset specifically designed for it and evaluate the proposed approach. The results demonstrate consistent performance across diverse scenarios, establishing learning-based methods as a promising direction for map forecasting in autonomous driving. Beyond demonstrating the feasibility of the task, we provide an extensive discussion of the method's limitations and identify key future research directions for scaling it to more complex driving conditions. Code is available at https://git-autopia.car.upm-csic.es/beyondformer.

I. INTRODUCTION

Traditional HD maps are costly to build and maintain, while onboard-sensor mapping cannot represent roads beyond the sensor range. The paper formulates beyond-view vectorized map generation and introduces BeyondFormer and a dataset to study it.

I. INTRODUCTION

  • Online vectorized mapping remains limited to the observable region, leaving road structure beyond the sensor field of view unexplored in vectorized representations.Traditional HD maps support long-range planning but are costly to construct and frequently need updates.Source: §I · p. 1
  • BeyondFormer frames beyond-view generation as predicting plausible vectorized map continuations from in-view maps, using a Transformer and MLP encoder with geometry and auxiliary topology heads.The model is evaluated as an exploratory first step, with controlled experiments intended to analyze its behavior and limitations rather than optimize performance.Source: §I · p. 1
  • The authors introduce the novel task and identify their work as the first to generate vectorized road representations beyond the vehicle’s sensed area.They describe it as a first step toward beyond-view map generation.Source: §I · p. 1
  • The authors release in-view and beyond-view map samples from 3DHD CityScenes because no existing dataset matched the task’s requirements.BeyondFormer is evaluated on the resulting dataset to analyze its continuation capability.Source: §I · p. 1

II. RELATED WORK

Prior work maps visible road structure or forecasts future views, but does not explicitly generate vectorized road geometry and topology beyond the sensor field of view. This paper studies that distinct task for autonomous driving.

Related work overview

  • Learning-based driving approaches are grouped into in-view vectorized map reconstruction and beyond-view trajectory or world-model generation.Both groups seek to reduce dependence on offline maps by using sensor observations and structural priors.Source: §II · p. 2

In-view HD map reconstruction

  • In-view mapping methods convert sensor readings into vectorized BEV maps; P-MapNet and StreamMapNet extend prediction range but remain within the observable region.P-MapNet uses SD and HD map encoders, while StreamMapNet uses a memory buffer to capture temporal information.Source: §II · p. 2

Beyond-view generation

  • Trajectory prediction and video or world-model generation forecast motion or pixel-level scene evolution, not vectorized road geometry and topology beyond observed regions.Pixel prediction also adds a computationally intensive process to vectorized map generation.Source: §II · p. 2
  • Outdoor beyond-view mapping faces substantial structural variability, including multiple lanes and directions, roundabouts, and lane splits.The paper identifies its study as the first, to the authors’ knowledge, to address beyond-view vectorized map generation for autonomous driving.Source: §II · p. 2

III. METHOD

The task infers road continuation outside the sensor field of view from a partial map. The method represents lane segments as normalized cubic Bézier curves and predicts them autoregressively.

A. Problem formulation

  • The task assumes a perfect in-view map covering approximately 70m around the vehicle and predicts plausible centerline continuations for another 100–150m.The goal is to extend the vehicle’s horizon beyond the sensed road structure.Source: A. Problem formulation · p. 2

B. In-view input map and ground truth map

  • Input samples contain lane centerline segments within a radius around a global coordinate; ground truth covers subsequent trajectory coordinates up to distance D.Centerlines are normalized to vehicle position and orientation, and lane lengths are unified to address sample irregularities.Source: B. In-view input map and ground truth… · pp. 2–3
  • Each lane centerline segment is represented as a cubic Bézier curve with four control points, allowing curves with up to two inflection points.This converts variable-length point sequences into a representation suited to learning-based algorithms.Source: B. In-view input map and ground truth… · p. 3
  • The ground truth groups new centerline segments by generation step, and the map continuation is produced autoregressively from those sets.Figure 2 contrasts raw samples with processed lanes, which offer a more balanced and realistic representation.Source: Figure 2 · p. 2, B. In-view input map and ground truth… · p. 3

C. System Overview

BeyondFormer encodes in-view maps and travel-direction endpoints separately, then uses a Transformer decoder to generate beyond-view maps with geometry and topology heads. A final fine-tuning step smooths predictions and checks connectivity.

C. System Overview

  • The system encodes the in-view map with a Transformer and travel-direction endpoints with an MLP before decoding their feature representations.The decoder uses cross-attention to combine the two input representations.Source: C. System Overview · p. 3
  • A geometry head decodes the beyond-view map, while an auxiliary topology head learns lane connectivity.The heads follow a Transformer decoder that processes the encoded inputs.Source: C. System Overview · p. 3
  • After end-to-end training, an additional fine-tuning step smooths predictions and checks their connectivity.This stage follows the geometry and topology prediction pipeline.Source: C. System Overview · p. 3

D. In-view map Encoder

The in-view map encoder embeds Bézier control points and uses positional encoding and Transformer blocks to encode lane geometry. A separate MLP encodes the endpoints of the last lane segments in the direction of travel to guide prediction.

D. In-view map Encoder

  • The Transformer encoder maps the in-view map’s Bézier control points into d_model embeddings, adds sinusoidal positional encoding, and uses self-attention blocks to encode lane geometry.The positional encoding represents sequence position; the encoder blocks extract global geometry.Source: D. In-view map Encoder · p. 3

E. End points Encoder

  • A three-layer MLP encodes the endpoints of the last lane segments in the direction of travel as features that guide where prediction starts.The endpoints are represented as d_model-sized embeddings.Source: E. End points Encoder · p. 3

F. Multi-head Map Decoder

The multi-head decoder combines encoded in-view-map and endpoint features with generation-step inputs to predict map geometry and lane connectivity. Predictions are generated autoregressively, with fine-tuning applied to ensure lane connectivity.

F. Multi-head Map Decoder

  • The decoder uses self-attention to refine predictions and cross-attention to fuse in-view-map features, endpoint features, and its own predictions.A geometry head produces Bézier curve points, while an auxiliary topology head predicts lane connectivity as a learned adjacency matrix.Source: F. Multi-head Map Decoder · p. 4
  • Scheduled training shifts from using true previous lane sets to generated sets, supporting autoregressive prediction over N generation steps.At inference, the model sequentially predicts the next lane set from its own past predictions.Source: F. Multi-head Map Decoder · p. 4

G. Fine tuning

  • Fine-tuning ensures lane connectivity by matching generated lane starts to the closest generated lane ends.Source: G. Fine tuning · p. 4

H. Loss Function

The loss combines geometry, topology, and coupling objectives. Geometry supervision covers curve shape, lane length, curvature, and continuity; topology and coupling losses supervise connectivity and consistency between the model heads.

H. Loss Function

  • The geometry loss combines vector, distance, curvature, and continuity losses to supervise matched centerline points, lane length, curvature, and lane-section connectivity.Ground-truth and predicted Bézier curves are discretized into K points; logarithmic curvature avoids loss explosion for straight lanes.Source: H. Loss Function · p. 4
  • The topology loss applies binary cross-entropy with logits to the ground-truth adjacency matrix, while the coupling loss aligns adjacent lane pairs across the auxiliary and main heads.Coupling minimizes the geometric distance between topologically adjacent lane end-to-start points.Source: H. Loss Function · p. 4

IV. EXPERIMENTAL EVALUATION

The evaluation presents BeyondFormer as a proof of concept for beyond-view map generation, using controlled experiments and ablations to assess behavior and challenges. Experiments use simplified samples from 3DHD CityScenes and compare curve predictions with common geometric metrics.

IV. EXPERIMENTAL EVALUATION

  • The evaluation frames BeyondFormer as a proof of concept and uses quantitative and qualitative results to assess feasibility.The exploratory study emphasizes controlled settings and ablations to analyze model behavior and identify challenges and limitations.Source: §IV · p. 4, A. Experimental setup · p. 4

A. Experimental setup

  • Experiments use 3,608 training, 171 validation, and 104 test samples from 3DHD CityScenes, excluding roundabouts and intersections.The in-view radius is R = 70m and the ground-truth distance is D = 150m.Source: A. Experimental setup · p. 4
  • The evaluation compares BeyondFormer with a classical baseline that extends Bézier curves using endpoint tangent and second-derivative matching at a fixed length.BeyondFormer has approximately 8M learnable parameters and was trained for 10k epochs with N = 4 generation steps.Source: A. Experimental setup · pp. 4–5
  • RMSE, Chamfer Distance, and Fréchet Distance measure pointwise deviation, bidirectional geometric similarity, and ordered curve coupling cost, respectively.Source: A. Experimental setup · p. 5

B. Beyond-view map generation performance

BeyondFormer achieves lower geometric error than the classical baseline, while qualitative outputs show better lane curvature but persistent difficulty with smooth parallel lanes and road splits. Errors rise with autoregressive horizon, especially for the baseline.

B. Beyond-view map generation performance

  • BeyondFormer predicts overall lane curvature, length, and distance correctly, but struggles to generate smooth parallel lanes; both methods miss road splits.The baseline produces cleaner lines but cannot predict curvature and direction changes.Source: B. Beyond-view map generation… · p. 5
  • 3.14m overall average error decrease over the baseline supports BeyondFormer's stronger geometric precision, with below 2% error over the approximate 120m, four-lane prediction length.The authors note that absolute errors still leave room for improvement before operational performance.Source: B. Beyond-view map generation… · p. 5, Table I · p. 5
  • RMSE increases with autoregressive prediction horizon for both methods, with a larger increase for the classical baseline.Each prediction step relies on previous predictions.Source: B. Beyond-view map generation… · p. 5, Table II · p. 5

C. Ablation study

Ablations report improvements from added geometry losses and model components, and inference measurements indicate real-time potential. The paper presents BeyondFormer as a reproducible starting point, while emphasizing feasibility rather than deployment readiness.

C. Ablation study

  • Adding Ldist, Lcurv, and Lcont to Lvec reduces RMSE by 23.60%, CD by 30.66%, and FD by 24.29%.The loss-function ablation considers geometry loss.Source: C. Ablation study · p. 5, Table III · p. 5
  • Adding the end-points encoder, auxiliary topology head, and output fine-tuning reduces RMSE by 16.97%, 38.55%, and 12.90%, respectively.These component additions guide the prediction.Source: C. Ablation study · p. 5, Table IV · p. 5

D. Real-time Evaluation

  • Inference averages 15.3 ± 1.2 ms with 221 MB peak GPU memory at batch size 1 on an NVIDIA GeForce RTX 4060.The authors describe these measurements as showing the method's real-time potential.Source: D. Real-time Evaluation · p. 5

V. CONCLUSIONS

  • BeyondFormer is a Transformer-based, reproducible starting point intended to demonstrate feasibility and support study of more complex driving scenarios, not deployment-ready performance.The conclusion describes the task as feasible under simplified road configurations while identifying challenges for reliable forecasting.Source: §V · p. 6

A. Limitations and Future Work

The evaluation is deliberately limited to controlled, simple road layouts and assumes accurate in-view maps. BeyondFormer still makes finer geometric and topological errors, motivating richer structural reasoning and representations.

A. Limitations and Future Work

  • The evaluation uses simple road layouts and assumes accurate in-view maps, leaving complex intersections, roundabouts, and robustness for future study.Controlled scenarios were chosen to isolate the proposed task's core challenges.Source: A. Limitations and Future Work · p. 6
  • Although BeyondFormer generates coherent continuations in simple scenarios, its maps remain inaccurate at finer geometric and topological levels.It struggles with lane splits and merges and often produces cluttered lane structures; the authors point to richer structural reasoning as a need for complex topology.Source: A. Limitations and Future Work · p. 6
  • Future directions include structural priors, improved losses, lane-aware representations, and richer HD map elements such as lane widths, road boundaries, and semantics.The authors mention SD map encoders as one possible structural prior.Source: A. Limitations and Future Work · p. 6
Loading 2609.07511v1…