Source-linked AI summary
OpenLane-V2: A Topology Reasoning Benchmark for Unified 3D HD Mapping
Huijie Wang, Tianyu Li, Yang Li, Li Chen, Chonghao Sima, Zhenbo Liu, Bangjun Wang, Peijin Jia, Yuting Wang, Shengyin Jiang, Feng Wen, Hang Xu, Ping Luo, Junchi Yan, Wei Zhang, Hongyang Li
TL;DR
Existing driving datasets often separate lane and traffic-signal perception, limiting representation of how signals relate to lanes. OpenLane-V2 presents a dataset and benchmark for scene-structure perception and topology reasoning, and evaluations report that centerline perception and topology predictions remain challenging. The dataset defines tasks and metrics for future research, while excluding planning and retaining the scale of its source datasets.
Problem
Separate lane and traffic-signal perception leaves limited representation of which traffic instructions apply to which lanes, especially when lane markings are absent.
Source: §1 · pp. 1–2Method
Results
Evaluations report that centerline detection and topology predictions remain challenging for the evaluated methods.
Source: §5.2 · p. 8Takeaways & Limitations
OpenLane-V2 provides tasks and metrics intended to support future research on scene-structure perception and reasoning.
Source: §6 · p. 9The dataset retains the scale of previous datasets and excludes planning, which requires movable-object knowledge for collision avoidance.
Source: §6 · p. 9
Abstract
from arXiv · showhide
Accurately depicting the complex traffic scene is a vital component for autonomous vehicles to execute correct judgments. However, existing benchmarks tend to oversimplify the scene by solely focusing on lane perception tasks. Observing that human drivers rely on both lanes and traffic signals to operate their vehicles safely, we present OpenLane-V2, the first dataset on topology reasoning for traffic scene structure. The objective of the presented dataset is to advance research in understanding the structure of road scenes by examining the relationship between perceived entities, such as traffic elements and lanes. Leveraging existing datasets, OpenLane-V2 consists of 2,000 annotated road scenes that describe traffic elements and their correlation to the lanes. It comprises three primary sub-tasks, including the 3D lane detection inherited from OpenLane, accompanied by corresponding metrics to evaluate the model's performance. We evaluate various state-of-the-art methods, and present their quantitative and qualitative results on OpenLane-V2 to indicate future avenues for investigating topology reasoning in traffic scenes.
1 Introduction
OpenLane-V2 addresses the separation of lane and traffic-signal perception by representing their topology, so scene understanding can account for which signals control which lanes. It combines 3D lane centerlines, traffic elements, and their relationships in a dataset and benchmark for scene-structure perception and reasoning.
1 Introduction
- Separate lane and traffic-signal datasets provide limited scene understanding, leaving ambiguity about lane guidance at intersections and which lanes a signal controls.The paper also notes that 2D lanelines can be insufficient for real-world requirements and that absent markings can leave vehicles without trajectory guidance.Source: §1 · pp. 1–2
- OpenLane-V2 unifies 3D lane centerlines, traffic elements, and topology relationships, with the OpenLane-V2 Score combining perception (DET) and reasoning (TOP).A lane–traffic-element correspondence is valid only when the traffic element controls that lane.Source: §1 · p. 2
- OpenLane-V2 is presented as the first autonomous-driving dataset focused on topology reasoning, with dedicated tasks and metrics for benchmarking scene understanding.The introduction says the benchmark builds on existing benchmarks and includes images collected from cities worldwide.Source: §1 · p. 3
- The dataset includes 2.1M instance-level annotations and 1.9M positive topology relationships, with annotations carefully validated.A development kit, framework plug-ins, test server, and leaderboard are also described as maintained resources.Source: §1 · p. 3
2 Related Work
Prior work spans 2D and 3D lane annotation, traffic-element recognition, and scene understanding, but the paper identifies limited coverage of lane–traffic-element relationships. OpenLane-V2 extends these lines of work with surrounding-area 3D lane centerlines and annotations of topology relationships.
2.1 3D Lane Detection
- OpenLane-V2 extends 3D lane annotation beyond visible front-view lanelines to conceptual centerlines around the vehicle, providing trajectory guidance in all directions.Earlier lane datasets largely label 2D lanelines; OpenLane was the first large-scale, real-world 3D laneline dataset.Source: §2.1 · p. 3
2.2 Traffic Element Recognition
- Earlier traffic-element datasets mainly emphasize object location, whereas OpenLane-V2 adds topology relationships between traffic elements and lanes.The table compares datasets by images, classes, annotations, tracking, and whether lane–traffic-element correspondences are annotated.Source: §2.2 · p. 4, Table 1 · p. 4
2.3 Scene Understanding
- Prior scene-understanding datasets and relationship tasks cover limited 2D object sets or movable-agent behavior, while this work emphasizes static-scene understanding that provides trajectory information.The related work contrasts this focus with datasets on human-object interactions and on the intentions or future behavior of agents.Source: §2.3 · p. 4
3 OpenLane-V2
OpenLane-V2 combines multi-view data from Argoverse 2 and nuScenes with annotations for lanes, traffic elements, and their relationships. Its lane annotations derive directed centerlines from HD maps, while traffic-element annotations record image locations, attributes, and lane correspondence.
3 OpenLane-V2
- The dataset uses 2,000 scene segments from Argoverse 2 and nuScenes, with high-quality annotations produced through experienced annotators and multiple validation stages.Scenes cover varied cities and conditions, including noon and night, sunny and rainy days, downtown and suburbs.Source: §3 · p. 4
3.1 Raw Data Acquisition
- Two subsets reflect the source datasets’ different sensor setups, with a unified right-handed coordinate system and camera and ego-pose parameters provided.subset_A contains scenes from six cities and subset_B from Boston and Singapore; their night and rain proportions differ.Source: §3.1 · p. 5
3.2 Centerlines and Their Connectivity
- Lane centerlines are regressed from HD-map boundaries, merged across single-predecessor or single-successor lanes, and split at intersections, forks, and merges.Each centerline is an ordered list of 3D points; subset_B’s z values are set to 0 because its HD maps lack height information. Lane direction runs from start to end, and connectivity links one lane’s endpoint to another’s start.Source: §3.2 · p. 5
3.3 Traffic Elements and Their Correspondence to Centerlines
- Traffic elements receive front-view 2D boxes and semantic attributes, while composite attributes are split into separate annotations sharing the same box.There are 13 attributes; elements with unobservable meaning are labeled unknown. U-turn annotations are much rarer than common red or green traffic lights.Source: §3.3 · p. 6
- Traffic-element correspondence is assigned by lane-driving rules: nondirectional elements associate with lanes permitting straight travel, and directional elements with lanes moving in the indicated direction.Traffic elements are associated only with centerlines outside intersections.Source: §3.3 · p. 6
4 Task Definition & Evaluation Metric
OpenLane-V2 evaluates scene structure perception and reasoning through three subtasks and an overall score. Its metrics assess 3D centerline detection, traffic-element recognition, and topology on lane and lane–traffic graphs.
4 Task Definition & Evaluation Metric
- The OpenLane-V2 Score averages metrics from 3D lane detection, traffic-element recognition, and topology recognition to describe primary-task performance.The primary task includes recognizing lanes and their dynamic drivable states, traffic-element attributes, and relationships among perceived entities.Source: §4 · p. 6
4.1 3D Lane Detection
- 3D lane detection perceives directed 3D lane centerlines from multi-view images covering a panoramic field of view.Centerline similarity uses discrete Fréchet distance, and DET_l is averaged over match thresholds {1.0, 2.0, 3.0}; thresholds are relaxed for distant lanes based on distance from the ego car.Source: §4.1 · pp. 6–7
4.2 Traffic Element Recognition
- Traffic-element recognition detects front-view locations and attributes together, with tiny element size identified as a challenge in large outdoor scenes.The evaluation uses IoU distance with a match threshold of 0.75 and averages DET_t over attributes.Source: §4.2 · p. 7
4.3 Topology Reasoning
- Topology reasoning evaluates directed lane connectivity and lane–traffic-element correspondence as separate graphs.TOP is an mAP metric on matched graphs, using entity-specific Fréchet and IoU similarities to project predicted vertices; TOP_ll and TOP_lt score lane connectivity and lane-to-traffic topology, respectively.Source: §4.3 · p. 7
5 Experiments
The experiments adapt several existing methods to OpenLane-V2 and report their quantitative and qualitative performance. Results expose challenges in centerline and traffic-element perception and in reasoning about their relationships.
5 Experiments
- The experiments adapt and evaluate multiple state-of-the-art methods, using visualizations and analysis to examine design choices.The qualitative results suggest that adding traffic-rule priors could be a potential way to improve reasoning about which traffic elements are associated with a lane.Source: §5 · p. 8, §5.2 · p. 8
5.1 Baselines
- The baselines include methods for centerline connectivity, point-sequence or point-set map prediction, and joint scene-structure understanding.STSU derives connectivity from detected centerlines; VectorMapNet predicts point sequences autoregressively; MapTR uses equivalent point-set permutations; TopoNet uses instance-level queries for centerlines and traffic elements.Source: §5.1 · p. 8
5.2 Results
- Existing methods struggle with centerline detection and connectivity, while tiny traffic elements also make recognition difficult.TOP requires matching predicted and ground-truth entities, so weak centerline perception also limits connectivity performance.Source: §5.2 · p. 8
- At a complex crossroad, all models approximate the intersection, but centerline positions, shapes, and topology remain imperfect.STSU produces many false-positive centerlines and fails to attach traffic information; MapTR's independent lane points do not form smooth driving trajectories, whereas VectorMapNet's autoregressive design produces more properly shaped centerlines. All three of VectorMapNet, MapTR, and TopoNet provide only partially correct centerline–traffic-element associations.Source: §5.2 · pp. 8–9
6 Conclusion
OpenLane-V2 benchmarks scene-structure perception and reasoning through lane centerlines, connectivity, traffic-element semantics, and lane associations. Its scope is bounded by the available dataset scale and excludes planning and direct real-world deployment.
6 Conclusion
- OpenLane-V2 represents lane networks with centerlines and connectivity, and traffic information with semantic labels and lane associations.The paper describes tasks and metrics and evaluates various methods on the dataset.Source: §6 · p. 9
- The dataset builds on existing datasets and remains at the same scale as previous datasets; the authors expect more driving scenes to increase diversity.Planning is not included because it also requires movable objects such as cars and pedestrians for collision avoidance; integrating static and dynamic entities is left for future work.Source: §6 · p. 9
- Models trained or evaluated on the dataset should not be used directly for deployment or real-world applications, and the dataset gives no safety-critical guarantees.Source: §6 · p. 9
OpenLane-V2: Supplementary Material
The supplementary material documents reproducibility, dataset responsibilities, and licensing. It also provides dataset documentation in Datasheet format.
A Overview
- The supplementary material includes author statements, licensing, benchmark implementation details, and Datasheet-format documentation.The documentation covers dataset distribution, maintenance, composition, and collection.Source: A Overview · p. 13
B Author Statement
- The authors accept responsibility for licensing, distribution, and maintenance of the dataset.Source: B Author Statement · p. 13
C Licensing
- The dataset uses CC BY-NC-SA 4.0, while the repository code uses the Apache License 2.0.Source: C Licensing · p. 13
D Datasheet
The datasheet describes OpenLane-V2's purpose, distribution, and maintenance. It presents annotations and topology relationships as resources for depicting traffic scenes and supporting autonomous-vehicle judgments.
D.1 Motivation
- The dataset aims to depict complex traffic scenes with instance and topology annotations, including directed centerlines, lane connectivity, semantic traffic elements, and lane-control associations.The datasheet says centerlines offer trajectory guidance and that traffic-element associations indicate lanes controlled under traffic rules.Source: §D.1 · p. 13
- The dataset was created by researchers from OpenDriveLab and Huawei, with funding from Shanghai AI Laboratory and Huawei.Source: §D.1 · p. 13
D.2 Distribution
- The dataset is publicly accessible online and is distributed through the OpenDriveLab OpenLane-V2 GitHub repository.Source: §D.2 · p. 13
D.3 Maintenance
- The authors support, host, and maintain the dataset; they plan to post updates, continue hosting older versions, and provide a mechanism for contributions.Source: §D.3 · p. 14
D.4 Composition
OpenLane-V2 records multi-view driving scenes with centerlines, traffic elements, and their lane connectivity and control relationships. The section describes dataset scale, annotation structure, collection, and supported tasks.
D.4 Composition
- Each frame identifies lane centerlines, traffic elements, centerline connectivity, and which traffic signals control particular lanes.A vehicle must follow a signal if and only if a topology relationship connects it to its lane.Source: §D.4 · p. 14
- The dataset contains 2,000 scenes and 72K frames, with about 1.8 million lane-centerline and 0.25 million traffic-element annotations.Frames are annotated at 2Hz, and each scene contains a varying number of consecutive frames.Source: §D.4 · p. 14
- Connectivity between lanes and relationships between lanes and traffic elements are supplied as per-frame adjacency matrices.Frames within a segment are consecutively sampled at 2Hz.Source: §D.4 · p. 14
- The dataset is partitioned into training, validation, and testing splits and redistributes reconstructed raw data from Argoverse 2 and nuScenes.The authors describe it as self-contained despite its source datasets.Source: §D.4 · p. 14
D.5 Collection Process
- Experienced annotators and multiple validation stages support annotation quality; contractors make and verify traffic-element and topology annotations.Source: §D.5 · p. 14
D.6 Use
- The dataset supports 3D lane detection, traffic-element recognition, and topology reasoning.Source: §D.6 · p. 15
E Evaluation Metric
The section illustrates how the topology metric aggregates ranked-neighbor precision across graph vertices. In the example, the resulting TOP score is approximately 0.036.
- For a vertex, the example ranks predicted neighbors by confidence against its ground-truth connected neighbors to calculate mAP.The example uses vertex 1, whose ground-truth neighbors are {2, 3, 5, 8}, and predicted neighbors are {4, 3, 5, 6, 7}.Source: E Evaluation Metric · p. 15
- The TOP calculation combines a correctness indicator for each predicted connection with the precision of predicted neighbors, then averages vertex mAPs.Source: E Evaluation Metric · p. 15
- The reported value is 7/192, approximately 0.036.Source: E Evaluation Metric · p. 15
F Implementation Details
The implementation section adapts existing perception methods to OpenLane-V2 and specifies shared training settings alongside model-specific changes. Its illustrations contrast centerline-based attributes and topology with alternative lane representations and associations.
F Implementation Details
- STSU, VectorMapNet, and MapTR are adapted to the benchmark alongside TopoNet, a method designed for scene-structure understanding.Source: F Implementation Details · p. 15
- All models use consistent input settings and are trained on both subsets; a typical 24-epoch run takes about 16 hours on eight NVIDIA A100 GPUs.Subset A images are 2048 × 1550 except the front view; subset B images are 1600 × 900.Source: F Implementation Details · p. 15
- TopoNet uses a ResNet-50 backbone, C3–C5 features, and a 200 × 100 BEV grid, with specified loss weights for traffic-element and lane-centerline tasks.Source: F Implementation Details · p. 15
- STSU predicts centerlines with a DETR head and connectivity with decoder queries; its adaptation concatenates multi-view BEV embeddings and adds TopoNet-style traffic-element and lane-topology heads.Bezier control points are interpolated into 11 equidistant points.Source: F Implementation Details · p. 15
- VectorMapNet uses decoder keypoints and autoregressive position generation; its benchmark adaptation predicts 2D BEV centerlines and omits height during evaluation.Its perception range is expanded to ±50m × ±25m, and centerlines are interpolated to match the benchmark setting.Source: F Implementation Details · p. 16
- MapTR uses averaged point queries as centerline instance queries and predicts 2D BEV centerlines, disregarding z during inference.The adaptation increases perception range to ±50m × ±25m and uses 11 point queries per lane.Source: F Implementation Details · p. 16
G Illustrations
- Figure 1 contrasts a red-light-controlled left-turn lane with a straight lane that can continue, while Figure 2 contrasts centerline attributes with separate laneline and road-element detection plus hand-crafted rules.Source: Figure 1 · p. 16, Figure 2 · p. 17
- Figures 3–5 depict centerline segmentation at lane splits and merges, a traffic element associated with three centerlines, and numerous centerlines in intersections.Source: Figure 3 · p. 17, Figure 4 · p. 18, Figure 5 · p. 18
Related papers
This paper cites
- Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations2016
- nuScenes: A multimodal dataset for autonomous driving2019
- The Open Images Dataset V4: Unified image classification, object detection, and visual relationship detection at scale2018
- Planning-oriented Autonomous Driving2022
- Argoverse 2: Next Generation Datasets for Self-Driving Perception and Forecasting2023
- Visual Semantic Role Labeling2015