Source-linked AI summary
Gen-LaneNet: A Generalized and Scalable Approach for 3D Lane Detection
Yuliang Guo, Guang Chen, Peitao Zhao, Weide Zhang, Jinghao Miao, Jingao Wang, Tae Eun Choe
TL;DR
Gen-LaneNet addresses 3D lane detection from a single image by aligning geometry-guided lane anchors with top-view features and decoupling image segmentation from 3D geometry learning. It surpasses 3D-LaneNet in AP and F-score across varied scene conditions while requiring fewer 3D lane labels.
Problem
Accurate real-time lane perception is fundamental for autonomous driving, while existing 3D lane methods can misalign ground-truth geometry with visual features and couple geometry reasoning to changing image appearance.
Method
Gen-LaneNet uses a geometry-guided lane-anchor representation with a geometric transformation for 3D points, plus a two-stage framework that separately learns image segmentation and 3D geometry.
Results
Gen-LaneNet significantly surpasses 3D-LaneNet in AP and F-score across multiple scene splits, including improvements of 0.8% AP and 1.7% F-score on balanced scenes.
Takeaways & Limitations
The decoupled framework supports a more scalable real-world solution by using cost-effective 2D lane annotations with limited expensive 3D labels.
Takeaways & Limitations
The method is evaluated against limitations arising from scene and appearance variation, including partial occlusions, illumination, weather, and uncommon test scenes.
Abstract
from arXiv · showhide
We present a generalized and scalable method, called Gen-LaneNet, to detect 3D lanes from a single image. The method, inspired by the latest state-of-the-art 3D-LaneNet, is a unified framework solving image encoding, spatial transform of features and 3D lane prediction in a single network. However, we propose unique designs for Gen-LaneNet in two folds. First, we introduce a new geometry-guided lane anchor representation in a new coordinate frame and apply a specific geometric transformation to directly calculate real 3D lane points from the network output. We demonstrate that aligning the lane points with the underlying top-view features in the new coordinate frame is critical towards a generalized method in handling unfamiliar scenes. Second, we present a scalable two-stage framework that decouples the learning of image segmentation subnetwork and geometry encoding subnetwork. Compared to 3D-LaneNet, the proposed Gen-LaneNet drastically reduces the amount of 3D lane labels required to achieve a robust solution in real-world application. Moreover, we release a new synthetic dataset and its construction strategy to encourage the development and evaluation of 3D lane detection methods. In experiments, we conduct extensive ablation study to substantiate the proposed Gen-LaneNet significantly outperforms 3D-LaneNet in average precision(AP) and F-score.
1. Introduction
Gen-LaneNet addresses the limitations of planar and prior monocular 3D lane detection with geometry-aligned anchors, decoupled learning, and a synthetic dataset. Its design targets generalization to unfamiliar scenes while reducing reliance on expensive 3D labels.
- 1. Introduction: Robust lane perception supports lateral vehicle control, accurate localization, and high-definition map construction.
- 1. Introduction: 2D lane methods commonly combine segmentation, projection to a flat ground plane, and curve fitting under a planar-road assumption.
- 1. Introduction: 3D-LaneNet unifies image encoding, spatial transformation, and 3D curve extraction, but inappropriate anchors and end-to-end coupling limit generalization.
- 1. Introduction: Gen-LaneNet detects 3D lanes from a single image using geometry-guided anchors, a two-stage framework, and a new synthetic dataset.The method aligns lane representations with top-view features, decouples image segmentation from geometry encoding, and supports development and evaluation through synthetic data.
- 1. Introduction: Gen-LaneNet’s two-stage design allows segmentation and geometry encoding to be learned independently, reducing the amount of expensive 3D labeling needed.
2. Related work
Prior lane-detection methods largely rely on 2D or multi-sensor approaches, while monocular 3D-LaneNet offers a unified alternative with important generalization and scalability limitations. Gen-LaneNet addresses these limitations through aligned anchors and decoupled learning.
- 2. Related work: Multi-sensor and stereo-camera methods provide alternatives to monocular inference but face high LiDAR cost, data sparsity, or reduced distant 3D accuracy.
- 2. Related work: 3D-LaneNet predicts 3D lanes from a single image by unifying image encoding, feature transformation, and 3D curve extraction.Its image-view features pass through projective transformations to a top-view pathway whose prediction head outputs 3D lane points.
- 2. Related work: 3D-LaneNet’s inappropriate coordinate frame misaligns ground-truth anchors with visual features, making generalization to partially novel scenes difficult.The mismatch is especially evident on hilly roads, where lanes projected to the virtual top-view appear nonparallel.
- 2. Related work: End-to-end coupling makes geometric encoding sensitive to image-appearance changes and increases dependence on costly 3D training labels.The cited variations include partial occlusion, illumination, and weather conditions; 3D ground truth may require multiple sensors, localization, calibration, and manual adjustment.
- 2. Related work: Gen-LaneNet resolves the anchor-alignment issue by representing lane points in a virtual top-view, where targets align with projected image features.
3. Gen-LaneNet
Gen-LaneNet combines geometry-guided lane anchors with a two-stage architecture to predict 3D lanes from a single image. It aligns lane representations with virtual top-view features, transforms predictions into the ego-vehicle frame, and decouples image segmentation from 3D geometry reasoning.
- 3.1. Geometry in 3D Lane Detection: Gen-LaneNet uses a geometric transformation to convert virtual top-view lane predictions into 3D points in the ego-vehicle coordinate frame.The transformation is derived from the co-linear relationship among a 3D lane point, its virtual top-view projection, and the camera center, and is independent of camera angles.
- 3.2. Geometry-guided anchor representation: Lane anchors are defined on equally spaced vertical lines and represent each lane with horizontal offsets, heights, visibility, and lane-existence attributes at fixed y-positions.The representation predicts lane lines as structured polylines in the virtual top-view, retaining only points with predicted visibility.
- 3.2. Geometry-guided anchor representation: Representing lane points in the virtual top-view aligns targets with projected visual features, supporting local patch-level encoding for unfamiliar scenes and partially visible lanes.The representation also adds visibility attributes so lanes that start or end midway can be handled more stably.
- 3.3. Two-stage framework with decoupled learning of image encoding and geometry reasoning: The proposed two-stage network first produces image-domain lane segmentation, then 3D-GeoNet predicts virtual top-view lanes before geometric transformation produces ego-vehicle 3D points.The architecture is illustrated as an image segmentation subnetwork followed by a top-view segmentation encoder and lane prediction head.
- 3.3. Two-stage framework with decoupled learning of image encoding and geometry reasoning: Decoupling image encoding from geometry reasoning reduces reliance on expensive real-world 3D labels and allows the framework to leverage larger 2D lane datasets.Training the geometry subnetwork with synthetic 3D data and the segmentation subnetwork with real 2D data is described as an optimal but future solution because of domain transfer issues.
4. Synthetic dataset and construction strategy
The authors construct a synthetic 3D lane dataset with diverse terrain, scene structure, illumination, and camera conditions, while explicitly modeling lane-point occlusion.
- The dataset uses Unity to render diverse 3D worlds aimed at evaluating generalization to rarely observed scenarios.It includes realistic background elements, varied scene structures, and diversified visual appearances.
- Three Silicon Valley-based maps provide highway, urban, and residential terrain with varied elevations and turnings.Images vary by location, day-time, lane-marker degradation, camera height, and pitch angle.
- Ground-truth depth and semantic maps identify foreground- and background-occluded lane points for reliable lane-detector evaluation.Background-occluded portions are intended to be discarded, whereas foreground-occluded portions should be recovered.
5. Experiments
Experiments evaluate Gen-LaneNet across balanced, rarely observed, and visually varied scenes using strict lane matching and comparisons against 3D-LaneNet and 3D-GeoNet.
- 5.1. Experimental setup: The three dataset splits test standard performance, generalization to complex urban scenes with elevation changes and sharp turns, and robustness to illumination variation.The visual-variation split trains the segmentation stage with all training images but excludes before-dawn 3D examples from geometry-subnetwork training.
- 5.2. Anchor effect: The new anchor design improves AP and F-score by 3% to 10% across all dataset splits for 3D-LaneNet, 3D-GeoNet, and Gen-LaneNet.The comparison keeps each candidate architecture fixed except for the integrated anchor representation.
- 5.1. Experimental setup: Evaluation formulates prediction-to-ground-truth assignment as minimum-cost-flow bipartite matching and reports AP, maximum F-score, precision, and recall.Partial matches use a Euclidean lane cost with dmax = 1.5m for positions covered by only one lane.
- 5.3. Upper bound of two-stage framework: 3D-GeoNet outperforms Gen-LaneNet across all three setups by around 3% on balanced scenes and 5% to 7% on rarely observed and visually varied scenes.It also exceeds 3D-LaneNet by 5% to 18% in F-score and AP, defining an upper bound for the two-stage framework.
- 5.4. Whole system evaluation: Gen-LaneNet improves over 3D-LaneNet by 0.8% AP and 1.7% F-score on balanced scenes, 6% AP and 4.4% F-score on rarely observed scenes, and around 13% in both metrics under visual variation.The visual-variation result is attributed to decoupled image encoding and 3D geometry reasoning.
- 5.4. Whole system evaluation: Gen-LaneNet maintains Euclidean error lower than or on par with 3D-LaneNet for matched lanes in both near and far ranges.The ranges are 0–40m and 40–100m, respectively.
6. Conclusion
The paper concludes that Gen-LaneNet combines geometry-guided anchors, decoupled two-stage learning, and synthetic data construction to improve 3D lane detection.
- Gen-LaneNet combines a geometry-guided anchor representation with a two-stage framework that decouples image segmentation from 3D lane prediction.The paper also introduces a synthetic-dataset construction strategy for 3D lane detection.
- Gen-LaneNet surpasses 3D-LaneNet in both AP and F-score across the evaluated perspectives.
A. Features investigation of 3D-LaneNet
The 3D-LaneNet pipeline separates image-view processing, road-plane prediction, top-view processing, and lane prediction, with projective transformations linking the first two pathways.
- 3D-LaneNet comprises image-view, road-plane prediction, top-view, and lane-prediction subnetworks.
- Image-view features are transferred to the top-view pathway through four projective transformation layers.The top-view pathway then processes transformed features to predict 3D lanes.
- For uphill roads, projected lane features diverge in the virtual top-view, and visualized top-view features show the same diverging pattern.This investigation examines whether top-view features align with projected ground-truth lane structure.
B. Algebraic derivation of the geometric transformation
The derivation establishes the geometric transformation between the ego-vehicle and virtual top-view coordinate frames, using projective geometry and a planar homography under a simplified camera setup.
- The derivation double-checks the geometric transformation between 3D and virtual top-view coordinate frames using algebraic manipulation.The resulting transformation agrees with the geometric proof in the main paper.
- The algebraic derivation assumes a simpler camera orientation in which only the pitch angle is involved.Camera height h and pitch angle θ determine the explicit forms of R and T.
- A 3D ego-vehicle point is projected to an image point, while its top-view counterpart reaches the same image point through a planar homography.The relationship is expressed using the rotation matrix R, translation vector T, and camera intrinsic matrix K.
- The derivation rewrites the transformation with s = sin θ and c = cos θ before expanding it into scalar equations.The scalar equations are then reorganized and substituted step by step to derive the final transformation.
- Substituting the derived coefficient α into the earlier equations produces the final transformation equations.These substitutions complete the algebraic verification of the coordinate-frame mapping.
C. Experiments on center lines
Center-line experiments evaluate the anchor representation, the two-stage framework’s upper bound, and the complete system across balanced, rarely observed, and visually varied scenes.
- Anchor effect: 7.1% and 11.1% F-score gains result from the new anchor on rarely observed scenes and scenes with visual variations, respectively.The anchor improves all candidate methods across all dataset splits and aligns visual features with lane labels.
- The upper bound of two-stage framework: 3D-GeoNet achieves the highest performance in all three splits, indicating the proposed two-stage framework’s upper bound with perfect image segmentation.The margin between 3D-LaneNet and 3D-GeoNet reaches 16% in both F-score and AP on visually varied scenes.
- The upper bound of two-stage framework: Gen-LaneNet gains significantly from more available 2D labels and a better segmentation network.
- Whole system comparison: 13% in both AP and F-score is Gen-LaneNet’s improvement over 3D-LaneNet on scenes with visual variations.Its localization errors remain lower or on par with 3D-LaneNet across close and far ranges, despite evaluating more matched lane pairs.
- Whole system comparison: Gen-LaneNet surpasses 3D-LaneNet across all three dataset splits in the whole-system center-line comparison.The reported splits are balanced scenes, rarely observed scenes, and scenes with visual variations.
D. Qualitative comparison
Qualitative comparisons examine the new anchor and the complete Gen-LaneNet system on challenging road geometries and illumination changes.
- Gen-LaneNet is more robust than 3D-LaneNet under unobserved illumination in qualitative whole-system comparisons.Figure 11 compares image and 3D predictions for lane lines and center lines across four examples.
- The new anchor consistently improves predictions on hilly and sharp-turning roads.Figure 10 compares the original 3D-LaneNet with its version using the new anchor on standard test examples.
- The qualitative evaluation contains separate comparisons for the anchor effect and for the complete-system behavior under visual variation.
- Figure 10 places image results on the left and 3D results on the right, with lane lines above center lines.
- Figure 11 uses the same image-versus-3D and lane-line-versus-center-line layout, coloring ground-truth lanes blue.