Source-linked AI summary
A Keypoint-based Global Association Network for Lane Detection
Jinsheng Wang, Yinchao Ma, Shaofei Huang, Tianrui Hui, Fei Wang, Chen Qian, Tianzhu Zhang
TL;DR
Lane detection needs flexible modeling of complex lane shapes, while point-by-point keypoint grouping is inefficient during postprocessing. GANet globally associates keypoints with lane starting points and supplements this with local feature aggregation, achieving state-of-the-art performance with higher speed.
Problem
Existing anchor-based methods lack flexibility for complex lane shapes, while keypoint-based methods can require inefficient point-by-point grouping during postprocessing.
Method
GANet globally regresses each keypoint toward its lane's starting point for parallel association, while LFA adaptively aggregates information from adjacent lane keypoints.
Results
GANet achieves state-of-the-art lane-detection performance with higher speed, including 79.63% F1 at 63 FPS on CULane for GANet-L.
Takeaways & Limitations
Global keypoint association offers a superior performance-efficiency trade-off for lane detection compared with prior methods.
Takeaways & Limitations
When output stride is 1, large offsets to lane starting points may be difficult to regress; the authors propose multi-level offset regression as future work.
Abstract
from arXiv · showhide
Lane detection is a challenging task that requires predicting complex topology shapes of lane lines and distinguishing different types of lanes simultaneously. Earlier works follow a top-down roadmap to regress predefined anchors into various shapes of lane lines, which lacks enough flexibility to fit complex shapes of lanes due to the fixed anchor shapes. Lately, some works propose to formulate lane detection as a keypoint estimation problem to describe the shapes of lane lines more flexibly and gradually group adjacent keypoints belonging to the same lane line in a point-by-point manner, which is inefficient and time-consuming during postprocessing. In this paper, we propose a Global Association Network (GANet) to formulate the lane detection problem from a new perspective, where each keypoint is directly regressed to the starting point of the lane line instead of point-by-point extension. Concretely, the association of keypoints to their belonged lane line is conducted by predicting their offsets to the corresponding starting points of lanes globally without dependence on each other, which could be done in parallel to greatly improve efficiency. In addition, we further propose a Lane-aware Feature Aggregator (LFA), which adaptively captures the local correlations between adjacent keypoints to supplement local information to the global association. Extensive experiments on two popular lane detection benchmarks show that our method outperforms previous methods with F1 score of 79.63% on CULane and 97.71% on Tusimple dataset with high FPS. The code will be released at https://github.com/Wolfwjs/GANet.
1. Introduction
Lane detection must represent complex lane shapes while distinguishing lane instances, but anchor-based and sequential keypoint methods impose flexibility, efficiency, or robustness limits. GANet globally associates keypoints with lane starts, while LFA supplements this process with local correlations.
- Lane detection must predict accurate lane shapes and distinguish lane instances for autonomous driving and ADAS.
- Anchor-based methods are efficient for lane discrimination but inflexible because predefined anchor shapes limit representation of varied lanes.
- Keypoint-based methods flexibly model lane shapes but sequential neighbor association is inefficient and can accumulate errors after a wrong assignment.
- GANet directly regresses each keypoint to its lane’s starting point, enabling independent parallel association and improving postprocessing efficiency.
- LFA aggregates features from adjacent lane points to supplement GANet’s global association with local information.
- GANet achieves state-of-the-art performance on two lane-detection benchmarks with faster speed and a favorable performance-efficiency trade-off.
2. Related Works
Related work includes segmentation-, detection-, and keypoint-based lane modeling, alongside deformable convolution for adaptive feature aggregation. GANet differs by using global keypoint association and lane-aware local aggregation.
- Segmentation-based methods: Segmentation-based methods formulate lane detection as per-pixel classification, using class distinctions and message passing to separate lane instances.
- Detection-based methods: Detection-based methods regress sampled lane points from predefined line-like anchors and apply NMS to select high-confidence lanes.
- Keypoint-based methods: Keypoint-based methods estimate lane points and associate them using embeddings or local geometry construction.
- Deformable convolution: Deformable convolution adaptively adjusts sampling locations through learned offsets, overcoming fixed-grid limitations for irregular structures.
- Deformable convolution: LFA adapts deformable aggregation to lane structure by restricting sampling to adjacent points on each lane.
3. Method
GANet extracts multi-scale features, estimates lane keypoints and their offsets to lane starting points, then groups keypoints into lane instances in parallel. Its LFA module aggregates local context from adjacent lane points to improve keypoint representations.
- Overall Architecture: GANet uses a CNN backbone, self-attention layer, and FPN neck to extract multi-level visual representations before decoding keypoint and offset maps.The decoder contains fully convolutional keypoint and offset heads.
- Keypoint Estimation: The keypoint head produces a confidence map whose values represent the probability that each location is a lane keypoint.Training samples K keypoints per lane and splats them onto the confidence map.
- Lane Construction: Lane construction first selects local maxima from the confidence map, identifies candidate starting points, and forms each lane as an ordered list of grouped keypoints.Starting-point candidates are deduplicated by selecting the geometric center of each local region.
- Starting Point Regression: Keypoints are associated globally by regressing offsets to lane starting points, so points estimating nearby starting-point coordinates can be grouped into the same lane.A point is associated when the estimated starting point lies within a predefined distance threshold, and matrix operations enable parallel association.
- Lane-aware Feature Aggregator: LFA predicts offsets to adjacent same-lane keypoints and uses deformable convolution to aggregate their features, enhancing each keypoint’s local context.An auxiliary loss supervises the predicted adjacent-point offsets against ground-truth lane offsets.
4. Experiments
Experiments evaluate GANet on two lane-detection benchmarks, then analyze dataset-specific results and module ablations.
- Experiments cover CULane and TuSimple benchmarks, followed by per-dataset results and ablation studies.
4.1. Experimental Setting
The evaluation uses CULane and TuSimple, with dataset-specific metrics and defined training configurations for several GANet backbone variants.
- Datasets: Experiments use the CULane and TuSimple lane-detection benchmarks.
- Datasets and Evaluation Metrics: CULane contains 88,880 training and 34,680 testing images across urban and highway scenarios, using F1 based on IoU greater than 0.5.
- Datasets and Evaluation Metrics: TuSimple contains 3,626 training and 2,782 testing images, with accuracy based on predicted points falling within 20 pixels of ground truth.
- Datasets and Evaluation Metrics: TuSimple predictions above 85% accuracy are treated as true positives, and F1 score is also reported.
- Implementation Details: GANet-S, GANet-M, and GANet-L use ResNet-18, ResNet-34, and ResNet-101 backbones, respectively.
4.2. Quantitative Results
GANet achieves strong results on both benchmarks, combining high accuracy with fast inference, while ablations show that LFA and its auxiliary loss contribute to performance.
- Results on CULane: 79.63% F1 at 63 FPS makes GANet-L state of the art on CULane and faster and more accurate than similarly sized LaneATT-ResNet122.
- Results on CULane: 78.79% F1 and 3.8× faster inference give GANet-S a comparable result to FOLOLane-ERF with a stronger performance-efficiency trade-off.
- Results on TuSimple: 97.71% F1 makes GANet-S the top-performing method on the TuSimple test set with high FPS.
- Ablation Study: The ablation compares a baseline without LFA, LFA without auxiliary loss, and the complete GANet.
- Ablation Study: LFA improves lane-line detection without auxiliary loss, while the auxiliary loss is described as vital for directing LFA toward key lane information.
4.3. Qualitative results
Qualitative comparisons indicate that LFA improves predictions under occlusion, suppresses background noise, and produces more lane-aligned aggregation points with auxiliary supervision.
- Visualization Results: With LFA, predicted lane points enhance one another and support correct predictions under vehicle occlusion.
- Visualization Results: Figure 6 compares input images, confidence maps and lane lines without and with LFA against ground-truth lane lines.
- Visualization Results: LFA suppresses background noise that may be introduced by global attention.
- Aggregation-Point Visualization: With auxiliary losses, LFA predicts aggregation points around straight lane lines, whereas the points are irregular without the auxiliary loss.
- Aggregation-Point Visualization: In curved-lane cases, LFA robustly models local lane structures, enhancing lane features and suppressing background noise.
5. Conclusion and Discussion
GANet globally associates lane keypoints with lane starting points and supplements this process with LFA, while experiments report higher speed than previous methods. The method’s main limitation is difficulty regressing large starting-point offsets at output stride 1.
- GANet directly regresses each keypoint to its lane’s starting point, while LFA supplements global association with local correlations between adjacent keypoints.The approach is reported to outperform previous methods with higher speed.
- Figure 7 visualizes LFA aggregation using red observation points, green predicted aggregation points, and light blue ground-truth lane points.
- At output stride 1, large offsets to lane starting points may be difficult to regress; the authors propose multilevel offset regression as future work.