Source-linked AI summary
Ultra Fast Structure-aware Deep Lane Detection
Zequn Qin, Huanyu Wang, Xi Li
TL;DR
Lane detection needs both lower computational cost and robustness when occlusion or lighting removes local visual clues. The paper replaces pixel-wise segmentation with row-based selection from global features and adds a structural loss, reporting state-of-the-art speed and accuracy, including 322.5 FPS for a lightweight version.
Problem
Existing segmentation-based lane detectors face computational expense and difficulty explicitly using lane rigidity and smoothness, while challenging scenes may lack local visual clues.
Method
The method selects lane locations at predefined rows using global features and applies structural loss to explicitly model lane prior information.
Results
The method achieves state-of-the-art accuracy and speed on challenging CULane, while a lightweight ResNet-18 version reaches 322.5 FPS at 288×800 resolution.
Takeaways & Limitations
Row-based selection and global features address the paper's speed and no-visual-clue targets, while structural loss explicitly models lane structure.
Abstract
from arXiv · showhide
Modern methods mainly regard lane detection as a problem of pixel-wise segmentation, which is struggling to address the problem of challenging scenarios and speed. Inspired by human perception, the recognition of lanes under severe occlusion and extreme lighting conditions is mainly based on contextual and global information. Motivated by this observation, we propose a novel, simple, yet effective formulation aiming at extremely fast speed and challenging scenarios. Specifically, we treat the process of lane detection as a row-based selecting problem using global features. With the help of row-based selecting, our formulation could significantly reduce the computational cost. Using a large receptive field on global features, we could also handle the challenging scenarios. Moreover, based on the formulation, we also propose a structural loss to explicitly model the structure of lanes. Extensive experiments on two lane detection benchmark datasets show that our method could achieve the state-of-the-art performance in terms of both speed and accuracy. A light-weight version could even achieve 300+ frames per second with the same resolution, which is at least 4x faster than previous state-of-the-art methods. Our code will be made publicly available.
1 Introduction
Lane detection must address both high computational cost and missing visual clues in occluded or severely lit scenes. The paper proposes row-based location selection using global features, plus structural loss, to target both challenges.
- Lane detection is important for ADAS and autonomous driving, but repeated execution across multiple camera inputs demands low computational cost.
- Deep segmentation methods are computationally expensive because they perform dense prediction, while their binary representation makes lane rigidity and smoothness difficult to model explicitly.
- Severe occlusion and extreme lighting can leave little or no visual clues, creating a need for higher-level semantic analysis and information from elsewhere in the image.
- The proposed formulation selects lane locations at predefined rows using global features instead of segmenting every pixel, significantly reducing computational cost.
- Global features provide a whole-image receptive field, allowing visual clues and messages from different locations to support detection when local target evidence is absent.
- A structural loss explicitly uses lane prior information, while the method reports state-of-the-art accuracy and speed on CULane and 300+ FPS for a lightweight version.
2 Related Work
Prior lane-detection research includes traditional image processing, deep segmentation, sequential prediction, and clustering formulations. These methods exploit visual, spatial, sequential, or grouped representations in different ways.
- Traditional methods use image-processing cues such as HSI color models and edge extraction, with tracking or graphical models used as post-processing when visual information is weak.
- Deep neural methods generally formulate lane detection as semantic segmentation and include designs such as vanishing-point guidance and sliced-feature information aggregation.
- Other approaches formulate lane detection as sequential prediction with LSTM or directional drawing, or as clustering of binary segments.
3 Method
The method formulates lane detection as selecting lane locations on predefined row anchors from global image features, reducing computation while incorporating broader context. Structural losses model continuity and lane shape through differentiable location representations.
- Definition of our formulation: Lane detection is represented as selecting cells across predefined row anchors rather than classifying every image pixel.The formulation uses h row anchors and w gridding cells, with an additional class for lane absence.
- Definition of our formulation: Global image features predict location distributions for each lane and row anchor, enabling the correct cell to be selected from those probabilities.The prediction is a (w + 1)-dimensional vector for each lane-row pair.
- How the formulation achieves fast speed: The formulation replaces H × W pixel classifications with C × h classifications, each over w + 1 cells, substantially reducing computational cost.The cited comparison reports 1.15×10^6 calculations for the segmentation formulation and describes the proposed formulation as extremely fast.
- Handling the no-visual-clue problem: Using global features gives the formulation a whole-image receptive field, allowing contextual information from other locations to address missing visual clues.The paper motivates this for cases such as lane occlusion, where information from other lanes, road shape, or vehicle direction can help locate the lane.
- Lane structural loss: The structural loss models lane relations by encouraging adjacent-row continuity and constraining second-order differences in lane locations.Second-order differences are zero for straight lanes and impose less influence than first-order differences when lanes are not straight.
- Lane structural loss: Expected locations computed from softmax probabilities provide a differentiable approximation for applying structural constraints to classification predictions.This expectation both remains differentiable and recovers a continuous location from a discrete location distribution.
4 Experiments
Experiments evaluate the method on two lane-detection benchmarks through ablations and comparisons with existing approaches. Results show strong accuracy and substantial speed advantages, including 322.5 FPS on CULane.
- Experimental settings: Experiments use TuSimple and CULane, with dataset-specific evaluation metrics and implementation settings.TuSimple uses evaluation accuracy, while CULane uses F1-measure based on IoU thresholds.
- Ablation studies: Expectation-based classification outperforms standard classification, while classification-based localization consistently outperforms regression-based alternatives.The comparison is conducted on the TuSimple dataset using REG, REG Norm, CLS, and CLS Exp variants.
- Ablation studies: The row-selecting formulation significantly improves performance over conventional segmentation, while structural loss and feature aggregation provide additional gains.The module comparison uses a ResNet-34 backbone on the TuSimple benchmark.
- Ablation studies: Similarity loss makes classification predictions smoother and improves performance.The qualitative comparison visualizes predicted distributions for points belonging to the same lane.
- Benchmark results: 41.7 times faster than SCNN, the method maintains comparable performance while substantially reducing TuSimple inference runtime.It is also more than 2 times faster than SAD, the second-fastest network in the comparison.
- Benchmark results: 322.5 FPS is achieved by the fastest model on CULane at 288×800 resolution, with the best performance in both accuracy and speed.The CULane experiments target challenging scenarios and compare F1-measure and runtime.
- Benchmark results: Qualitative results show that the method performs well under various conditions on both TuSimple and CULane.The visualizations are presented in Fig. 8.
5 Conclusion
The paper proposes row-based lane selection with global features and a structural loss, achieving state-of-the-art accuracy and speed. Its lightweight ResNet-18 version reaches 322.5 FPS at comparable performance and resolution.
- Conclusion: The formulation treats lane detection as row-based selection using global features to address speed and no-visual-clue challenges.Lanes are represented as selected locations on predefined image rows rather than segmentation maps.
- Conclusion: Structural loss explicitly models lane prior information within the proposed formulation.The conclusion identifies structural loss as a component whose effectiveness is supported by qualitative and quantitative experiments.
- Conclusion: 322.5 FPS is achieved by the lightweight ResNet-18 version at comparable performance and the same resolution.The conclusion also reports state-of-the-art accuracy and speed for the ResNet-34 model.