Source-linked AI summary
Keep your Eyes on the Lane: Real-time Attention-guided Lane Detection
Lucas Tabelini, Rodrigo Berriel, Thiago M. Paixão, Claudine Badue, Alberto F. De Souza, Thiago Oliveira-Santos
TL;DR
Lane detection needs robustness and real-time efficiency for autonomous driving, but existing approaches often struggle with speed, accuracy, or reproducibility. LaneATT combines anchor-based pooling with global attention in a single-stage model and reports strong accuracy-efficiency results across three benchmarks.
Problem
Lane detection must handle adverse conditions while meeting real-time requirements, yet many methods struggle with efficiency and related practical issues.
Method
LaneATT is an anchor-based single-stage lane detector using anchor-based feature pooling, a lightweight backbone, and global attention features.
Results
Across TuSimple, CULane, and LLAMAS, LaneATT is reported to outperform state-of-the-art methods in both efficacy and efficiency, including 250 FPS and almost an order of magnitude fewer MACs than the previous state-of-the-art.
Takeaways & Limitations
The model offers accuracy-efficiency trade-offs intended for practical real-time lane detection, supported by benchmark comparisons and ablation analysis.
Takeaways & Limitations
Comparisons of lane-detector efficiency are constrained by missing or partially public source code and by differences between MACs and FPS as efficiency measures.
Abstract
from arXiv · showhide
Modern lane detection methods have achieved remarkable performances in complex real-world scenarios, but many have issues maintaining real-time efficiency, which is important for autonomous vehicles. In this work, we propose LaneATT: an anchor-based deep lane detection model, which, akin to other generic deep object detectors, uses the anchors for the feature pooling step. Since lanes follow a regular pattern and are highly correlated, we hypothesize that in some cases global information may be crucial to infer their positions, especially in conditions such as occlusion, missing lane markers, and others. Thus, this work proposes a novel anchor-based attention mechanism that aggregates global information. The model was evaluated extensively on three of the most widely used datasets in the literature. The results show that our method outperforms the current state-of-the-art methods showing both higher efficacy and efficiency. Moreover, an ablation study is performed along with a discussion on efficiency trade-off options that are useful in practice.
1. Introduction
Lane detection must handle adverse conditions and real-time constraints, while existing methods often struggle with efficiency, robustness, reproducibility, or accuracy. LaneATT addresses these challenges with an anchor-based attention model evaluated across three benchmarks.
- Lane detection must remain accurate under extreme lighting, weather, occlusion, and missing lane markings.
- Real-time inference is required in several applications, but many lane detection models struggle to meet this constraint.
- Segmentation-based approaches can achieve real-time performance but often restrict backbone choices and may require heuristic post-processing.
- LaneATT is an anchor-based single-stage model using a lightweight backbone and a novel attention mechanism to aggregate global information.
- The method targets higher accuracy than existing real-time state-of-the-art methods and is evaluated on TuSimple, CULane, and LLAMAS.
- 250 FPS and almost an order of magnitude fewer MACs than the previous state-of-the-art are reported for the proposed model.
2. Related work
Deep lane detection research has centered on segmentation and row-wise classification, alongside faster alternatives, but comparisons are hindered by post-processing and reproducibility issues. LaneATT responds by publishing reproducible code while targeting both speed and accuracy.
- Deep lane detectors predominantly use segmentation or row-wise classification, while other approaches pursue alternative lane representations.
- Segmentation-based methods: Segmentation methods make per-pixel lane predictions and require post-processing, with SCNN reported at 7.5 FPS.
- Row-wise classification methods: Row-wise classification predicts lane cells across image rows and also requires post-processing to construct complete lanes.
- Other approaches: FastDraw and PolyLaneNet avoid some clustering steps and achieve high speed, but are not reported to outperform state-of-the-art accuracy.
- Reproducibility: Unpublished or partially public code hinders comparisons, while MACs do not always proxy the FPS that determines practical speed.
- LaneATT publishes full reproducibility code while aiming to be faster and more accurate than existing state-of-the-art methods.
3. Proposed method
LaneATT detects lanes with an anchor-based single-stage architecture that combines anchor-local features with global attention features. Its lane representation uses fixed vertical coordinates and line-based anchors to produce final lane predictions.
- LaneATT receives front-facing RGB images and predicts lane boundary lines with an anchor-based single-stage model.
- A CNN backbone generates feature maps, from which each anchor’s features are pooled and combined with global attention features.
- Classification and regression layers use the concatenated pooled and attention features to produce final predictions.
- Combining local and global features lets the model use information from other lanes in cases such as occlusion or invisible lane markings.
- Lanes are represented by x-coordinates associated with fixed, equally spaced y-coordinates.
- Start and end indices define the valid contiguous portion when a lane does not span the full image vertically.
- Each line-based anchor is defined by an origin on an image border and a direction, providing references for lane proposals.
3.2. Backbone
The backbone stage extracts and reduces CNN feature maps before anchor-specific pooling. Each anchor samples a line of feature-map locations, producing local feature vectors with boundary padding.
- Feature extraction uses a generic CNN backbone, such as ResNet, to generate the feature map for later anchor-specific pooling.
- A 1 × 1 convolution reduces the backbone feature map’s channel dimensionality to reduce computational costs.
- Each anchor identifies feature-map points intercepted by its rasterized virtual line for pooling.
- The pooled representation for every anchor is a feature vector containing local feature information.
- Anchor portions outside the feature-map boundaries are zero-padded during pooling.
- Unlike proposal-based RoI pooling, the method pools directly from anchors and does not require a fixed-size RoI pooling layer.
3.4. Attention mechanism
LaneATT augments anchor-local features with attention-derived global information. A fully connected layer assigns inter-anchor weights, whose weighted feature aggregation produces global features efficiently via matrix multiplication.
- Local pooled features may be insufficient for predicting lane existence and position, especially with occlusion.
- The attention layer processes each anchor’s local feature and outputs a probability weight for every other anchor.
- Self-attention is excluded by setting the anchor’s weight to zero, while other weights come from a softmax output.
- The weighted local features are combined to produce a global feature vector with the same dimensions as the local feature matrix.
- Global features for all anchors are computed efficiently as matrix multiplication using the anchor weight matrix and local-feature matrix.
3.5. Proposal prediction
LaneATT predicts one lane proposal per anchor using classification probabilities, horizontal offsets, and proposal length. It concatenates local and global features before parallel classification and regression heads generate the final proposal outputs.
- Each anchor produces a proposal containing K + 1 class probabilities, Npts horizontal offsets, and a valid proposal length l.The classes include K lane types and one background or invalid-proposal class.
- The proposal start-index s is determined directly by the anchor origin’s y-coordinate.
- Local and global features are concatenated into an augmented feature vector for final proposal generation.
- Parallel fully connected classification and regression layers predict the proposal’s class probabilities and regression outputs.
3.6. Non-maximum Supression (NMS)
LaneATT applies non-maximum suppression during training and testing to reduce false positives. The procedure uses a lane-distance metric computed over the y-coordinate range shared by two lanes.
- NMS is applied in both training and testing to reduce the number of false positives.
- The lane distance between two lanes is computed using their common valid indices, or y-coordinates.
- The common-index range starts at max(sa, sb) and ends at min(ea, eb).
3.7. Model training
Training assigns anchors to positive, negative, or disregarded groups using their distance from ground-truth lanes. The retained anchors and proposals are optimized with a multi-task loss using classification and regression outputs and targets.
- Anchors closer than τp to a ground-truth lane are positives, while anchors farther than τn are negatives.
- Anchors whose distance lies between the positive and negative thresholds are disregarded.
- The remaining positive and negative anchors are used in a multi-task loss.
- The loss uses anchor classification and regression outputs together with their corresponding targets.
- Regression loss uses the proposal-ground-truth common indices and the ground-truth end-index to select supervised x-coordinates and length.
3.8. Anchor filtering for speed efficiency
LaneATT reduces the anchor set to improve speed by retaining anchors most frequently marked positive during training.
- 2,782 anchors comprise the full set, and this elevated number limits the model’s speed.The method identifies the anchor count as a main speed constraint.
- Anchors are filtered by counting how often each training-set anchor is marked positive.The same positivity criterion used during training determines which anchors are retained or discarded.
- Only the top-Nanc anchors are retained for both training and testing.
4. Experiments
LaneATT is evaluated on TuSimple, CULane, and LLAMAS using accuracy and efficiency measures. It achieves strong accuracy while maintaining high speed, with trade-offs depending on anchors and input size.
- Evaluation setup: LaneATT is evaluated on TuSimple, CULane, and LLAMAS using dataset-specific metrics and efficiency measures.Reported efficiency metrics include frames per second (FPS) and multiply-accumulate operations (MACs).
- TuSimple: TuSimple results are on par with state-of-the-art methods, while LaneATT is much faster than most alternatives.TuSimple results are already saturated because its scenes are less complex and its metric is permissive.
- TuSimple: 5.64% is LaneATT’s highest false-positive rate with ResNet-122, compared with 18.91% for the cited method’s lowest rate.
- CULane: CULane results show the highest F1 among compared methods while maintaining high efficiency, reaching 250 FPS.The ResNet-18 model surpasses the cited ResNet-34 model by almost 3% F1 while running at 250 versus 175 FPS.
- LLAMAS: LaneATT achieves F1 greater than 90% with all three backbones on LLAMAS.Only PolyLaneNet is available for comparison on this recent benchmark.
- Efficiency trade-offs: Reducing anchors can improve efficiency without a large F1 drop, but excessive reduction causes considerable F1 degradation.Input-size changes show similar trade-offs, with larger MAC reductions; anchor count affects training time more than inference because NMS is a bottleneck.
- Ablation study: Removing anchor-based pooling causes a massive performance drop, while the proposed attention mechanism significantly increases model performance.The pooling procedure enables use of a lightweight backbone, and Focal Loss outperforms Cross Entropy in the ablation.
5. Conclusion
LaneATT is a real-time single-stage lane detector that combines effective accuracy with high efficiency across three benchmarks. Its anchor-based attention improves performance, while reported trade-offs support practical deployment choices.
- LaneATT outperforms state-of-the-art models in extensive comparisons while remaining real-time and efficient.
- 171 vs. 30 FPS: on TuSimple, LaneATT achieves the second-highest reported F1, only 0.02% below the top-F1 method.
- +4.38% of F1: on CULane, LaneATT establishes a new state-of-the-art among similarly fast real-time methods.
- +93% F1: LaneATT achieves high performance on LLAMAS with all three evaluated backbones.
- The novel anchor-based attention mechanism significantly increases F1, and the paper reports efficiency trade-offs useful in practice.