PaperlayerView exact revision on arXiv

Source-linked AI summary

CLRNet: Cross Layer Refinement Network for Lane Detection

Tu Zheng, Yifei Huang, Yang Liu, Wenjian Tang, Zheng Yang, Deng Cai, Xiaofei He

arXiv:2203.10350v1cs.CV

TL;DR

Accurate lane detection must reconcile high-level semantics with detailed local patterns, especially when lanes are occluded or visually weak. CLRNet detects lanes from high-level features, refines them with low-level features, adds ROIGather and Line IoU loss, and outperforms current state-of-the-art methods on three benchmarks.

  • Problem

    Lane detection remains challenging because high-level semantics and low-level localization detail are both needed, while lanes may lack visible evidence under occlusion or extreme lighting.

  • Method

    CLRNet progressively refines lane priors from high-level to low-level features, uses ROIGather for global context, and applies Line IoU loss to regress each lane as a whole unit.

  • Results

    CLRNet outperforms current state-of-the-art lane-detection methods on the CULane, LLAMAS, and Tusimple benchmarks.

  • Takeaways & Limitations

    Combining semantic context, local detail, global feature relations, and whole-lane regression improves lane-detection performance and localization accuracy.

Abstract

from arXiv · show

Lane is critical in the vision navigation system of the intelligent vehicle. Naturally, lane is a traffic sign with high-level semantics, whereas it owns the specific local pattern which needs detailed low-level features to localize accurately. Using different feature levels is of great importance for accurate lane detection, but it is still under-explored. In this work, we present Cross Layer Refinement Network (CLRNet) aiming at fully utilizing both high-level and low-level features in lane detection. In particular, it first detects lanes with high-level semantic features then performs refinement based on low-level features. In this way, we can exploit more contextual information to detect lanes while leveraging local detailed lane features to improve localization accuracy. We present ROIGather to gather global context, which further enhances the feature representation of lanes. In addition to our novel network design, we introduce Line IoU loss which regresses the lane line as a whole unit to improve the localization accuracy. Experiments demonstrate that the proposed method greatly outperforms the state-of-the-art lane detection approaches.

1. Introduction

Lane detection requires both global semantic context and local detail: low-level features can confuse landmarks with lanes, while high-level features may localize lanes imprecisely. CLRNet addresses this complementarity with cross-layer refinement, ROIGather, and Line IoU loss, achieving strong benchmark performance.

  • Motivation: Low-level features can mistake landmarks for lanes because they lack global context, while high-level features can produce inaccurate lane localization.Lane semantics and local appearance are complementary requirements for accurate detection.
  • Motivation: Occlusion by vehicles and extreme lighting can leave little visual evidence for lane presence.These cases are illustrated as difficult detection scenarios in Figure 1.
  • Proposed approach: CLRNet first detects lanes using high-level semantic features, then progressively refines their locations with low-level detailed features.The framework is designed to fully use complementary feature levels for lane detection.
  • Proposed approach: ROIGather enhances lane-feature representations by building relations between ROI lane features and the whole feature map to capture global context.The module is also described as plug-compatible with other networks.
  • Proposed approach: Line IoU loss regresses lane lines as whole units and improves performance compared with standard smooth-l1 loss.The loss is tailored to lane detection and targets localization accuracy.
  • Results: CLRNet achieves state-of-the-art accuracy across the CULane, Tusimple, and LLAMAS benchmarks, while the study adopts mF1 for localization comparison.The reported evaluation covers three lane-detection benchmarks.

2. Related Work

Lane-detection methods are commonly organized by representation: segmentation-based, anchor-based, and parameter-based approaches. The proposed CLRNet overview emphasizes refining lane priors across feature levels, contrasting with these established strategies.

  • Method categories: CNN-based lane detection methods are categorized as segmentation-based, anchor-based, or parameter-based according to lane representation.This taxonomy distinguishes pixel-wise prediction, reference-based regression, and curve-parameter regression.
  • Segmentation-based methods: Segmentation-based methods typically use pixel-wise prediction, with SCNN gathering spatial context but remaining slow for real-time applications.RESA targets real-time feature aggregation, while CurveLane-NAS uses computationally expensive neural architecture search.
  • Anchor-based methods: Anchor-based methods include line-anchor and row-anchor approaches that regress lanes from predefined references or predict cells across image rows.Line-CNN, LaneATT, SGNet, UFLD, and CondLaneNet exemplify these variants.
  • Parameter-based methods: Parameter-based methods regress curve parameters, offering fewer regression parameters but sensitivity to predicted parameters.PolyLaneNet uses polynomial regression, while LSTR incorporates road structure and camera pose with a transformer.
  • CLRNet overview: CLRNet’s overview shows lane priors refined from high-level to low-level features, with contextual feature extraction and Line IoU-based regression.The figure presents the network’s feature pyramid, refinement process, and lane-prior heads.

3. Approach

CLRNet combines high-level semantics for coarse lane detection with low-level detail for refinement, while ROIGather adds global context and Line IoU loss regresses lanes as whole units.

  • Lane Representation: Lane Prior represents each lane as equally vertically sampled 2D points with foreground/background probabilities, length, start coordinates, angle, and point offsets.This representation is designed for the strong shape prior of thin, long lanes.
  • Cross Layer Refinement: Cross-layer refinement detects lanes from high-level features, then progressively refines their parameters using increasingly detailed feature levels.The refinement sequence starts at L0 and approaches L2; each refinement uses ROI lane features and two fully connected layers.
  • ROIGather: ROIGather computes attention between lane-prior ROI features and the global feature map, adds the aggregated global context back to the ROI features.This supplies contextual information when lanes are occupied or blurred and local visual evidence is insufficient.
  • Line IoU Loss: Line IoU loss treats the lane as a whole unit by integrating IoU over sampled positions of extended predicted and ground-truth segments.Unlike pointwise distance loss, it is designed for overall lane localization and is differentiable.
  • Training: Training combines focal classification loss, smooth-L1 regression for geometric parameters, and Line IoU loss on assigned samples.An auxiliary segmentation loss may be added during training without inference cost.

4. Experiment

Experiments evaluate CLRNet on three lane-detection benchmarks using F1 and mF1 metrics, reporting state-of-the-art accuracy, efficiency, qualitative behavior, and component ablations.

  • Datasets: Three benchmarks—CULane, Tusimple, and LLAMAS—are used to evaluate lane-detection performance.CULane includes challenging categories, Tusimple contains highway scenes, and LLAMAS is a recently released large-scale benchmark.
  • Evaluation metrics: mF1 averages F1 measured at IoU thresholds from 0.5 through 0.95, emphasizing localization accuracy at higher overlaps.The metric is introduced alongside conventional F1 evaluation for CULane and LLAMAS.
  • Benchmark results: 80.47 F1@50 is achieved on CULane, while CLRNet also reaches 206 FPS on an NVIDIA 1080Ti GPU with TensorRT.The reported CULane results include gains over CondLaneNet and LaneATT in both F1@50 and mF1.
  • Benchmark results: 7.7 F1@50 over PolyLaneNet and 2.4 F1@50 over LaneATT are reported on the LLAMAS test set, with nearly 2 points higher mF1 than LaneAF.LaneAF is reported at near 20 FPS on the validation dataset, whereas the comparison emphasizes CLRNet’s localization performance.
  • Benchmark results: 0.6% F1 over the previous state of the art is achieved on Tusimple, where performance differences among methods are otherwise very small.The authors characterize the Tusimple result as a new state of the art on a nearly saturated benchmark.
  • Ablation studies: 51.90 to 52.80 mF1 follows adding Line IoU loss, refinement raises mF1 to 54.74, and ROIGather adds a further 0.5%.The ablation attributes consistent metric improvements to combining high- and low-level features, while ROIGather enhances lane representation using global context.

5. Conclusion

CLRNet combines high-level semantic features with low-level local detail for lane detection, while ROIGather and Line IoU loss address contextual representation and whole-lane regression.

  • CLRNet predicts lanes from high-level features and progressively refines their locations using fine-detail features.This design uses semantic information for detection and local features for precise localization.
  • ROIGather enhances lane-feature representations by building relations with all pixels to address missing visual evidence.
  • Line IoU loss regresses the lane line as a whole unit and improves performance compared with smooth-l1 loss.
  • Experiments on CULane, LLamas, and Tusimple show that CLRNet outperforms current state-of-the-art lane detection methods.
Loading 2203.10350v1…