PaperlayerView exact revision on arXiv

Source-linked AI summary

Ultra Fast Deep Lane Detection with Hybrid Anchor Driven Ordinal Classification

Zequn Qin, Pengyi Zhang, Xi Li

arXiv:2206.07389v1cs.CV

TL;DR

Lane detection needs both efficient inference and robustness when occlusion or extreme lighting removes visible cues. The paper uses sparse coordinates on hybrid anchors with global-feature ordinal classification, and reports state-of-the-art speed and accuracy, including a lightweight model exceeding 300 FPS.

  • Problem

    Dense segmentation-based lane detection struggles with efficiency, while severe occlusions and extreme lighting can remove the visual information needed to locate lanes.

  • Method

    The method represents lanes with sparse coordinates on hybrid row and column anchors and learns those coordinates through top-down ordinal classification using global features.

  • Results

    The method achieves state-of-the-art speed and accuracy, with a lightweight ResNet-18 version reaching 300+ FPS.

  • Takeaways & Limitations

    Anchor-driven global classification provides a formulation intended to address both computational efficiency and lane detection without visible lane clues.

  • Takeaways & Limitations

    Anchor arrangements remain fixed and handcrafted, while the method assumes each horizontal or vertical line intersects each lane instance at most once.

Abstract

from arXiv · show

Modern methods mainly regard lane detection as a problem of pixel-wise segmentation, which is struggling to address the problems of efficiency and challenging scenarios like severe occlusions and extreme lighting conditions. Inspired by human perception, the recognition of lanes under severe occlusions and extreme lighting conditions is mainly based on contextual and global information. Motivated by this observation, we propose a novel, simple, yet effective formulation aiming at ultra fast speed and the problem of challenging scenarios. Specifically, we treat the process of lane detection as an anchor-driven ordinal classification problem using global features. First, we represent lanes with sparse coordinates on a series of hybrid (row and column) anchors. With the help of the anchor-driven representation, we then reformulate the lane detection task as an ordinal classification problem to get the coordinates of lanes. Our method could significantly reduce the computational cost with the anchor-driven representation. Using the large receptive field property of the ordinal classification formulation, we could also handle challenging scenarios. Extensive experiments on four lane detection datasets show that our method could achieve state-of-the-art performance in terms of both speed and accuracy. A lightweight version could even achieve 300+ frames per second(FPS). Our code is at https://github.com/cfzd/Ultra-Fast-Lane-Detection-v2.

1 INTRODUCTION

Lane detection must balance fast inference on constrained vehicle hardware with robustness when occlusion or lighting removes visible lane cues. The paper addresses both through sparse hybrid anchors and global-feature ordinal classification, reporting state-of-the-art speed and performance.

  • Challenges: Lane detection is fundamental to autonomous driving and ADAS, but dense segmentation pipelines make fast inference difficult on constrained computing devices.The task must be executed heavily to provide instant perception results for downstream systems.
  • Challenges: Severe occlusions and extreme lighting can leave little or no visible lane information, requiring contextual clues such as road shape and vehicle-heading trends.The formulation seeks to use global and long-range information when direct visual evidence is absent.
  • Proposed formulation: Sparse coordinates on predefined anchors reduce computational cost, while global-feature classification provides a receptive field spanning the whole input.The method represents lanes with key points and learns their coordinates in a classification-based manner.
  • Hybrid anchor system: Hybrid anchors assign row anchors to ego lanes and column anchors to side lanes because each anchor orientation localizes its corresponding lane type more accurately.This design addresses the magnified localization error observed when one anchor system is used for all lanes.
  • Ordinal classification: Ordinal classification combines base and expectation losses so predicted coordinates preserve ordinal relationships and improve lane localization.The method uses mathematical expectation rather than argmax to obtain continuous predicted classes.
  • Results: The fastest model exceeds 300 FPS with performance comparable to the state of the art, and the paper reports stronger results with a 6.3-point improvement at the same speed.The authors evaluate the formulation on four major lane detection datasets.

2 RELATED WORK

Lane detection research has largely used bottom-up image processing, heat-map, and segmentation formulations, while this work presents a top-down approach using row and hybrid anchor representations.

  • Traditional lane detection methods use low-level image processing in a bottom-up manner, including color models, edge extraction, and geometric techniques.
  • Deep learning approaches commonly formulate lane detection with heat maps or segmentation networks, with later methods exploring affinity fields, neural architecture search, and local pattern modeling.
  • Other formulations model lanes with LSTM-based line structures, sequential direction prediction, instance segmentation, or differential geometry.
  • This work differs from previous bottom-up methods by using top-down modeling with row and hybrid anchor representations to reduce learning difficulty and speed up detection.

3 ULTRA FAST LANE DETECTION

The method represents lanes sparsely on hybrid row and column anchors, then predicts anchor coordinates and lane existence through ordinal classification. This design reduces localization error and computational cost while using expectation-based inference for coordinates.

  • Lane Representation with Anchors: Hybrid anchors assign each lane to the more vertical anchor type, reducing the magnified localization error caused by poorly aligned row or column anchors.Row anchors are weak for horizontal side lanes, while column anchors are weak for vertical ego lanes.
  • Lane Representation with Anchors: Lanes are encoded as fixed-size coordinate targets from intersections with predefined anchors, with -1 marking anchors lacking a lane intersection.The target is divided into row-anchor and column-anchor components.
  • Anchor-driven Network Design: The network flattens backbone features into a classifier with localization and existence branches for row and column anchors.The localization branch predicts anchor coordinates, while the existence branch filters absent lanes.
  • Ordinal Classification Losses: Ordinal classification maps anchor coordinates to ordered classes, allowing the coordinate-learning problem to use classification losses that exploit neighboring-class relationships.The method combines a base classification loss with an expectation loss.
  • Ordinal Classification Losses: Expectation inference obtains lane coordinates from the predicted class distributions rather than argmax, and expectation loss pushes their mathematical expectation toward ground truth.The expectation loss reduces mean average localization error relative to conventional classification.
  • Analysis and Discussion: For CULane settings, the classification head requires 0.04 GMac versus 30.86 GMac for the segmentation head, while the proposed head uses less than 5% of inference time.The reported ideal calculations are 1.54 × 10^4 for classification and 2.56 × 10^6 for segmentation.

4 EXPERIMENTS 1

Experiments evaluate the method on four lane-detection datasets, including varied lighting, urban scenarios, curved lanes, and map-collected data. Ablations support hybrid anchors, ordinal losses, and selected classification dimensions and anchor counts, while benchmark results show strong speed and accuracy.

  • Experimental settings: Experiments cover TuSimple, CULane, CurveLanes, and LLAMAS, with CULane spanning nine urban scenarios and CurveLanes emphasizing curved cases.TuSimple uses stable highway lighting, whereas CULane includes conditions such as dazzle light, night, occlusion-related settings, shadows, and missing lines.
  • Ablation studies: The hybrid anchor system significantly improves performance over row-only and column-only anchors.The ablation is conducted with a ResNet-18 backbone under the same settings as the main experiments.
  • Ablation studies: Expectation-based classification outperforms standard classification, while classification-based methods consistently outperform regression-based methods.The expectation loss also reduces mean average localization error relative to conventional classification.
  • Ablation studies: Classification dimensions create a trade-off: increasing them first improves and then reduces performance as classification difficulty and localization precision change.Smaller dimensions simplify classification but represent broader location ranges; larger dimensions narrow those ranges but make classification harder.
  • Ablation studies: Increasing row-anchor counts generally improves performance but gradually reduces detection speed, leading to Nrow=18 and Ncol=40 for ResNet-18 models.For ResNet-34 models, the selected settings are Nrow=72 and Ncol=80.
  • Results on four datasets: The fastest model achieves 300+ FPS, while the method delivers comparable performance to state-of-the-art methods and is 41.7 times faster than SCNN.On CurveLanes, it improves performance while maintaining faster speed than CurveLane-S; on LLAMAS, it achieves the best performance and fastest speed.

5 CONCLUSION

The paper proposes a hybrid-anchor, ordinal-classification formulation that learns sparse lane coordinates from global features to achieve speed and accuracy. A lightweight ResNet-18 version reaches 300+ FPS, while anchor placement remains fixed and handcrafted.

  • 5 CONCLUSION: The proposed formulation combines a hybrid anchor system with an ordinal classification pipeline for sparse lane-coordinate prediction from global features.The stated goal is to achieve both remarkable speed and accuracy.
  • 5 CONCLUSION: A lightweight ResNet-18 version achieves 300+ FPS.
  • 5 CONCLUSION: Anchor arrangements remain fixed and handcrafted, motivating automatic, dynamic, rotatable, and nonuniform anchors as potential future directions.

1 OVERVIEW

The supplementary materials extend the paper with additional evidence, discussion, visualization, and implementation-related detail.

  • 1 OVERVIEW: The supplementary materials add statistics, discussions, illustrations, and tables covering error bands, hardware FPS, hybrid-anchor advantages, and post-processing details.

2 VALIDATING THE MAGNIFIED LOCALIZATION PROBLEM

The paper validates the magnified localization problem by comparing localization errors across lane angles with a mathematically derived error band. The measured trends fit the predicted ε/sinθ relationship.

  • 2 VALIDATING THE MAGNIFIED LOCALIZATION PROBLEM: Localization-error statistics across lane angles fit the mathematically derived ε/sinθ error band.The comparison is presented for a model using row anchors.
  • 2 VALIDATING THE MAGNIFIED LOCALIZATION PROBLEM: The reported agreement between measured trends and the derived error band supports the existence of the magnified localization problem.

3 THE SPEED ON THE CPUS AND EMBEDD CHIPS

The method achieves real-time or near-real-time speed across CPUs and embedded hardware, including the Nvidia Jetson TX2, at both tested resolutions.

  • The method is tested on AMD EPYC 7282, Intel Xeon E5-2678, Intel i9 10990K, and NVIDIA Jetson TX2 devices.
  • The authors report real-time or near-real-time speed even on CPUs and embedded chips.

4 DEMONSTRATING OF THE ADVANTAGE OF HYBRID ANCHOR

Hybrid anchors combine row and column anchor systems to model lanes according to their orientation. Row anchors better model vertical lanes, while column anchors better fit horizontal lanes.

  • Row anchors are more appropriate for vertical lanes because vertical lanes vary substantially in the vertical direction and intersect row anchors more often.
  • Column anchors are more appropriate for horizontal lanes because horizontal lanes have the corresponding orientation-specific anchor preference.
  • With six anchors, row anchors model vertical lines better while column anchors fit horizontal lines better.

5 THE HIDDEN PREMISE OF THE ANCHOR-BASED METHOD

The anchor-based method assumes that each horizontal or vertical line intersects each lane instance only once. Under multiple intersections, both representations remain valid, although the bottom-first result can be shorter.

  • The method assumes each horizontal or vertical line has only one intersection point with each lane instance.This premise is examined for extremely curved or complex roads with multiple intersections.
  • Top-first representation preserves the top intersection, whereas bottom-first representation preserves the bottom intersection.
  • Under multiple intersections, both restored-lane representations remain valid despite being slightly inaccurate or incomplete.
  • The bottom-first result is slightly shorter than the ground truth, but its major lower lane portions remain intact; the method uses bottom-first representation.

6 POST-PROCESSING

Post-processing removes very short invalid lane predictions, using different thresholds for the original row-based and hybrid-anchor methods. This threshold difference explains the reported performance discrepancy between 68.4 and 66.09.

  • The original row-based method uses a loose post-processing threshold of at least two lane points to remove very short invalid predictions.Horizontal lanes may have few intersections with row anchors, motivating the loose threshold.
  • Multiple-intersection examples distinguish extreme curved lanes, anchors with two intersections, and bottom-first restored lanes.
  • The hybrid-anchor method can use a more aggressive threshold because it represents horizontal and vertical lanes with denser lane points.
  • 68.4 is the row-based method with loose post-processing, whereas 66.09 uses aggressive hybrid-anchor post-processing.
Loading 2206.07389v1…