PaperlayerView exact revision on arXiv

Source-linked AI summary

Rethinking Efficient Lane Detection via Curve Modeling

Zhengyang Feng, Shaohua Guo, Xin Tan, Ke Xu, Min Wang, Lizhuang Ma

arXiv:2203.02431v2cs.CVcs.AIcs.LG

TL;DR

Existing lane detectors often rely on indirect local representations or heuristics, while polynomial curve methods are difficult to optimize. The paper introduces an end-to-end Bézier curve detector with deformable feature flip fusion for lane symmetry. It reports favorable performance across three datasets, including a new state-of-the-art result on LLAMAS, while exceeding 150 FPS with fewer than 10 million parameters.

  • Problem

    Existing segmentation- and point detection-based lane detectors commonly require heuristic decoding or anchors, while polynomial curve coefficients are difficult to optimize.

  • Method

    BézierLaneNet models lanes with parametric on-image Bézier curves and uses deformable convolution-based feature flip fusion to exploit scene symmetry.

  • Results

    The method achieves favorable performance on three datasets, defeats existing methods on LLAMAS, runs above 150 FPS, and has fewer than 10 million parameters.

  • Takeaways & Limitations

    BézierLaneNet provides a fast, lightweight, fully end-to-end baseline for parametric curve-based lane detection.

  • Takeaways & Limitations

    Curvature coefficients remain difficult to generalize when training data are highly biased toward straight lanes, and feature flip fusion assumes scene symmetry from a front-mounted camera.

Abstract

from arXiv · show

This paper presents a novel parametric curve-based method for lane detection in RGB images. Unlike state-of-the-art segmentation-based and point detection-based methods that typically require heuristics to either decode predictions or formulate a large sum of anchors, the curve-based methods can learn holistic lane representations naturally. To handle the optimization difficulties of existing polynomial curve methods, we propose to exploit the parametric Bézier curve due to its ease of computation, stability, and high freedom degrees of transformations. In addition, we propose the deformable convolution-based feature flip fusion, for exploiting the symmetry properties of lanes in driving scenes. The proposed method achieves a new state-of-the-art performance on the popular LLAMAS benchmark. It also achieves favorable accuracy on the TuSimple and CULane datasets, while retaining both low latency (> 150 FPS) and small model size (< 10M). Our method can serve as a new baseline, to shed the light on the parametric curves modeling for lane detection. Codes of our model and PytorchAutoDrive: a unified framework for self-driving perception, are available at: https://github.com/voldemortX/pytorch-auto-drive .

1. Introduction

Lane detection methods commonly use local, indirect representations, while polynomial curve methods struggle to optimize abstract coefficients. The paper proposes a Bézier curve detector with symmetry-aware feature fusion and reports strong accuracy alongside speed and compactness.

  • Lane detection supports autonomous-driving decisions such as lane keeping, centering, and changing, and can use RGB images from front-mounted cameras.
  • Segmentation- and point detection-based methods represent lanes through indirect proxies, while polynomial curves use abstract coefficients that are difficult to optimize.
  • Bézier curves use on-image control points that provide stable, flexible lane parameterization and enable end-to-end learning with sparse proposals.
  • Feature flip fusion aggregates a feature map with its horizontally flipped version to exploit lane symmetry in front-mounted-camera scenes.
  • The method outperforms existing methods on LLAMAS with a lightweight ResNet-34 backbone and is evaluated across three popular lane-detection datasets.

2. Related Work

Related work spans segmentation, point detection, and curve-based lane representations. The paper argues that holistic curves avoid some post-processing costs but that prior polynomial approaches face optimization and performance limitations addressed by Bézier modeling.

  • Segmentation-based Lane Detection: Segmentation methods predict lane pixels and typically require heuristic decoding, clustering, or other post-processing to form line instances.
  • Point Detection-based Lane Detection: Point detection methods predict dense lane points but often depend on dataset-specific anchors and Non-Maximum Suppression.
  • Curve-based Lane Detection: Holistic curve representations can eliminate occlusions, post-processing, and fixed lane counts, but prior methods lag on challenging datasets and may converge slowly.
  • Curve-based Lane Detection: The paper replaces abstract polynomial coefficients with Bézier control points defined in image coordinates to improve optimization.

3. B´ezierLaneNet

BézierLaneNet models lanes as cubic Bézier curves and uses feature flip fusion to incorporate complementary lane information. Its sampling-based loss and bipartite matching support end-to-end prediction without heuristic decoding or NMS.

  • Bézier curve modeling: Cubic Bézier curves provide sufficient flexibility and stability for lane modeling, while higher-order curves add instability without substantial gains.Control-point coordinates are defined relative to image size.
  • Overall architecture: The pipeline strengthens encoder features with feature flip fusion, pools them to 1D proposals, applies two 1D convolutions, and predicts curves through classification and regression branches.The architecture uses a ResNet feature map, with 50 proposals on CULane.
  • Feature flip fusion: Feature flip fusion combines original and horizontally flipped feature maps, using deformable convolution offsets conditioned on the original map to align them.An auxiliary binary segmentation branch improves performance when combined with feature fusion by supporting more spatially accurate alignment.
  • Curve fitting and loss: Sampling loss measures curve distance by evaluating Bézier curves at sampled t values, avoiding the weak geometric correspondence of control-point L1 distance.The formulation uses 100 sampled points and is reported to ease convergence and reduce sensitivity to hyperparameters.
  • Curve fitting and loss: Training uses least-squares fitting to generate Bézier control points from annotated lane points, then performs one-to-one optimal bipartite matching between labels and predictions.Matching quality combines sampled curve distance and class score and is solved with the Hungarian algorithm.
  • Curve fitting and loss: The overall objective is a weighted sum of curve regression, lane-existence classification, and auxiliary segmentation losses.The weights are λ1 = 1, λ2 = 0.1, and λ3 = 0.75.

4. Experiments

Experiments evaluate B´ezierLaneNet on TuSimple, CULane, and LLAMAS, showing strong accuracy alongside speed, compactness, and robustness across comparisons and analyses.

  • Datasets and metrics: Experiments use TuSimple, CULane, and LLAMAS, with F1 metrics for CULane and LLAMAS and Accuracy, FPR, and FNR for TuSimple.CULane and LLAMAS match lines using 30-pixel widths and pixel IoU over 0.5; TuSimple accuracy counts correctly predicted online points.
  • Comparisons: B´ezierLaneNet advances curve-based state of the art by 6.85% on CULane and 6.77% on LLAMAS.It outperforms previous curve-based methods across all three datasets.
  • Comparisons: Over 2× faster than LSTR, B´ezierLaneNet also converges 4-5× faster.LSTR reports 98 and 97 FPS for its 1× and 2× models under the paper’s FPS evaluation.
  • Comparisons: B´ezierLaneNet outperforms segmentation-based methods in speed and accuracy, including SCNN by 0.92% on LLAMAS with ResNet-34.On CULane, its small models surpass large ResNet-101 RESA and SCNN models by 1-2%.
  • Comparisons: Compared with UFLD, B´ezierLaneNet is 3-5% better overall, faster on ResNet-34, and avoids UFLD’s large fully connected layers.UFLD is 0.9% lower on CULane Normal and 7.4%, 3.0%, and 3.2% worse on Shadow, Crowd, and Night.
  • Efficiency and analysis: The method remains fully end-to-end, exceeds 150 FPS, uses fewer than 10 million parameters, and maintains high accuracy across datasets.The analysis also reports robustness under adverse local-texture conditions and identifies curvature-distribution bias as a limitation.

5. Conclusions

The conclusion presents B´ezierLaneNet as a fully end-to-end detector using parametric B´ezier curves and feature flip fusion, with favorable accuracy, speed, and model size.

  • Conclusions: B´ezierLaneNet models lane lines with on-image parametric B´ezier curves in a fully end-to-end pipeline.The representation models continuous lane geometry without recurrent feature aggregation or heuristic anchors.
  • Conclusions: Feature flip fusion exploits driving-scene symmetry while deformable convolution accommodates slight asymmetries.The module is designed for front-mounted-camera settings.
  • Conclusions: The method achieves favorable performance on three datasets and defeats existing methods on the LLAMAS benchmark.The conclusion reports this outcome together with real-time speed and compact model size.
  • Conclusions: B´ezierLaneNet runs above 150 FPS and has fewer than 10 million parameters.FPS is measured using repeated single-image inference on a fixed platform, with warm-up procedures before trials.

B. Specifications for Compared Methods

The paper standardizes implementations and profiling for compared methods, documenting model-specific training, testing, post-processing, and evaluation choices.

  • Segmentation-based methods: The segmentation baseline is an optimized DeeplabV1 implementation without CRF, with separate lane classes and a lane-existence branch.Its training and testing scheme incorporates recent advances, and post-processing uses the lane-existence prediction.
  • Segmentation-based methods: The re-implemented SCNN adds spatial-CNN initialization and learning-rate warm-up to prevent gradient explosion from recurrent feature aggregation.The improved implementation achieves better performance than the original version.
  • Segmentation-based methods: SCNN is reported using the same-data re-implemented setting rather than a competition result trained with external data.The paper therefore excludes the externally trained 96.53 accuracy report from its comparison.
  • Segmentation-based methods: RESA omits dataset-specific no-lane-area cutout to preserve a generalizable comparison across methods.Other implementation differences were validated to improve CULane validation performance.
  • Point and grid-based methods: UFLD is profiled under the paper’s standard for parameter count and FPS, while its gridding-cell predictions use expected locations for point coordinates.Its TuSimple result is not reported because source-code training produced an almost 20% false-positive rate.
  • Curve-based methods: PolyLaneNet requires no post-processing and uses large random rotations, horizontal flips, and random crops during augmentation.Its reported profiling follows the paper’s standardized parameter and FPS scripts.
  • Point detection-based methods: LaneATT uses customized CUDA NMS, while the paper removes CPU B-Spline interpolation because it is slow and improves CULane by only about 0.2%.The profiling scripts were adjusted to match the paper’s parameter and FPS standards.

B.7. LSTR

The LSTR reimplementation uses dataset-specific training settings and reports improved CULane performance after tuning and correcting an implementation issue. Its polynomial formulation remains numerically unstable, with substantial fp32 training failures on CULane.

  • LSTR uses learning rate 2.5 × 10^-4 with batch size 20, training for 150 epochs on CULane and 2000 on TuSimple.Prediction thresholds are 0.95 for CULane and 0.5 for TuSimple.
  • 4% better CULane performance is achieved than the authors’ trial after hyper-parameter tuning and correcting the classification branch output channels.The original branch had three output channels instead of the correct two.
  • LSTR’s unbounded polynomial coefficients cause numerical instability and a roughly 30% fp32 training failure rate on CULane.The instability is especially problematic because bipartite matching requires precision.
  • The LSTR-Bezier ablation divides curve-loss weighting by 10 because its 100 sampled points with X and Y coordinates produce a loss scale about ten times larger.This modulation is intended to produce a loss landscape similar to the original LSTR loss.

B.8. B´ezierLaneNet

BézierLaneNet is implemented without post-processing and uses affine and horizontal-flip augmentation, but polynomial ablations fail to converge sufficiently for the CULane metric.

  • BézierLaneNet requires no post-processing and uses a 0.95 default prediction threshold, reduced to 0.5 for TuSimple.Its augmentations remove random lighting and use random affine transforms and horizontal flips.
  • The augmentation pipeline uses rotations up to 10 degrees, translations up to 50 pixels horizontally and 20 vertically, and scale changes up to 20%.
  • Polynomial ablations predict six coefficients for a third-order polynomial, including curve coefficients and start/end Y coordinates.Additional L1 losses are added for the start and end Y coordinates.
  • Even with the full BézierLaneNet setup and 150 CULane epochs, polynomial models do not converge well enough to pass the CULane metric.A sampling loss of 0.02 corresponds to a 32.8-pixel average X-coordinate error on the training set.

C. B´ezier Curve Implementation Details

The implementation samples Bézier curves efficiently and computes curve IoU through convex-polygon operations, while noting unresolved numerical and fitting limitations for some loss formulations.

  • Fast Sampling: Bézier curves are fast-sampled by precomputing Bernstein-basis values for fixed t values, leaving one matrix multiplication at runtime.
  • Fast Sampling: The four displayed sampling rows evaluate cubic Bézier curves at fixed parameter values using control points P0 through P3.The formula is efficiently implemented with matrix multiplication, with u0 = 1 − t0 and u1 = 1 − t1.
  • IoU Computation: Bézier-curve IoU is converted into convex-polygon IoU by forming curve convex hulls, sorting vertices, finding intersections, and summing polygon areas.The intersection polygon contains edge-intersection vertices and vertices lying inside or on both polygons.
  • Intersections and Insiders: Polygon edges are represented as ax + by = c, and pairwise line intersections are computed algebraically before retaining points on the respective segments.
  • Intersections and Insiders: A convex polygon contains a point when the point lies on the same side of every edge; the sign formula distinguishes right, left, and on-edge cases.The implementation uses matrix operations and indexing for batched computation.
  • GIoU: The proposed GIoU implementation uses a smallest enclosing convex object, while its PyTorch IoU computation runs at about five times rectangle-IoU runtime.The enclosing object can be a convex hull or an enclosing rectangle.
  • GIoU: The Bézier IoU loss does not converge on lane datasets because mostly straight lanes yield extremely small convex-hull areas and numerical instabilities.It succeeded in a toy polygon-fitting experiment but not in lane-dataset optimization.

E. GT and Prediction Matching Prior

The matching prior restricts matched predictions to local maxima in classification logits, producing activations with a spatial pattern resembling driving scenes.

  • A local maximum prior restricts matched predictions to locations with locally maximal classification logits.The prior is used instead of a centerness prior.
  • The learned classification-logit activations exhibit a structure similar to an actual driving scene.

F. Extra Results

The extra results include a feature-flip-fusion ablation and discuss scope boundaries related to viewing geometry and runtime stability.

  • Ablation Study: Table 9 measures Feature Flip Fusion on TuSimple Accuracy and LLAMAS F1 before and after adding the module.Results are averaged three times with the ResNet-34 backbone because the ablations were not sufficiently stable for a clear single-run difference.
  • Discussions: Feature flip fusion is not suited to lateral-view lane detection used for estimating drivable-area border distance.The paper states that this application contradicts the module’s underlying symmetry assumption and is outside the paper’s focus.
  • Discussions: Complex conditional lane decoding can produce unstable runtime depending on the input image, which is undesirable for real-time systems.

H. Qualitative Results

Qualitative results show accurate predictions across several driving conditions, while also exposing threshold-, occlusion-, illumination-, and annotation-related failure cases.

  • TuSimple and CULane: The model fits highway curves and predicts lanes under congestion and shadows, but occlusion causes inaccurate CULane predictions.On TuSimple, errors appear mainly on the far side where projection destroys image detail.
  • TuSimple and CULane: Low-threshold settings produce a high false-positive rate on TuSimple, although some false positives correspond to meaningful lanes omitted from center-line annotations.
  • Figure Encoding: The figure marks false positives red, true positives green, ground truth blue, and Bézier control points with solid circles.Blue ground-truth lines that are barely visible are precisely covered by green predictions; images are slightly resized and intended for color viewing at 2× scale.
  • LLAMAS: LLAMAS results are accurate for clear straight lines and large curvatures, but low illumination can cause failures and confuse LIDAR- and HD-map-based annotations.A white arrow misleads the unsupervised annotation in one low-illumination scene.
Loading 2203.02431v2…