Source-linked AI summary
LaneAF: Robust Multi-Lane Detection with Affinity Fields
Hala Abualsaud, Sean Liu, David Lu, Kenny Situ, Akshay Rangesh, Mohan M. Trivedi
TL;DR
Lane detection must handle diverse road infrastructure, changing lane topology, and unclear markings while distinguishing separate lane instances. LaneAF predicts binary masks with horizontal and vertical affinity fields, then decodes them row by row into lane clusters, achieving state-of-the-art performance on LLAMAS and strong benchmark results on TuSimple.
Problem
Lane detection must distinguish lane instances despite diverse infrastructure, merging and splitting lanes, and worn or unclear markings.
Method
LaneAF predicts binary lane segmentation masks and horizontal and vertical per-pixel affinity fields, which are decoded row by row to cluster foreground pixels into lane instances.
Results
LaneAF achieves state-of-the-art LLAMAS performance with an F1 score of 96.07% and a TuSimple false positive rate of 0.0280.
Takeaways & Limitations
Affinity fields support clustering lanes of arbitrary widths and help the model maintain accuracy for lane points farther from the camera.
Abstract
from arXiv · showhide
This study presents an approach to lane detection involving the prediction of binary segmentation masks and per-pixel affinity fields. These affinity fields, along with the binary masks, can then be used to cluster lane pixels horizontally and vertically into corresponding lane instances in a post-processing step. This clustering is achieved through a simple row-by-row decoding process with little overhead; such an approach allows LaneAF to detect a variable number of lanes without assuming a fixed or maximum number of lanes. Moreover, this form of clustering is more interpretable in comparison to previous visual clustering approaches, and can be analyzed to identify and correct sources of error. Qualitative and quantitative results obtained on popular lane detection datasets demonstrate the model's ability to detect and cluster lanes effectively and robustly. Our proposed approach sets a new state-of-the-art on the challenging CULane dataset and the recently introduced Unsupervised LLAMAS dataset.
I. INTRODUCTION
Lane detection supports autonomous driving and remains challenging across varied road infrastructure and lane configurations. LaneAF addresses the difficulty of separating lanes by combining binary segmentation with affinity-field clustering that supports an unknown number of lane instances.
- Motivation: Lane detection influences vehicle guidance, steering, interaction with road users, and trajectory prediction for autonomous driving.The paper also links precise lane detection to handling lane endings, merges, splits, and unclear markings.
- Motivation: Varied infrastructure, lane markings, topology changes, and worn markings make reliable lane detection difficult.Different line types also carry implications for how vehicles may interact with them.
- Contributions: LaneAF predicts binary masks and affinity fields, then decodes them into multiple lane instances instead of assigning each lane a separate segmentation class.This contrasts with standard anchor-free multi-class lane segmentation.
- Problem and approach: Binary segmentation detects lane pixels but cannot distinguish separate lane entities, motivating a clustering scheme based on affinity fields.The proposed horizontal and vertical affinity fields encode lane structure for instance separation.
- Contributions: The affinity-field representation is not tied to a predetermined lane count, making the model agnostic to how many lanes appear on the road.The paper identifies efficient affinity-field generation and decoding into an unknown number of clustered lane instances as a contribution.
II. RELATED RESEARCH
Prior lane-detection methods span feature-based, model-based, segmentation-based, instance-clustering, and anchor-based formulations. LaneAF builds on binary lane segmentation and clustering while using affinity fields to associate pixels into lane instances.
- Prior approaches: Traditional feature-based and model-based lane detectors are impractical in real-world scenes because they require ideal road conditions.Data-driven methods subsequently became common for detecting lane boundaries and regions.
- Segmentation approaches: Deep lane detection is commonly formulated as semantic or multi-class segmentation, with each lane represented as a separate class.Related methods include CNN-RNN models, embedding losses, and GAN-based approaches.
- Instance clustering: Other approaches perform binary segmentation of all lanes before clustering pixels into individual lane instances.This line of work addresses instance separation after detecting lane foreground pixels.
- LaneAF: LaneAF predicts binary lane segmentation masks and per-pixel horizontal and vertical affinity fields, then clusters foreground pixels into lanes during post-processing.The affinity fields encode directions used for horizontal and vertical association.
- LaneAF: LaneAF generates affinity fields from ground-truth segmentation masks and decodes predicted fields row by row from the bottom upward.The paper provides separate procedures for creating and decoding the fields.
A. Network Backbone
LaneAF uses a DLA-34 fully convolutional backbone to predict binary masks and two affinity fields. During decoding, horizontal affinities form row-wise clusters and vertical affinities associate them with lanes from bottom to top.
- A. Network Backbone: The DLA-34 backbone provides hierarchical feature aggregation for fusing resolutions and refining spatial representations.The architecture uses iterative and hierarchical deep aggregation mechanisms.
- B. Affinity Fields: The model predicts a binary segmentation mask plus HAF and VAF vector fields, with HAF clustering pixels horizontally and VAF clustering them vertically.Each field assigns a unit vector to image locations, and HAF supports lanes of arbitrary widths.
- B. Affinity Fields: The decoding algorithm scans foreground pixels row by row, applies a clustering threshold, updates lane endpoints, and spawns new lanes when clusters remain unassigned.The binary mask, HAF, VAF, and threshold are the principal decoder inputs.
- B. Affinity Fields: Ground-truth affinity fields are created row by row from bottom to top using lane segmentation masks.The HAF uses mean lane position within a row, while the VAF points toward the lane’s mean location in the previous row.
- B. Affinity Fields: At test time, foreground pixels are first grouped horizontally using HAF predictions, then assigned to existing lanes using VAF predictions.The decoder repeats this process from the bottom row upward and can create new lanes for unassigned clusters.
C. Losses
The model uses separate losses for binary lane segmentation and affinity-field prediction, then sums them into the total training loss.
- Weighted binary cross-entropy trains the segmentation branch for imbalanced foreground and background pixels.The sigmoid-normalized output is compared with the target, with foreground pixels receiving increased penalty through a weight.
- An additional intersection over union loss is used for the imbalanced binary segmentation branch.
- L1 regression is applied to foreground locations in both the horizontal and vertical affinity-field branches.
- The total training loss is the sum of the individual losses for the model’s prediction heads.
A. Implementation Details
Implementation rescales inputs and targets to match the DLA-34 output resolution, accelerates row-wise decoding, and uses scheduled Adam optimization with early stopping.
- Rescaling inputs to half resolution and targets to one-eighth resolution matches the backbone’s downsampling and speeds decoding.The DLA-34 outputs are downsized by a factor of 4, while decoding processes only an eighth of the original rows.
- Decoding typically takes about 15-20ms on a modern CPU without code optimizations.The operation is entirely CPU-based and is reported not to affect overall latency.
- Training uses Adam with a 0.0001 learning rate, 0.001 weight decay, 40 epochs, and fivefold learning-rate reductions every 10 epochs.
- The foreground loss weight is set to 9.6, reflecting the approximate background-to-foreground pixel ratio in public datasets.Early stopping retains the model parameters that perform best on the validation set.
B. Datasets
The study benchmarks on TuSimple, CULane, and LLAMAS, which cover different lane-detection conditions and dataset characteristics.
- TuSimple contains highway scenes with up to five lanes across good and fair weather, daytime lighting, and traffic conditions.
- CULane provides more data and nine test categories containing complex scenarios, including challenging lighting conditions.
- LLAMAS is a newer dataset with many highway-recording images generated from an unspecified source in the supplied passage.
C. Metrics
Evaluation uses dataset-specific lane-detection metrics, including TuSimple accuracy with false-positive and false-negative rates and F1 for CULane and LLAMAS.
- TuSimple evaluation reports accuracy, false-positive rate, and false-negative rate using the benchmark’s official metrics.
- TuSimple accuracy is based on correctly predicted lane points relative to ground-truth lane points.Npred denotes correctly predicted lane points, while Ngt denotes ground-truth lane points.
- CULane uses F1 as its only metric, with F1 based on intersection over union.
- The LLAMAS lane-approximations benchmark also uses the F1 measure.The measure uses precision and recall derived from true positives, false positives, and false negatives.
D. Ablation Experiments
Ablations on TuSimple validate the LaneAF design, showing that its losses, training transformations, output resolution, and affinity-based clustering materially affect performance.
- Removing IoU loss, weighted BCE loss, or random transformations decreased accuracy and increased false-positive and false-negative rates.Without weighted BCE, F1 dropped significantly.
- The experiments were conducted on the TuSimple validation set and reported in Table II.
- Down-sampling factor 4 achieved the best results, while factor 8 caused the largest accuracy degradation and factor 2 increased runtime.
- The ablation results validate binary segmentation followed by separate affinity-field clustering over standard multi-class segmentation.
E. Results
LaneAF achieves strong benchmark performance across TuSimple, CULane, and LLAMAS while using affinity-field decoding to separate nearby lane instances. Qualitative results show robust clustering in converging lanes and challenging road conditions, with horizon proximity remaining a limitation.
- Benchmark results: LaneAF achieves a false positive rate of 0.0280 on TuSimple, although its accuracy remains slightly below PINet, ENet-SAD, and SCNN.Six training runs produced an accuracy standard deviation of 0.12, indicating consistent results.
- Qualitative analysis: Affinity-field decoding separates lane instances even when lanes converge, using horizontal and vertical vectors to cluster pixels row by row.HAF vectors point toward lane centers, while VAF vectors point toward the next row’s mean lane-pixel location.
- Benchmark results: 77.41% F1 on CULane surpasses models of similar size and LaneATT with its largest backbone, while setting new benchmarks in difficult categories.LaneAF leads in categories including Dazzle, Shadow, No line, Curve, and Night.
- Benchmark results: 96.07% F1 on LLAMAS surpasses LaneATT’s best model by over 2%, with the gap attributed to higher recall.The result is reported as state-of-the-art performance on LLAMAS.
- Qualitative analysis: Far-away lane points show little to no degradation, but horizon clusters can occasionally be assigned to non-optimal lanes when lane lines are close.The horizon error is linked to the proximity of lane lines.
- Qualitative analysis: Qualitative results span curved, merging, and splitting highways, difficult lighting, and additional highway scenes across TuSimple, CULane, and LLAMAS.One TuSimple example falsely detects an airplane contrail as a lane line.
V. CONCLUDING REMARKS
LaneAF combines binary masks with horizontal and vertical affinity fields to cluster pixels into lane instances through lightweight, interpretable decoding. It achieves strong benchmark results, including state-of-the-art performance on CULane and LLAMAS.
- Binary masks and per-pixel affinity fields enable lightweight row-by-row clustering into variable-width lane instances without assuming a fixed or maximum lane count.
- The interpretable clustering process can be analyzed to identify and correct error sources, while ablations validate its effectiveness over standard multi-class segmentation.
- 77.41% total F1 on CULane sets a new state-of-the-art, surpassing deeper and more complex models.
- LaneAF achieves the lowest reported TuSimple false positive rate of 0.0280 and a state-of-the-art LLAMAS F1 score by +2%.