Source-linked AI summary
End-to-End Lane Marker Detection via Row-wise Classification
Seungwoo Yoo, Heeseok Lee, Heesoo Myeong, Sungrack Yun, Hyoungwoo Park, Janghoon Cho, Duck Hoon Kim
TL;DR
Lane marker detection is important for camera-based positioning, but conventional pixel-level segmentation relies on complex post-processing. The paper proposes end-to-end row-wise classification with horizontal reduction for direct vertex prediction, achieving state-of-the-art performance on TuSimple and CULane while showing failures under reflections, severe curves, and occlusions.
Problem
Lane marker detection is a key camera-perception and positioning task, while conventional pixel-level segmentation requires complex post-processing to impose lane-marker structure.
Method
E2E-LMD treats detection as row-wise classification, uses horizontal reduction to model lane-marker representations, and directly outputs vertices for argmax-based localization.
Results
The proposed method achieves state-of-the-art performance without complex post-processing and is on par with or outperforms recent methods on TuSimple and CULane.
Takeaways & Limitations
The framework captures lane-marker representations efficiently and may also apply to general polygon prediction and semantic or instance segmentation.
Takeaways & Limitations
The method often fails under bonnet reflections, severe curves, or occlusions.
Abstract
from arXiv · showhide
In autonomous driving, detecting reliable and accurate lane marker positions is a crucial yet challenging task. The conventional approaches for the lane marker detection problem perform a pixel-level dense prediction task followed by sophisticated post-processing that is inevitable since lane markers are typically represented by a collection of line segments without thickness. In this paper, we propose a method performing direct lane marker vertex prediction in an end-to-end manner, i.e., without any post-processing step that is required in the pixel-level dense prediction task. Specifically, we translate the lane marker detection problem into a row-wise classification task, which takes advantage of the innate shape of lane markers but, surprisingly, has not been explored well. In order to compactly extract sufficient information about lane markers which spread from the left to the right in an image, we devise a novel layer, which is utilized to successively compress horizontal components so enables an end-to-end lane marker detection system where the final lane marker positions are simply obtained via argmax operations in testing time. Experimental results demonstrate the effectiveness of the proposed method, which is on par or outperforms the state-of-the-art methods on two popular lane marker detection benchmarks, i.e., TuSimple and CULane.
1. Introduction
Lane marker detection is important for camera-based autonomous driving systems but conventional pixel-level segmentation requires complex post-processing. E2E-LMD instead directly predicts lane marker vertices through row-wise classification and achieves competitive benchmark performance without complex post-processing.
- Lane marker detection supports camera perception and positioning for applications such as lane keeping and lane-change assist.
- Conventional methods typically segment lane-marker pixels first, then use post-processing to impose structural constraints such as uniqueness or straightness.
- E2E-LMD directly predicts lane marker vertices by treating detection as multiple row-wise classification tasks.
- The framework compresses and models horizontal components before separately modeling each lane marker and obtaining positions through argmax operations.
- The method is on par with or outperforms recent state-of-the-art methods on TuSimple and CULane without complex post-processing.
2. Related Work
Prior lane marker detectors use hand-crafted features, segmentation, GANs, auxiliary structural cues, regression, or horizontal representations. These approaches address lane detection from different perspectives but commonly involve segmentation and post-processing.
- Traditional lane marker detectors rely on hand-crafted low-level features followed by post-processing, while newer methods learn features with deep networks.
- Segmentation-based methods include instance clustering, spatial message propagation, pixel-wise clustering, and lane-specific weight-map prediction.
- GAN-based approaches improve segmentation outcomes or environmental adaptability by conditioning generation on lane labels or low-light scenes.
- Other methods use vanishing-point supervision or formulate lane marker detection and classification as regression problems.
- Column-wise representations have been used for free-space recognition, with successive vertical pooling to regress free-space boundaries.
3. Proposed Method
E2E-LMD reformulates lane marker detection as row-wise classification and uses successive horizontal reduction to predict lane vertices end to end. Its architecture combines shared and lane marker-wise representations with confidence branches and classification-based training losses.
- Problem formulation: E2E-LMD treats each lane marker as row-wise horizontal-position classification, with final positions obtained by argmax during testing.The representation encodes lane markers as vertices across image rows.
- Network architecture: The architecture extends an encoder-decoder network with successive shared and lane marker-wise horizontal reduction modules.Shared modules compress width while preserving the vertical dimension; dedicated modules then produce a single vector for each row and lane marker.
- Prediction branches: The row-wise vertex-location branch classifies horizontal position, while vertex-wise and lane marker-wise branches predict existence confidence.The location branch operates on features whose channel size matches the target horizontal resolution.
- Horizontal reduction module: The horizontal reduction module combines residual down-sampling with horizontal pixel unshuffle to move spatial information into channels while compressing width.A convolution reduces the increased channel count after rearrangement, supporting efficient spatial compression.
- Feature representation: An SE block recalibrates channel-wise responses using global information, making lane representations more discernible for separation.PCA visualizations report more distinctive and disentangled representations in later shared HRM layers, while SE outputs make lane representations easier to separate.
- Training objective: Training optimizes a weighted total loss combining lane marker vertex location, vertex confidence, and lane marker-wise confidence losses.The location loss can use cross-entropy, KL-divergence, or PL-loss; confidence losses use binary cross-entropy.
4. Experiments
The experiments evaluate E2E-LMD on TuSimple and CULane using standard dataset-specific metrics, compare it with prior methods, and analyze failures and design choices. Results are competitive on TuSimple and consistently outperform state-of-the-art methods across CULane scenarios, while failures occur under reflections, severe curves, and occlusions.
- Datasets and metrics: The evaluation uses TuSimple and CULane, with accuracy, false-positive, and false-negative scores for TuSimple and IoU-based F1-measure for CULane.CULane treats lane markings as 30-pixel-wide lines and counts predictions with IoU above 0.5 as true positives.
- Datasets and metrics: TuSimple contains 6,408 highway images, while CULane contains 133,235 frames spanning urban, rural, highway, and nine challenging driving scenarios.The datasets use different image resolutions and train/validation/test splits.
- Benchmark results: The proposed method achieves competitive performance on TuSimple, notably with a low false-positive ratio without complex post-processing such as RANSAC.The authors suggest that heavier networks can overfit because TuSimple has relatively few training images.
- Benchmark results: The method consistently outperforms state-of-the-art methods across CULane scenarios, including comparison with a method using CycleGAN to augment insufficient scenario data.Table 3 reports F1-measure except for the Cross scenario, where only false positives are shown.
- Qualitative analysis: Qualitative results show successful localization at night, in shadows, and under tunnels, but failures arise from bonnet reflections, severe curves, and occlusions.Figure 6 presents failed examples, while Figure 7 presents qualitative test-image results.
- Ablation experiments: Ablations examine SE-block placement, shared HRM depth, and loss functions; Post-SE performs best, shared HRM count controls speed–accuracy trade-offs, and simple CE loss is preferable in these experiments.The authors attribute CE's advantage to spatial information incorporated by the horizontal reduction module.
5. Conclusion
The method detects lane-marker vertices end-to-end by classifying lane markers and using a novel module for horizontal reduction, achieving state-of-the-art performance without complex post-processing.
- The method classifies each lane marker and obtains its vertices in an end-to-end manner.
- A novel module performs effective horizontal reduction within the proposed architecture.
- The method achieves state-of-the-art performance without any complex post-processing.
- The architecture may also be used for general polygon prediction and semantic or instance segmentation.