Source-linked AI summary
VPGNet: Vanishing Point Guided Network for Lane and Road Marking Detection and Recognition
Seokju Lee, Junsik Kim, Jae Shin Yoon, Seunghak Shin, Oleksandr Bailo, Namil Kim, Tae-Hee Lee, Hyun Seok Hong, Seung-Hoon Han, In So Kweon
TL;DR
Lane and road-marking recognition lacks robust evidence and benchmarks for adverse weather and low illumination. The paper introduces a diverse benchmark and a vanishing-point-guided multi-task network, which jointly detects and classifies markings while predicting the vanishing point. VPGNet is reported as robust across conditions and real-time, with a 30 ms forward pass in the benchmark comparison.
Problem
Lane and road-marking detection research and public datasets have been largely limited to fine-weather, simple road conditions, leaving poor-weather and low-illumination recognition insufficiently addressed.
Method
VPGNet is a unified end-to-end multi-task network that jointly detects and classifies lanes and road markings while using quadrant-based vanishing-point prediction to encode global scene context.
Results
The model shows significantly better lane detection than FCN-8s in each bad-weather scenario, with a 30 ms forward pass versus 130 ms for FCN-8s.
Takeaways & Limitations
The benchmark and VPGNet provide a single-pass approach for lane and road-marking detection, recognition, and vanishing-point prediction under varied weather and illumination.
Abstract
from arXiv · showhide
In this paper, we propose a unified end-to-end trainable multi-task network that jointly handles lane and road marking detection and recognition that is guided by a vanishing point under adverse weather conditions. We tackle rainy and low illumination conditions, which have not been extensively studied until now due to clear challenges. For example, images taken under rainy days are subject to low illumination, while wet roads cause light reflection and distort the appearance of lane and road markings. At night, color distortion occurs under limited illumination. As a result, no benchmark dataset exists and only a few developed algorithms work under poor weather conditions. To address this shortcoming, we build up a lane and road marking benchmark which consists of about 20,000 images with 17 lane and road marking classes under four different scenarios: no rain, rain, heavy rain, and night. We train and evaluate several versions of the proposed multi-task network and validate the importance of each task. The resulting approach, VPGNet, can detect and classify lanes and road markings, and predict a vanishing point with a single forward pass. Experimental results show that our approach achieves high accuracy and robustness under various conditions in real-time (20 fps). The benchmark and the VPGNet model will be publicly available.
1. Introduction
The paper addresses lane and road-marking recognition that remains unreliable in poor weather and illumination, where suitable datasets are also scarce. It proposes a benchmark and a vanishing-point-guided multi-task network for robust recognition across challenging scenes.
- Research gap: Recognition methods have largely been evaluated in fine weather, with performance dropping under challenging weather and illumination conditions.Existing CNN and hand-crafted-feature approaches remain limited to fine weather and simple road conditions.
- Research gap: Public lane and road-marking datasets are limited, small, and generally omit adverse-weather scenarios.The cited Caltech Lanes and Road Marking datasets contain 1,225 and 1,443 images, respectively, and were captured on sunny days.
- Approach: The authors collect data under varied circumstances and propose a network that recognizes global context for challenging-scene recognition.The paper identifies a large, diverse dataset and stronger network capability as requirements for robust recognition.
- Approach: A vanishing-point prediction task is introduced to embed geometric context because driver gaze direction correlates with road direction.The task is motivated by humans locating lanes even when lane markings are difficult to see.
- Contributions: The benchmark contains about 20,000 images and 17 manually annotated lane and road-marking classes, while the unified network jointly performs recognition and vanishing-point prediction.The authors report robustness across weather conditions and real-time performance, and suggest the vanishing-point task can help detect unseen lanes.
2. Related Work
Prior lane and road-marking methods use hand-crafted features or deep networks, but detection under poor conditions remains unresolved. The paper positions its approach as addressing bad weather and low illumination within a monocular road-scene setting.
- Problem setting: The paper adopts a monocular vision setup for its road-scene detection problem and related-work comparisons.The setup is stated explicitly at the beginning of the related-work section.
- Hand-crafted methods: Related work commonly uses hand-crafted edges, colors, textures, geometric fitting, or splines to localize lanes and road markings.These methods perform well in certain situations but poorly in unfamiliar conditions.
- Deep learning methods: Deep learning approaches apply CNNs, CNN-RNN combinations, and dual-view networks to lane detection.The described methods use learned geometric information, recurrent processing, or front-view and top-view inputs.
- Open problem: Despite promising deep-learning results, detection under poor conditions remains unsolved in the reviewed literature.The paper targets performance across bad weather and low-illumination conditions.
- Deep learning methods: General object-detection networks achieve strong benchmark results but lose performance on smaller and thinner objects such as lanes and road markings.The related-work discussion contrasts large-object benchmarks with the thin structures relevant to this paper.
3. Benchmark
The benchmark combines urban driving imagery from multiple weather and illumination scenarios with pixel-, grid-, class-, and vanishing-point annotations. Grid projection is used to preserve thin lane information through the network.
- Data collection: Images are categorized into no-rain, rain, heavy-rain, and night scenarios collected from vehicle-mounted cameras in urban driving.Night images represent challenging low illumination, while daytime images cover three precipitation levels.
- Annotation: Lane and road-marking objects receive polygon-based pixel masks with class labels assigned at the pixel level.Manual corner points are connected to form each object polygon.
- Annotation: Thin annotations can vanish during convolution, pooling, and resizing, so pixel-level labels are projected onto an 8×8 grid.For a 640×480 input and 80×60 output, a grid cell receives a class label when any original annotated pixel lies inside it.
- Dataset statistics: The benchmark reports scenario frame counts and per-class instance counts through separate dataset-statistics tables.Table 1 summarizes frames by scenario, while Table 2 summarizes instances by class.
- Annotation: Vanishing points are manually localized where parallel lanes supposedly meet and labeled EASY, HARD, or NONE by scene difficulty or absence.Cluttered scenes are HARD, clear straight roads are EASY, and intersections may have no vanishing point.
- Dataset statistics: The dataset contains about 20,000 images from three weeks of driving in Seoul, with 17 annotated lane and road-marking classes.Raw 30-fps video was sampled at 1 Hz to generate the image data.
4. Neural Network
VPGNet is a unified multi-task network that jointly detects and recognizes lanes and road markings while predicting a vanishing point. Its architecture combines complementary task modules, a quadrant-based VPP design, staged training, and task-specific post-processing.
- Network Architecture: VPGNet combines grid regression, object detection, multi-label classification, and vanishing point prediction in one forward pass.The modules jointly detect and classify lane and road markings while predicting the vanishing region.
- Vanishing Point Prediction: The network uses vanishing point prediction to embed global geometric context for locating lanes that are difficult to see.The vanishing point is defined by lane convergence and is intended to guide robust detection under occlusion and extreme illumination.
- Vanishing Point Prediction: The quadrant VPP method divides the image into four sections whose intersection represents the vanishing point, using four quadrant channels plus an absence channel.This design covers global scene structure and enriches gradient information compared with binary classification.
- Training: VPGNet trains first on VPP alone and then jointly trains all tasks to reduce task dependence and balance shared learning.The first phase initializes shared kernels with global context; the second phase optimizes all task losses together.
- Training: The total training loss sums grid regression, object detection, multi-label, and VPP losses with weights adjusted from their initial scales.The regression branch uses L1 loss, while the other listed branches use cross entropy losses; weights are recalibrated when scale differences grow.
- Post-Processing: Post-processing samples and clusters lane points before VP-guided quadratic regression, while road markings use grid sampling and box clustering.Inverse perspective mapping separates sampled lane points near the VP and supports both straight and curved roads.
5. Results
Experiments evaluate task cooperation, metrics, detection and recognition across weather scenarios, and vanishing-point prediction. VPGNet achieves robust lane performance in adverse conditions and operates at 20 Hz.
- 5.1. Experimental Settings: 20 Hz is achieved with a single forward pass; inference takes about 30 ms and post-processing takes 20 ms or less.
- 5.2. Analysis of Multi Task Learning: Adding tasks activates more shared-network neurons, especially around roadway boundaries.
- 5.4. Lane Detection and Recognition: VPGNet achieves the best F1 score on the Caltech Lanes Dataset, although all three task variants perform similarly in easy daytime scenes.
- 5.4. Lane Detection and Recognition: VPGNet shows significantly better lane detection than FCN-8s in each bad-weather benchmark scenario.VPGNet's forward pass is 30 ms, compared with 130 ms for FCN-8s.
- 5.4. Lane Detection and Recognition: Grid-level annotation improves FCN-8s performance over pixel-level annotation by strengthening gradients around thin lane and road-marking regions.
- 5.4. Lane Detection and Recognition: The VPP task improves lane finding without a visible vanishing point and improves single-white-line recognition in rainy and night conditions.
- 5.5–5.6. Road Marking and Vanishing Point Evaluation: VPP-guided multi-task learning substantially improves other road-marking classes and helps vanishing-point prediction after joint training.Stop lines are the exception among the reported road-marking classes, while the second-phase VPP prediction is highly improved.
6. Conclusions
The paper introduces a four-scenario benchmark and a VP-guided multi-task network for lane and road-marking detection and classification. Evaluation reports robustness across weather conditions, real-time performance, and benefits from the VPP task.
- The benchmark covers daytime no-rain, rain, heavy-rain, and night scenarios.
- VPGNet jointly detects and classifies lanes and road markings while using vanishing-point guidance.
- The evaluation reports robustness under different weather conditions and real-time performance.
- The VPP task enhances lane and road-marking detection and classification by strengthening activations for markings and roadway boundaries.