Source-linked AI summary
Embedding Structured Contour and Location Prior in Siamesed Fully Convolutional Networks for Road Detection
Qi Wang, Junyu Gao, Yuan Yuan
TL;DR
Road detection requires accurate per-pixel boundary segmentation despite occlusion, irregular road shapes, and limitations of RGB-only features. The paper proposes s-FCN-loc, which integrates RGB images, semantic contours, and location priors in a siamesed FCN. It reports competitive KITTI performance and faster convergence than the original FCN, while noting weaker generalization when camera-dependent location priors differ.
Problem
Road detection remains difficult because roads are occluded, lack definite shapes, and require accurate pixel-level boundaries.
Method
s-FCN-loc simultaneously integrates RGB images, semantic contours, and a location-prior feature map in a siamesed FCN.
Results
The method achieves a competitive second-place Max F-measure 93.26% on the KITTI Urban Road category and converges faster than the original FCN.
Takeaways & Limitations
The integrated model learns road-boundary features and location priors without postprocessing, with experiments verifying the effectiveness of its components.
Abstract
from arXiv · showhide
Road detection from the perspective of moving vehicles is a challenging issue in autonomous driving. Recently, many deep learning methods spring up for this task because they can extract high-level local features to find road regions from raw RGB data, such as Convolutional Neural Networks (CNN) and Fully Convolutional Networks (FCN). However, how to detect the boundary of road accurately is still an intractable problem. In this paper, we propose a siamesed fully convolutional networks (named as ``s-FCN-loc''), which is able to consider RGB-channel images, semantic contours and location priors simultaneously to segment road region elaborately. To be specific, the s-FCN-loc has two streams to process the original RGB images and contour maps respectively. At the same time, the location prior is directly appended to the siamesed FCN to promote the final detection performance. Our contributions are threefold: (1) An s-FCN-loc is proposed that learns more discriminative features of road boundaries than the original FCN to detect more accurate road regions; (2) Location prior is viewed as a type of feature map and directly appended to the final feature map in s-FCN-loc to promote the detection performance effectively, which is easier than other traditional methods, namely different priors for different inputs (image patches); (3) The convergent speed of training s-FCN-loc model is 30\% faster than the original FCN, because of the guidance of highly structured contours. The proposed approach is evaluated on KITTI Road Detection Benchmark and One-Class Road Detection Dataset, and achieves a competitive result with state of the arts.
I. INTRODUCTION
Road detection is a challenging per-pixel segmentation problem because roads lack definite shapes and accurate boundaries matter for autonomous driving. Existing deep models learn visual features but may overlook spatial structure, motivating contour and location information.
- Accurate road-region detection is important for autonomous driving because it supports navigation and other street-scene tasks.
- Road detection is per-pixel classification, not bounding-box localization, because road surfaces are often occluded and lack definite shapes.
- Traditional approaches use 3D point clouds or location information from extra sensors, while this paper focuses on extracting deeper vision information.
- CNN and FCN methods learn discriminative features for image understanding and semantic segmentation, but conventional architectures can ignore spatial structure and location information.
- Semantic contours represent object boundaries and whole-scene structure, motivating a contour-processing stream for road detection.
2) Location Priors:
The paper incorporates whole-image location priors into the proposed network as feature maps, avoiding patch-specific prior preprocessing and targeting false detections. The approach combines this design with RGB and contour processing.
- Location Priors: Road regions usually appear near the image bottom, so location priors can help remove false detections.
- Location Priors: A 2-channel location map represents normalized x- and y-coordinate priors for the whole image.
- Location Priors: The proposed pipeline generates a semantic contour map, processes RGB and contour inputs in siamesed FCN streams, and outputs the road region.
- Location Priors: The location prior is directly concatenated as a feature map to s-FCN, reducing mistaken detection more simply than using different priors for different patches or superpixels.
- Location Priors: Structured priors such as shape, edge, contour, and location have been used across prior detection methods.
III. APPROACH
The approach builds on FCN for dense per-pixel prediction and uses fused deep and shallow features to improve spatially finer segmentation. The implementation adopts VGG16-based FCN-16s.
- Fully Convolutional Network: FCN replaces fully connected layers with convolutional layers to produce arbitrary-size outputs for dense prediction.
- Fully Convolutional Network: FCN-16s fuses deep and shallow feature maps to recover location and edge clues for finer per-pixel results.
- Fully Convolutional Network: The paper adopts VGG16-net’s FCN-16s architecture, which fuses pool4 and conv7 feature maps by summation.
B. Semantic Contour Map
Semantic contour maps provide structured, intensity-valued boundary information that complements RGB data. The proposed siamesed FCN processes both inputs with shared convolutional parameters to learn boundary features.
- Semantic Contour Map: Semantic contour maps encode spatial structure and skeleton clues more effectively than traditional edge detectors.
- Semantic Contour Map: Contour maps are grayscale rather than binary, so contour intensity is quantified.
- Semantic Contour Map: Human subjects could segment street-scene objects using semantic contour maps without special training.
- Semantic Contour Map: The model generates contours with Structured Forests and adds a neural-network stream to process contour information.
- Semantic Contour Map: The siamesed FCN processes RGB images and semantic contour maps simultaneously while sharing convolutional parameters between streams.
D. Incorporating Location Priors in s-FCN
The method uses spatial regularities in street scenes by appending normalized coordinate maps to s-FCN, aiming to reduce unreasonable detections. The section also situates this design within the paper’s evaluation criteria and comparative experiments.
- Road regions usually occur near image bottoms, while buildings and trees tend to appear on both sides, providing a location prior for detection.
- The location prior is a two-channel x/y feature map with normalized coordinates in [0, 1], resized and concatenated with s-FCN’s final feature map.
- The experiments compare original FCN-16s, s-FCN, and s-FCN-loc across the KITTI and One-Class Road Detection datasets.
- The evaluation uses precision, accuracy, F-measure, and max F-measure, with KITTI ranking methods by max F-measure.
B. Dataset
The study evaluates its road-detection approach on the KITTI Vision Benchmark Suite and the One-Class Road Detection Dataset.
- The proposed approach is evaluated on the KITTI road detection dataset and the One-Class Road Detection Dataset.
1) KITTI Dataset:
The KITTI dataset provides road-detection scenes with perspective images, benchmark evaluation in bird’s-eye-view space, and a training/validation setup used for component analysis.
- KITTI contains 579 images, split into 289 training and 290 testing images, at 375 × 1242-pixel resolution.
- The benchmark ranks submitted methods by max F-measure in bird’s-eye-view space under a flat-world transformation.
- The dataset includes color stereo images, GPS information, and Velodyne laser scans for each scene.
- Images are resized to 500 × 500, while contour maps use Structured Forests with one decision tree.
- The location map is 44 × 44, dropout is 0.5 in conv6 and conv7, and the classification threshold is 0.5.
- Evaluation metrics are computed in bird’s-eye-view space for KITTI and in perspective images for the One-Class dataset.
D. KITTI Dataset: Performance on Validation Set
On KITTI, s-FCN-loc achieves the strongest validation performance among the compared stepwise models and competitive benchmark performance, while improving road-boundary segmentation and reducing location-inconsistent false detections.
- Validation-set comparison: s-FCN-loc achieves the best validation result across F1-measure, accuracy, precision, and recall among the compared models.The reported improvement applies to every criterion except recall rate.
- Validation-set comparison: Visualizations show that s-FCN and s-FCN-loc produce cleaner road boundaries than FCN, while s-FCN-loc reduces building regions mistakenly classified as road.The location prior particularly alleviates false detections in positions where roads rarely occur.
- KITTI benchmark: 93.26% Max F-measure places s-FCN-loc second in the Urban Road category, close to DDN’s 93.43% best result.The method also ranks fourth in runtime among the 11 listed algorithms.
- KITTI benchmark: The proposed model obtains the 8-th prize among all 52 KITTI server submissions, including anonymous entries.The paper separately compares against the first ten real-name submissions.
- KITTI benchmark: The model generalizes from training to testing data, but small road corners remain difficult because the network’s receptive field is too large for independent small regions.This limitation is visible in exemplar KITTI results.
- KITTI benchmark: In BEV evaluation, errors in distant road regions are magnified relative to the perspective-image results.The same phenomenon appears across multiple KITTI exemplars.
F. OC Dataset: Performance
On the OC testing set, s-FCN-loc provides the strongest overall performance among the compared models, while location priors trade fewer false detections for more missed detections. Visual examples show improved boundary segmentation and suppression of implausible road regions.
- s-FCN-loc achieves the best result on the first three criteria in the OC comparison, while s-FCN has the best recall.
- Adding location priors reduces false detection but increases missing detection on the OC dataset.
- Visual results show s-FCN and s-FCN-loc alleviating the original FCN’s inaccurate road-boundary segmentation.
- Location priors greatly alleviate false road regions in examples where the original FCN and s-FCN identify rarely occupied road positions.
G. Analysis of Convergent Speed
The proposed models converge faster than the original FCN, with structured contours guiding training and location maps using finer spatial priors. However, location priors can weaken cross-dataset generalization when camera-dependent road distributions differ.
- Analysis of Convergent Speed: 30% faster overall training convergence is achieved by s-FCN and s-FCN-loc than by the original FCN on KITTI, with similar trends on OC.
- Analysis of Convergent Speed: 80,000 iterations are needed for s-FCN and s-FCN-loc to converge on KITTI, versus 240,000 for the original FCN.
- Analysis of Convergent Speed: Highly structured contour maps guide more effective semantic representation learning, which the paper associates with quicker convergence.
- Location Priors: 44 × 44 location maps provide more accurate priors than 16 × 16 maps for distant roads and small corner regions.
- Generalization Analysis: s-FCN and s-FCN-loc generalize better than FCN overall, but s-FCN-loc is weaker than s-FCN on KITTI and CamVid.
- Generalization Analysis: The authors attribute weaker s-FCN-loc generalization to location priors differing across camera properties and dataset road distributions.
J. The Effects of Contour Maps vs. Depth Maps
Contour maps outperform depth maps as the auxiliary input in the reported Cityscapes comparison, while also supporting faster training. The paper presents contour streams as extensible to other dense-prediction networks and tasks.
- The Effects of Contour Maps vs. Depth Maps: Contour maps are described as highly structured global edge features that significantly improve road-boundary segmentation and are easy to generate in real time.
- The Effects of Contour Maps vs. Depth Maps: s-FCN thoroughly defeats s-FCN with depth across all four reported Cityscapes metrics.
- The Effects of Contour Maps vs. Depth Maps: s-FCN trains in less time than s-FCN with depth, whose depth input requires a lower learning rate and fails to converge at 10^-10.
- Future Directions: The contour stream can be added to CNN and DeConv NN architectures and is proposed for future saliency detection and semantic image segmentation.