Source-linked AI summary
FastDraw: Addressing the Long Tail of Lane Detection by Adapting a Sequential Prediction Network
Jonah Philion
TL;DR
FastDraw addresses the challenge of lane detection under long-tailed sensor conditions, where models must represent variable lane structures, run in real time, and adapt to new scenes. It integrates iterative lane decoding into a convolutional network and uses unsupervised style transfer for adaptation, achieving competitive accuracy across standard and difficult conditions while running at 90 FPS. The paper assumes lanes are curve segments that can be drawn by conditioning on the previous lane pixel.
Problem
Lane detection models must represent arbitrary lane structures, operate in real time, and adapt to new domains despite limited public annotations.
Method
FastDraw uses a fully convolutional iterative decoder that predicts local lane relationships and applies unsupervised style transfer to adapt training images across visual conditions.
Results
Competitive accuracy was achieved on standard and difficult lane-detection conditions, including Tusimple and CULane, while running at 90 frames per second.
Takeaways & Limitations
Lane detection can be decoded within the network and adapted to difficult environments without additional human annotation.
Takeaways & Limitations
The approach assumes lanes are curve segments that are functions of image height and can be drawn by conditioning only on the previous lane pixel.
Abstract
from arXiv · showhide
The search for predictive models that generalize to the long tail of sensor inputs is the central difficulty when developing data-driven models for autonomous vehicles. In this paper, we use lane detection to study modeling and training techniques that yield better performance on real world test drives. On the modeling side, we introduce a novel fully convolutional model of lane detection that learns to decode lane structures instead of delegating structure inference to post-processing. In contrast to previous works, our convolutional decoder is able to represent an arbitrary number of lanes per image, preserves the polyline representation of lanes without reducing lanes to polynomials, and draws lanes iteratively without requiring the computational and temporal complexity of recurrent neural networks. Because our model includes an estimate of the joint distribution of neighboring pixels belonging to the same lane, our formulation includes a natural and computationally cheap definition of uncertainty. On the training side, we demonstrate a simple yet effective approach to adapt the model to new environments using unsupervised style transfer. By training FastDraw to make predictions of lane structure that are invariant to low-level stylistic differences between images, we achieve strong performance at test time in weather and lighting conditions that deviate substantially from those of the annotated datasets that are publicly available. We quantitatively evaluate our approach on the CVPR 2017 Tusimple lane marking challenge, difficult CULane datasets, and a small labeled dataset of our own and achieve competitive accuracy while running at 90 FPS.
1. Introduction
FastDraw addresses lane-detection challenges in variable lane structure, real-time inference, and adaptation to new environments by integrating decoding into a convolutional network and using style-transfer augmentation.
- Motivation: Previous lane detectors estimate pixel likelihoods, threshold candidates, and cluster them into lanes during post-processing.This structure inference is generally nondifferentiable and makes performance dependent on the initial segmentation and post-processing heuristics.
- Model: FastDraw integrates lane decoding into the network by predicting local lane shape and following predicted contours to recover global lanes.The model draws lanes iteratively rather than delegating structure inference to a separate heuristic decoder.
- Requirements: The design targets arbitrary numbers and lengths of lanes while preserving curved lane geometry instead of reducing lanes to fixed-dimensional polynomials.Polynomial approximations can lose accuracy on tight curves, where localization matters for safe driving.
- Model: FastDraw uses a convolutional iterative decoder to satisfy real-time constraints while providing a natural uncertainty estimate through autoregressive predictions.The convolutional backbone enables operation at 90 frames per second on a GTX 1080.
- Adaptation: Unsupervised style transfer adapts FastDraw to images far from publicly annotated training distributions by encouraging invariance to low-level texture differences.The approach targets corner-case weather and environmental conditions without additional human annotation.
2. Related Work
Related work commonly separates lane-feature extraction from clustering, while FastDraw connects lane structure prediction to learned decoding and draws on style-transfer and structured-prediction research.
- Lane Detection: Traditional and contemporary lane detectors commonly extract lane-marking features with learned or hand-crafted methods, then cluster pixels during post-processing.Recent approaches also encode lane identities as segmentation channels or regularize curve fitting through learned homographies.
- Lane Detection: Lane-detection results remain sensitive to post-processing heuristics, motivating methods that reduce reliance on separate clustering stages.This limitation is distinct from improving the initial segmentation alone.
- Broader Applications: Lane models have been adapted beyond dashcam imagery to lidar point clouds, open street maps, and satellite imagery.Semantic-segmentation progress has supported these extensions across dense prediction settings.
- Style Transfer: Prior style-transfer work includes supervised image translation, unsupervised back-translation, and models such as MUNIT that generate distributions of possible translations.FastDraw builds on this line of work to augment lane-detection training images.
- Structured Drawing: Structured-prediction research has inferred relationships between parts for human pose and iteratively drawn object polygons in Polygon-RNN and Sketch-RNN.FastDraw similarly predicts which pixels belong to the same lane while preserving lane structure.
3. Model
FastDraw models lane detection as direct polyline decoding rather than relying on heuristic post-processing. Its fully convolutional network predicts local lane relationships, decodes lanes greedily, and is trained with perturbation and style-transfer techniques for robustness.
- Lane representation: FastDraw maximizes polyline likelihoods, avoiding heuristic clustering during lane decoding.The model preserves lane annotations as sequences of height–width coordinates rather than reducing them to per-pixel likelihoods alone.
- Lane representation: The network predicts lane membership and categorical width offsets linking neighboring pixels, including an end token for lane termination.The categorical support spans offsets from −L to L plus end, allowing multimodal local transitions rather than forcing regression to an average path.
- Decoding: Lanes are decoded by selecting an initial pixel, greedily following predicted arrows upward and downward, then concatenating both trajectories until termination.This convolutional decoding avoids the recurrent neural network dependency used by a direct sequential predictor.
- Architecture: The architecture uses a ResNet semantic-segmentation backbone with skip connections and three two-layer convolutional heads.The heads predict the lane mask and directional pairwise distributions used for decoding.
- Training: Training combines segmentation and sequence objectives with dynamically learned task-specific uncertainty weights.The authors retain dynamic weighting in all reported results to avoid tuning loss-balance hyperparameters, although fixed weights performed similarly.
- Training: Noise-based lane denoising and MUNIT style transfer augment training to reduce exposure bias and adapt annotated data to new weather and lighting conditions.The style-transfer procedure assumes geometric lane content is preserved while image appearance changes, and uses unlabeled target-domain images to generate additional examples.
4. Experiments
The experiments evaluate FastDraw on public lane datasets and Massachusetts driving data, including style-transferred images for adaptation. They also examine uncertainty and training-example generation.
- Datasets: FastDraw is evaluated on the Tusimple and CULane lane-detection datasets, plus Massachusetts dash-cam data collected across varied weather conditions.Tusimple contains 3,626 annotated images, while CULane includes 88,880 training images, 9,675 validation images, and 34,680 test images.
- Adaptation: Massachusetts adaptation uses MUNIT to translate Tusimple training footage into the authors’ imagery, generating 10,000 sampled images.The authors also record 10,000 Massachusetts dash-cam images and adjust original annotations to develop D′.
- Uncertainty: FastDraw’s predicted uncertainty increases where lane locations are ambiguous, including wide markings, shadows, possible false positives, and occlusions.Figure 5 visualizes the predicted standard deviation with a color map thresholded at 0 and 9 pixels.
- Training configuration: FastDraw training examples use L = 6 pixels and σ = 2 pixels at 128 × 256 resolution, and L = 16 pixels and σ = 5 pixels at 352 × 640.These parameters are used for the two reported image-size settings.
5. Results
FastDraw achieves competitive lane-detection performance while using a learned decoder, uncertainty estimates, and style-transfer adaptation. The results span Tusimple, CULane, Massachusetts imagery, runtime, and decoder behavior.
- Tusimple: FastDraw is competitive with EL-GAN on all reported Tusimple metrics, using fewer layers.EL-GAN improves segmentation with an adversarial loss but still requires a heuristic decoder for structured lanes.
- Tusimple: Adapted FastDraw achieves high accuracy on Tusimple but also has the highest false positive rate.The authors attribute this to a stronger lane-shape prior and style segmentation that may omit lanes, causing roadside false positives.
- Uncertainty: FastDraw’s uncertainty rises in occluded and shadowy conditions and can be propagated through the self-driving stack to discourage reckless driving.The uncertainty is computed from the predicted categorical distribution over local lane-width changes.
- Decoder analysis: The learned decoder is generally distinct from a simple heuristic decoder, although their outputs are correlated.The comparison evaluates how often decoded pixels agree with the highest-likelihood pixel in the preceding row.
- CULane: FastDraw is competitive with the CULane ResNet-50 baseline and outperforms it by a wide margin on curves.The authors link the curve advantage to preserving lane polylines instead of representing lanes as cubic polynomials.
- Massachusetts adaptation: Style-transferred training enables qualitative generalization to night, evening, and rainy scenes without additional human annotation.The augmented model also produces smoother curves, recognizes lane ends better, and finds all lanes in Massachusetts imagery.
- Massachusetts adaptation: Adaptation markedly improves the precision/recall trade-off and makes models more robust to scene changes.The comparison uses FastDraw models trained with and without style-transfer adaptation.
6. Conclusion
FastDraw provides accurate lane detection across standard and difficult environments without additional human annotation, under assumptions about lane geometry and sequential drawing.
- Conclusion: FastDraw adapts lane detection to difficult environments without additional human annotation.The approach assumes lanes are curve segments that function over image height and can be drawn iteratively from the previous lane pixel.