Source-linked AI summary
Learning Lightweight Lane Detection CNNs by Self Attention Distillation
Yuenan Hou, Zheng Ma, Chunxiao Liu, Chen Change Loy
TL;DR
Sparse lane annotations make deep lane detection difficult, especially in occluded and poorly lit scenes. The paper introduces Self Attention Distillation, which uses a network’s own contextual attention maps for representation learning without extra labels or inference cost. Across three benchmarks, lightweight SAD models show consistent gains, with ENet-SAD outperforming baselines on BDD100K and remaining competitive on TuSimple and CULane.
Problem
Deep lane detection is challenged by subtle, sparse lane annotations and difficult scenes such as severe occlusion and poor lighting.
Method
Self Attention Distillation uses activation-based attention maps from deeper network layers as distillation targets for lower layers during training.
Results
SAD produces consistent performance gains across TuSimple, CULane, and BDD100K, while ENet-SAD outperforms all baselines on BDD100K and remains compelling on the other benchmarks.
Takeaways & Limitations
SAD improves visual attention and detection performance in lightweight CNNs without increasing inference time, providing a training-only alternative to additional supervision.
Abstract
from arXiv · showhide
Training deep models for lane detection is challenging due to the very subtle and sparse supervisory signals inherent in lane annotations. Without learning from much richer context, these models often fail in challenging scenarios, e.g., severe occlusion, ambiguous lanes, and poor lighting conditions. In this paper, we present a novel knowledge distillation approach, i.e., Self Attention Distillation (SAD), which allows a model to learn from itself and gains substantial improvement without any additional supervision or labels. Specifically, we observe that attention maps extracted from a model trained to a reasonable level would encode rich contextual information. The valuable contextual information can be used as a form of 'free' supervision for further representation learning through performing topdown and layer-wise attention distillation within the network itself. SAD can be easily incorporated in any feedforward convolutional neural networks (CNN) and does not increase the inference time. We validate SAD on three popular lane detection benchmarks (TuSimple, CULane and BDD100K) using lightweight models such as ENet, ResNet-18 and ResNet-34. The lightest model, ENet-SAD, performs comparatively or even surpasses existing algorithms. Notably, ENet-SAD has 20 x fewer parameters and runs 10 x faster compared to the state-of-the-art SCNN, while still achieving compelling performance in all benchmarks. Our code is available at https://github.com/cardwing/Codes-for-Lane-Detection.
1. Introduction
Lane detection is difficult because lane annotations are sparse and real-world scenes contain occlusion, poor lighting, and irrelevant markings. SAD addresses this by using a network’s own attention maps as training supervision, improving lightweight CNN representations without deployment-time cost.
- Motivation: Lane detection is challenging because long, thin lanes produce sparse supervisory pixels amid much larger background regions.Dense prediction methods assign binary lane labels per pixel, making the annotation signal subtle for deep-model training.
- Motivation: Existing multi-task learning and message passing schemes add context, but multi-task learning requires extra annotations and message passing can be expensive.Examples of additional annotations include scene segmentation maps, vanishing points, and drivable areas.
- Self-Attention Distillation: SAD uses attention maps from a network’s own layers as distillation targets for lower layers, complementing segmentation-based supervision without additional labels or external supervision.The approach is applied during training and does not increase inference time.
- Self-Attention Distillation: After SAD, lower-layer attention maps capture richer scene context, while improved lower-layer representations reinforce deeper layers, including blocks without direct distillation targets.Without SAD, attention maps across layers show little improvement despite continued training.
- Results: ENet-SAD achieves state-of-the-art lane detection performance on TuSimple, CULane, and BDD100K, supporting SAD’s usefulness for small networks.The authors also investigate layer-wise mimicking paths and when to introduce SAD during training.
2. Related Work
Lane detection research has moved from hand-crafted features toward end-to-end deep learning and dense prediction. Related approaches add scene context through multi-task learning, geometric constraints, or auxiliary generative modeling.
- Lane Detection: Traditional lane detection uses specialized hand-crafted features to extract and group lane segments, but is less robust and suited mainly to easier driving scenarios.These methods require complex feature-selection processes.
- Lane Detection: Deep-learning lane detectors learn features end-to-end and commonly formulate lane detection as pixel-wise semantic segmentation.Each pixel receives a label indicating whether it belongs to a lane.
- Contextual Supervision: Multi-task learning and message passing complement lane supervision by incorporating richer scene context.Examples include jointly segmenting lane boundaries and road areas while adding geometric constraints relating them.
3. Methodology
The methodology formulates lane detection as joint semantic segmentation and lane-existence prediction, then adds self-attention distillation to transfer contextual information from deeper to earlier layers. Attention maps are generated from activations, aligned across layers, and used with task losses during training without adding inference-time computation.
- Lane detection maps an input image to a segmentation map and a binary lane-existence prediction.
- Self Attention Distillation: SAD derives attention maps from the network itself and distills deeper-layer context into lower layers without external labels or inference-time cost.The method complements segmentation-based supervision with layer-wise and top-down attention distillation.
- Self Attention Distillation: Activation-based attention maps are formed by reducing each layer’s channel-wise activations to a spatial map, with G2_sum reported as the most effective mapping function.The mapping alternatives differ in how they weight channel activations; the experiments report the G2_sum choice as yielding the most performance gains.
- Self Attention Distillation: Successive layer-wise distillation compares transformed attention maps with an L2-style loss, using the deeper map as the target.The example uses M = 4 layers and applies a uniform weighting scheme across SAD paths.
- Self Attention Distillation: The total training objective combines cross-entropy segmentation, IoU, lane-existence, and distillation losses, balanced by α, β, and γ.The IoU term targets overlap between predicted and ground-truth lane pixels, while the existence term uses binary cross entropy.
- Architecture Design: The ENet adaptation adds a lane-existence branch, dilated convolutions, and E4–E3 feature concatenation to enlarge receptive fields and preserve resolution-related information.Inference for CULane additionally smooths probability maps, thresholds lane existence, samples positions, and connects them with cubic splines.
4. Experiments
Experiments evaluate SAD across three lane-detection benchmarks and lightweight CNNs, showing strong performance alongside reduced model complexity. Ablations further examine qualitative behavior, training timing, and distillation paths.
- Datasets: TuSimple, CULane, and BDD100K provide evaluations spanning standard and challenging driving scenarios, including crowded roads and poor lighting.CULane includes crowded-road and poor-lighting conditions; BDD100K also contains severe occlusion and poor lighting.
- Results: ENet-SAD outperforms all baselines on BDD100K while achieving compelling performance on TuSimple and CULane.The comparison includes state-of-the-art algorithms and accounts for model runtime and parameter count.
- Results: Applying SAD to ResNet-101 increases CULane F1-measure from 70.8 to 71.8 and BDD100K accuracy from 34.45% to 35.56%.This shows gains also occur in a deeper model.
- Qualitative results: ENet-SAD produces more precise lane detections than ENet on TuSimple and CULane, while its BDD100K probability maps are more compact and less noisy.The qualitative comparisons include poor-lighting conditions.
- Ablation study: SAD remains effective without supervision from the lane-existence branch, reaching 96.61% on TuSimple, 70.8 on CULane, and 36.54% on BDD100K.The experiment blocks the P1 branch gradient from the main branch.
- Ablation study: Low-level distillation degrades performance, whereas neighboring-layer paths and later training-stage introduction are more beneficial.The authors attribute the low-level degradation to interference with local-feature learning and report that later introduction benefits earlier representations.
5. Discussion
The discussion concludes that SAD consistently improves CNN lane-detection models across architectures and benchmarks, while suggesting extensions to other fine-grained visual tasks.
- Discussion: SAD improves representation learning and delivers consistent performance gains across ENet, ResNet-18, ResNet-34, and ResNet-101 on three benchmarks.The benchmarks are TuSimple, CULane, and BDD100K.
- Discussion: The authors suggest extending SAD to tasks requiring fine-grained attention, including image saliency detection and image matting.This is presented as a direction for future work.
A. Details of Architecture
The architecture details describe lane-existence branches for SAD models and modifications that expand receptive fields without increasing parameter count.
- Architecture: Dilated convolution increases the receptive field without increasing the number of parameters in the lane-existence prediction branch.The modification is described for the original ENet design.
- Architecture: Table 7 summarizes lane-existence prediction branches for ENet-SAD, ResNet-18-SAD, and ResNet-34-SAD.For the ResNet variants, dilated convolutions replace the original convolutions in the last two blocks.
B. Lane Post-processing in CULane
CULane inference converts probability maps and lane-existence predictions into final lane outputs through smoothing, thresholding, row-wise peak search, and spline connection.
- Post-processing: A 9 × 9 kernel first smooths the multi-channel probability maps during CULane inference.The image produces probability maps and a lane-existence vector before post-processing.
- Post-processing: For lanes with existence probability above 0.5, the method searches every 20 rows for the highest-probability position and connects positions with cubic splines.This procedure produces the final lane output.
C. More Qualitative Results in Lane Detection
Qualitative comparisons show ENet-SAD produces more precise and cleaner lane detections than ENet and SCNN across challenging conditions.
- ENet-SAD detects lanes more precisely than ENet on TuSimple and CULane.
- ENet-SAD is less affected by irrelevant road objects than SCNN.
- ENet-SAD produces more compact probability maps than SCNN in poor lighting conditions.
- ENet-SAD probability maps contain less noise than SCNN in poor lighting conditions.