Aerial LaneNet: Lane Marking Semantic Segmentation in Aerial Imagery using Wavelet-Enhanced Cost-sensitive Symmetric Fully Convolutional Neural Networks

I. INTRODUCTION

• Airborne imagery is presented as a way to cover large areas quickly for high-precision map creation, complementing ground-based mapping that faces limited coverage, sensor drift, GPS shadows, and traffic occlusions.Such maps support autonomous driving, infrastructure monitoring, lane-wise traffic management, and urban planning.Source: §I · pp. 1–2

A. Challenges

• Shadows and washed-out markings complicate localization, while rare classes such as speed-limit, bus, bike, and disabled-parking signs are unevenly represented.Completely washed-out markings are ignored; partial occlusion creates challenges for prediction and annotation.Source: A. Challenges · p. 2
• Lane markings vary across many classes, and their pixel sizes can be very small in airborne imagery, with some separator signs measuring 5 × 5 px.The listed classes include boundaries, intersections, signs, no-parking zones, and pedestrian crossings.Source: A. Challenges · p. 2, Figure 3 · p. 2
• Aerial segmentation must distinguish lane markings from similar-looking backgrounds, including sport-field lines, railways, and building roofs.Partial or full occlusion by trees, branches, or bridges can also change or hide markings.Source: A. Challenges · p. 2, Figure 4 · p. 3

B. Related work

• Earlier aerial approaches often extract roads before detecting markings, and can produce false positives or perform poorly without road extraction.Other methods use thresholds, geometric cues, or additional data; some target only selected marking types.Source: B. Related work · p. 3
• Ground-imagery lane-marking datasets and methods do not address the tiny markings in airborne images, which can be as small as 3 × 3 px.The authors report no public dataset specifically for lane-marking localization in remote-sensing imagery.Source: B. Related work · p. 4
• The proposed approach uses DWT decompositions within FCNNs to preserve high-frequency information for small-marking segmentation, alongside a weighted loss and symmetric FCNN.Unlike the cited DWT-CNN classification work, this work focuses on segmentation, investigates insertion locations, and keeps RGB imagery as network input.Source: B. Related work · p. 4
• The work introduces what the authors describe as the first high-quality annotated dataset for lane-marking semantic segmentation in airborne imagery.They position FCNNs as a learned alternative to road-first, edge-detection-based methods.Source: B. Related work · p. 4

C. Our contribution

• Aerial lane-marking segmentation combines FCNNs with DWT to provide representations at different scales and full-spectral analysis.The authors identify FCNNs as their semantic-segmentation approach and DWT as a way to analyze inputs at multiple scales.Source: C. Our contribution · p. 5, §II · pp. 5–6
• The work introduces a high-quality pixel-wise annotated aerial lane-marking dataset and uses a cost-sensitive loss to penalize lane-marking errors more than background errors.The dataset is presented as a resource to encourage future work in the area.Source: C. Our contribution · p. 5

II. ARIAL LANENET: WAVELET-ENHANCED COST-SENSITIVE SYMMETRIC FULLY CONVOLUTIONAL NEURAL NETWORK

• Aerial LaneNet processes high-resolution images as sliding-window patches, predicts binary lane-marking masks, and stitches patches into a full-resolution output.The patch workflow is motivated by image resolution and hardware memory constraints.Source: §II · p. 5
• First-level DWT splits the grayscale input into four half-size decomposition parts, while further levels provide representations at different resolutions.The decomposition figure contrasts conventional FCNN low-frequency analysis with DWT full-spectral analysis.Source: Figure 6 · p. 6, Figure 7 · p. 6, §II · p. 6

A. Discrete Wavelet Transform (Background)

• DWT repeatedly applies low- and high-pass filters followed by downsampling, producing a multilevel filter-bank decomposition.In 2D, decomposition proceeds across image directions and continues on the low-frequency quadrant.Source: A. Discrete Wavelet Transform… · pp. 6–7
• The method uses the first-order Daubechies Haar wavelet, while the Daubechies family is denoted dbN by order.The paper identifies db1 as Haar and notes its lower computational cost and fewer filter-bank coefficients.Source: A. Discrete Wavelet Transform… · pp. 6–7
• Four-level DWT inputs are computed from grayscale imagery and injected at progressively smaller resolutions; the DWT parameters remain fixed during training.The resulting sizes are H/2 × W/2, H/4 × W/4, H/8 × W/8, and H/16 × W/16; the first level produces approximate, horizontal, vertical, and diagonal components.Source: A. Discrete Wavelet Transform… · p. 7
• The network combines wavelet decompositions with a symmetric FCNN and uses a cost-sensitive loss that weights lane-marking errors more heavily than background errors.The architecture adds convolution and dropout after upsampling, includes an FCN4s variant, and uses max pooling; first-level fusion follows the first pooling operation because its resolution matches the wavelet output.Source: A. Discrete Wavelet Transform… · pp. 7–8

III. EXPERIMENTS

• The section covers the experimental dataset, procedures, quantitative and qualitative results, and their discussion.Source: §III · p. 8

A. AerialLanes18 Dataset

• 20 RGB images measuring 5616 × 3744 px were selected from a Munich-area aerial campaign, with approximately 13 cm ground sampling distance.The campaign used the German Aerospace Center's 3K camera system on April 26, 2012, at about 1,000 m above ground.Source: A. AerialLanes18 Dataset · p. 8
• The images depict urban and partly rural areas, including highways, first- and second-order roads, crossings, and congestion.Source: A. AerialLanes18 Dataset · p. 8

B. Annotation of AerialLanes18

• Human experts manually marked lane markings on roads and highways, including lines, turn signs, speed-limit signs, and bus and disabled-parking signs.Washed-out lane markings were ignored during annotation.Source: B. Annotation of AerialLanes18 · p. 8

C. Implementation Details

• ImageNet-pretrained networks initialize training because the dataset has few images and annotating lane markings is difficult; fine-tuning is used to reduce overfitting.FCN32s is the wavelet-fusion baseline with VGG16 as the main backbone; AlexNet, GoogleNet, and ResNet-101 are also considered.Source: C. Implementation Details · pp. 8–9
• The network takes 1024 × 1024 px patches; training uses an 800 px horizontal and vertical cropping step, while testing uses a 1000 px step, with random flipping during training augmentation.One random image provides a validation set of 24 patches; the test set has 240 patches. Final training uses the training and validation sets.Source: C. Implementation Details · p. 9
• The final network is trained end-to-end for about 10 epochs with Adam, a 0.0001 learning rate, and batch size 1.No road segmentation or third-party data such as OpenStreetMap is used; training runs on one Nvidia Titan X Pascal GPU using TensorFlow.Source: C. Implementation Details · p. 9

• Predictions are evaluated per image against ground truth, with each pixel classified as lane marking or background.Source: §IV · p. 9
• Mean IoU and Dice are used alongside pixel accuracy, frequency weighted IoU, recall, and precision; Dice is included because the dataset is heavily imbalanced.IoU measures overlap between predictions and ground truth; Dice is described as unaffected by the number of pixels in each class.Source: §IV · p. 11
• Weighted loss raises the FCN32s-AlexNet baseline’s mean IoU by almost 2 percent over its 51.0% mean IoU when trained from scratch.The loss penalizes lane-marking pixel errors more than background pixel errors.Source: §IV · p. 11

• VGG16 outperforms the shallower AlexNet and slightly outperforms GoogleNet among the evaluated backbones.Source: 1) Different Base Network Investigation · p. 11
• Pixel accuracy and frequency weighted IoU exceed 99% because background pixels dominate, making mean IoU and Dice more reliable for this task.The authors attribute background overfitting to the small dataset and the class imbalance.Source: 1) Different Base Network Investigation · p. 11

2) The Effect of

• λlane = 400 gives the best reported result, rather than the training-set pixel ratio of 389, and performance degrades at the test-set ratio of 308.The authors note that this hyper-parameter has no automatic tuning approach and suggest further research on finding it automatically.Source: 2) The Effect of · pp. 11–12, Table III · p. 12

3) The Importance of Symmetric FCNN:

• The symmetric FCNN improves mean IoU by almost 3 percent, while added convolution, dropout, and up-sampling layers each contribute around 1 percentage point.The authors report that deeper networks still struggle with small lane markings, which can be smoothed and removed by convolution and average pooling.Source: 3) The Importance of Symmetric FCNN · p. 12, Table IV · p. 12

4) The Effect of DWT:

• Combining the first four DWT decomposition levels performs best; adding a fifth worsens results, and the authors also evaluate a ResNet-101 backbone with DWT at selected stages.They omit the fifth level at stage 5 based on the VGG16 fifth-level result.Source: 4) The Effect of DWT · p. 12, Table V · p. 13

5) The Effect of DWT Components:

• Horizontal and vertical DWT components each improve mean IoU by almost 5 percent, while diagonal details add almost 2 percentage points and the approximation component worsens performance.The authors report lower performance with higher Daubechies orders, including 1.45 mean IoU for db2.Source: 5) The Effect of DWT Components · pp. 12, 14, Table VI · p. 13

6) Varied Possible Fusions:

• Fusion immediately after pooling layers performs best; fusion before pooling decreases mean IoU, though the reported degradation is not significant.The authors suggest that subsequent convolutions extract higher-level features after post-pooling fusion, while max pooling can retain high-valued lane pixels.Source: 6) Varied Possible Fusions · p. 14

7) Confusion Matrix Investigation:

• The best-performing configuration achieves 71.55% lane-marking pixel accuracy despite the heavily imbalanced dataset.Source: 7) Confusion Matrix Investigation · p. 14, Table IX · p. 14
• Background classification yields 0.1% false positives and 99.8% true negatives, but false negatives remain high.Source: 7) Confusion Matrix Investigation · p. 14
• False negatives are common for straight and dot-shaped markings, whose narrow boundaries and small size are difficult to segment; shadows and rare classes also remain challenging.Dot-shaped objects can be as small as 5 × 5 px, and the authors report that infrequent classes lack enough training samples for strong performance.Source: 7) Confusion Matrix Investigation · p. 14

8) Comparison with the state-of-the-art:

• Aerial LaneNet outperforms the compared networks, with a 4% mIOU improvement over DeepLabv3+.The comparison includes FCN-8s, DeepLab, UNet, and DeepLabv3; the qualitative figure compares predictions with ground truth and DeepLabv3/DeepLabv3+.Source: 8) Comparison with the state-of-the-art · pp. 14–15, Figure 16 · p. 15

9) Qualitative Analysis:

• Qualitatively, it segments straight and dashed highway lines, but shadows, narrow or tiny markings, and look-alike structures drive false positives or negatives.It sometimes localizes markings absent from the ground truth, while shadow areas remain difficult.Source: 9) Qualitative Analysis · p. 15

10) Cross-domain Generalization:

• Across flights with varied GSD, camera angles, and illumination, the network shows good generalization and robustness to most earlier challenges.Tests include a new dataset and examples with small markings and soccer-field lines resembling lane markings.Source: 10) Cross-domain Generalization · p. 15

V. CONCLUSIONS

• The conclusion attributes tiny-object segmentation to retaining high- and low-frequency pixel information, while noting shadows, semantic road signs, and washed-out markings as limits.The authors introduce AerialLanes18 as a high-quality benchmark dataset and plan to investigate different processing of shadow areas.Source: §V · pp. 15–16