PaperlayerView exact revision on arXiv

Source-linked AI summary

Inter-Region Affinity Distillation for Road Marking Segmentation

Yuenan Hou, Zheng Ma, Chunxiao Liu, Tak-Wai Hui, Chen Change Loy

arXiv:2004.05304v1cs.CV

TL;DR

Road-marking segmentation poses a challenge for small models because sparse supervision makes scene structure difficult to learn. IntRA-KD transfers a teacher’s structural knowledge by having the student match inter-region affinity graphs. It consistently outperforms prior distillation methods across three benchmarks and reports strong results with a substantially smaller, faster ERFNet than ResNet-101.

  • Problem

    Small road-marking models must learn scene structure from sparse labels, while existing distillation methods do not adequately transfer that structural knowledge.

    Source: §1 · p. 1

  • Method

    IntRA-KD represents scene regions as feature-distribution nodes linked by pairwise similarities, then trains the student to match the teacher’s affinity graph.

    Source: §1 · pp. 1–2

  • Results

    IntRA-KD consistently outperforms previous distillation methods across ApolloScape, CULane, and LLAMAS, with reported gains across lightweight student models.

    Source: §1 · p. 2, §5 · p. 8

  • Takeaways & Limitations

    The method generalizes across ERFNet, ENet, and ResNet-18 student architectures, with encouraging results also reported on Cityscapes.

    Source: §1 · p. 2, §4.1 · p. 7

Abstract

from arXiv · show

We study the problem of distilling knowledge from a large deep teacher network to a much smaller student network for the task of road marking segmentation. In this work, we explore a novel knowledge distillation (KD) approach that can transfer 'knowledge' on scene structure more effectively from a teacher to a student model. Our method is known as Inter-Region Affinity KD (IntRA-KD). It decomposes a given road scene image into different regions and represents each region as a node in a graph. An inter-region affinity graph is then formed by establishing pairwise relationships between nodes based on their similarity in feature distribution. To learn structural knowledge from the teacher network, the student is required to match the graph generated by the teacher. The proposed method shows promising results on three large-scale road marking segmentation benchmarks, i.e., ApolloScape, CULane and LLAMAS, by taking various lightweight models as students and ResNet-101 as the teacher. IntRA-KD consistently brings higher performance gains on all lightweight models, compared to previous distillation methods. Our code is available at https://github.com/cardwing/Codes-for-IntRA-KD.

1. Introduction

Road marking segmentation is difficult for small models because sparse labels, tiny markings, poor lighting, and occlusions make scene structure hard to learn. IntRA-KD transfers a teacher’s scene-structure knowledge through inter-region affinity graphs and consistently outperforms other distillation methods across three benchmarks.

1. Introduction

  • Road marking segmentation is difficult because markings are tiny and sparse in labels, while poor lighting and vehicle occlusions further impede learning scene structure.These challenges become especially consequential for small models deployed on vehicles with limited computational resources.Source: §1 · p. 1
  • Existing distillation methods rarely exploit scene structure, leaving small models less able to distinguish visually similar but functionally different road markings.The paper frames structural relationships as an important constraint against sparse supervision.Source: §1 · p. 1
  • IntRA-KD represents regions as graph nodes encoding feature-distribution statistics and connects node pairs by similarity, then updates the student by matching the teacher’s graph.Figure 1 depicts intermediate student and teacher activations and affinity graphs whose edges represent cosine similarities.Source: §1 · pp. 1–2, Figure 1 · p. 2
  • IntRA-KD consistently outperforms previous distillation methods across ApolloScape, CULane, and LLAMAS, and ERFNet uses 21× fewer parameters and runs 16× faster than ResNet-101.The reported ERFNet comparison is 2.49 M versus 52.53 M parameters and 10.2 ms versus 171.2 ms.Source: §1 · p. 2

2. Related Work

Road marking segmentation moved from hand-crafted segment features to end-to-end dense prediction, but compact models still face limited capacity and sparse supervision. Prior distillation transfers logits, features, attention, or sample relations, while the cited approaches do not model relationships among regions within one scene.

2. Related Work

  • Hand-crafted road-marking features required sophisticated engineering and worked well only in simple highway scenarios.These methods classified segments after feature-based segmentation.Source: §2 · p. 2
  • Deep dense-prediction methods assign a category to each pixel, while lightweight models address memory and inference needs but remain constrained by context and sparse supervision.Prior responses include self-distillation with deep-layer attention [8], SCNN message passing [14], and joint lane-area and boundary prediction.Source: §2 · p. 2
  • Knowledge distillation transfers logits [6], feature maps [4] [9], or similarity maps, while sample-relation methods [15] [17] [23] do not model inter-region structure within one image.The paper positions IntRA-KD as incorporating relationships between different areas of a single sample.Source: §2 · p. 2

3. Methodology

The methodology formulates road-marking segmentation as pixelwise prediction and transfers teacher scene structure by having a student match inter-region affinity graphs. Regions are represented through areas of interest and feature-distribution statistics, whose pairwise similarities form the graph.

3.1. Problem Formulation

  • The student and teacher construct inter-region graphs whose nodes represent regional feature-distribution statistics and whose edges encode pairwise feature-distribution similarity.The student is trained to mimic the teacher's graph.Source: §3.1 · p. 3, Figure 2 · p. 4

3.2. Inter-Region Affinity Knowledge Distillation

  • Smoothed, thresholded ground-truth maps create AOI masks that include surroundings around road markings, unlike masks restricted to the marked pixels.The authors report that naïve distillation within ground-truth areas was ineffective for transferring contextual information.Source: §3.2 · p. 3
  • AOI masks extract class-region features, and moment pooling summarizes their distributions with first-, second-, and third-order statistics.These correspond to mean, variance, and skewness; higher-order moments bring marginal gains while requiring heavier computation.Source: §3.2 · pp. 3–4
  • The affinity graph combines mean, variance, and skewness sub-graphs, using class-pair moment-vector similarities as the high-level knowledge learned by the student.The method uses cosine similarity between moment vectors rather than matching vectors directly, avoiding extra operations or parameters for feature-dimension consistency.Source: Figure 2 · p. 4, §3.2 · p. 5, Figure 5 · p. 5
  • Affinity distillation is independent of feature-map dimensions and reduces target size from h×w×n to n^2, with n usually thousands of times smaller than h×w.The method also gathers AOI features from foreground and background areas.Source: §3.2 · p. 5

4. Experiments

Experiments compare IntRA-KD with prior methods across three road-marking benchmarks and find that ERFNet-IntRA-KD outperforms the baselines on all three. Additional tests examine other student backbones, qualitative predictions, and the contributions of individual loss terms.

4. Experiments

  • ApolloScape, CULane, and LLAMAS differ in label scope, and LLAMAS testing uses a split of its validation set because an official test server was unavailable.ApolloScape labels road markings by function and has more classes; the experiments use mIoU for ApolloScape, F1-measure for CULane, and mAP for LLAMAS.Source: §4 · pp. 5–6
  • The experiments use ERFNet, ENet, or ResNet-18 as students and ResNet-101 as teacher, comparing IntRA-KD with five contemporary distillation methods.The distillation targets include high- and middle-level teacher features; ERFNet block 2 and block 3 mimic ResNet-101 block 3 and block 5, respectively.Source: §4 · p. 6

4.1. Results

  • ERFNet-IntRA-KD outperforms all baselines and previous distillation methods on ApolloScape, CULane, and LLAMAS; on CULane it has 21× fewer parameters and runs 16× faster than ResNet-101.The comparisons are summarized in Tables 2–4; runtime was averaged across 100 samples on a single GeForce GTX TITAN X Maxwell GPU.Source: Table 2 · p. 6, §4.1 · p. 6, Table 3 · p. 7
  • IntRA-KD brings performance gains over BiFPN on ApolloScape with ENet and ResNet-18, with encouraging gains also reported on CULane and LLAMAS.Detailed results for ENet and ResNet-18 are provided in the supplementary material.Source: §4.1 · p. 7, Figure 7 · p. 7
  • ERFNet-IntRA-KD predicts long, thin markings more accurately and produces more complete predictions than ERFNet and ERFNet-BiFPN in crowded-road and poor-light examples.Its feature embeddings are more structured and distinctly clustered by class, with a distribution similar to the ResNet-101 teacher.Source: §4.1 · p. 7, Figure 8 · p. 7, Figure 9 · p. 8

4.2. Ablation Study

  • 43.2 mIoU is reached when all three affinity moments are combined with attention-map distillation; results remain similar across the tested loss coefficients.The figure compares the full combination with ERFNet without distillation, reported at 40.4 mIoU; the tested coefficients are 0.05, 0.10, and 0.15.Source: §4.2 · p. 8

5. Conclusion

IntRA-KD transfers scene-structural knowledge from teacher to student through inter-region affinity graphs. The paper reports consistent gains over contemporary distillation methods on three large-scale road-marking benchmarks.

5. Conclusion

  • IntRA-KD represents scene structure with inter-region affinity graphs that capture similarities in feature distributions across regions.The conclusion describes the method as transferring structural knowledge from a teacher model to a student model.Source: §5 · p. 8
  • 43.2 mIoU is reported for ERFNet-IntRA-KD using the three affinity-moment losses and attention-map loss, versus 40.4 mIoU for ERFNet without distillation.These values are shown for ApolloScape-test in the loss-term comparison.Source: Figure 10 · p. 8
  • IntRA-KD brings consistent performance gains over contemporary distillation methods across three large-scale road-marking segmentation benchmarks.The conclusion says Cityscapes results are provided in the supplementary material.Source: §5 · p. 8

Related papers

Loading 2004.05304v1…