Source-linked AI summary
DABNet: Depth-wise Asymmetric Bottleneck for Real-time Semantic Segmentation
Gen Li, Inyoung Yun, Jonghyun Kim, Joongkyu Kim
TL;DR
Real-time semantic segmentation requires balancing accuracy with inference speed and model capacity. DABNet addresses this challenge with a depth-wise asymmetric bottleneck combining depth-wise asymmetric and dilated convolutions, achieving 70.1% Mean IoU at 104 FPS with 0.76 million parameters on Cityscapes.
Problem
Real-time semantic segmentation must balance competitive accuracy with low energy, memory use, and inference speed.
Method
DABNet uses a depth-wise asymmetric bottleneck that combines depth-wise asymmetric and dilated convolutions to extract local and contextual features jointly.
Results
70.1% Mean IoU is achieved on the Cityscapes test set at 104 FPS with 0.76 million parameters, without pretraining or post-processing.
Takeaways & Limitations
DABNet provides a lightweight real-time segmentation model with fast inference speed and competitive results across Cityscapes and CamVid benchmarks.
Takeaways & Limitations
DABNet does not depend on any backbone and is designed from scratch with less than 0.76 million parameters.
Abstract
from arXiv · showhide
As a pixel-level prediction task, semantic segmentation needs large computational cost with enormous parameters to obtain high performance. Recently, due to the increasing demand for autonomous systems and robots, it is significant to make a tradeoff between accuracy and inference speed. In this paper, we propose a novel Depthwise Asymmetric Bottleneck (DAB) module to address this dilemma, which efficiently adopts depth-wise asymmetric convolution and dilated convolution to build a bottleneck structure. Based on the DAB module, we design a Depth-wise Asymmetric Bottleneck Network (DABNet) especially for real-time semantic segmentation, which creates sufficient receptive field and densely utilizes the contextual information. Experiments on Cityscapes and CamVid datasets demonstrate that the proposed DABNet achieves a balance between speed and precision. Specifically, without any pretrained model and postprocessing, it achieves 70.1% Mean IoU on the Cityscapes test dataset with only 0.76 million parameters and a speed of 104 FPS on a single GTX 1080Ti card.
1 Introduction
DABNet targets real-time semantic segmentation’s tradeoff between accuracy, capacity, and inference speed by combining depth-wise asymmetric and dilated convolutions. The resulting network reports competitive benchmark performance with very low parameter count and fast inference.
- 1 Introduction: Real-time semantic segmentation must balance competitive accuracy with low energy, memory use, and inference speed.
- 1 Introduction: Dilated convolution enlarges the receptive field, while depth-wise separable convolution reduces parameters but can degrade performance when used as a simple replacement.
- 1 Introduction: DAB combines depth-wise asymmetric convolution with dilated convolution to jointly extract local and contextual information while reducing parameters.
- 1 Introduction: DABNet is designed with fewer parameters, comparable accuracy, and faster inference speed than existing state-of-the-art real-time segmentation methods.
- 1 Introduction: 70.1% Mean IoU on the Cityscapes test dataset is achieved at 104 FPS with 0.76 M parameters, without a context module, pretrained model, or post-processing.
2 Related Work
Related real-time segmentation methods use architectural strategies to balance prediction quality and inference speed, while dilated convolution expands receptive fields without reducing feature-map resolution. These approaches remain open to further improvement.
- 2 Related Work: Real-time segmentation networks seek a tradeoff between high-quality prediction and high inference speed.
- 2 Related Work: ENet, ICNet, ERFNet, ESPNet, and BiSeNet use filter reduction, multiresolution branches, factorized convolutions, spatial pyramids, or dual paths for efficient segmentation.
- 2 Related Work: Dilated convolution inserts zeros between kernel elements to enlarge the effective receptive field without increasing padding-related resolution loss.
- 2 Related Work: Existing real-time networks achieve speed–performance tradeoffs, but the paper identifies further room for improvement.
3 Proposed Network
The proposed network builds DABNet around a two-branch depth-wise asymmetric bottleneck that combines local and contextual feature extraction. Architectural choices limit resolution loss and model capacity while retaining receptive-field expansion.
- 3 Proposed Network: DABNet uses the DAB module as its core component and designs the final architecture around it.
- 3.1 Depth-wise Asymmetric Bottleneck: The bottleneck halves channels before restoring them with a point-wise convolution, while using a 3 × 3 convolution at the module beginning.
- 3.1 Depth-wise Asymmetric Bottleneck: Factorized depth-wise convolution replaces an N × N kernel with N × 1 followed by 1 × N convolution, reducing per-pixel complexity from O(N2) to O(N).
- 3.1 Depth-wise Asymmetric Bottleneck: The two DAB branches separately provide local information and contextual information, then are added and fused with a 1 × 1 point-wise convolution.
- 3.2 Network Architecture Design: DABNet uses three downsampling operations to obtain 1/8-resolution features while preserving spatial information and details.
- 3.2 Network Architecture Design: The two DAB blocks contain 3 and 6 consecutive DAB modules, with dilation rates in the second block set to 4, 4, 8, 8, 16, 16.
- 3.2 Network Architecture Design: The model is trained from scratch without post-processing and uses less than 0.76 million parameters.
4 Experiments
Experiments evaluate DABNet on Cityscapes and CamVid using controlled training and runtime protocols, ablations, benchmark comparisons, and qualitative examples. The results support a speed–accuracy balance with small model capacity.
- Datasets and protocol: Cityscapes uses 2,975 training, 500 validation, and 1,525 test images with 19 categories; CamVid uses 367 training, 101 validation, and 233 test images with 11 categories.Experiments use fine annotations for Cityscapes and report runtime on a single GTX 1080Ti.
- Ablation study: 2.3% lower accuracy for DABNet-(r=4) than DABNet indicates that increasing dilation rates is important in the tested configuration.Coprime dilation rates also perform worse than DABNet.
- Ablation study: 104.2–104.5 FPS is maintained when dilated convolutions are removed or reduced, whereas dilation rate 2 in standard convolution lowers speed from 104.2 to 85.6 FPS.Applying dilation to depth-wise convolution has nearly no harmful effect on speed in this experiment.
- State-of-the-art comparison: 70.1% mIoU on the Cityscapes test set is achieved with 0.76 million parameters, significantly outperforming existing real-time segmentation work.The evaluation compares accuracy and parameter counts on the Cityscapes test set.
- State-of-the-art comparison: DABNet uses 9.7% of ICNet’s parameters while achieving a better result, and about 1% of the parameters of PSPNet and DeepLabV2 for comparable performance.On CamVid, it processes 360 × 480 images at 146 FPS.
5 Conclusions
DABNet combines a Depth-wise Asymmetric Bottleneck module with a lightweight architecture to jointly extract local and contextual features. It achieves competitive accuracy, fast inference, and low capacity on Cityscapes.
- DABNet uses a Depth-wise Asymmetric Bottleneck module to extract local and contextual features jointly.
- 70.1% Mean IoU on Cityscapes test set is achieved with only 0.76 million parameters.
- 104 fps is achieved on 512× 1024 high-resolution images.
- The network shows significant improvements in accuracy, speed, and capacity compared with state-of-the-art methods.