PaperlayerView exact revision on arXiv

Source-linked AI summary

Efficient Dense Modules of Asymmetric Convolution for Real-Time Semantic Segmentation

Shao-Yuan Lo, Hsueh-Ming Hang, Sheng-Wei Chan, Jing-Jhih Lin

arXiv:1809.06323v3cs.CV

TL;DR

Real-time semantic segmentation needs accurate predictions without the high computational cost and slow inference of many existing networks. EDANet combines asymmetric convolution, dilated convolution, and dense connectivity in an efficient architecture. It achieves 67.3% mIoU at 108.7 FPS on Cityscapes using a GTX 1080Ti, without extra context or post-processing modules.

  • Problem

    Real-time semantic segmentation requires balancing accuracy with efficiency because accuracy-focused networks can be costly and speed-focused networks may lose accuracy.

  • Method

    EDANet combines asymmetric convolution, dilated convolution, and dense connectivity in an efficient network without extra decoder, context, or post-processing modules.

  • Results

    67.3% mIoU and 108.7 FPS were achieved on the Cityscapes test set using a single GTX 1080Ti.

  • Takeaways & Limitations

    EDANet provides a reported trade-off between segmentation reliability and inference efficiency at relatively small computational cost.

Abstract

from arXiv · show

Real-time semantic segmentation plays an important role in practical applications such as self-driving and robots. Most semantic segmentation research focuses on improving estimation accuracy with little consideration on efficiency. Several previous studies that emphasize high-speed inference often fail to produce high-accuracy segmentation results. In this paper, we propose a novel convolutional network named Efficient Dense modules with Asymmetric convolution (EDANet), which employs an asymmetric convolution structure and incorporates dilated convolution and dense connectivity to achieve high efficiency at low computational cost and model size. EDANet is 2.7 times faster than the existing fast segmentation network, ICNet, while it achieves a similar mIoU score without any additional context module, post-processing scheme, and pretrained model. We evaluate EDANet on Cityscapes and CamVid datasets, and compare it with the other state-of-art systems. Our network can run with the high-resolution inputs at the speed of 108 FPS on one GTX 1080Ti.

1 National Chiao Tung University 2 Industrial Technology Research Institute

The paper focuses on semantic segmentation, real-time processing, and fast network design.

  • Semantic segmentation is a central topic.
  • Real-time operation is a key focus.
  • Fast network design is emphasized.

1 INTRODUCTION

Real-time semantic segmentation requires balancing accuracy and efficiency because accurate networks can be computationally costly, while fast networks may sacrifice accuracy. EDANet addresses this trade-off through an efficient architecture using asymmetric convolution, dense connectivity, and no extra processing modules.

  • Real-time applications require effective networks that combine segmentation accuracy with fast inference.
  • High-performing methods such as PSPNet and SegModel achieve accuracy at the expense of computational cost and inference speed.
  • Speed-focused methods such as ENet and ESPNet improve efficiency but experience notable accuracy drops.
  • EDANet is designed to achieve both high efficiency and accuracy, placing it among systems exceeding 30 FPS.
  • Asymmetric convolution factorizes an n×n kernel into n×1 and 1×n convolutions, reducing parameters with little performance degradation.
  • EDANet omits extra decoder, context, and post-processing modules while retaining high accuracy and inference speed.

2 RELATED WORK

Prior semantic segmentation research includes accuracy-oriented, speed-oriented, and densely connected network designs, each with distinct efficiency or complexity trade-offs.

  • CNN-based segmentation: FCN pioneered CNN-based semantic segmentation by replacing fully connected layers with convolutions for pixel-level prediction.
  • High accuracy networks: U-Net and DeconvNet use decoder structures to recover spatial information, but their heavy decoders impose substantial computational cost.
  • High accuracy networks: Dilation10 and DeepLab introduce dilated-convolution context mechanisms for aggregating multi-scale information.
  • High inference speed networks: ENet and ESPNet greatly improve efficiency but significantly sacrifice accuracy.
  • High inference speed networks: ICNet and BiSeNet improve the balance between inference speed and segmentation performance, though further improvement remains possible.
  • Densely connected networks: FC-DenseNet and SDN extend DenseNet to segmentation but add decoder complexity and computational cost.

3 METHOD

EDANet combines asymmetric and dilated convolution with module-level dense connectivity to balance segmentation efficiency, accuracy, and spatial detail. Its architecture uses lightweight downsampling, omits a decoder, and upsamples class maps directly.

  • EDANet architecture: EDANet comprises three downsampling blocks, two EDA blocks, and a projection layer, with 5 and 8 densely connected EDA modules in the two EDA blocks.
  • EDA module: Each EDA module begins with a 1×1 point-wise convolution and contains two pairs of asymmetric convolution layers, concatenating input with newly produced features.
  • Asymmetric convolution: 33% of parameters and computational cost are saved for kernel size 3, while performance degradation is often very small.
  • Dilated convolution: Dilated convolution enlarges receptive fields without increasing parameter count and is inserted in the second asymmetric pair of selected EDA modules.The dilation rates are 2, 2, 2, 2, 4, 4, 8, 8, 16, and 16.
  • Dense connectivity: Module-level dense connectivity lets each module acquire only a few new features while aggregating information from different receptive fields at low computational cost.
  • Network design choices: EDANet uses only three downsampling operations, retains a 1/8 feature-size ratio, and replaces a decoder with 1×1 projection plus 8× bilinear interpolation.This strategy reduces accuracy slightly but saves many computations.

4 EXPERIMENTS

EDANet is evaluated on Cityscapes and CamVid through dataset-specific experiments, ablations, and comparisons with existing real-time segmentation systems. The results examine architectural choices and report high accuracy alongside high inference speed.

  • Datasets: Experiments use fine-annotated Cityscapes and CamVid datasets, with Cityscapes inputs downsampled to 512×1024 for training and testing.Cityscapes has 19 classes and 5000 fine-annotated images; CamVid has 11 classes, 367 training images, and 233 testing images.
  • Core Module: 1.18% lower accuracy for EDA-non-dense than EDANet supports the effectiveness of dense connectivity.EDA-non-asym has almost the same accuracy but 27% more computational cost, indicating the efficiency advantage of asymmetric convolution.
  • Decoder: EDA-ERFdec improves accuracy by 0.46% but increases computational cost by 44%, motivating the omission of a decoder.The decoder adds two deconvolution-plus-ERF blocks and a final deconvolution layer.
  • Downsampling Block: EDANet attains 3.47% higher accuracy than EDA-DenseDown with only a little more computational cost.This compares EDANet’s downsampling design with a DenseNet-inspired alternative.
  • Evaluation Results: EDANet achieves 67.3% mIoU at 108.7 FPS on a single GTX 1080Ti for Cityscapes.It also reaches 81.3 FPS on a Titan X without multi-crop or multi-scale testing.
  • Evaluation Results: EDANet processes 360×480 CamVid images at 163 FPS on one GTX 1080Ti.The paper reports outstanding efficiency and accuracy on CamVid.

5 CONCLUSION

EDANet is presented as a real-time semantic segmentation network designed to balance segmentation reliability with computational efficiency. The proposed architecture achieves accurate results at relatively low computational cost compared with other state-of-the-art systems.

  • EDANet combines efficient dense modules with asymmetric convolution for real-time semantic segmentation.
  • The experiments support accurate segmentation with relatively small computational cost compared with other state-of-the-art systems.
  • The resulting architecture provides a trade-off between reliability and efficiency for scene-understanding applications.

ACKOWLEDGMENTS

The authors acknowledge contributors who assisted with project discussions and paper illustration, as well as institutional funding support.

  • Ping-Rong Chen is thanked for helpful discussions during the project.
  • Shang-Wei Hung is thanked for drawing the paper.
  • The work received support from the Mechanical and Mechatronics Systems Research Lab., ITRI, under Grant 3000547822.

A. NETWORK DETAILS

The appendix documents EDANet and its ablation variants through layer-disposal tables and implementation details. It standardizes the appendix input size and identifies the structures used for each variant.

  • Architecture documentation: The appendix provides detailed network architectures for EDANet and all variants used in the ablation study.
  • Architecture documentation: Tables 4–10 correspond to EDANet, EDA-non-asym, EDA-non-dense, EDA-shallow, EDA-ASPP, EDA-ERFdec, and EDA-DenseDown.
  • Implementation setting: All appendix architecture tables use input sizes of 512×1024.
  • Module structures: The appendix identifies structures for the EDA module, non-asymmetric and non-dense modules, downsampling block, and ASPP.
  • Variant relationships: EDA-non-dense uses dilation-rate placement consistent with ERFNet.
  • Variant relationships: EDA-ASPP and EDA-ERFdec use structures consistent with DeepLabv3 and ERFNet, respectively.

B. RESULTS ON THE CITYSCAPES AND THE CAMVID DATASETS

Additional evaluation reports class-level IoU scores on Cityscapes and CamVid and provides qualitative visual results for both datasets. Tables and figures organize the quantitative and visual evidence separately.

  • Quantitative results: Class-level IoU scores are reported for the Cityscapes and CamVid datasets.
  • Quantitative results: Table 11 presents IoU scores on the Cityscapes test set.
  • Quantitative results: Table 12 presents IoU scores on the CamVid test set.
  • Visual results: Figure 6 shows sample EDANet visual results on the Cityscapes validation set.
  • Visual results: Figure 7 shows sample EDANet visual results on the CamVid test set.
Loading 1809.06323v3…