PaperlayerView exact revision on arXiv

Source-linked AI summary

CGNet: A Light-weight Context Guided Network for Semantic Segmentation

Tianyi Wu, Sheng Tang, Rui Zhang, Yongdong Zhang

arXiv:1811.08201v2cs.CV

TL;DR

Semantic segmentation models must provide high accuracy while remaining deployable on mobile devices, but existing approaches often emphasize either accuracy or compactness. CGNet addresses this gap with Context Guided blocks that combine local, surrounding, and global context across all network stages. It reports 64.8% mean IoU on Cityscapes with fewer than 0.5 M parameters, without pre-processing, post-processing, or complex upsampling.

  • Problem

    Mobile semantic segmentation requires both high accuracy and a small memory footprint, whereas existing high-accuracy and lightweight models do not simultaneously satisfy these needs.

  • Method

    CGNet uses Context Guided blocks to learn joint local-surrounding features, refine them with global context, and capture context at semantic and spatial levels across all stages.

  • Results

    64.8% mean IoU is achieved on the Cityscapes test set with less than 0.5 M parameters and without pre-processing, post-processing, or complex upsampling.

  • Takeaways & Limitations

    Experiments on Cityscapes and CamVid show CGNet provides a lightweight and memory-efficient approach for high-quality semantic segmentation under resource limitations.

Abstract

from arXiv · show

The demand of applying semantic segmentation model on mobile devices has been increasing rapidly. Current state-of-the-art networks have enormous amount of parameters hence unsuitable for mobile devices, while other small memory footprint models follow the spirit of classification network and ignore the inherent characteristic of semantic segmentation. To tackle this problem, we propose a novel Context Guided Network (CGNet), which is a light-weight and efficient network for semantic segmentation. We first propose the Context Guided (CG) block, which learns the joint feature of both local feature and surrounding context, and further improves the joint feature with the global context. Based on the CG block, we develop CGNet which captures contextual information in all stages of the network and is specially tailored for increasing segmentation accuracy. CGNet is also elaborately designed to reduce the number of parameters and save memory footprint. Under an equivalent number of parameters, the proposed CGNet significantly outperforms existing segmentation networks. Extensive experiments on Cityscapes and CamVid datasets verify the effectiveness of the proposed approach. Specifically, without any post-processing and multi-scale testing, the proposed CGNet achieves 64.8% mean IoU on Cityscapes with less than 0.5 M parameters. The source code for the complete system can be found at https://github.com/wutianyiRosun/CGNet.

1. Introduction

CGNet targets accurate semantic segmentation on mobile devices by combining contextual modeling with an architecture designed for low parameter and memory costs.

  • Motivation: Mobile deployment requires semantic segmentation models to balance high accuracy with a small memory footprint.Existing high-accuracy models can have tens of millions of parameters, while small-footprint methods may achieve low segmentation accuracy.
  • Efficiency: CGNet uses a deep-and-thin architecture and channel-wise convolutions to reduce parameters and memory footprint.It contains 51 layers with 32, 64, and 128 channels across its three stages.
  • Results: 64.8% mean IoU is achieved on the Cityscapes test set with less than 0.5 M parameters and without pre-processing, post-processing, or complex upsampling.The introduction reports this result as evidence of CGNet’s efficiency and effectiveness.
  • Context Guided block: CGNet’s Context Guided block learns joint features from local features and surrounding context, then refines them with global context.The block models spatial dependency and semantic contextual information through local, surrounding, and global context.
  • Architecture: CGNet applies Context Guided blocks in all stages to capture contextual information from both semantic and spatial levels.This design differs from architectures that capture context only after encoding or ignore contextual information.
  • Results: Under an equivalent number of parameters, CGNet significantly outperforms existing segmentation networks such as ENet and ESPNet.The comparison is reported as a contribution of the proposed architecture.

2. Related Work

Prior lightweight segmentation models trade accuracy against resource use, while contextual models improve feature understanding through expanded or propagated context. CGNet instead integrates contextual information throughout its architecture using Context Guided blocks.

  • Small semantic segmentation models: Small semantic segmentation models seek a balance between segmentation accuracy and model parameters or memory footprint.ENet, ICNet, and ESPNet target embedded or resource-constrained deployment, but many such models follow image-classification design principles.
  • Context Guided block: The CG block combines local, surrounding, and global context through dedicated feature extractors.Its surrounding-context extractor uses atrous or dilated convolution to obtain a relatively large receptive field.
  • Residual learning: Residual learning in the CG block is implemented through Local Residual Learning and Global Residual Learning connections.These connections bridge the input with the joint-feature and global-feature extraction paths.
  • Contextual information models: Several contextual-information models enlarge receptive fields or add modules to capture multi-scale, adaptive, or global context.Examples include Dilation8, SAC, DenseASPP, PSPNet, and global-residual refinement approaches.
  • Contextual information models: Other approaches construct information-propagation models to capture dense and global pairwise relationships in images.SPN is described as using row- and column-based linear propagation.
  • Attention models: Attention models learn to reweight features or channel responses when predicting semantic labels.The related work discusses attention mechanisms across input scales and channel-wise feature recalibration.

3. Proposed Approach

The proposed approach uses Context Guided blocks to combine local features, surrounding context, and global context, then builds a lightweight network that aggregates context across stages while preserving spatial information.

  • Context Guided Block: The Context Guided block combines local features, surrounding context, joint features, and global context to improve semantic segmentation representations.It uses standard convolution for local features and atrous/dilated convolution for surrounding context.
  • Architecture Configuration: CGNet’s architecture is specified for Cityscapes with stage-2 and stage-3 block counts M and N, while its surrounding-context extractor uses atrous/dilated convolution rate r.The supplied architecture table caption identifies these configuration variables but does not provide their selected values.
  • Context Guided Block: The CG block uses local and global residual learning connections to support complex feature learning and gradient back-propagation.Local residual learning connects the input to the joint feature extractor, while global residual learning connects the input to the global feature extractor.
  • Context Guided Network: CGNet follows a “deep and thin” design with 51 convolutional layers, small channel numbers, three down-sampling stages, and 1/8 feature-map resolution.The architecture is designed to preserve discriminative spatial information while reducing memory footprint.
  • Context Guided Network: CGNet applies Context Guided blocks throughout stages 2 and 3 to aggregate contextual information from shallow spatial levels to deep semantic levels.This design differs from methods that add context modules only after encoding.
  • Context Guided Network: Channel-wise convolutions in the local and surrounding context extractors reduce cross-channel computational cost and memory footprint.The design is intended to reduce parameters while retaining the separate local and surrounding-context processing paths.

4. Experiments

Experiments evaluate CGNet through ablations and benchmark comparisons on Cityscapes and CamVid, showing that its context mechanisms improve accuracy while maintaining a small resource footprint.

  • Experimental Settings: Cityscapes contains 5,000 street-scene images with 19 annotated classes, while CamVid provides 367 training, 101 validation, and 233 testing images across 11 classes.Cityscapes performance uses IoU; CamVid reports pixel IoU averaged across 11 classes.
  • Ablation Studies: The surrounding context extractor improves accuracy by 5.1% when used in all blocks, compared with 0.8% when used only in the last block.The full configuration is therefore reported as beneficial for segmentation accuracy.
  • Ablation Studies: Global context improves accuracy from 58.9% to 59.7%, supporting its use for refining the joint feature.The ablation directly evaluates the global context extractor fglo(∗).
  • Ablation Studies: Input injection improves accuracy from 59.4% to 59.7% by down-sampling the input image into stages 2 and 3.The reported interpretation is increased information flow through the network.
  • Ablation Studies: PReLU improves performance from 58.1% to 59.7% compared with ReLU, so the proposed model uses PReLU.This comparison is performed in CGNet M3N15.
  • Comparison with State-of-the-Arts: CGNet has FLOPS and parameter counts close to ENet, improves over ENet by 6.5%, and is 4.5% more accurate than ESPNet.Its memory requirement is reported as 334.0 M, versus 3997.5 M for DenseASPP.
  • Comparison with State-of-the-Arts: 14.3% higher accuracy than ENet is reported on CamVid, while CGNet is only 1.9% below FCN (Res101) with 110× fewer parameters.The comparison uses 0.5 M versus 56.8 M parameters for CGNet and FCN (Res101), respectively.

5. Conclusions

The paper rethinks semantic segmentation around image recognition and object localization, then introduces CGNet to combine contextual modeling with memory-efficient inference for resource-limited settings.

  • Conclusions: The Context Guided block learns joint features from local and surrounding context, then further improves them with global context.This design targets the combined recognition and localization characteristics of semantic segmentation.
  • Conclusions: CGNet applies Context Guided blocks across network stages in a light-weight architecture for semantic segmentation.The paper reports experiments on Cityscapes and CamVid as evidence of its effectiveness under resource limitations.
Loading 1811.08201v2…