Source-linked AI summary
Smoothed Dilated Convolutions for Improved Dense Prediction
Zhengyang Wang, Shuiwang Ji
TL;DR
Dilated convolutions suffer from gridding artifacts that limit dense prediction performance. This paper smooths individual dilated convolutions through decomposition-based degridding and introduces an SS output layer, yielding consistent improvements with negligible extra training parameters.
Problem
Dilated convolutions can create gridding artifacts because adjacent outputs use separate input units, producing inconsistent local information and limiting dense prediction performance.
Method
The paper proposes decomposition-based methods that smooth individual dilated convolutions, generalizes them as separable and shared operations, and introduces an SS output layer.
Results
The proposed methods consistently improve dense prediction performance with negligible extra training parameters, while the SS output layer significantly improves performance by replacing only the output layer.
Takeaways & Limitations
The methods provide efficient degridding for dilated-convolution networks, including whole-network smoothing through an output-layer replacement.
Abstract
from arXiv · showhide
Dilated convolutions, also known as atrous convolutions, have been widely explored in deep convolutional neural networks (DCNNs) for various dense prediction tasks. However, dilated convolutions suffer from the gridding artifacts, which hampers the performance. In this work, we propose two simple yet effective degridding methods by studying a decomposition of dilated convolutions. Unlike existing models, which explore solutions by focusing on a block of cascaded dilated convolutional layers, our methods address the gridding artifacts by smoothing the dilated convolution itself. In addition, we point out that the two degridding approaches are intrinsically related and define separable and shared (SS) operations, which generalize the proposed methods. We further explore SS operations in view of operations on graphs and propose the SS output layer, which is able to smooth the entire DCNNs by only replacing the output layer. We evaluate our degridding methods and the SS output layer thoroughly, and visualize the smoothing effect through effective receptive field analysis. Results show that our methods degridding yield consistent improvements on the performance of dense prediction tasks, while adding negligible amounts of extra training parameters. And the SS output layer improves the performance significantly and is very efficient in terms of number of training parameters.
1 INTRODUCTION
Dilated convolutions enlarge receptive fields while preserving spatial resolution, but can create gridding artifacts that disrupt local information and hamper DCNN performance. This work smooths individual dilated convolutions with parameter-efficient degridding methods and proposes an SS output layer that smooths entire networks by replacing only the output layer.
- Dilated convolutions enlarge receptive fields while keeping spatial resolution and have been widely used for semantic segmentation, object detection, audio generation, video modeling, and machine translation.
- For dilation rates above one, adjacent output units use completely separate input units, causing inconsistent local information known as gridding artifacts that hampers DCNN performance.
- The proposed degridding methods smooth the dilated convolution itself rather than a stacked block, allowing any single dilated convolutional layer to be replaced independently.
- The methods add minimal extra parameters and are based on a decomposition-based interpretation of dilated convolutional operations [5],, [23].
- Graph-based analysis motivates the SS output layer, which smooths DCNNs with dilated convolutions by replacing only the output layer and better aggregates information from large receptive fields than original dilated-convolution output layers.
2 BACKGROUND AND RELATED WORK
Dilated convolutions expand receptive fields by inserting zeros between filter weights, but cascaded layers can produce gridding artifacts when dilation rates share factors. Prior solutions mainly address this problem across blocks of stacked layers, including hybrid dilation approaches.
- Dilated convolutions: Dilated convolutions insert r −1 zeros between adjacent filter weights, with r = 1 recovering standard convolution; larger rates expand the sampling pattern.They are also called atrous convolutions, and higher-dimensional cases generalize the one-dimensional definition.
- Related work: Recent semantic-segmentation studies [4], [7], [8] primarily address gridding by designing solutions for blocks of stacked dilated convolutional layers, including hybrid dilated convolution (HDC) [7].These approaches reflect the prevalent use of dilated convolutions in cascade within DCNNs.
- Gridding problem: Cascaded dilated convolutions intensify gridding when continuously stacked dilation rates share a common factor of 2, while parallel dilated convolutions have also been explored for output layers [5], [6].The cascade pattern is used in DCNNs for tasks including semantic image segmentation, where dilation can preserve receptive-field size after downsampling removal.
- Gridding problem: Dilation rates larger than one create gridding artifacts because adjacent outputs are computed from completely separate input units and therefore have different actual receptive fields.In cascaded layers, a common-factor relationship such as 2, 2, 2 or 2, 4, 8 propagates the problem through the block [7].
3 SMOOTHED DILATED CONVOLUTIONS
The section decomposes dilated convolution into periodic subsampling, shared standard convolution, and reinterlacing, explaining gridding as inconsistent information across independent groups. It proposes two smoothing approaches, generalizes them as separable-and-shared operations, and introduces an SS output layer for efficient network-wide smoothing and aggregation.
- 3.1 Decomposition: Dilated convolution decomposes into periodic subsampling, shared standard convolution, and reinterlacing, offering an alternative view to zero-inserted filters.The shared convolution uses the same filters for every reduced-resolution group, which is then reinterlaced to the original resolution.
- 3.1 Decomposition: Gridding arises because the r^d intermediate groups lack dependencies and can therefore collect inconsistent local information.The proposed remedies add dependencies among these groups at different decomposition stages.
- 3.2–3.3 Degridding methods: The methods are related through separable and shared operations: SS convolutions use one spatial filter shared across all input-output channel pairs, unlike separable convolutions’ channel-specific filters.An SS convolution can be inserted before decomposition, while the other degridding method operates between shared convolution and reinterlacing; the approaches correspond to SS operations around the dilated convolution.
- 3.4 SS operations: SS operations can be interpreted as directed graph subgraphs, with different graph constructions producing different operations; related weight-generation methods include MoNet, GraphSAGE, GAT [30], and LGCN.The SS convolution updates the center node by incorporating information from all nodes in its scanning window, unlike the SS block-wise fully-connected layer.
- 3.5 SS output layer: The SS output layer replaces output-layer dilated convolutions with an SS operation that smooths and aggregates information, potentially requiring only one operation after a cascaded encoder.Within each window it uses the SS-convolution subgraph and GAT [30] to generate scalar weights; sufficiently large windows can aggregate global information.
4 EXPERIMENTAL STUDIES
Experiments on PASCAL VOC 2012 and Cityscapes show that the proposed degridding methods consistently improve dilated-convolution DCNNs, while ERF analysis visualizes their smoothing effect. Replacing only DeepLabv2’s output layer with an SS output layer also improves performance efficiently.
- Experimental setup: The proposed methods address gridding artifacts in the last two encoder blocks while comparisons use different output layers and two datasets.The baseline is DeepLabv2 with a ResNet-101 encoder modified using dilation rates r = 2 and r = 4.
- Degridding results: The Multigrid variant does not improve performance in this implementation, possibly because prior evaluations combined it with other modifications.The cited explanation mentions dense upsampling convolution and deeper encoders as additional changes in prior studies [7] and [6].
- Degridding results: Both proposed degridding methods improve mIoU over the DeepLabv2 baseline with and without MS-COCO pre-training, indicating dataset-independent gains.The methods also improve IoU for most classes under both settings.
- ERF analysis: ERF analysis verifies that the proposed improvements arise from degridding by visualizing the smoothing effect in selected DCNN blocks.The first visualization examines the encoder’s final 3×3 dilated convolution with dilation rate r = 4.
- SS output layer: The SS output layer improves DeepLabv2 performance when only the output layer is replaced, and its efficiency is evaluated through output-layer parameter counts.Experiments vary the SS output layer’s window size, with dk and do set to 512 and 8 graph-attention heads.
5 CONCLUSIONS
The paper proposes efficient SS-based degridding methods that smooth dilated convolutions directly, requiring negligible extra parameters. An SS output layer further smooths the entire network by replacing only the output layer and improves dense prediction.
- 5 CONCLUSIONS: The two degridding methods address gridding artifacts within a single dilated convolution rather than across cascaded layers, while requiring negligible extra parameters.Experimental results show that the methods improve DCNNs with dilated convolutions.
- 5 CONCLUSIONS: The SS output layer smooths the entire network by replacing only the output layer and obtains improved dense prediction.This extends the proposed SS-operation direction beyond the individual degridding methods.
- 5 CONCLUSIONS: Overall, the proposed SS-based degridding methods are efficient and effective for improving DCNNs with dilated convolutions.The conclusion combines the methods’ negligible parameter overhead with their reported performance improvements.