Source-linked AI summary
Delving Deeper into Anti-aliasing in ConvNets
Xueyan Zou, Fanyi Xiao, Zhiding Yu, Yong Jae Lee
TL;DR
Aliasing during ConvNet downsampling can distort signals and make predictions sensitive to shifts, while a universal blur may not suit features with different frequencies. The paper introduces content-aware low-pass filtering that predicts filters by spatial location and channel group, and reports improved accuracy and shift consistency across multiple tasks.
Problem
Aliasing affects ConvNets during downsampling, and applying one filter everywhere is suboptimal because feature frequencies vary across spatial locations and channels.
Method
The proposed adaptive low-pass layer predicts separate filter weights for each spatial location and channel group before downsampling.
Results
Across classification and segmentation evaluations, the method outperforms competitive baselines in accuracy and shift consistency while adapting to different feature frequencies.
Takeaways & Limitations
Adaptive filtering can avoid aliasing while preserving useful information for recognition across multiple visual tasks.
Abstract
from arXiv · showhide
Aliasing refers to the phenomenon that high frequency signals degenerate into completely different ones after sampling. It arises as a problem in the context of deep learning as downsampling layers are widely adopted in deep architectures to reduce parameters and computation. The standard solution is to apply a low-pass filter (e.g., Gaussian blur) before downsampling. However, it can be suboptimal to apply the same filter across the entire content, as the frequency of feature maps can vary across both spatial locations and feature channels. To tackle this, we propose an adaptive content-aware low-pass filtering layer, which predicts separate filter weights for each spatial location and channel group of the input feature maps. We investigate the effectiveness and generalization of the proposed method across multiple tasks including ImageNet classification, COCO instance segmentation, and Cityscapes semantic segmentation. Qualitative and quantitative results demonstrate that our approach effectively adapts to the different feature frequencies to avoid aliasing while preserving useful information for recognition. Code is available at https://maureenzou.github.io/ddac/.
1 Introduction
Aliasing can make small input shifts produce drastically different sampled signals, while universal blur filters may remove useful detail because frequencies vary across locations and channels. The paper proposes adaptive filtering and evaluates it across recognition tasks.
- Motivation: Aliasing distorts high-frequency signals during subsampling, and a one-position shift can produce a completely different sampled signal.The Nyquist theorem requires a sampling rate at least twice the highest signal frequency to prevent aliasing.
- Motivation: Universal Gaussian filtering is potentially suboptimal because natural-image and feature-map frequencies vary across spatial locations and channels.The paper motivates stronger blur for high-frequency background noise and weaker blur for lower-frequency edges.
- Contribution: The method predicts separate low-pass filters for spatial locations and channel groups, adapting blur strength to feature frequencies while preserving useful content.It is presented as an adaptive, architecture-independent anti-aliasing layer for ConvNets.
- Evaluation: The paper introduces shift-consistency metrics for semantic and instance segmentation to measure robustness to aliasing effects from input shifts.These metrics complement accuracy-based evaluation across the paper’s multiple tasks.
- Evaluation: Across ImageNet, segmentation, and domain-generalization experiments, the method outperforms competitive baselines in both accuracy and shift consistency.The reported evaluations include ImageNet, ImageNet VID, MS-COCO, PASCAL VOC, and Cityscapes.
- Evaluation: Qualitative results show that the module is interpretable across different spatial locations and channel groups.The qualitative contribution is stated alongside the quantitative multi-task evaluation.
2 Related Work
Related work studies robustness to input variation and integrates fixed or content-aware filters into neural networks. This paper focuses anti-aliasing analysis on pixel-level segmentation tasks, where excessive blur can damage object boundaries.
- Network robustness: Prior robustness methods address adversarial attacks, input translation, and natural perturbations because small input variations can cause large prediction changes.Existing approaches include specialized losses, architectures, and shift-invariant representation learning.
- Image filtering: Box and Gaussian filters use fixed weights, whereas bilateral and guided filters adapt to content by preserving edges while removing noise.The paper notes that integrating such filters into neural networks requires input-dependent hyperparameter tuning.
- Pixel classification: Semantic and instance segmentation require precise object boundaries, so excessive or insufficient blur can harm pixel-level predictions through edge loss or aliasing.The paper investigates anti-aliasing in these tasks beyond prior work focused mainly on image classification.
3 Approach
The approach predicts content-aware low-pass filters for spatial locations and channel groups, then applies them before downsampling to adapt blur to feature frequencies. Analyses show spatially varying blur and grouped features capture different frequency content while reducing computational cost.
- 3 Approach: The module predicts filters for each spatial location and feature channel group, then applies them to input features for content-aware anti-aliasing.The predicted filters produce anti-aliased features before downsampling.
- 3 Approach: Spatially adaptive filters blur higher-frequency content more and lower-frequency content less, reducing aliasing while preserving useful information.Filter-weight variance indicates the degree of blur across image locations.
- 3 Approach: Channel grouping assigns one predicted filter to each group, allowing channels with different frequency content to receive different filtering without predicting a filter for every channel.Features within groups are more similar, while different groups capture distinct aspects such as edges or color blobs.
- 3 Approach: A convolutional block generates k × k filter weights for every spatial location and group, with positive weights normalized to sum to one so the filters are low-pass.The output has dimensions n × g × k^2 × h × w, where g is the number of channel groups.
- 3 Approach: The method's channel-group analysis shows groups specialize in different frequencies, with smooth areas represented separately from sharp intensity transitions while saving computation.The visualizations support interpreting the learned filters across spatial locations and feature groups.
4 Experiments
Across classification and segmentation tasks, adaptive anti-aliasing improves accuracy and shift consistency over fixed-filter and non-anti-aliased baselines, while ablations show benefits from spatially and channel-varying filters.
- Image Classification: 1.3 points higher ImageNet top-1 accuracy than ResNet-101 without anti-aliasing and 0.6 points higher than LPF, with additional consistency gains.The method reaches 79.0 versus 77.7 and 78.4 top-1 accuracy, respectively, and improves consistency by +0.2/+0.4 over LPF for 3×3 and 5×5 filters.
- Domain Generalization: On ImageNet VID, the method generalizes better than ResNet-101 and LPF, improving top-1 accuracy by +2.3% and +1.1%, respectively.The paper associates this with representations less sensitive to downsampling and shifts.
- Instance Segmentation: Adaptive anti-aliasing improves MS-COCO mask and box mAP over LPF by +0.4/+0.5 and mAISC by +1.0/+1.0.Compared with the baseline, mAISC increases by +5.1/+4.7 for mask/box predictions, while fixed Gaussian filtering provides smaller mAP gains.
- Semantic Segmentation: Semantic segmentation improves by 1.8 and 1.0 mIOU points on PASCAL VOC and Cityscapes, with mASSC gains of +0.5 and +0.3.These consistency gains occur despite baseline mASSC scores of 95.5/96.0.
- Ablation Studies: Increasing channel groups generally improves top-1 accuracy, but performance saturates beyond 8 groups, while spatially and channel-adaptive filters perform best overall.The saturation is hypothesized to result from overfitting; the filter ablation favors predicting filters that vary across both locations and channel groups.
- Qualitative Results: Qualitative analyses show better-preserved segmentation edges and learned filters that enlarge thin structures to make their contours easier to preserve.Examples include clearer road edges, light-pole delineation, and thicker tree branches produced by spatially varying filter weights.
5 Conclusion
The paper proposes an adaptive content-aware low-pass filtering layer that predicts separate filter weights for each spatial location and channel group, adapting to feature frequencies to avoid aliasing while preserving useful recognition information.
- The proposed layer predicts separate filter weights for each spatial location and channel group, adapting filtering to different feature frequencies.