Source-linked AI summary
Wavelet Integrated CNNs for Noise-Robust Image Classification
Qiufu Li, Linlin Shen, Sheng Guo, Zhihui Lai
TL;DR
CNN image-classification outputs can change drastically under small noise variations. The paper integrates DWT and IDWT into CNNs, replacing standard down-sampling while discarding high-frequency components. WaveCNets improve classification accuracy and noise-robustness across evaluated CNN architectures and datasets, with limitations for some architectures and an analysis scoped to orthogonal wavelets and 1D signals.
Problem
Small image variations can cause drastic CNN output changes, motivating methods that improve CNN noise-robustness.
Method
WaveCNets replace max-pooling, average-pooling, and strided-convolution down-sampling with DWT-based layers that retain low-frequency components and drop high-frequency components.
Results
WaveCNets achieve higher classification accuracy and better noise-robustness than original CNN architectures across the reported evaluations.
Takeaways & Limitations
Wavelet-based down-sampling preserves basic object structures while suppressing data noise during inference.
Takeaways & Limitations
Noise-robustness does not significantly improve for VGG16bn, and the layer analysis is presented for orthogonal wavelets and 1D signals.
Abstract
from arXiv · showhide
Convolutional Neural Networks (CNNs) are generally prone to noise interruptions, i.e., small image noise can cause drastic changes in the output. To suppress the noise effect to the final predication, we enhance CNNs by replacing max-pooling, strided-convolution, and average-pooling with Discrete Wavelet Transform (DWT). We present general DWT and Inverse DWT (IDWT) layers applicable to various wavelets like Haar, Daubechies, and Cohen, etc., and design wavelet integrated CNNs (WaveCNets) using these layers for image classification. In WaveCNets, feature maps are decomposed into the low-frequency and high-frequency components during the down-sampling. The low-frequency component stores main information including the basic object structures, which is transmitted into the subsequent layers to extract robust high-level features. The high-frequency components, containing most of the data noise, are dropped during inference to improve the noise-robustness of the WaveCNets. Our experimental results on ImageNet and ImageNet-C (the noisy version of ImageNet) show that WaveCNets, the wavelet integrated versions of VGG, ResNets, and DenseNet, achieve higher accuracy and better noise-robustness than their vanilla versions.
1. Introduction
CNN outputs can change drastically under small input variations, motivating wavelet-based integration to suppress noise and improve classification robustness. WaveCNets replace conventional down-sampling with DWT and retain low-frequency feature information while removing high-frequency components.
- Small input variations can cause drastic changes in CNN image-classification outputs, reflecting weak noise-robustness.
- WaveCNets integrate DWT and IDWT as general network layers and replace conventional CNN down-sampling with DWT.
- During down-sampling, WaveCNets discard high-frequency feature components and extract high-level features from the retained low-frequency component.
- The study evaluates WaveCNets with various wavelets and CNN architectures on ImageNet and ImageNet-C for accuracy and noise-robustness.
- WaveCNets achieve increased classification accuracy and better noise-robustness on ImageNet than original CNN architectures.
2. Related works
Prior work identifies weaknesses in CNN sampling and feature extraction, including noise sensitivity, structure loss, and imperfect detail recovery. Wavelets have been used in preprocessing and neural networks, but existing approaches differ in architecture and computational cost.
- CNN noise can amplify through layers and contribute to wrong predictions, partly because down-sampling operations may ignore the classic sampling theorem.
- Max-pooling, average-pooling, and strided-convolution remain widely used despite potentially erasing or diluting image details.
- Anti-aliased CNNs add filtering before down-sampling, but retain the Max operation and differ from WaveCNets in their low-pass filters.
- Up-sampling operations such as transposed convolution and max-unpooling cannot precisely recover original data and perform poorly for restoring image details.
- Wavelets decompose data across frequency intervals, and IDWT can reconstruct data from wavelet outputs.
- Earlier wavelet-neural-network approaches include shallow wavelet-search networks, computationally costly deeper classifiers, and wavelet-based image-restoration networks.
3. Our method
The method converts DWT and IDWT into network layers and uses wavelet decomposition for denoising and down-sampling. WaveCNets retain low-frequency feature maps, discard high-frequency components, and replace standard pooling or strided-convolution operations.
- DWT and IDWT layers: The DWT/IDWT layer analysis is presented for orthogonal wavelets and 1D signals, with generalization to other wavelets and 2D/3D signals described as requiring slight changes.
- DWT and IDWT layers: DWT decomposes signals using low-pass and high-pass filters followed by down-sampling, while IDWT reconstructs the original signal from low- and high-frequency components.
- DWT and IDWT layers: For 2D data, DWT applies one-dimensional transforms across rows and columns, with corresponding inverse operations for reconstruction.
- DWT and IDWT layers: The layers implement DWT and IDWT channel by channel for multi-channel data in PyTorch.
- WaveCNets: Wavelet denoising decomposes noisy data, filters high-frequency components, and reconstructs the data with IDWT.
- WaveCNets: WaveCNets use DWTll as the simplest denoising operation by dropping high-frequency components during down-sampling.
- WaveCNets: Max-pooling and average-pooling are replaced directly by DWTll, while strided-convolution becomes stride-1 convolution followed by DWTll.
- WaveCNets: DWTll halves feature-map size while preserving main information in the low-frequency component and helping maintain basic object structure.
4. Experiments
WaveCNets replace conventional CNN down-sampling with wavelet-based operations and are evaluated on ImageNet and ImageNet-C across architectures and wavelets. Results show improved clean-image accuracy, noise robustness, feature-map quality, and segmentation performance, with important variation across wavelets and architectures.
- ImageNet classification: WaveCNets replace max-pooling, average-pooling, and strided-convolution with DWT without adding learnable parameters.The evaluated architectures include WVGG16bn, WResNets, and WDenseNet121.
- ImageNet classification: Haar and Cohen wavelets improve classification accuracy across CNN architectures, while longer asymmetric Daubechies filters can reduce accuracy.For WResNet18, top-1 accuracy decreases from 71.48% to 68.74% with the cited longer-filter configuration.
- ImageNet classification: 1.71% higher final classification accuracy accompanies consistently lower validation loss for WResNet18(Haar) than ResNet18.With the same number of learnable parameters, WResNet18(Haar) also has training loss about 0.08 lower throughout training.
- ImageNet classification: WaveCNet feature maps have cleaner backgrounds and more complete object structures than CNN feature maps under strong noise.The examples show WaveCNets preserving clock structures that become fuzzy or broken in corresponding ResNet feature maps.
- Noise-robustness: 80.38 noise mCE for WResNet18(ch2.2), down from the ResNet18 baseline of 88.97, demonstrates improved noise robustness; VGG16bn is an exception.Across ResNet18, ResNet34, and ResNet50, all tested wavelets improve noise robustness, even when clean-image accuracy decreases for some wavelets.
- Image segmentation: 64.23% mIoU is achieved by the Haar WaveUNet on CamVid using only 367 training images.WaveUNets obtain higher mIoU than the comparably trained SegNet models, and visual examples show better preservation and restoration of object details.
5. Conclusions
WaveCNets convert DWT and IDWT into general network layers and use them to build wavelet-integrated CNNs for image classification, preserving object structures while suppressing noise during inference.
- WaveCNets transform DWT and IDWT into general network layers for image classification.
A. Wavelets
The paper supports DWT/IDWT layers for orthogonal and biorthogonal wavelets, while framing down-sampling as a source of structural damage and noise accumulation that wavelets can mitigate.
- Wavelet families: Daubechies wavelets are orthogonal, with filter lengths determined by approximation order.
- Wavelet families: Cohen wavelets are symmetric biorthogonal wavelets using analysis filters for decomposition and dual filters for reconstruction.
- General layers: The DWT and IDWT layers apply to discrete orthogonal or biorthogonal wavelets and can be modified for other wavelet tools.
- Aliasing effects: Conventional deep-network down-sampling can break object structures and accumulate noise through aliasing effects.
- Wavelet integration: WaveCNets apply DWT to maintain basic object structures and resist noise propagation in feature maps.
C. The detailed results on ImageNet-C
ImageNet-C evaluates WaveCNets on common corruptions using corruption-error metrics, with detailed CE and mCE results reported across six WaveCNet architectures.
- Evaluation protocol: ImageNet-C contains ImageNet validation images corrupted by 15 visual corruptions at five severity levels.
D. Shift-invariance of WaveCNets
The paper evaluates shift-invariance by comparing whether WaveCNets and original CNNs produce the same prediction for an image under two different shifts.
- Evaluation criterion: The shift-invariance criterion checks how often a classifier gives the same prediction for one image under two different shifts.
- Evaluation: WaveCNets and original CNNs are evaluated using an expectation-based shift-invariance criterion, with results presented in Table 12.
E. The architectures of SegNet and WaveUNets
SegNet and WaveUNets use matched encoder-decoder architectures based on VGG16bn, with corresponding layers aligned by channel counts. Their configurations and ImageNet-C corruption-error results are documented in the associated tables.
- Architecture: SegNet and WaveUNets adopt an encoder-decoder architecture based on the first 13 convolutional layers of VGG16bn.The encoder contains 13 convolutional layers, and the decoder contains the same number.
- Architecture: Corresponding encoder and decoder convolutional layers use the same channel counts, except at the network’s first and last layers.
- Architecture: Every listed convolutional layer includes Batch Normalization and ReLU, with table columns specifying encoder input channels and decoder output channels.
F. The amount of multiply-adds in 2D DWT/IDWT
The paper analyzes the computational cost of replacing CNN down-sampling operations with DWT and IDWT layers. It reports multiply-add counts and wavelet-operation ratios for WaveCNets and WaveUNets, alongside related architecture and robustness tables.
- DWT/IDWT cost: For a 2D tensor of size M × N with channel count C, the paper defines the multiply-add cost of 2D DWT inference.
- Related evaluations: Additional supplied figures and tables cover feature maps, blur, weather, and digital mCE, shift-invariance, deep-network configurations, and encoder-decoder architectures.
- Operation ratios: Table 14 reports wavelet-related multiply-adds as ratios of total operations for WaveCNets and WaveUNets with 3×224×224 input.For WaveCNets, only DWTll multiply-adds are counted.