PaperlayerView exact revision on arXiv

Source-linked AI summary

WaveCNet: Wavelet Integrated CNNs to Suppress Aliasing Effect for Noise-Robust Image Classification

Qiufu Li, Linlin Shen, Sheng Guo, Zhihui Lai

arXiv:2107.13335v1cs.CV

TL;DR

CNN predictions can be disrupted by image noise, and conventional down-sampling can alias frequency components. The paper integrates DWT/IDWT layers into CNNs as WaveCNets, reporting improved classification robustness and COCO detection performance.

  • Problem

    CNN predictions can change under spatial or adversarial noise, while common down-sampling can alias frequency components and damage object structures.

    Source: A. Noise-robustness · p. 2, §V · p. 14

  • Method

    WaveCNets replace common CNN down-sampling with DWT, passing low-frequency components forward and dropping high-frequency components.

    Source: §I · p. 2, C. WaveCNets · p. 6

  • Results

    WaveCNets improve ImageNet classification accuracy, noise and adversarial robustness, and COCO detection performance with Faster R-CNN and RetinaNet backbones.

    Source: §I · p. 2, §V · pp. 14–15

  • Takeaways & Limitations

    The reported gains span image classification robustness and object detection on COCO.

    Source: §V · pp. 14–15

Abstract

from arXiv · show

Though widely used in image classification, convolutional neural networks (CNNs) are prone to noise interruptions, i.e. the CNN output can be drastically changed by small image noise. To improve the noise robustness, we try to integrate CNNs with wavelet by replacing the common down-sampling (max-pooling, strided-convolution, and average pooling) with discrete wavelet transform (DWT). We firstly propose general DWT and inverse DWT (IDWT) layers applicable to various orthogonal and biorthogonal discrete wavelets like Haar, Daubechies, and Cohen, etc., and then design wavelet integrated CNNs (WaveCNets) by integrating DWT into the commonly used CNNs (VGG, ResNets, and DenseNet). During the down-sampling, WaveCNets apply DWT to decompose the feature maps into the low-frequency and high-frequency components. Containing the main information including the basic object structures, the low-frequency component is transmitted into the following layers to generate robust high-level features. The high-frequency components are dropped to remove most of the data noises. The experimental results show that %wavelet accelerates the CNN training, and WaveCNets achieve higher accuracy on ImageNet than various vanilla CNNs. We have also tested the performance of WaveCNets on the noisy version of ImageNet, ImageNet-C and six adversarial attacks, the results suggest that the proposed DWT/IDWT layers could provide better noise-robustness and adversarial robustness. When applying WaveCNets as backbones, the performance of object detectors (i.e., faster R-CNN and RetinaNet) on COCO detection dataset are consistently improved. We believe that suppression of aliasing effect, i.e. separation of low frequency and high frequency information, is the main advantages of our approach. The code of our DWT/IDWT layer and different WaveCNets are available at https://github.com/CVI-SZU/WaveCNet.

I. INTRODUCTION

The introduction links CNN noise vulnerability to down-sampling that aliases frequency components and damages object structures. It proposes DWT-based WaveCNets and reports improved classification robustness and detection performance.

B. Down-sampling

  • Max-pooling, average-pooling, and strided convolution can alias frequency components, leaving residual noise and damaging basic object structures.The paper contrasts this with anti-aliased CNNs, which target shift-invariance using low-pass filters, while WaveCNets target noise robustness using wavelets.Source: §I · pp. 1–2, B. Down-sampling · p. 3

I. INTRODUCTION

  • WaveCNets replace common CNN down-sampling with DWT, carrying low-frequency components forward and dropping high-frequency components.The paper also introduces general DWT/IDWT layers and evaluates the networks on ImageNet and COCO.Source: §I · p. 2
  • WaveCNets improve ImageNet classification accuracy, noise robustness, and adversarial robustness; Faster R-CNN and RetinaNet backbones also improve COCO detection performance.The introduction reports detection improvements for small, medium, and large objects.Source: §I · p. 2

A. Noise-robustness

  • ImageNet-trained CNNs can amplify noise through their layers, and both spatial and adversarial noise can change their predictions.The introduction notes data augmentation and filtering alternatives, but says augmentation can increase training time or reduce clean-image accuracy, while broad-frequency filtering can damage object structures.Source: §I · p. 1, A. Noise-robustness · p. 2

C. Wavelets in deep learning

Earlier wavelet-based networks differ in their learnability, task scope, and evaluation scale. This paper presents general DWT/IDWT layers, grounded in wavelet decomposition and reconstruction, and describes filter families used to implement them.

III. OUR METHOD

  • DWT separates data into components, while IDWT reconstructs the original data; the paper applies these transforms to suppress aliasing in CNNs.The method section presents general DWT/IDWT layers for use in wavelet-integrated networks.Source: C. Wavelets in deep learning · p. 3, §III · p. 3

C. Wavelets in deep learning

  • ScatNet cascades wavelet transforms with pooling and nonlinear operations, but is a hand-designed feature extractor without learnable parameters.The paper also notes that parameterized wavelet networks can be difficult to train because of increased computation.Source: C. Wavelets in deep learning · p. 3
  • Prior wavelet-based deep networks include restoration, SAR-image, and classification methods, but some have limited depth, evaluation, or wavelet coverage.The cited classification networks were not systematically evaluated on ImageNet, and the lack of general transform layers limited many methods to one or two wavelets.Source: C. Wavelets in deep learning · p. 3

A. Wavelet

  • Daubechies wavelets are orthogonal and use approximation order p, with filter length 2p.The table lists low-pass filters; the corresponding high-pass filters are derived from them.Source: Table I · p. 4, A. Wavelet · p. 4
  • Cohen wavelets are symmetric biorthogonal wavelets with separate filters for decomposition and reconstruction.Their two order parameters are p and p̃.Source: Table II · p. 4, A. Wavelet · p. 4

B. Wavelet transform layers

The section develops DWT/IDWT as network layers and describes their forward and backward propagation. In 1D, DWT filters and downsamples data, while IDWT reconstructs it from the low- and high-frequency components.

B. Wavelet transform layers

  • The analysis treats forward and backward propagation for general DWT/IDWT layers, with the discussion extending beyond orthogonal wavelets and 1D/2D data.The text says the analysis can be generalized to other discrete wavelets and 3D data with slight changes.Source: B. Wavelet transform layers · p. 4

1) 1D DWT/IDWT:

  • 1D DWT applies low- and high-pass filters and downsamples data into low- and high-frequency components.The filters are denoted l and h; IDWT reconstructs x using xlow and xhigh.Source: 1) 1D DWT/IDWT · pp. 4–5
  • The 1D layer treatment also derives backward propagation for DWT and IDWT from their forward operations.The text identifies gradients with respect to xlow as relevant for DWT backpropagation and gradients with respect to xlow and xhigh for IDWT.Source: 1) 1D DWT/IDWT · p. 5

2) 2D DWT/IDWT:

The 2D transform applies 1D DWT along rows and columns, yielding components that separate the main information from directional details. The section also describes network-layer implementations and the reduced data size of the DWTll denoising choice.

2) 2D DWT/IDWT:

  • 2D DWT is formed by applying 1D DWT to each row and column, and the 2D IDWT is implemented correspondingly.Source: 2) 2D DWT/IDWT · p. 5
  • Xll is the low-frequency component carrying the main information and basic object structures; Xlh, Xhl, and Xhh retain horizontal, vertical, and diagonal details.Source: 2) 2D DWT/IDWT · p. 5
  • DWTll halves the data size, whereas the general wavelet-denoising approach keeps it unchanged.Source: Figure 2 · p. 5
  • The DWT/IDWT network layers support orthogonal and biorthogonal wavelets, including Haar, Daubechies, and Cohen, and process multi-channel data channel by channel.Source: 2) 2D DWT/IDWT · p. 5

C. WaveCNets

WaveCNets replace common CNN down-sampling with DWTll, retaining low-frequency feature-map information while dropping high-frequency components. This design is intended to suppress aliasing and resist noise propagation.

C. WaveCNets

  • The method uses the simplest wavelet-based denoising choice: decompose with DWT and drop the high-frequency components rather than reconstructing the data.Source: C. WaveCNets · p. 6
  • WaveCNets replace pooling with DWTll and replace strided convolution with stride-1 convolution followed by DWTll in VGG16bn, ResNets, and DenseNet121.Source: C. WaveCNets · p. 6
  • DWTll halves feature-map size while removing high-frequency components and retaining low-frequency information for identifiable features.Source: C. WaveCNets · p. 6
  • The authors describe DWTll as maintaining basic object structures and resisting noise propagation by suppressing aliasing in CNN feature maps.Source: C. WaveCNets · p. 6, Figure 3 · p. 6

D. Computation for DWT/IDWT

Wavelet transforms add computation to WaveCNets without adding learnable parameters. The section estimates the multiply-add overhead associated with the transforms and reports that DWTll is more computationally involved than common down-sampling.

D. Computation for DWT/IDWT

IV. EXPERIMENTS

ImageNet experiments compare WaveCNets with vanilla CNNs across architectures and wavelets. Symmetric wavelets improved accuracy broadly, while feature maps and training curves showed cleaner representations and faster early loss reduction.

A. ImageNet classification

  • 71.62% was the best WResNet18 top-1 accuracy with Cohen “ch2.2”; symmetric wavelets improved all CNNs, while higher-order asymmetric Daubechies could reduce accuracy.For ResNet18, symmetric wavelets improved accuracy by about 1.50%.Source: A. ImageNet classification · pp. 6–7, Table IV · p. 7
  • In the first 10 epochs, WResNet18(Haar) loss fell faster than ResNet18 loss; its final training loss was about 0.08 lower, with the same learnable-parameter count.Its validation loss was also always lower, and classification accuracy increased by 1.71%.Source: A. ImageNet classification · p. 7, Figure 4 · p. 7
  • Across illustrated architectures and images, WaveCNet feature maps had cleaner backgrounds and more complete object structures than the corresponding CNN maps.The examples include a ResNet50 wall clock whose structures remained visible in WaveCNet maps at both 56×56 and 28×28.Source: A. ImageNet classification · pp. 7–8, Figure 5 · p. 8
  • The authors suggest that common down-sampling can break object structures and accumulate noise, while DWT relieves these effects in feature maps.They relate this interpretation to prior observations that CNNs can favor textures over shapes [8] and accuracy gains after low-pass filtering [27].Source: A. ImageNet classification · pp. 7–8

B. Noise-robustness

On ImageNet-C noise corruptions, WaveCNets generally improved noise robustness over the original CNNs, with VGG16bn an exception. Example feature maps show that WaveCNet retained object structure under noise where ResNet18 maps did not.

B. Noise-robustness

  • The evaluation used ImageNet-C noise corruptions with classifiers trained on clean ImageNet images.The noise subset is described as 750K images.Source: B. Noise-robustness · pp. 8–9
  • Under Gaussian, shot, and impulse noise, WResNet18 feature maps retained basic vase and moped structures that often vanished in ResNet18 maps.The examples compare feature maps generated from clean images and their noisy versions.Source: B. Noise-robustness · p. 9
  • VGG16bn gained 1.03% accuracy but showed no significant noise-robustness improvement, limiting the method’s reported robustness benefit for that architecture.The paper describes VGG16bn as potentially unsuitable for noise robustness despite accuracy similar to ResNet34.Source: B. Noise-robustness · p. 9

C. Wavelet denoising block

The section evaluates soft-threshold wavelet denoising of noisy inputs and compares it with DWT integration in CNNs. Denoising lowered mCE across tested networks, while the reported comparison favored DWT integration over preprocessing alone.

C. Wavelet denoising block

  • The denoising block applies soft thresholding to DWT high-frequency components; the experiments use one DWT/IDWT level and set λ to 0.1.The tested high-frequency components are Xlh, Xhl, and Xhh.Source: C. Wavelet denoising block · p. 9
  • 81.31 to 79.45 mCE for vanilla ResNet50 and 70.46 to 68.50 for Haar WResNet101; denoising lowered mCE across networks, and Cohen(2,2) WResNet101 reached 64.01.The table reports lower mCE as better.Source: C. Wavelet denoising block · pp. 9–10, Table V · p. 10
  • WResNet18(ch2.2) on noisy input had mCE 80.38, lower than ResNet18’s 87.77 on denoised input, supporting a stronger reported improvement from DWT integration.The comparison is between the original noisy input for WResNet18 and denoised input for ResNet18.Source: C. Wavelet denoising block · p. 10
  • The compared wavelet down-sampling modes that add high-frequency components to the low-frequency component may damage useful information; WResNet18 using DWTll performed best among compared approaches.The comparison reports generally better accuracy and noise mCE for wavelet-based down-sampling.Source: Table VI · p. 11, Figure 8 · p. 11

D. Comparison with other wavelet based down-sampling

Wavelet down-sampling generally improves ResNet18 classification accuracy and ImageNet-C noise mCE over the original network. On adversarial samples, WaveCNets improve accuracy over clean-trained CNNs, though they remain below specially trained defenses.

D. Comparison with other wavelet based down-sampling

  • Wavelet-based down-sampling generally improves ImageNet accuracy and ImageNet-C noise mCE over original ResNet18 (69.76% accuracy and 88.97 mCE).The comparison includes modes that average or concatenate DWT components alongside the proposed mode.Source: D. Comparison with other wavelet based… · p. 11
  • WResNet18 A performs worst among the wavelet down-sampling networks; adding high-frequency components damages low-frequency information in the presence of noise.Its parameter count matches original ResNet18.Source: D. Comparison with other wavelet based… · p. 11
  • WResNet18 C uses 21.62 × 10^6 parameters and raises WResNet18 accuracy by 0.11% to 0.77%, but does not evidently improve noise-robustness.Its parameter count is nearly WResNet34's 21.80 × 10^6, while WResNet34 is clearly better in accuracy and noise mCE.Source: D. Comparison with other wavelet based… · p. 11

E. Adversarial robustness

  • On PGD samples, WResNet101 with ch2.2 is about 4.35% higher in accuracy than ResNet101 (53.94%).The classifiers are trained only on clean ImageNet training data; wavelet variants generally achieve about 3% higher accuracy for this attack.Source: E. Adversarial robustness · p. 11, Table VII · p. 12
  • WaveCNets resist the tested attackers consistently, but perform below specially trained defense methods.The authors contrast this with WaveCNets' high-frequency denoising without adversarial training.Source: E. Adversarial robustness · p. 11

F. Object detection

With 12-epoch training, wavelet-integrated COCO detectors outperform original detectors and match or exceed 24-epoch baselines. The gains include small, medium, and large objects, with a modest detection-speed cost from DWT.

F. Object detection

  • With 12-epoch training, wavelet integration improves detection of small, medium, and large objects; for Faster R-CNN/ResNet101 on test-dev, gains are 1.1%, 1.7%, and 1.2%, respectively.By contrast, 24-epoch baseline training raises APM and APL but lowers APS by 0.7%.Source: F. Object detection · p. 12, Table VIII · p. 13
  • On test-dev, Faster R-CNN with ResNet101 AP rises from 39.8% to 41.2% with ch3.3 after 12 epochs, exceeding the 40.5% 24-epoch baseline.The improvement is 1.4 percentage points; the table reports detector results on COCO.Source: F. Object detection · p. 12, Table VIII · p. 13
  • In four examples, wavelet-integrated detectors correctly detect target objects that 12-epoch baseline detectors falsely detect or miss, including small objects.The comparison also includes original detectors trained for 24 epochs.Source: F. Object detection · pp. 12–13
  • DWT requires more multiply-add operations and lowers Faster R-CNN speed from 18.1 to 16.3 FPS when Haar is added to ResNet50 on an Nvidia Tesla V100.The experiments replace ResNet50/ResNet101 detector backbones with WResNet50/WResNet101 and apply DWT during down-sampling.Source: F. Object detection · pp. 12–13

G. 3D object classification

On ModelNet40, 3D WaveCNets improve classification with symmetric Haar and Cohen wavelets, while Daubechies improvements are not guaranteed. The paper concludes that WaveCNets also improve image classification robustness and COCO detection, and identifies image-to-image and 3D medical imaging as future applications.

G. 3D object classification

  • The experiments convert ModelNet40 point clouds to 32 × 32 × 32 tensors and apply 3D DWT to down-sample 3D feature maps in 3D WVGG9 and WResNet16.The 3D networks are trained from scratch and compared with 3D VGG9 and 3D ResNet16.Source: G. 3D object classification · p. 13, Figure 10 · p. 15
  • On ModelNet40, Cohen ch5.5 lifts 3D VGG9 accuracy from 87.08% to 88.25%, and ch2.2 lifts 3D ResNet16 from 84.32% to 85.90%; symmetric Haar/Cohen gains are consistent, unlike Daubechies.The table summarizes ModelNet40 classification results for 3D WaveCNets.Source: G. 3D object classification · pp. 13–14, Table IX · p. 14

V. CONCLUSIONS

  • The conclusion attributes weak CNN noise-robustness to aliasing that accumulates noise and breaks object structures, and reports WaveCNet gains in ImageNet accuracy, noise-robustness, adversarial robustness, and COCO detection.It names Faster R-CNN and RetinaNet as the improved COCO detectors.Source: §V · pp. 14–15
  • Future work will study DWT/IDWT in image-to-image tasks and 3D DWT/IDWT networks for 3D medical image processing.Source: §V · p. 15
Loading 2107.13335v1…