Source-linked AI summary
Generic Deep Networks with Wavelet Scattering
Edouard Oyallon, Stéphane Mallat, Laurent Sifre
TL;DR
The paper examines whether fixed scattering transforms can capture useful image properties for classification of complex objects with structural variability and clutter. It uses two cascaded wavelet layers, with spatial filtering first and joint spatial, angular, and scale filtering in the second. On Caltech databases, the authors report accuracy similar to double-layer neural networks pretrained on ImageNet.
Problem
The paper asks whether scattering can handle Caltech object datasets with complex structural variability and clutter, beyond earlier applications to digit recognition and texture discrimination.
Source: §1 · p. 1Method
Results
The authors report similar Caltech accuracy for a two-layer scattering network and double-layer neural networks pretrained on ImageNet.
Source: §4 · p. 3Takeaways & Limitations
The experiments indicate that scattering can help explain the first two convolution-network layers and provide an efficient initialization for those layers.
Source: §4 · p. 3The experiments are preliminary, and further improvements may come from optimizing scaling invariance.
Source: §4 · p. 3
Abstract
from arXiv · showhide
We introduce a two-layer wavelet scattering network, for object classification. This scattering transform computes a spatial wavelet transform on the first layer and a new joint wavelet transform along spatial, angular and scale variables in the second layer. Numerical experiments demonstrate that this two layer convolution network, which involves no learning and no max pooling, performs efficiently on complex image data sets such as CalTech, with structural objects variability and clutter. It opens the possibility to simplify deep neural network learning by initializing the first layers with wavelet filters.
1. Introduction
The paper asks whether scattering transforms can capture generic image properties associated with deep networks’ early layers. It applies a two-layer transform to complex CalTech object datasets and distinguishes itself through average pooling, no inner renormalization, and a focus on the first two layers.
1. Introduction
- Scattering transforms are proposed as a way to capture generic image properties and build affine-transform invariants through cascaded wavelet convolutions and modulus nonlinearities.Prior work had applied scattering to digit recognition and texture discrimination; this paper targets more complex object variability and clutter.Source: §1 · p. 1
- The CalTech-101 and CalTech-256 experiments adapt scattering to object variability with spatial, rotation, and scale wavelets, separable convolutions, and independently processed YUV channels.The datasets feature more complex structural variability and clutter than the earlier digit and texture settings described in the paper.Source: §1 · p. 1
- Unlike most deep networks, the method uses average pooling without ad-hoc inner renormalization, and the authors report that average pooling does not harm results in cluttered environments.The paper says average pooling guarantees mathematical stability of the representation.Source: §1 · p. 1
- The study concentrates on two layers because a third layer did not reduce classification errors; improving later layers may require learning their filters.The first two layers address translation, rotation, and scaling variability.Source: §1 · p. 1
2. Scattering along Translations, Rotations and Scales
The scattering representation cascades spatial wavelet transforms and modulus operations, then averages coefficients to produce orders zero through two. Its second transform acts jointly across spatial, rotation, and scale variables.
2. Scattering along Translations, Rotations and Scales
- The first transform filters the image with a low-pass filter and scaled, rotated complex wavelets; taking coefficient moduli defines the first layer.The mother wavelet is a well-localized complex Morlet function, and the averaging filter is Gaussian.Source: §2 · pp. 1–2
- Figure 1 depicts successive modulus operations on wavelet coefficients followed by average pooling.The depicted sequence uses the first- and second-layer wavelet transforms before the final averaging.Source: §2 · p. 2, Figure 1
- The first transform computes an averaged image and modulus wavelet coefficients, with spatial subsampling and an aggregated index over position, angle, and scale.The representation indexes first-layer coefficients by u1 = (u, w1).Source: §2 · p. 2
- The second wavelet transform convolves first-layer coefficients with separable wavelets along spatial, rotation, and scale variables.Its wavelet family is chosen as a tight frame, yielding an invertible linear operator that preserves the norm.Source: §2 · p. 2
- Average pooling produces first- and second-order scattering coefficients, and the representation concatenates orders zero, one, and two.The second-order coefficients are obtained by low-pass filtering the second-layer coefficients; higher orders are not computed.Source: §2 · p. 2
- The network uses no nonlinear inner renormalization; at classification, output coefficients are standardized by setting their variance to 1.The first- and second-order scattering coefficients are each formed by low-pass filtering their corresponding layer coefficients.Source: §2 · p. 2
3. Numerical Classification Results
The experiments compare scattering with convolutional networks on Caltech-101 and Caltech-256, using standardized coefficients and a linear SVM. Scattering has similar performance to ConvNets restricted to one or two layers, while pooling comparisons vary with the wavelets used.
3. Numerical Classification Results
- The evaluation resizes images to 128 by 128 pixels, processes YUV channels separately, and uses spatial pooling at scale 2J = 32 before a linear SVM.Scattering coefficients are standardized using means and variances computed on the training databases.Source: §3 · p. 2
- 85.5% accuracy on Caltech-101 and 72.6% on Caltech-256 are reported for a 7-layer ImageNet-pretrained ConvNet, versus 66.2% and 39.6% for its 2-layer version.The respective comparisons use 30 and 60 training images per class.Source: §3 · p. 2
- Scattering has similar performance to ConvNets restricted to 1 and 2 layers on Caltech-101 and Caltech-256.The table covers classification accuracies by layer count, with 30 training images per class for Caltech-101 and 60 for Caltech-256.Source: Table 1 · p. 3, §3 · p. 3
- 73.4% on Caltech-101 and 47.7% on Caltech-256 are reported for LLC, which outperforms the fixed architecture in the comparison through unsupervised data-set-adapted optimization.LLC uses a first layer of SIFT features and an unsupervised dictionary optimization in its second layer.Source: §3 · p. 3
- With complex wavelets, max pooling reaches 60.5% without overlap and 62.0% with overlap on Caltech-101, below the paper’s average-pooling results; real Haar wavelets give 66.9% max versus 67.0% average pooling.The authors say the differing behaviors with real and complex wavelets are not well understood.Source: §3 · p. 3
4. Conclusion
A two-layer scattering network without learning achieves Caltech accuracy similar to double-layer networks pretrained on ImageNet. The authors see wavelet scattering as a way to understand and efficiently initialize the first two convolutional layers, while noting that scaling invariance could be optimized further.
4. Conclusion
- The two-layer scattering network, which involves no learning, provides similar accuracy on the Caltech databases to double-layer neural networks pretrained on ImageNet.The transform linearizes variability relative to translations and rotations and provides translation invariance with average pooling.Source: §4 · p. 3
- Wavelet scattering transforms provide an efficient initialization for the first two convolutional layers and an approach to understanding them for complex image classification.The authors describe these conclusions as indicated by preliminary experiments.Source: §4 · p. 3
- Further improvements to the first scattering layers remain possible, particularly by optimizing scaling invariance.Source: §4 · p. 3