Source-linked AI summary
Invariant Scattering Convolution Networks
Joan Bruna, Stéphane Mallat
TL;DR
Image classification needs representations that suppress uninformative variability while remaining stable to deformations, and Fourier-modulus invariants do not provide that stability. The paper develops wavelet scattering networks that cascade wavelet transforms with modulus and averaging, producing complementary invariant information and state-of-the-art results on handwritten digits and textures. It also shows that scattering captures higher-order moments beyond the Fourier power spectrum.
Problem
Image classification requires representations invariant to rigid transformations but continuous to deformations, while Fourier-modulus invariants are unstable at high frequencies.
Method
The paper constructs translation-invariant, deformation-stable representations by cascading predefined wavelet transforms with modulus operators and average pooling.
Results
State-of-the-art classification results were obtained for handwritten digit recognition and texture discrimination using SVM and PCA classifiers.
Takeaways & Limitations
Additional scattering layers provide complementary invariant information beyond first-layer SIFT-type descriptors, while higher-order moments distinguish textures with identical Fourier power spectra.
Takeaways & Limitations
Complex variability not approximated by finite-group actions requires unsupervised optimization of representations from training data.
Abstract
from arXiv · showhide
A wavelet scattering network computes a translation invariant image representation, which is stable to deformations and preserves high frequency information for classification. It cascades wavelet transform convolutions with non-linear modulus and averaging operators. The first network layer outputs SIFT-type descriptors whereas the next layers provide complementary invariant information which improves classification. The mathematical analysis of wavelet scattering networks explains important properties of deep convolution networks for classification. A scattering representation of stationary processes incorporates higher order moments and can thus discriminate textures having the same Fourier power spectrum. State of the art classification results are obtained for handwritten digits and texture discrimination, using a Gaussian kernel SVM and a generative PCA classifier.
1 INTRODUCTION
Image classification requires representations that remove uninformative rigid variability while remaining stable to non-rigid deformations. Wavelet scattering addresses this through cascaded wavelet convolutions and modulus-based averaging, yielding complementary invariant information for classification and texture discrimination.
- Rigid translations, rotations, and scaling create within-class variability that should be eliminated from image representations.
- Full deformation invariance can reduce discrimination, so representations should instead remain continuous under small deformations.
- Fourier-modulus invariants are unstable to deformations, motivating localized wavelets and nonlinear operators for translation-invariant representations.
- Wavelet scattering cascades wavelet transforms, modulus operators, and average pooling to build deformation-stable translation-invariant representations.The first layer produces SIFT- or Daisy-type descriptors, while later layers recover information lost through local averaging.
- Scattering representations capture higher-order moments and can discriminate non-Gaussian textures sharing the same Fourier power spectrum.
- State-of-the-art results were obtained for handwritten digit recognition and texture discrimination using Gaussian-kernel SVM and PCA-based generative classifiers.
2 TOWARDS A CONVOLUTION NETWORK
The paper develops translation-invariant representations by combining localized wavelets with nonlinear modulus and averaging, while retaining deformation stability. Iterated wavelet-modulus paths recover information lost by averaging and provide higher-order scattering coefficients.
- 2.1 Fourier and Registration Invariants: Fourier-modulus and registration invariants are translation invariant but unstable to small deformations, especially at high frequencies.The Fourier modulus can also fail to discriminate very different signals with identical Fourier-modulus values.
- 2.2 Scattering Wavelets: Wavelets provide localized, deformation-stable filters, but their transform is translation covariant rather than invariant.A nonlinear operator is therefore introduced before averaging to construct nontrivial translation-invariant coefficients.
- 2.3 Scattering Convolution Network: Averaging removes nonzero-frequency information, while subsequent wavelet-modulus layers recover those frequencies and produce complementary coefficients.The first layer is related to SIFT- and Daisy-type descriptors, whereas deeper layers add higher-order information.
- 2.2 Scattering Wavelets: The scattering transform iterates wavelet convolutions and modulus operators along paths, then integrates the resulting coefficients.Its coefficients are translation invariant and Lipschitz continuous to deformations.
- 2.3 Scattering Convolution Network: Scattering networks use predefined wavelets and output coefficients at all layers rather than learning filters or retaining only the deepest layer.The wavelet family is multiscale and directional, with filters covering the relevant image-frequency plane.
- 2.3 Scattering Convolution Network: First-order scattering coefficients can coincide for very different images, whereas second-order coefficients discriminate them.The frequency subsets are organized as rotated quadrants at order one and subdivided partitions at order two.
3 SCATTERING PROPERTIES
The scattering transform is a contractive, norm-preserving representation under suitable wavelet conditions, with energy concentrated in shorter frequency-decreasing paths. This concentration explains why network depth can be limited with negligible signal-energy loss.
- The scattering transform is computed by cascading wavelet modulus operators, and its properties depend on the underlying wavelet transform.The analysis gives wavelet conditions under which the transform is contractive and preserves signal norm.
- Under suitable wavelet conditions, scattering energy decreases quickly with path length and remains significant mainly on frequency-decreasing paths.Restricting computation to these paths defines the convolutional network architecture.
- The modulus and wavelet transform are contractive, so their composition is contractive; when the wavelet transform is unitary, the modulus stage preserves signal norm.The full scattering transform inherits contractivity through iterative application of the contractive operator.
- As maximum path length increases, the energy in the last network layer converges to zero.This result underlies the rapid decay of higher-order scattering energy.
- The network depth can therefore be limited with negligible loss of signal energy.This conclusion follows from the vanishing energy of sufficiently high-order scattering terms.
- For a first-order scattering coefficient, the large-scale limit relates its squared norm to the squared norm of the corresponding wavelet coefficient.Specifically, the scaled limit 2^2J∥SJ[λ]x∥2 equals ∥φ∥2∥x⋆ψλ∥2.
1. The more sparse x⋆ψλ(u) the smaller ∥x⋆ψλ∥2
Scattering energy propagates toward lower frequencies along frequency-decreasing paths, while higher-order coefficients capture structure beyond second-order statistics. The representation can therefore compress efficiently while retaining information useful for discriminating textures.
- Higher-order scattering energy is below 1% for path lengths m ≥3 on typical Caltech101 images.The energy decays exponentially with path length because scattering energy converges to zero as m increases.
- The modulus demodulates wavelet coefficients, sending scattering energy along frequency-decreasing paths while other paths have negligible energy.For λ = 2^j r, a subsequent wavelet is non-negligible only at lower frequency, with 2^j′ < 2^j.
- Scattering coefficients of path length m depend on moments up to order 2m, allowing textures with identical power spectra to be distinguished.Power spectra encode second-order moments, whereas higher-order scattering coefficients capture additional moment information.
- For two textures with the same power spectrum, first-order scattering gives ρ(1) = 1, whereas second-order scattering gives ρ(2) = 5.The first-order coefficients are nearly equal, while order-2 coefficients are highly dissimilar because they depend on moments up to order 4.
- For Brodatz textures, most expected scattering energy is concentrated in paths of length m ≤3.
- A separable DCT across scattering scales and orientations nearly decorrelates coefficients, with the lowest-frequency one-third absorbing more than 99% of scattering energy on natural images.The cosine transform further compresses the reduced representation produced by convolution, modulus, and subsampling with O(N log N) operations.
4 CLASSIFICATION USING SCATTERING VEC-
Scattering representations combine translation invariance and deformation stability with PCA or SVM classifiers for handwritten digits and textures. Higher-order coefficients improve discrimination, while classification performance depends on training size, scale, and scattering order.
- Scattering representations eliminate translation variability and remain stable to deformations, supporting classification with PCA and SVM classifiers.
- 4.1 PCA Affine Scattering Space Selection: The PCA classifier represents each class with an affine space built from its mean scattering vector and leading covariance eigenvectors, then selects the smallest approximation error.
- Second-order scattering coefficients reduce classification errors by about 30%, whereas third-order coefficients add only marginal improvements while substantially increasing computation.
- 4.2 Handwritten Digit Recognition: 0.43% error is obtained on the original MNIST dataset with the scattering SVM when larger training sets make its flexibility advantageous over the affine classifier.
- 4.2 Handwritten Digit Recognition: 2.3% error is obtained on USPS with standard-deviation equalization, compared with 2.6% using supremum normalization.
- 4.3 Texture Discrimination: 0.5% texture error with first-order coefficients falls to 0.2% with second-order coefficients, which incorporate fourth-order moments beyond the Fourier power spectrum.
5 CONCLUSION
The conclusion presents wavelet scattering as a computationally manageable representation that combines translation invariance with deformation stability. It reports state-of-the-art results for handwritten digits and textures, while identifying more complex variability as a boundary requiring learned representations.
- A reduced cosine scattering transform is at most three times larger than a SIFT descriptor and requires O(N log N) operations.
- State-of-the-art results are reported for handwritten digit recognition and texture discrimination using SVM or PCA classifiers.
- Variability not approximated by finite-group actions, such as in CalTech101 or Pascal, requires unsupervised optimization of representations from training data.