Source-linked AI summary
Classification with Scattering Operators
Joan Bruna, Stéphane Mallat
TL;DR
The paper addresses image classification with descriptors that retain multiscale and multidirectional structure while remaining locally translation invariant and deformation-stable. It constructs scattering vectors from cascaded wavelet and modulus operations, then classifies them through PCA-based affine model selection. The approach obtains state-of-the-art results on handwritten digit recognition and texture discrimination, including small-training-set settings.
Problem
Averaging-based invariant descriptors reduce variability and provide translation invariance but lose information needed to characterize complex image structures.
Method
The method computes scattering vectors with cascaded wavelet transforms, modulus operators, and averaging, then selects PCA affine models for classification.
Results
State-of-the-art results are obtained for handwritten digit recognition and texture discrimination, including settings with small training sets and substantial rotation or illumination variability.
Takeaways & Limitations
Scattering combines local translation invariance and deformation linearization with co-occurrence coefficients that characterize texture and complex structures.
Abstract
from arXiv · showhide
A scattering vector is a local descriptor including multiscale and multi-direction co-occurrence information. It is computed with a cascade of wavelet decompositions and complex modulus. This scattering representation is locally translation invariant and linearizes deformations. A supervised classification algorithm is computed with a PCA model selection on scattering vectors. State of the art results are obtained for handwritten digit recognition and texture classification.
1. Introduction
Scattering operators address information loss from averaging-based invariant descriptors by recovering multiscale, multidirectional co-occurrences. PCA model selection on scattering vectors achieves state-of-the-art results for handwritten digits and texture discrimination under substantial variability and small training sets.
- Averaging wavelet coefficient amplitudes improves local translation invariance and reduces variability but also removes information.Scattering operators are motivated as a way to recover information lost by this averaging.
- Scattering operators recover high frequencies as multiscale, multidirectional co-occurrence coefficients, enriching descriptors of corners, junctions, and textures.The representation remains locally translation invariant and linearizes small deformations.
- PCA selects an affine approximation model for each class's scattering transform to classify images.The method uses scattering vectors as inputs to supervised model selection.
- State-of-the-art results are reported for handwritten digit recognition and texture discrimination with rotation and illumination variability and small training sets.
2. Scattering
Scattering represents images through cascaded wavelet convolutions, modulus operations, and averaging, producing locally translation-invariant descriptors with multiscale co-occurrence information. The construction is contractive, stable to small deformations, computationally efficient, and recovers information lost by first-order averaging.
- 2.2. Scattering Coefficients: The representation restores high frequencies removed by averaging while preserving local translation invariance through modulus and low-pass averaging.Successive wavelet convolutions recover finer-scale information before it is averaged again.
- 2.2. Scattering Coefficients: Scattering coefficients encode co-occurrences across wavelet scales and orientations, distinguishing corners and junctions from edges and characterizing texture structure.Only pairs with 2j2 < 2j1 are retained because higher-order scale interactions are negligible.
- 2.2. Scattering Coefficients: A scattering vector is built by iterating wavelet convolutions, complex modulus operators, low-pass averaging, and subsampling across layers.The cascade resembles a convolutional network and includes coefficients through a maximum order m.
- 2.2. Scattering Coefficients: With FFT computation, the overall scattering representation has complexity O(N log N).
- 2.3. Scattering Distance and Deformation Stability: Order-2 coefficients contain about 20% of the energy of all order-1 coefficients and have an important impact on classification results.
- 2.3. Scattering Distance and Deformation Stability: The scattering distance has a translation error term 2^-J|τ|∞ and deformation terms dominated by |∇τ|∞, becoming essentially proportional to deformation amplitude under the stated scale condition.The condition is 2J ≥ |τ|∞/|∇τ|∞.
3. Classification by Affine Model Selection
The classifier represents each class with a PCA-derived affine model of scattering vectors and assigns images to the best-fitting model. Across digit and texture benchmarks, this approach achieves strong results, including a 0.09% CUReT error rate and improvements over cited baselines.
- 3. Classification by Affine Model Selection: The classifier models each class with an affine space around its scattering-vector centroid, using the leading PCA variability directions.The model is Ad,i = E{SJFi} + Vd,i, where d is selected to approximate within-class scattering variability.
- 3. Classification by Affine Model Selection: Cross-validation jointly selects the affine-space dimension d and scattering scale 2J using held-out training images.Classification minimizes approximation error while excluding the first d principal variability directions from the residual.
- 3.1. Handwritten digit recognition: On MNIST, second-order scattering improves first-order results, while third-order coefficients provide only marginal additional gains; PCA outperforms the scattering SVM up to 60000 samples.Below 5 103 training samples, the scattering PCA classifier also improves on deep-learning convolutional networks.
- 3.1. Handwritten digit recognition: Scattering PCA yields smaller error rates than a convolutional network on rotated MNIST and is designed to handle deformation, rotation, scaling, and illumination variation.For rotated MNIST, increasing d incorporates rotation invariance by removing principal variability directions associated with rotations.
- 3.2. Scattering Texture Classification: For Brodatz textures with matched power spectra, first-order scattering distances vanish up to estimation error, while higher orders remain discriminative.This demonstrates that orders q > 1 capture texture distinctions not determined by the power spectrum alone.
- 3.2. Scattering Texture Classification: 0.09% error on CUReT is a factor-25 improvement over the 2.46% state-of-the-art Markov Random Field result.Results are averaged over 10 random splits, using either 23 or 46 training images per class.
4. Conclusion
Scattering representations provide locally translation-invariant, deformation-linearizing descriptors with texture-characterizing co-occurrence coefficients. PCA model selection on these representations achieves state-of-the-art results for handwritten digit recognition and texture discrimination with small training sets.
- PCA model selection on scattering representations yields state-of-the-art results for handwritten digit recognition and texture discrimination with small training sets.
- Scattering transforms provide locally translation-invariant representations that linearize small deformations and characterize textures through co-occurrence coefficients.
- Combining scattering with wavelet transforms on a compact Lie group extends invariance beyond translations.