Source-linked AI summary
Rigid-Motion Scattering for Texture Classification
Laurent SIfre, Stéphane Mallat
TL;DR
Texture classification requires representations that handle geometric variability while discriminating stationary, non-Gaussian processes from a single image. The paper develops deep rigid-motion scattering with joint translation–rotation operators and scale handling, obtaining state-of-the-art results on multiple texture databases. Its joint representation preserves information that separable invariants can lose.
Problem
Texture classification must distinguish non-Gaussian, non-Markovian stationary processes from a single realization despite translation, rotation, and scaling variability.
Method
The paper constructs deep scattering networks with convolutions and wavelets on the rigid-motion group, jointly processing spatial translation and orientation variables and adding scale invariance through linear projectors.
Results
Rigid-motion scattering provides state-of-the-art classification results on most texture datasets, while rigid-motion and scale invariance considerably improve results over translation scattering on databases with non-rigid deformations.
Takeaways & Limitations
Joint translation–rotation operators provide tighter invariants than separable operators, which can lose information, supporting texture classification under large geometric transformations.
Takeaways & Limitations
Separable translation and rotation invariants can confuse structurally different images because they ignore joint location and orientation structure; registration invariants are not stable to deformations.
Abstract
from arXiv · showhide
A rigid-motion scattering computes adaptive invariants along translations and rotations, with a deep convolutional network. Convolutions are calculated on the rigid-motion group, with wavelets defined on the translation and rotation variables. It preserves joint rotation and translation information, while providing global invariants at any desired scale. Texture classification is studied, through the characterization of stationary processes from a single realization. State-of-the-art results are obtained on multiple texture data bases, with important rotation and scaling variabilities.
1 Introduction
The paper develops scattering representations that jointly handle translation and rotation variability while preserving discriminative information. It applies these rigid-motion invariants to texture classification, including scaling variability and single-image stationary-process analysis.
- Image classification seeks representations that reduce intra-class variability while preserving discriminative information across classes.
- Rigid-motion scattering constructs convolutional representations invariant to translations and rotations by operating on the joint rigid-motion group.Separating translation and rotation variables can lose important joint information and reduce discrimination.
- Texture classification must address translation, rotation, and scaling variability in non-Gaussian, non-Markovian stationary processes observed from a single image.Single realizations make high-order moments impractical because their estimators have excessive variance.
- The paper extends translation-invariant scattering to joint translation and rotation invariance and incorporates scaling variability through linear projectors.The resulting invariants provide effective representations for texture classification across a large range of texture databases.
- The paper introduces wavelet transforms and convolutions on the rigid-motion group, with experiments reproducible through the ScatNet MATLAB toolbox.
2 Invariance to Translations, Rotations and Deformations
The section develops translation-invariant representations that remain stable to deformations, then shows why separable translation–rotation invariants lose joint information and motivates rigid-motion scattering.
- 2.1 Translation Invariance and Deformation Stability: Translation invariance alone is insufficient because deformation invariance must preserve information that discriminates different classes.The paper distinguishes desired class-specific deformation invariance from invariance to arbitrary deformations, which could confuse different digits or images.
- 2.2 Wavelet Transform Invariants: Averaging produces local translation invariance but removes high frequencies, which wavelet transforms recover through localized, deformation-stable coefficients.Morlet wavelets decompose signals across orientations and scales while remaining stable under deformation.
- 2.3 Transation Invariant Scattering: Scattering iterates wavelet modulus operators in a convolutional network, producing averaged invariant coefficients and covariant coefficients at successive layers.The final representation concatenates coefficients across layers, while the transform is non-expansive and stable to deformations.
- 2.4 Separable Versus Joint Rigid Motion Invariants: Separable invariants over translations and rotations can confuse textures whose orientation components have different relative locations because they discard joint position–orientation structure.This failure occurs because separate processing of orientation components does not connect their spatial relationships.
- 2.4 Separable Versus Joint Rigid Motion Invariants: Rigid-motion scattering constructs translation–rotation invariants directly on the rigid-motion group to preserve information between positions and orientations.This joint construction addresses the information loss exhibited by separable scattering and related separable invariants.
3 Rigid-motion Scattering
Rigid-motion scattering extends scattering to the special Euclidean group SE(2), jointly processing translations and rotations rather than separating them. It uses wavelets on spatial and angular variables, then iterated rigid-motion modulus operators to produce invariant scattering coefficients.
- Group-based scattering: Scattering operators are extended to Lie groups by calculating wavelet transforms on the group, with SE(2) used for translations and rotations.The resulting representation is invariant to the action of the group.
- Group-based scattering: SE(2) parameterizes a rigid motion by a translation v ∈ R2 and a rotation rθ with θ ∈ [0,2π).Its group product is noncommutative and combines the translation and rotation components.
- Wavelets on SE(2): Wavelets on SE(2) are constructed from separable products of spatial wavelets in L2(R2) and angular wavelets in L2(SO(2)).The spatial component uses dilated wavelets and a rotationally symmetric averaging function, while the angular component uses periodic wavelets and circular convolutions.
- Scattering architecture: The rigid-motion wavelet transform is a bounded linear operator with energy-conservation conditions, and its modulus operators are non-expansive.Non-expansiveness follows from the linear operator and modulus inequalities.
- Scattering architecture: Rigid-motion scattering first applies a spatial wavelet modulus, then iterates rigid-motion wavelet modulus operators along the group variable.Each modulus layer outputs scattering coefficients and computes the next propagated representation; coefficients are concatenated across orders 0 ≤ m ≤ M.
4 Fast Rigid-Motion Scattering
The fast rigid-motion scattering implementation uses successive filter-bank cascades over spatial and orientation variables. This reduces computation while preserving the multistage structure of the rigid-motion wavelet transform.
- First- and second-layer scattering suffice for state-of-the-art texture-classification results.
- 4.1 Wavelet Filter Bank Implementation: The spatial wavelet transform uses fixed-support filters and convolution-downsampling cascades across scales.Low-pass filtering produces low frequencies, while oriented filters produce high-frequency coefficients.
- 4.1 Wavelet Filter Bank Implementation: O(CNP) operations and O(CN) memory implement the spatial cascade for C orientations, image size N, and filter size P.The cascade applies 1+C convolutions at each progressively downsampled resolution.
- 4.2 Rigid Motion Wavelet Filter Bank Implementation: The rigid-motion wavelet transform first filters and downsamples spatially, then retransforms each leaf along the orientation variable.The orientation cascade uses low- and high-pass filters over periodic orientations.
- 4.2 Rigid Motion Wavelet Filter Bank Implementation: The full rigid-motion wavelet transform costs O(CLN(P + logC)) operations and O(CLN) memory.Its spatial stage alone costs O(CLNP) operations and O(CLN) memory.
5 Image Texture Classification
The paper evaluates rigid-motion scattering and scale-invariant extensions for texture classification across datasets with varied geometric transformations. Joint rigid-motion operators and partial scale invariance improve or preserve classification performance, reaching state-of-the-art results across the reported databases.
- Datasets and evaluation: Texture classification is evaluated on KTH-TIPS, UIUC, UMD, and FMD datasets spanning controlled and uncontrolled geometric variability.Experiments compare progressively more invariant scattering representations with published state-of-the-art algorithms.
- Classification method: The PCA classifier averages log scattering transforms across dilated training and test images, then classifies by distance from each class’s projected affine space.Class spaces are estimated with SVD from centered training examples, while dilation augmentation provides partial scale invariance.
- Representation: Rigid-motion scattering preserves joint position–orientation information that separable translation and rotation invariants lose.The method uses convolutions and wavelets on the rigid-motion group rather than treating translation and rotation independently.
- KTH-TIPS: 84.3±3.1, 98.3±0.9, and 99.4±0.4 accuracy are obtained with log and scale invariance for training sizes 5, 20, and 40 on KTH-TIPS.The corresponding translation-scattering accuracies are 69.1±3.5, 94.8±1.3, and 98.0±0.8.
- UIUC and UMD: Rigid-motion and scale invariance provide considerable improvements over translation scattering on UIUC and UMD, with overall results often matching or exceeding state of the art.These datasets contain uncontrolled deformations including shear, perspectivity effects, and non-rigid deformations.
- FMD: 52.2 % accuracy is achieved on FMD using rigid-motion log scattering on concatenated YUV channels with a one-versus-all linear SVM.The paper identifies this as the best result for a single feature, while multiple features and feature selection can improve it further.
6 Conclusion
Rigid-motion scattering constructs stable translation and rotation invariants through joint spatial and orientation wavelet cascades. Applied to textures with substantial geometric variability, it achieves state-of-the-art classification on most datasets, with extensions toward dilation and broader vision tasks.
- Conclusion: Rigid-motion scattering produces stable translation and rotation invariants through wavelet cascades over spatial and orientation variables.Its fast implementation uses two downsampling and filtering cascades.
- Conclusion: Joint operators provide tighter invariants than separable operators, which can lose excessive information about positions and orientations.The conclusion links this design choice to preserving useful joint structure.
- Conclusion: Rigid-motion scattering achieves state-of-the-art classification results on most texture datasets containing large geometric transformations.The paper also notes extensions to dilation and promising results for object recognition.