Source-linked AI summary
Kymatio: Scattering Transforms in Python
Mathieu Andreux, Tomás Angles, Georgios Exarchakis, Roberto Leonarduzzi, Gaspar Rochette, Louis Thiry, John Zarka, Stéphane Mallat, Joakim andén, Eugene Belilovsky, Joan Bruna, Vincent Lostanlen, Muawiz Chaudhary, Matthew J. Hirn, Edouard Oyallon, Sixin Zhang, Carmine Cella, Michael Eickenberg
TL;DR
Many signal-processing and machine-learning tasks require representations invariant to translations and deformations. Kymatio implements the wavelet scattering transform in Python across 1D, 2D, and 3D, with framework compatibility, GPU execution, and reduced memory requirements. The package limits its scope to second-order coefficients and is released as free, open-source software under a 3-clause BSD license.
Problem
Signal-processing and machine-learning tasks need representations invariant to translations and deformations while preserving information relevant to the task.
Method
Kymatio provides a unified Python implementation of scattering transforms in 1D, 2D, and 3D, integrated with automatic differentiation and scientific Python frameworks.
Results
Kymatio provides a well-documented, user-friendly, fast implementation usable with PyTorch and TensorFlow/Keras across diverse applications.
Takeaways & Limitations
Kymatio makes scattering transforms accessible within scientific Python and deep learning workflows, including hybrid deep learning, generative modeling, and 3D chemistry applications.
Takeaways & Limitations
The implementation restricts its scope to second-order coefficients because higher-order coefficients are typically low-energy but computationally intensive.
Abstract
from arXiv · showhide
The wavelet scattering transform is an invariant signal representation suitable for many signal processing and machine learning applications. We present the Kymatio software package, an easy-to-use, high-performance Python implementation of the scattering transform in 1D, 2D, and 3D that is compatible with modern deep learning frameworks. All transforms may be executed on a GPU (in addition to CPU), offering a considerable speed up over CPU implementations. The package also has a small memory footprint, resulting inefficient memory usage. The source code, documentation, and examples are available undera BSD license at https://www.kymat.io/
1. Introduction
Scattering transforms represent signals with invariance to translations and deformations while preserving task-relevant information. Kymatio provides a unified, efficient implementation across dimensions and modern computational environments.
- 1. Introduction: Scattering transforms target classification and regression tasks involving signal invariance to translations and deformations.The paper cites images, audio recordings, and electronic densities as examples.
- 1. Introduction: The representation is a convolutional network with fixed wavelet and lowpass averaging filters coupled with modulus nonlinearities.
- 1. Introduction: Kymatio unifies 1D, 2D, and 3D scattering transforms through a single API compatible with automatic differentiation libraries.
- 1. Introduction: Depth-first traversal reduces stored intermediate signals, enabling efficient processing in limited-memory environments such as GPUs.
2. Implementing the Scattering Transform
The implementation builds scattering coefficients from wavelet filtering, nonlinearities, and lowpass averaging, with computational optimizations for memory and speed. Kymatio restricts the presented scope to second-order coefficients and supports dimension-specific wavelet constructions.
- 2. Implementing the Scattering Transform: Signals are defined on d-dimensional grids with d = 1, 2, 3, and periodic convolution is used between signals on the grid.
- 2. Implementing the Scattering Transform: Wavelet filter banks, a lowpass filter at averaging scale 2^J, and a nonlinearity together define the scattering transform.
- 2. Implementing the Scattering Transform: The zeroth-order coefficient is a local average, while first-order coefficients apply wavelet convolution, a nonlinearity, and lowpass averaging.
- 2. Implementing the Scattering Transform: The first-wavelet modulus demodulates signals by shifting energy toward low frequencies, after which a second filter bank recovers remaining frequencies.
- 2. Implementing the Scattering Transform: Second-order coefficients use only a subset Λ2(λ1) of second-layer frequencies because the first nonlinear output is low frequency.
- 2. Implementing the Scattering Transform: Higher-order coefficients are typically low-energy but computationally intensive, so the implementation restricts its scope to second-order coefficients.
- 2. Implementing the Scattering Transform: Intermediate low-frequency results are downsampled, while depth-first traversal and periodic fast Fourier transforms reduce computational load and stored intermediates.
3. Project vision
Kymatio’s project vision emphasizes stable, usable, Python-native scattering software with GPU support, broad framework integration, and differentiability for downstream applications.
- Code quality and community: Kymatio prioritizes stable, easy-to-use software through unit tests, few dependencies, intuitive interfaces, and extensive documentation.Its development model also aims to foster an open-source community and maintain contributors.
- Python ecosystem: Kymatio provides a completely Pythonic implementation that integrates with the scientific Python ecosystem.This addresses the prevalence of MATLAB implementations among existing scattering packages.
- GPU support: Kymatio offers easy-to-use GPU implementations for scattering transforms in 1D, 2D, and 3D.The paper identifies GPU compatibility as critical for many data science workloads.
- Framework integration: Frontends for NumPy, scikit-learn, PyTorch, and TensorFlow/Keras support integration across scientific and deep learning pipelines.PyTorch and TensorFlow/Keras frontends enable inclusion in deep learning workflows.
- Applications and comparison: Differentiability simplifies applications including reconstruction and generative modeling.Table 1 compares Kymatio with existing scattering transform packages.
4. User Interface and Documentation
Kymatio presents a consistent interface across dimensionalities and frontends, with PyTorch-style execution and examples spanning reconstruction, learning, molecular regression, and speech classification.
- Interface: Users construct scattering objects by specifying the averaging scale J and input signal shape.The interface includes Scattering1D, Scattering2D, and HarmonicScattering3D classes.
- Interface: The resulting scattering object behaves like a PyTorch nn.Module and is applied through calls to the transform object.This provides a consistent execution pattern for the PyTorch frontend.
- Device execution: GPU and CPU execution are selected through nn.Module-style calls, including S.cuda() and S.cpu().The API exposes explicit device-switching methods.
- Package context: Kymatio’s title identifies it as a Python package for scattering transforms.The section’s interface examples are presented in the context of Kymatio: Scattering Transforms in Python.
- Documentation and examples: Documentation examples cover image reconstruction and generation, hybrid scattering with CNNs, molecular-property regression, and spoken-digit classification.These examples illustrate applications across images, deep learning, chemistry, and audio.
5. Conclusion
Kymatio delivers a well-documented, user-friendly, fast scattering-transform implementation compatible with major deep learning frameworks and supporting diverse applications.
- Conclusion: Kymatio provides a well-documented, user-friendly, and fast implementation of the scattering transform.The package is presented as a practical software implementation rather than a new scattering representation.
- Conclusion: PyTorch and TensorFlow/Keras compatibility supports applications including hybrid deep learning, generative modeling, and 3D chemistry.Future work targets speed, flexibility, and backend support.