Source-linked AI summary
Asteroid: the PyTorch-based audio source separation toolkit for researchers
Manuel Pariente, Samuele Cornell, Joris Cosentino, Sunit Sivasankaran, Efthymios Tzinis, Jens Heitkaemper, Michel Olvera, Fabian-Robert Stöter, Mathieu Hu, Juan M. Martín-Doñas, David Ditter, Ariel Frank, Antoine Deleforge, Emmanuel Vincent
TL;DR
Deep-learning source-separation research lacks consistently complete and reproducible tooling across models, datasets, and pipeline stages. Asteroid addresses this gap with a PyTorch toolkit and Kaldi-style end-to-end recipes, whose implementations outperform reference results in many cases, by up to 2.6 dB on four WHAMR tasks.
Problem
Existing source-separation toolkits and isolated implementations often lack data preparation, complete pipelines, broad task support, or easy experiment configuration.
Method
Asteroid provides a PyTorch-based toolkit with native neural building blocks, configurable components, and Kaldi-style recipes covering data download, preparation, training, and evaluation.
Results
Asteroid’s recipes outperform original results on all four WHAMR tasks by up to 2.6 dB and exceed original results for most evaluated architectures.
Takeaways & Limitations
Asteroid provides a reproducible, extensible toolkit whose implementations are competitive across several datasets and architectures.
Takeaways & Limitations
The paper’s permutation-invariant loss formulation assumes the loss can be decomposed into pairwise source losses followed by a reduce function.
Abstract
from arXiv · showhide
This paper describes Asteroid, the PyTorch-based audio source separation toolkit for researchers. Inspired by the most successful neural source separation systems, it provides all neural building blocks required to build such a system. To improve reproducibility, Kaldi-style recipes on common audio source separation datasets are also provided. This paper describes the software architecture of Asteroid and its most important features. By showing experimental results obtained with Asteroid's recipes, we show that our implementations are at least on par with most results reported in reference papers. The toolkit is publicly available at https://github.com/mpariente/asteroid .
1. Introduction
Asteroid is a PyTorch-based open-source toolkit designed to make deep-learning source-separation research more complete, extensible, configurable, and reproducible. It addresses gaps in existing toolkits by providing broader end-to-end support and recipes, while its implementations match or exceed reported reference results.
- Deep learning-based approaches now largely outperform classical open-source toolkits for single-channel source separation.
- Existing toolkits often omit data preparation, limit configurability, support only selected models or tasks, or provide incomplete pipelines.
- Asteroid is a PyTorch-based open-source toolkit for deep-learning audio source separation and speech enhancement, designed for researchers and practitioners.
- Its principles include using native PyTorch code, supporting third-party code imports with minimal changes, and covering data preparation through evaluation.
- Asteroid recipes are configurable from the command line and are organized to support reproducible experimentation.
2. General framework
Asteroid’s general framework targets single-channel source separation by decomposing a noisy mixture into source estimates. Its typical neural architecture encodes the mixture, estimates source masks, and decodes the masked representations.
- Asteroid focuses primarily on single-channel source separation, where a recording contains multiple sources and additive noise.
- The source-separation goal is to estimate each source signal from the observed mixture.
- The encoder-masker-decoder architecture transforms the mixture into a representation, estimates one mask per source, and reconstructs time-domain estimates.
- Figure 1 depicts the typical encoder-masker-decoder architecture used by neural source-separation systems.
3. Functionality
Asteroid provides modular neural source-separation components, training and evaluation infrastructure, and reproducible recipes across diverse datasets. Its PIT implementation efficiently handles source-order ambiguity by reducing computation from J! to J^2.
- Filterbanks: Asteroid separates filterbanks from encoder and decoder objects, supporting multiple filterbank types and PyTorch-compatible interfaces.Supported options include free, DFT, analytic free, parameterized sinc, and multi-phase Gammatone filterbanks, with pseudo-inverse and dynamic-filter support.
- Masker networks: Asteroid implements widely used masker networks, including TasNet LSTMs, Conv-TasNet temporal convolutional networks, DPRNNs, and Open-Unmix.The toolkit supports Conv-TasNet variants with or without skip connections and Open-Unmix for music source separation.
- Losses and reconstruction: Asteroid supports reconstruction and optimization components, including multiple separation losses, Griffin-Lim and MISI inversion, and NumPy and torchaudio interfaces.Available losses include MSE, SI-SDR, SDR, SNR, PESQ, and affinity loss for deep clustering.
- Permutation-invariant training: J! to J^2: Asteroid accelerates PIT loss computation by precomputing the pairwise loss matrix before selecting the minimum-cost permutation.PITLossWrapper can convert a pairwise loss or permutation-dependent loss into a PIT loss.
- Recipes and tooling: Asteroid provides baseline recipes for speech, music, noisy, reverberant, and arbitrary-sound separation datasets, alongside distributed training and standard evaluation tools.Supported datasets include wsj0-2mix, WHAMR, LibriMix, FUSS, DNS, SMS-WSJ, Kinect-WSJ, and MUSDB18; evaluation supports SDR, SIR, SAR, SI-SDR, PESQ, and STOI.
4. Implementation
Asteroid’s Kaldi-style recipes organize the full workflow from data acquisition through evaluation, while configuration and experiment outputs are managed through a standard recipe structure.
- Recipes run through stages for downloading data, generating mixtures, preparing DataLoader metadata, training, and evaluating separation systems.
- Data preparation uses dataset authors’ official scripts, optional augmentation, and text files containing filenames, lengths, and speaker IDs.
- Recipe directories separate generated data, logs, and experiment outputs into data/, logs/, and exp/ locations.
- A YAML configuration file supplies most constants, generates a command-line argument parser, and saves modified configurations for reuse.
- The run.sh entry point passes command-line settings to training scripts, including configuration overrides and experiment-level options.
5. Example results
Asteroid recipes were evaluated on wsj0-2mix and WHAMR against reported results for established source-separation systems. The implementations generally matched or exceeded original results, with WHAMR gains reaching 2.6 dB, while recipes were designed for rapid experimentation.
- Datasets and evaluation: The evaluation compares Asteroid implementations with reported state-of-the-art results on wsj0-2mix and four WHAMR tasks.wsj0-2mix measures single-channel two-speaker separation, while WHAMR extends the setting to noisy and reverberant conditions.
- Results: For most architectures, Asteroid implementations outperform the original wsj0-2mix results.Table 2 reports SI-SDRi on the wsj0-2mix test set for several well-known systems.
- Results: On all four WHAMR tasks, Asteroid recipes achieve better results than originally reported, by up to 2.6 dB.The tasks are anechoic, noisy anechoic, reverberant, and noisy reverberant separation.
- Training choices: Shorter training segments for memory-intensive architectures and 10^-5 weight decay for recurrent architectures increased final performance.These choices were used to increase batch size, stabilize gradients, and improve recurrent systems.
- Usability: Writing the TasNet recipe used in Table 3 took less than a day, and its results were generated through a command-line invocation.The example specifies the GPU with the --id argument.
6. Conclusion
The paper introduces Asteroid as an open-source audio source separation toolkit for researchers and practitioners. Comparative experiments show competitive results across several datasets and architectures, while the toolkit is designed for rapid extension.
- Asteroid is an open-source audio source separation toolkit designed for researchers and practitioners.
- Comparative experiments show competitive Asteroid results on several datasets and architectures.
- The toolkit is designed to be quickly extended with new network architectures or benchmark datasets.