Source-linked AI summary
Cutting Music Source Separation Some Slakh: A Dataset to Study the Impact of Training Data Quality and Quantity
Ethan Manilow, Gordon Wichern, Prem Seetharaman, Jonathan Le Roux
TL;DR
Music source separation research is constrained by scarce labeled stems and coarse existing instrument categories, making it difficult to separate data effects from model effects. The paper introduces Slakh, a synthesized dataset of rendered MIDI mixtures and stems, and evaluates it alongside Flakh for training-data studies. Slakh augments existing data and supports new instrument categories, while its purely instrumental scope limits direct comparisons involving vocals.
Problem
Music source separation lacks abundant labeled stems and granular instrument categories, complicating comparisons between model architectures and training-data strategies.
Method
The paper renders Lakh MIDI files with professional-grade sample-based virtual instruments to create Slakh, and uses FluidSynth-rendered Flakh to study synthesis quality.
Results
Slakh augments MUSDB18 to improve separation performance, with Slakh2100 outperforming Flakh2100 and enabling evaluation of bass, drums, guitar, and piano separation.
Takeaways & Limitations
Slakh provides substantially more training data and enables source-separation benchmarks for instrument categories previously grouped into “Other.”
Takeaways & Limitations
Slakh contains no isolated vocal sources because MIDI does not support vocals, making direct comparisons with existing datasets incomplete.
Abstract
from arXiv · showhide
Music source separation performance has greatly improved in recent years with the advent of approaches based on deep learning. Such methods typically require large amounts of labelled training data, which in the case of music consist of mixtures and corresponding instrument stems. However, stems are unavailable for most commercial music, and only limited datasets have so far been released to the public. It can thus be difficult to draw conclusions when comparing various source separation methods, as the difference in performance may stem as much from better data augmentation techniques or training tricks to alleviate the limited availability of training data, as from intrinsically better model architectures and objective functions. In this paper, we present the synthesized Lakh dataset (Slakh) as a new tool for music source separation research. Slakh consists of high-quality renderings of instrumental mixtures and corresponding stems generated from the Lakh MIDI dataset (LMD) using professional-grade sample-based virtual instruments. A first version, Slakh2100, focuses on 2100 songs, resulting in 145 hours of mixtures. While not fully comparable because it is purely instrumental, this dataset contains an order of magnitude more data than MUSDB18, the {\it de facto} standard dataset in the field. We show that Slakh can be used to effectively augment existing datasets for musical instrument separation, while opening the door to a wide array of data-intensive music signal analysis tasks.
1. INTRODUCTION
Music source separation needs substantial labeled data, but music datasets are scarce and coarse, complicating fair comparisons between models and training strategies. The paper introduces Slakh, a synthesized dataset designed to expand training data and instrument coverage.
- Music source separation models typically require large amounts of labeled mixtures and corresponding clean sources.
- 10 hours from 150 songs makes MUSDB18 relatively small compared with datasets used for other deep learning tasks.
- More training data is associated with higher objective quality measures, but this confounds architectural advances with augmentation and training strategies.
- Existing music datasets use coarse source categories, limiting granular separation of musical instruments.
- Slakh provides high-quality instrumental mixtures and sources rendered from the Lakh MIDI dataset, with Slakh2100 covering 2100 songs.
2. SLAKH DATASET CREATION AND ANALYSIS
Slakh2100 is built by rendering selected Lakh MIDI files into separate instrument stems and automatically mixed instrumental recordings, with a FluidSynth counterpart for studying synthesis quality. It substantially expands open multi-track data while retaining an important scope limitation: no isolated vocal sources.
- Dataset creation: Slakh2100 contains 2100 automatically mixed tracks, accompanying MIDI files, and 145 hours of mixture data.
- MIDI selection: The Lakh MIDI dataset contains over 170,000 unique MIDI files whose instrument tracks can be isolated and synthesized.
- MIDI selection: Files are selected using MIDI instrument program numbers and must contain piano, bass, guitar, and drums tracks.
- Rendering: Each MIDI track is split into a separate file and assigned a matching patch before rendering.
- Rendering and mixing: Tracks are loudness-normalized, summed instantaneously, and gain-adjusted automatically to prevent clipping.
- Flakh: FluidSynth renders the same 2100 MIDI files as Flakh, using a simpler synthesis process that cannot be mixed identically because of boosted low-amplitude noise.
- Dataset analysis: Slakh2100 is larger than all other existing open multi-track music source-separation datasets combined by mixtures and total duration.
- Dataset analysis: Slakh and Flakh have similar overall spectral shapes but stronger mid-range harmonic peaks, less low-frequency energy, and a steeper high-frequency decline than MUSDB18.
3. EXPERIMENTAL VALIDATION
The experiments use controlled BLSTM-based separation models to compare training-set quantity, synthesis quality, and mixture construction. Results show that Slakh augmentation improves MUSDB18 separation, while dataset mismatch affects cross-dataset performance.
- 3.1. Model and setup: Four bidirectional LSTM layers with 600 units per direction estimate real-valued masks from log-magnitude spectrogram inputs.The models use a sigmoid output layer and are trained with the truncated phase sensitive approximation objective.
- 3.2. Training sets: Coherent mixtures combine time-aligned sources from the same song, whereas incoherent augmentation samples separate 10-second clips for each source.Coherent clips require the target source to exceed a -30 dB RMS threshold; incoherent mixtures can draw sources from different songs and offsets.
- 3.3. MUSDB18 test set results: Slakh2100 augmentation improves bass and drums SI-SDR over non-augmented MUSDB18, and outperforms Flakh2100, indicating synthesis realism matters.The MUSDB18 + Slakh and MUSDB18 + MUSDB18-incoh. gains differ by only 0.1 dB for both bass and drums.
- 3.4. Slakh2100 test set results: Slakh-trained models outperform MUSDB18-trained models on bass and drums in the Slakh2100 test set, while guitar and piano separation remains comparatively low.The paper attributes the guitar and piano difficulty to overlapping mid-frequency instruments and suggests automatic equalization as a possible improvement.
- 3.4. Slakh2100 test set results: Models trained without Slakh data perform poorly when separating bass on the Slakh2100 test set.This result, together with the MUSDB18 evaluation, demonstrates that training-data domain affects separation performance.
4. CONCLUSIONS AND FUTURE WORK
The paper concludes that Slakh is an open dataset of synthesized instrumental mixtures and stems that increases training data and supports new instrument categories. It also identifies realism improvements and dataset expansion as important future directions.
- 4. CONCLUSIONS AND FUTURE WORK: Slakh provides high-quality instrumental mixtures and corresponding stems synthesized from MIDI with professional-grade sample-based virtual instruments.The dataset is released as an open resource for music source separation research.
- 4. CONCLUSIONS AND FUTURE WORK: Increasing training data with Slakh improves separation performance, while Slakh also enables evaluation of instrument categories absent from existing datasets.The paper contrasts this with existing datasets, where guitar and piano were grouped into “Other” categories.
- 4. CONCLUSIONS AND FUTURE WORK: Unrealistic augmentations can achieve results similar to Slakh, but augmentation of existing datasets saturates comparatively quickly.The authors describe Slakh as having more room for growth through additional mixtures, instruments, and effects.
- 4. CONCLUSIONS AND FUTURE WORK: Future work targets more realistic Slakh mixes through better equalization, isolated vocals, and small deviations in note tuning.The paper also proposes using Slakh for stronger source-separation benchmarks and other data-intensive music analysis tasks.