Source-linked AI summary
Moisesdb: A dataset for source separation beyond 4-stems
Igor Pereira, Felipe Araújo, Filip Korzeniowski, Richard Vogl
TL;DR
Fine-grained music source separation lacks public data with detailed stem taxonomies, while many existing datasets use only four stems. MoisesDB contributes 240 annotated multitrack songs organized into stems and reports baselines for publicly available separation models.
Problem
Many stem datasets use only four categories, and available multitrack collections may be unsuitable for source-separation research or restricted by licensing.
Source: §1 · pp. 1–2Method
The paper introduces MoisesDB, a 240-track dataset with annotated individual sources organized into stems, and provides a Python library for working with it.
Source: §1 · p. 2, §3.3 · p. 4Results
The paper reports baseline performance results for HT-Demucs, Spleeter, and masking-based oracle methods.
Source: §5 · p. 6Takeaways & Limitations
The dataset is intended to support training and testing robust source-separation systems capable of separating more stems.
Source: §1 · p. 2MoisesDB songs are unmastered, with lower loudness and higher dynamic range than commercial recordings, raising concerns about distributional shift.
Source: §3.2 · p. 4
Abstract
from arXiv · showhide
In this paper, we introduce the MoisesDB dataset for musical source separation. It consists of 240 tracks from 45 artists, covering twelve musical genres. For each song, we provide its individual audio sources, organized in a two-level hierarchical taxonomy of stems. This will facilitate building and evaluating fine-grained source separation systems that go beyond the limitation of using four stems (drums, bass, other, and vocals) due to lack of data. To facilitate the adoption of this dataset, we publish an easy-to-use Python library to download, process and use MoisesDB. Alongside a thorough documentation and analysis of the dataset contents, this work provides baseline results for open-source separation models for varying separation granularities (four, five, and six stems), and discuss their results.
1. INTRODUCTION
Music source separation supports practical applications, but training data is difficult to obtain and common stem datasets provide limited granularity. MoisesDB addresses this with annotated multitrack recordings organized into stems and accompanying baseline results.
1. INTRODUCTION
- Source separation supports applications such as music education, play-along tracks, by-ear transcription, and karaoke backing tracks.The paper also notes its use as preprocessing for tasks such as voice separation for f0 tracking.Source: §1 · p. 1
- Training supervised systems requires substantial data, while legal, studio-confidentiality, and production-workflow barriers hinder multitrack and stem dataset release.Preparing, exporting, and organizing stems from recording projects can itself be considerable work.Source: §1 · p. 1
- Many existing stem datasets use only vocals, drums, bass, and other, while practical applications may need guitars, keys, and strings.Task-specific datasets such as vocal-separation collections have limited relevance to the broader task of splitting tracks into stems.Source: §1 · pp. 1–2
- Existing multitrack collections may not be ready for source separation or may have license restrictions, and some recent models rely largely on non-public data.The paper identifies a need for more free data with a detailed taxonomy for training and testing systems that separate more stems.Source: §1 · p. 2
- MoisesDB provides 240 music tracks totaling over 14 hours, with annotations and a taxonomy that groups individual tracks into stems.The dataset is offered free of charge for non-commercial research use.Source: §1 · p. 2
2. RELATED WORK
Existing music-separation datasets either organize multitrack recordings inconveniently for separation or use a coarse four-stem taxonomy. MoisesDB addresses the need for data organized to support finer-grained separation.
2. RELATED WORK
- MedleyDB supplies multitrack audio and rich annotations, but its numbered stems and sound-production-based groupings make it cumbersome for source separation out of the box.Stems are not meaningfully labeled across songs, and instruments are grouped by how they produce sound rather than their role in a mix.Source: §2 · pp. 2–3
- MUSDB18 provides four stems—vocals, drums, bass, and other—but that limited taxonomy prevents more granular separation systems.Source: §2 · p. 3
- MoisesDB offers 240 previously unreleased songs in a source-separation-focused taxonomy, intended to support researchers building more granular separation systems.The paper identifies limited stem counts and datasets not organized for separation as barriers to granular research.Source: §2 · p. 3
3. DATASET
MoisesDB contains 240 songs across twelve genres, organized as individual tracks grouped into semantically labeled stems. The paper describes the dataset's stem distribution, recording conditions, and accompanying Python library.
3. DATASET
- The dataset contains 240 songs by 47 artists across twelve genres, with a total duration of 14 hours, 24 minutes, and 46 seconds.Songs average 3:36, with a standard deviation of 66 seconds; artists and genres follow a power-law-like distribution.Source: §3 · pp. 3–4
3.1 Stem Taxonomy
- MoisesDB groups individual tracks into semantically labeled stems using a taxonomy reflecting recording and mixing, and songs can have different numbers of stems.This organization supports separation into more stems, on-the-fly stem mixing for augmentation, and separation of individual tracks within a stem.Source: §3.1 · p. 4
- Vocals, drums, and bass appear on virtually every song, while wind is rare; the authors anticipate imbalance may challenge training for underrepresented stems.They note that “other plucked” tracks may remain difficult to distinguish from guitar when training solely on MoisesDB, while the data can help characterize such errors.Source: §3.1 · p. 4
3.2 Recording and Mastering
- Tracks are professionally recorded in stereo and mixed without mastering, yielding lower loudness and higher dynamic range than professionally mastered commercial songs.The authors flag a possible distribution shift, while noting that HT-Demucs generalizes reasonably to mastered recordings when trained on unmastered data.Source: §3.2 · p. 4
3.3 Python Library
- The accompanying Python library parses metadata and automatically builds stems and mixes; its example workflow also saves individual stems.Source: §3.3 · pp. 4, 6
4. BENCHMARKING
The benchmark reports SDR results for oracle and open-source separation methods across four-, five-, and six-stem groups. HT-DEMUCS exceeds oracle methods on bass and drums in four- and six-stem tracks, but is slightly worse on other, piano, and guitar in six-stem tracks.
4. BENCHMARKING
- SDR benchmarks cover IBM, IRM, and MWF oracle methods and HT-DEMUCS and Spleeter, across four-, five-, and six-stem groups where each model supports them.Spleeter results cover four and five stems; HT-DEMUCS results cover four and six.Source: §4 · p. 6
- SDR measures how much true-source energy is preserved in an estimated source after separation, with results expressed in decibels.The calculation compares the true signal s(n) with its estimate at each time n.Source: §4 · p. 6
- The benchmark compares SDR distributions for vocals, bass, drums, other, piano, and guitar across the three stem groups.Tracks were selected to contain the stems in each group; additional tracks were merged into “other.”Source: §4 · p. 6
- HT-DEMUCS outperforms oracle methods on bass and drums in four- and six-stem tracks, but performs slightly worse on other, piano, and guitar in six-stem tracks.IRM and MWF show no significant SDR difference across stem groups.Source: §4 · p. 6
5. CONCLUSION
The paper presents MoisesDB as a multitrack dataset with a hierarchical taxonomy for source separation beyond four stems. It documents the dataset, provides public-model and oracle performance results, and offers a Python library for access and machine-learning integration.
5. CONCLUSION
- MoisesDB is a multitrack dataset with a hierarchical taxonomy designed for source separation beyond four stems.The paper analyzes its taxonomy, track-duration and genre distributions, and the availability of songs by stem and source.Source: §5 · p. 6
- The paper reports results for two publicly available separation methods and three masking-based oracle methods, which indicate theoretical performance limits for mask-based separation.It also provides a Python library for data access and fast integration with machine-learning libraries.Source: §5 · p. 6
- The authors aim for MoisesDB to support development of extended separation models and other uses, including automatic mixing and generative accompaniment systems.Source: §5 · p. 6