Source-linked AI summary
FLOMSS: Generative Music Source Separation by Flow Matching
Shuyang Cui, Zhi Zhong, Keisuke Toyama, Koichi Saito, Qiyu Wu, Yukara Ikemiya, Junghyun Koo, Kin Wai Cheuk, Woosung Choi, Giorgio Fabbro, Wei-Hsiang Liao, Shusuke Takahashi, Yuki Mitsufuji
TL;DR
Flow matching has not been systematically evaluated for predefined-stem, full-band stereo music source separation. FLOMSS studies conditional flow matching and evaluates it with signal, spectral, embedding-based, and listening-test measures; it improves spectral and embedding-based comparisons while SDR remains lower than strong discriminative methods.
Problem
Flow matching has not been systematically studied for predefined-stem music source separation in full-band stereo mixtures.
Source: §1 · p. 1Method
FLOMSS applies supervised conditional flow matching to four-source full-band stereo MSS and systematically studies prediction targets, paths, model configurations, and mixture-consistency strategies.
Source: §1 · p. 1Results
Takeaways & Limitations
The results establish flow matching as a promising direction for generative MSS, with a distributional–reconstruction trade-off in the evaluated setting.
Source: §6 · p. 4Hard mixture consistency degrades overall performance for the evaluated waveform- and STFT-domain paths, suggesting it may require an MSS-specific flow path and training strategy.
Source: §5.3 · p. 4
Abstract
from arXiv · showhide
Music source separation (MSS) has been conventionally addressed by discriminative methods. Recently, flow matching has emerged as a strong generative paradigm for modeling complex data distributions, with successful applications across diverse domains. These advances motivate us to systematically investigate flow matching for MSS in a four-source (Vocals, Bass, Drums, Other) setting in full-band stereo. Specifically, we investigate prediction targets, flow paths, single-source versus multi-source modeling, and mixture consistency constraints. Beyond the signal-level metrics commonly used for MSS, such as signal-to-distortion ratio, we further evaluate MSS methods using embedding-based metrics, as well as a listening test, providing a comprehensive benchmark. On MUSDB18-HQ, our proposed model exhibits competitive performance in both objective and subjective tests, highlighting the potential of flow matching for generative MSS. Demo: https://ik4sumii.github.io/FLOMSS/
1. INTRODUCTION
MSS has been dominated by discriminative methods, while existing generative approaches do not explicitly learn mixture-to-source mappings in full-band stereo. This paper studies conditional flow matching for four-source stereo MSS and reports competitive results across signal- and embedding-based evaluation.
1. INTRODUCTION
- Earlier generative MSS methods use source priors or conditional inpainting rather than explicitly learning mixture-to-source generation, and do not produce full-band stereo audio.The paper identifies full-band stereo as essential to high-fidelity music production.Source: §1 · p. 1
- Flow matching learns continuous transports by regressing velocity fields along probability paths and has begun to appear in audio separation, but its use in full-band stereo MSS remains unstudied systematically.The paper distinguishes predefined-stem music mixtures from common speech-separation settings involving multiple speakers in monaural mixtures.Source: §1 · p. 1
- The study trains supervised conditional flow-matching models for Vocals, Bass, Drums, and Other on MUSDB18-HQ and evaluates signal-, spectral-, and embedding-based metrics.It also benchmarks against prior discriminative and generative methods.Source: §1 · p. 1
- FLOMSS surpasses prior generative baselines and improves spectral and embedding-based metrics over a same-backbone discriminative baseline while remaining competitive with state-of-the-art discriminative methods.The systematic study covers prediction targets, flow paths, single- and multi-source modeling, and mixture-consistency strategies.Source: §1 · p. 1
2. BACKGROUND
The background formalizes supervised music source separation as predicting individual sources from a mixture. It then introduces flow matching and the conditional-flow-matching parameterizations and paths used in the paper.
2.1. Music Source Separation
- A supervised MSS model predicts source signals from the observed mixture using either source-specific predictors or one joint predictor, with reconstruction objectives such as waveform or STFT losses.The formulation includes time-frequency masking and waveform-to-waveform separators.Source: §2.1 · p. 1
2.2. Flow Matching
- Flow matching trains a continuous normalizing flow by regressing velocity fields along a prescribed probability path; conditional flow matching uses tractable sample-conditioned velocities.The stated conditional objective has the same gradient as the marginal flow-matching objective.Source: §2.2 · pp. 1–2
- CFM can predict either the target velocity or target sample; these parameterizations share a prescribed path but may optimize differently with finite model capacity and data.Source: §2.2 · p. 2
- The study considers optimal-transport-inspired CFM and Schrödinger-bridge-inspired CFM, with the latter using fixed endpoints and nonzero intermediate variance.Source: §2.2 · p. 2
3. MUSIC SOURCE SEPARATION BY CONDITIONAL FLOW MATCHING
FLOMSS formulates full-band stereo source separation as conditional flow matching in a compressed complex-STFT domain. It studies single- and joint-source prediction, flow paths, and mixture-consistency constraints.
3.1. Problem Definition
- MSS is framed as estimating a target source’s conditional distribution from the observed mixture, with a Gaussian path transporting samples from the mixture representation to the source representation.The path uses a linear mean interpolation, and the model estimates its time-dependent velocity field.Source: §3.1 · p. 2
3.2. Single-source Modeling
- Single-source models compare OT-CFM and SB-CFM paths and velocity- versus data-prediction objectives, alongside a same-backbone discriminative baseline.The generative objectives use mixture-conditioned inputs; the discriminative baseline predicts the target directly from the mixture.Source: §3.2 · p. 2
3.3. Multi-source Modeling
- Joint modeling predicts all four sources with one model, using expanded input and output channels, and evaluates soft and hard mixture-consistency constraints.Soft MC adds an MSE penalty to the data-prediction loss; hard MC enforces exact mixture consistency along the probability path in waveform or complex-STFT space.Source: §3.3 · p. 2
4. EXPERIMENTS
Experiments use MUSDB18-HQ stereo audio and compare models with signal, perceptual, distributional, and listening-test measures. The setup specifies the model configurations, training protocol, and evaluation procedures.
4.1. Experimental Setup
- The experiments use a 16M-parameter NCSN++ U-Net backbone, with a 67M-parameter version for scaling, and 10-step solvers for both OT-CFM and SB-CFM.The STFT uses a 1534-sample Hann window and 384-sample hop; OT-CFM uses Midpoint ODE, while SB-CFM uses an exponential integrator.Source: §4.1 · p. 2
- Models train on the official 44.1 kHz stereo MUSDB18-HQ split with online remix augmentation, using 93,600 steps and an effective batch size of 32.Training segments contain 97,920 samples; inference uses 10% overlap and triangular crossfades.Source: §4.1 · pp. 2–3
4.2. Evaluation Metrics
- Evaluation includes spectral and waveform distances, SDR, embedding-based perceptual and distributional metrics, and a WebMUSHRA listening test with 23 participants.The listening test rates ten 8-second excerpts on a 0–100 scale; Fig. 1 tracks target-prediction metrics across ten Midpoint ODE steps for Vocals, with lower values better.Source: §4.2 · p. 3, Figure 1 · p. 3
5. RESULTS
FLOMSS shows competitive benchmark and listening-test results, alongside trade-offs between waveform reconstruction and spectral or embedding-based similarity. Across the experiments, single-source and multi-source variants reveal different quality and efficiency trade-offs.
5.1. Ablation Study with the Single-source Modeling
- The discriminative baseline has the best waveform distance and SDR, while OT-CFM variants reduce Mel and STFT distances and improve most embedding-based metrics.Within OT-CFM, v-prediction scores best on embedding metrics, whereas x-prediction performs better on signal-based metrics.Source: §5.1 · p. 3
5.2. Stepwise Behaviors
- Early Vocal estimates approach the discriminative baseline in Mel distance and SDR, while later estimates generally improve spectral and embedding metrics at the expense of SDR.This reflects a trade-off between spectral and embedding similarity and waveform reconstruction fidelity.Source: §5.2 · pp. 3–4
5.3. Multi-Source Modeling
- A single unconstrained model produces all four stems with a nearly unchanged backbone size, but its separation quality—especially on embedding metrics—is worse than the single-source model.Soft mixture consistency recovers most of the SDR loss while preserving Mel and LPAPS; hard consistency degrades overall results on the evaluated waveform- and STFT-domain paths, except with MERT embeddings.Source: §5.3 · p. 4
5.4. Benchmarking with Prior Methods
- FLOMSS-M improves Mel and FD over SCNet and HTDemucs on average across embeddings, but has weaker SDR and slightly worse LPAPS.FLOMSS-S improves Mel and all embedding-based metrics over its same-backbone discriminative baseline while yielding slightly lower SDR; comparisons involve methods trained on different datasets.Source: §5.4 · p. 4, Table 3 · p. 4
- FLOMSS-M ranked second overall in the listening test, with first place in Bass, second in Vocals, and third in Drums.Overall ranks exclude reference conditions and systems without results for all four sources.Source: §5.4 · p. 4, Table 4 · p. 4
6. CONCLUSION
FLOMSS is an open-source flow-matching framework for full-band stereo music source separation, evaluated on MUSDB18-HQ. Its results show gains on spectral and embedding-based metrics alongside lower SDR than strong discriminative methods, supporting flow matching as a promising direction for generative MSS.
6. CONCLUSION
- On MUSDB18-HQ, FLOMSS surpasses prior generative methods and a same-backbone discriminative baseline on spectral and embedding-based metrics, while SDR remains lower than strong discriminative methods.The study systematically examines prediction targets, flow paths, single- and multi-source modeling, and mixture consistency.Source: §6 · p. 4