Source-linked AI summary
DAMSEP: Distance-Aware Monaural Source Separation using Multi-RIR Estimation
Wen Wen, Qiang Zhou, Yu Xi, Haoyu Li, Bohan Li, Kai Yu
TL;DR
Monaural source separation typically recovers audio without estimating source-specific RIRs, leaving distance cues unmodeled. DAMSEP jointly estimates sources and their RIRs from a single-microphone mixture; on HETMIXR, it outperforms evaluated baselines in separation, RIR estimation, and distance ordering.
Problem
Monaural separation models typically recover audio without estimating source-specific RIRs, although those responses encode cues to relative source distance.
Source: §1 · p. 1Method
DAMSEP jointly estimates separated sources and source-specific RIRs from a single-microphone mixture using source supervision and reverberant reconstruction.
Source: §1 · p. 1Results
On HETMIXR, DAMSEP achieves better separation, more accurate RIR estimation, and higher distance-ordering accuracy than evaluated baselines.
Source: §5 · p. 4Takeaways & Limitations
The DRRs of estimated source-specific RIRs provide cues for determining the relative near/far order of separated sources.
Source: §1 · p. 1
Abstract
from arXiv · showhide
Although room impulse responses (RIRs) encode source-distance cues, conventional monaural source separation focuses on recovering audio content without estimating source-specific RIRs, losing the associated spatial information. To address this limitation, we propose Distance-Aware Monaural Source Separation using Multi-RIR Estimation (DAMSEP), the first end-to-end framework that is jointly trained for source separation and multi-source RIR estimation from a single-microphone mixture. DAMSEP integrates a separation backbone with shared dereverberation and RIR estimation modules to jointly recover clean sources and source-specific complex convolutive transfer functions under source estimation and reverberant reconstruction objectives, enabling relative near/far ordering through the direct-to-reverberant ratios of the corresponding RIRs. For comprehensive evaluation, we introduce HETMIXR, which spans heterogeneous source content and diverse simulated room conditions with source-specific RIRs and geometric distance annotations. Experiments on HETMIXR demonstrate superior performance in source separation, RIR estimation, and distance ordering. Ablation studies reveal the complementary benefits of source supervision and reverberant reconstruction, while additional evaluations show generalization to single-speaker inputs and mixtures generated using measured RIRs from an unseen room. Our code and dataset are available at https://github.com/Wenanzhi/DAMSEP.
1. INTRODUCTION
Monaural separation typically recovers audio without estimating source-specific room responses, leaving relative distance information unmodeled. DAMSEP jointly estimates sources and their RIRs, and its evaluations report gains across separation, RIR estimation, and distance ordering.
1. INTRODUCTION
- Conventional monaural separation generally omits source-specific RIR estimation, while blind RIR estimators are not designed to recover distinct responses from overlapping mixtures.Cascaded estimation can propagate separation errors, and independent training prevents RIR supervision from guiding the separator.Source: §1 · p. 1
- Experiments on HETMIXR show DAMSEP outperforms task-specific baselines in separation, RIR estimation, and distance ordering.Source: §1 · p. 1
- DAMSEP jointly recovers sources and associated RIRs from a single-microphone mixture, using source supervision and source-specific reverberant reconstruction.The estimated RIRs’ DRRs provide cues for relative near/far ordering.Source: §1 · p. 1
- HETMIXR pairs reverberant mixtures with source signals, RIRs, and geometric source-microphone distances across diverse source types.Source: §1 · p. 1
2. DISTANCE-AWARE SEPARATION VIA MULTI-RIR ESTIMATION
DAMSEP jointly estimates clean sources and source-specific room responses from a monaural mixture. Its shared modules and training objectives support reverberant reconstruction, while estimated RIRs provide relative near/far ordering.
2.1. Problem Formulation
- DAMSEP estimates each source’s clean spectrum, reverberant spectrum, and associated complex convolutive transfer function, focusing primarily on two-source mixtures.The clean waveform is obtained from the clean spectrum, and the time-domain RIR is recovered from the predicted CTF.Source: §2.1 · p. 2
2.2. DAMSEP Architecture
- A separation module predicts reverberant spectra, then shared dereverberation and fusion modules produce clean spectra and source-specific CTFs.The CTF decoder maps the fused response representation to each source’s CTF.Source: §2.2 · p. 2
2.3. Joint Training and Distance-Aware Inference
- PIT selects a source assignment from the clean-source waveform negative-SNR objective, and that permutation is shared across clean-source, reverberant-source, and reconstruction losses.The reconstruction loss compares reverberant targets with CTF-filtered clean references.Source: §2.3 · p. 2
- At inference, each predicted CTF is converted to a time-domain RIR using a synthetic sweep and inverse filtering; the RIRs’ DRRs provide relative near/far ordering.The DRR comparison yields ordinal ordering rather than an absolute distance estimate.Source: §2.3 · pp. 2–3
3. EXPERIMENTAL SETUP
The experimental setup introduces HETMIXR and a measured-RIR test set, then specifies task-specific comparison protocols, implementation settings, and metrics for separation, RIR fidelity, and distance ordering.
3.1. Datasets and Evaluation Scenarios
- HETMIXR provides heterogeneous two-source reverberant mixtures with source-specific RIRs, geometric distances, and train/validation/test splits; a separate measured-RIR set tests zero-shot evaluation in an unseen room.HETMIXR uses WSJ0 speech, music, TV audio, and NCSSD dialogue; 2,801 test mixtures remain after excluding clips shorter than 4 seconds.Source: §3.1 · p. 3
3.2. Baselines and Comparison Protocols
- DAMSEP is compared against retrained separation and RIR-estimation baselines, while distance ordering uses oracle inputs or DAMSEP-separated reverberant signals for cascade comparisons.Oracle estimators receive ground-truth reverberant sources; cascade estimators receive separated reverberant sources, allowing comparison with DAMSEP’s jointly trained response branch.Source: §3.2 · p. 3
3.3. Implementation details
- DAMSEP is trained on 4-second segments at 8 kHz using a 256-sample Hann window, 64-sample hop, 256-point FFT, and 60 CTF taps.Training uses Adam with an initial learning rate of 10^-3, loss weights λrvb = 0.1 and λrecon = 0.5, and early stopping.Source: §3.3 · p. 3
3.4. Evaluation Metrics
- Evaluation measures separation improvement and interference/artifacts, RIR waveform and spectral fidelity, and near/far ordering accuracy against geometric distances.D-Acc. is the percentage of source pairs whose predicted DRR-based order matches the geometric reference; ground-truth-RIR ordering agrees for all pairs in both test sets.Source: §3.4 · p. 3
4. RESULTS AND ANALYSIS
DAMSEP performs strongly across separation, source-specific RIR estimation, and distance ordering on HETMIXR. Ablations and additional evaluations examine objective contributions, single-source inputs, and transfer to measured-RIR mixtures from an unseen room.
4.1. Source Separation Performance
- DAMSEP achieves the best HETMIXR performance across all four separation metrics and improves SI-SDRi by 0.65 dB over TF-Locoformer; on WHAMR!, its separation is comparable to SPMamba.The comparison evaluates separation while DAMSEP also estimates source-specific RIRs.Source: §4.1 · pp. 3–4
4.2. Simulated RIR Estimation
- DAMSEP has lower RIR-50, LSD, DRR error, and C50 error than the compared estimators, and the highest distance-ordering accuracy, including over cascade baselines.The results cover response reconstruction, acoustic-parameter estimation, and relative distance ordering on HETMIXR.Source: Table 2 · p. 4, §4.2 · p. 4
- Trained only on two-source mixtures, DAMSEP achieves better response and spectral errors, waveform correlation, and DRR and C50 estimates than external estimators on identical single-source inputs.It is evaluated without retraining or fine-tuning.Source: §4.2 · p. 4, Table 3 · p. 4
4.3. Ablation Study
- Adding reconstruction supervision improves separation, DRR and C50 errors, and distance ordering; retaining it, the reverberant-source objective provides further gains, indicating complementary objectives.Without reconstruction supervision, CTF-specific parameters receive no task gradients, and relatively high ordering accuracy does not imply accurate RIR recovery.Source: Table 4 · p. 4, §4.3 · p. 4
4.4. Zero-shot Evaluation on the Measured-RIR Test Set
- On measured-RIR mixtures from an unseen room, DAMSEP achieves the lowest RIR-50 and LSD and highest RIR correlation, with more pronounced gains in distance ordering over cascade baselines.DAMSEP was trained exclusively on simulated data; its RIR-fidelity gains modestly outperform Rec-RIR.Source: Table 5 · p. 4, §4.4 · p. 4
5. CONCLUSIONS
DAMSEP jointly separates sources and estimates a source-specific RIR from monaural multi-speaker mixtures. On HETMIXR it outperforms evaluated baselines in separation, RIR estimation, and distance ordering, with further evaluations on single-speaker inputs and measured-RIR mixtures from an unseen room.
5. CONCLUSIONS
- DAMSEP jointly separates sources and estimates each source's RIR from a monaural multi-speaker mixture, with better separation, RIR estimation, and distance-ordering results than evaluated baselines.The estimated RIRs' DRRs provide relative near/far order; further evaluations report RIR recovery on single-speaker inputs and zero-shot transfer to mixtures rendered with measured RIRs from an unseen room.Source: §5 · p. 4
6. COMPLIANCE WITH ETHICAL STANDARDS
The study uses public audio datasets and simulated and measured room impulse responses; it recruited no human participants and conducted no animal experiments.
6. COMPLIANCE WITH ETHICAL STANDARDS
- No new human participants were recruited or recorded, and no animal experiments were conducted.The source datasets were used under their access and licensing terms, and no additional institutional ethical approval was required.Source: §6 · p. 5