PaperlayerView exact revision on arXiv

Source-linked AI summary

Deep neural networks for single channel source separation

Emad M. Grais, Mehmet Umut Sen, Hakan Erdogan

arXiv:1311.2746v1cs.NEcs.LG

TL;DR

Single-channel source separation has limited mixture information, and earlier classifier-based methods assign hard time-frequency masks. This paper uses a trained DNN to assess estimated source spectra within an energy-minimization separation framework, initialized by NMF; it reports better separation than regular NMF across tested speech-to-music ratios.

  • Problem

    Single-channel source separation is challenging because the mixture contains limited information, and prior classifier-based separation used hard time-frequency masks rather than checking estimated source spectra.

    Source: §1 · p. 1, §1.1 · p. 1

  • Method

    A trained joint DNN scores estimated source spectra within an energy-minimization framework that fits source models and reconstructs the mixture, with NMF providing initialization.

    Source: §1.2 · p. 1

  • Results

    Across speech-to-music ratios from -5 to 5 dB, DNN outperformed regular NMF, with SDR and SNR improvements usually around 1-1.5 dB and SIR improvement up to 3 dB.

    Source: §5 · p. 4

  • Takeaways & Limitations

    The approach is reported to work regardless of energy-scale differences between source signals in training and separation.

    Source: §1.2 · p. 1

    The magnitude-spectrum formulation ignores STFT phase angles and approximates the mixture magnitude as the sum of source magnitudes.

    Source: §2 · p. 2

Abstract

from arXiv · show

In this paper, a novel approach for single channel source separation (SCSS) using a deep neural network (DNN) architecture is introduced. Unlike previous studies in which DNN and other classifiers were used for classifying time-frequency bins to obtain hard masks for each source, we use the DNN to classify estimated source spectra to check for their validity during separation. In the training stage, the training data for the source signals are used to train a DNN. In the separation stage, the trained DNN is utilized to aid in estimation of each source in the mixed signal. Single channel source separation problem is formulated as an energy minimization problem where each source spectra estimate is encouraged to fit the trained DNN model and the mixed signal spectrum is encouraged to be written as a weighted sum of the estimated source spectra. The proposed approach works regardless of the energy scale differences between the source signals in the training and separation stages. Nonnegative matrix factorization (NMF) is used to initialize the DNN estimate for each source. The experimental results show that using DNN initialized by NMF for source separation improves the quality of the separated signal compared with using NMF for source separation.

1. INTRODUCTION

Single-channel source separation is challenging because the mixture contains limited information, motivating source models trained on data. The paper proposes a joint DNN-based approach that checks estimated source spectra during separation and uses NMF for initialization.

1. INTRODUCTION

  • Limited information in a single-channel mixture makes source separation challenging, so source-specific training data are commonly used to model sources.The paper introduces nonlinear source models trained with a deep neural network.Source: §1 · p. 1

1.1. Related work

  • Probabilistic source models usually assume training and mixture sources have the same energy levels, while dictionary-only models can admit estimates that reduce separation quality.Combining dictionaries with cluster models was introduced to address energy scaling and model fit.Source: §1.1 · p. 1

1.2. Contributions

  • The proposed joint DNN classifies estimated spectra rather than assigning hard time-frequency masks, encouraging source-model fit and mixture reconstruction through energy minimization.NMF supplies initial source estimates, and the approach is designed to handle energy differences between training and separation.Source: §1.2 · p. 1

2. PROBLEM FORMULATION

The paper formulates two-source, single-channel separation in the STFT domain. It estimates the unknown source magnitude spectrograms from the mixture and training data, approximating their sum after ignoring phase.

2. PROBLEM FORMULATION

  • The formulation assumes two sources mixed on one channel and represents the mixture in the STFT domain, where t indexes frames and f indexes frequency.The source STFTs are unknown and must be estimated.Source: §2 · p. 2
  • Because phase angles are ignored, the magnitude spectrum of the mixture is approximated as the sum of the source magnitude spectra.This approximation is presented within the cited framework.Source: §2 · p. 2
  • In matrix form, the unknown source magnitude spectrograms must be estimated using the observed mixture and training data.The paper introduces this matrix representation after stating the magnitude-spectrum approximation.Source: §2 · p. 2

3. NMF FOR SUPERVISED SOURCE SEPARATION

The section presents supervised NMF as a way to obtain source estimates for initializing the DNN separation method. It fits nonnegative spectrograms with dictionaries and gains, then applies the trained dictionaries to the mixture.

3. NMF FOR SUPERVISED SOURCE SEPARATION

  • NMF represents a nonnegative spectrogram matrix as a dictionary multiplied by a gain matrix, with the factors fitted by minimizing Itakura-Saito divergence.The basis vectors are optimized so the data can be approximated by linear combinations of dictionary columns.Source: §3 · p. 2
  • The IS-NMF factors are updated alternately and multiplicatively; the factors are usually initialized with positive random values.The cited update description specifies element-wise operations for multiplication, division, and squaring.Source: §3 · p. 2
  • For initialization, NMF decomposes each source's training frames, holds the trained dictionaries fixed for the mixture, estimates gains, and forms source magnitude estimates for the DNN.The resulting magnitude spectrogram estimates initialize the sources in the DNN separation stage.Source: §3 · p. 2

4. THE METHOD

The method replaces NMF’s nonnegative-cone source model with a DNN that scores source spectra, then estimates source spectra and gains by energy minimization. The estimates are encouraged to fit the DNN, sum to the mixture, and remain nonnegative.

4. THE METHOD

  • NMF models sources in a nonnegative dictionary cone, motivating nonlinear source models to better accommodate source variability.The method uses a DNN to model source spectra.Source: §4 · p. 2

4.1. Training the DNN

  • The DNN classifies normalized magnitude-spectrum frames with one output per source, using RBM initialization followed by supervised backpropagation.Its output scores measure how much separated spectra carry each source’s characteristics.Source: §4.1 · pp. 2–3

4.2. Source separation using DNN and energy minimization

  • The separation objective jointly estimates source spectra and gains, encouraging DNN fit, mixture reconstruction, and nonnegative magnitude spectra.DNN-fit costs are combined with the error between the mixture and the weighted sum of source estimates.Source: §4.2 · p. 3
  • Because the optimization is non-convex, source spectra and gains are initialized from NMF estimates before energy minimization.The source spectra are normalized by their ℓ2 norms, and gains are initialized from the NMF estimates’ norms relative to the mixture.Source: §4.2 · p. 3
  • The optimized parameters are used in a Wiener filter to produce final source-spectrum estimates.Source: §4.2 · p. 3

5. EXPERIMENTS AND DISCUSSION

Across tested speech-to-music ratios, DNN with energy minimization improved separation over regular NMF, and using three neighboring frames improved results over single-frame input. Experiments used TIMIT speech and piano recordings.

5. EXPERIMENTS AND DISCUSSION

  • Tests used TIMIT male speech and 39 piano pieces from one artist for training, with one piano piece held out; mixtures combined test music with 20 test speech files.The audio was sampled at 16 kHz, and the piano training set totaled approximately 185 minutes.Source: §5 · p. 4
  • Separation was evaluated using average SDR and SIR over 20 test utterances, with output SNR as an additional criterion.Source: §5 · p. 4
  • Using L = 3 neighboring spectral frames improved results over single-frame DNN input; the third hidden layer used 500 nodes instead of 200.The authors conjecture that more neighboring frames could yield better results.Source: §5 · p. 4
  • Across SMR -5 to 5 dB, DNN with energy minimization beats regular NMF: SDR and SNR improvements are usually around 1-1.5 dB, while SIR improvement can be as high as 3 dB.The authors associate the SIR improvement with decreased remaining music in reconstructed speech.Source: Table 1 · p. 4, Table 2 · p. 4, §5 · p. 4

6. CONCLUSION

The paper uses a trained DNN to model each source within an energy-minimization approach to single-channel source separation. The framework also estimates each source’s scale in the mixture, and the authors identify parameter adjustments and alternative DNN architectures as directions for improvement.

6. CONCLUSION

  • The trained DNN guides source separation in an energy-minimization framework while the method estimates each source’s scale in the mixture.Source-signal training data are used to train the DNN.Source: §6 · p. 4
  • The authors identify model-parameter adjustments as a way to improve the proposed single-channel source separation approach.Source: §6 · p. 4
  • Deep autoencoders and deep recurrent neural networks are proposed for future testing, with the latter able to handle temporal structure in source signals.Source: §6 · p. 4
  • The authors anticipate that further work may improve the approach’s performance.Source: §6 · p. 4
Loading 1311.2746v1…