PaperlayerView exact revision on arXiv

Source-linked AI summary

Speaker Recognition from Raw Waveform with SincNet

Mirco Ravanelli, Yoshua Bengio

arXiv:1808.00158v3eess.AScs.LGcs.SDeess.SP

TL;DR

Speaker recognition systems have largely relied on i-vectors and hand-crafted features, which may hinder extraction of narrow-band speaker characteristics. SincNet constrains waveform filters through parametrized sinc functions, and experiments show faster convergence, better performance than standard CNNs, and superiority to i-vector systems under the considered setting.

  • Problem

    Hand-crafted features may hinder extraction of narrow-band speaker characteristics such as pitch and formants, motivating waveform-based speaker recognition.

  • Method

    SincNet processes waveforms with parametrized sinc band-pass filters whose learned parameters are only the low and high cutoff frequencies.

  • Results

    SincNet converges faster and achieves better end-task performance than a standard CNN, while also outperforming an i-vector system under the considered experimental setting.

  • Takeaways & Limitations

    SincNet learns filters tuned to speaker characteristics such as pitch and formants and is more computationally efficient through filter symmetry.

  • Takeaways & Limitations

    The study targeted speaker recognition only, with evaluation under a specific experimental setting and no bounds imposed to force f2 below the Nyquist frequency.

Abstract

from arXiv · show

Deep learning is progressively gaining popularity as a viable alternative to i-vectors for speaker recognition. Promising results have been recently obtained with Convolutional Neural Networks (CNNs) when fed by raw speech samples directly. Rather than employing standard hand-crafted features, the latter CNNs learn low-level speech representations from waveforms, potentially allowing the network to better capture important narrow-band speaker characteristics such as pitch and formants. Proper design of the neural network is crucial to achieve this goal. This paper proposes a novel CNN architecture, called SincNet, that encourages the first convolutional layer to discover more meaningful filters. SincNet is based on parametrized sinc functions, which implement band-pass filters. In contrast to standard CNNs, that learn all elements of each filter, only low and high cutoff frequencies are directly learned from data with the proposed method. This offers a very compact and efficient way to derive a customized filter bank specifically tuned for the desired application. Our experiments, conducted on both speaker identification and speaker verification tasks, show that the proposed architecture converges faster and performs better than a standard CNN on raw waveforms.

1. INTRODUCTION

SincNet addresses weaknesses in waveform-based CNNs by constraining the first layer to learn compact band-pass filters through cutoff frequencies. Across challenging speaker-recognition conditions, it converges faster and performs better than standard CNNs while also outperforming an i-vector system.

  • Motivation: Hand-crafted features such as FBANK and MFCC may smooth narrow-band speaker cues, motivating direct processing of spectrograms or raw waveforms.The cited motivation highlights pitch and formants as potentially important characteristics.
  • Motivation: The first convolutional layer is critical because standard waveform CNNs can learn noisy, incongruous multi-band filters, especially with limited training data.These filters may not yield an efficient speech representation.
  • SincNet approach: SincNet constrains the first-layer filters to parametrized sinc functions implementing band-pass filters, learning only low and high cutoff frequencies.This leaves flexibility while focusing learning on parameters with broad effects on filter shape and bandwidth.
  • Experiments: The experiments use minimal speaker training data of 12-15 seconds and short test sentences lasting 2 to 6 seconds.The paper characterizes these conditions as challenging but realistic.
  • Results: Across varied datasets, SincNet converges faster and achieves better end-task performance than a standard CNN.The experiments evaluate the proposed architecture under the challenging conditions described above.
  • Results: Under the considered experimental setting, SincNet also outperforms a traditional speaker-recognition system based on i-vectors.

2. THE SINCNET ARCHITECTURE

SincNet replaces unconstrained first-layer waveform filters with parametrized sinc band-pass filters whose learned cutoff frequencies define a compact, interpretable filter bank. The architecture remains differentiable and supports standard CNN components after the sinc-based convolution.

  • Filter design: The first layer of a standard CNN performs time-domain convolutions with FIR filters whose L taps are learned directly from data.SincNet instead uses a predefined function g that depends on relatively few learnable parameters.
  • Filter design: SincNet constructs rectangular band-pass filters in the frequency domain as the difference between two low-pass filters, then obtains the time-domain function using the inverse Fourier transform.The cutoff frequencies f1 and f2 are learned, while rect(·) denotes the rectangular magnitude-frequency function.
  • Filter design: SincNet convolves waveforms with parametrized sinc functions implementing band-pass filters, learning only low and high cutoff frequencies rather than every filter tap.The learned cutoff frequencies are f1 and f2; the sinc function is defined as sinc(x) = sin(x)/x.
  • Filter implementation: The truncated sinc filters are windowed to reduce edge discontinuities, and the implementation uses a Hamming window for high frequency selectivity.The authors report no significant performance difference when using Hann, Blackman, or Kaiser windows; symmetric filters avoid phase distortions and permit efficient computation.
  • Network integration: SincNet cutoff frequencies are jointly optimized with the rest of the network using gradient-based learning, after which standard pooling, normalization, activation, dropout, and additional layers can be applied.The architecture can stack standard convolutional, fully connected, or recurrent layers before speaker classification with a softmax classifier.
  • Model properties: SincNet reduces first-layer parameters from F · L in a standard CNN to 2F, yielding 8k versus 160 parameters when F = 80 and L = 100.Doubling filter length doubles the standard CNN count but leaves SincNet's count unchanged; the compact design is suitable for few-sample regimes and produces more interpretable feature maps.

3. RELATED WORK

Prior waveform and spectrogram approaches seek richer speech representations than hand-crafted features, while this study applies sinc filters directly to raw waveforms for speaker recognition.

  • Spectrogram-based CNN systems retain more information than hand-crafted features but require tuning frame and frequency-bin hyper-parameters.These include frame duration, overlap, window typology, and the number of frequency bins.
  • SincNet is presented as the first study to demonstrate sinc filters for time-domain audio processing from raw waveforms with CNNs.The application focus is speaker recognition rather than speech recognition.
  • The compact filters are intended for speaker recognition with few seconds of training data per speaker and short test sentences.The paper describes this as a realistic evaluation scenario.

4. EXPERIMENTAL SETUP

The experiments evaluate SincNet across multiple speaker-recognition corpora and compare it with raw-waveform CNNs, hand-crafted-feature systems, and an i-vector baseline using specified classification and verification setups.

  • Experiments use publicly available data, including TIMIT and Librispeech, to evaluate speaker recognition across corpora with different numbers of speakers.TIMIT contains 462 speakers and Librispeech 2484 speakers in the stated setup.
  • Speech sentences are split into 200 ms waveform chunks with 10 ms overlap before processing by SincNet.The first layer uses 80 sinc filters of length L = 251 samples, followed by two convolutional layers with 60 filters of length 5.
  • Frame-level speaker classification uses a softmax classifier, while sentence-level classification averages frame predictions and selects the speaker with the highest average posterior.
  • Speaker verification is evaluated with either cosine distance between last-hidden-layer d-vectors or the softmax posterior for the claimed identity.The latter setup is denoted DNN-class.
  • Baselines include a standard raw-waveform CNN, CNN and MLP systems using FBANK and MFCC features, and an i-vector system for speaker verification.The standard CNN uses the same architecture as SincNet but replaces sinc-based convolution with standard convolution.

5. RESULTS

Experiments compare SincNet with standard CNNs and other speaker-recognition systems on identification and verification tasks. SincNet learns more meaningful frequency-selective filters, converges faster, and achieves stronger reported performance.

  • 5.1. Filter Analysis: SincNet learns rectangular band-pass filters, whereas standard CNN filters can be noisy or multi-band.Filter inspections on Librispeech show clearer frequency responses for SincNet.
  • 5.1. Filter Analysis: SincNet’s learned frequency bands align with pitch and first- and second-formant regions, unlike the standard CNN’s less meaningful pattern.The main SincNet peaks occur near pitch, 500 Hz, and 900–1400 Hz.
  • 5.1. Filter Analysis: SincNet filters are, on average, more selective than CNN filters, possibly capturing narrow-band speaker clues more effectively.
  • 5.2. Speaker Identification: 33.0% FER for SincNet versus 37.7% for the CNN baseline on TIMIT, with SincNet decreasing error faster over training epochs.The comparison uses Frame Error Rate (FER%).
  • 5.2. Speaker Identification: SincNet outperforms competing systems on TIMIT and Librispeech classification, including a 4% relative improvement on Librispeech with convergence at 1200 versus 1800 epochs.Standard FBANKs are comparable on TIMIT but significantly worse on Librispeech.
  • 5.3. Speaker Verification: SincNet achieves about an 11% relative performance improvement over the standard CNN in Librispeech speaker verification.All DNN models achieve EER below 1% in the reported verification experiments.
  • 5.3. Speaker Verification: Under the study’s challenging conditions, the best i-vector system reaches EER=1.1%, while neural networks achieve better generalization.The conditions use minimal training material and short test sentences.

6. CONCLUSIONS AND FUTURE WORK

The paper concludes that SincNet is an efficient waveform-processing architecture whose constrained filters improve convergence and speaker-recognition performance. It also identifies broader time-series applications as future work.

  • 6. CONCLUSIONS AND FUTURE WORK: SincNet directly processes waveform audio using efficiently parameterized constraints on filter shapes.
  • 6. CONCLUSIONS AND FUTURE WORK: SincNet shows performance benefits across the considered speaker identification and verification corpora.
  • 6. CONCLUSIONS AND FUTURE WORK: SincNet improves convergence speed and computational efficiency over a standard CNN through exploitation of filter symmetry.
  • 6. CONCLUSIONS AND FUTURE WORK: Learned filters are tuned to extract speaker characteristics including pitch and formants.
  • 6. CONCLUSIONS AND FUTURE WORK: Future work will evaluate SincNet on VoxCeleb and extend it to speech, emotion, separation, and music-processing tasks.
Loading 1808.00158v3…