PaperlayerView exact revision on arXiv

Source-linked AI summary

Towards End-to-End Synthetic Speech Detection

Guang Hua, Andrew Beng Jin Teoh, Haijian Zhang

arXiv:2106.06341v1eess.AS

TL;DR

Synthetic speech detection has traditionally relied on pre-transforms and hand-crafted features, while increasingly realistic neural speech synthesis creates new detection challenges. This paper replaces that pipeline with lightweight end-to-end waveform DNNs, showing strong ASVspoof2019 performance and promising generalization to unseen data.

  • Problem

    Improved neural speech synthesis has made synthetic voices difficult to distinguish from human speech, while detection has largely relied on feature-extraction and classifier pipelines with information-discarding handcrafted features.

  • Method

    The paper proposes TSSDNet, a lightweight end-to-end neural network that detects synthetic speech directly from waveforms using ResNet- or Inception-style structures and standard DNN components.

  • Results

    TSSDNet achieved better ASVspoof2019 synthetic speech detection results than heavier feature-engineered, pre-transform, and system-fusion approaches, while also generalizing promisingly to an unseen dataset.

  • Takeaways & Limitations

    The findings support the potential of lightweight end-to-end DNNs as an alternative to hand-crafted features for synthetic speech detection.

Abstract

from arXiv · show

The constant Q transform (CQT) has been shown to be one of the most effective speech signal pre-transforms to facilitate synthetic speech detection, followed by either hand-crafted (subband) constant Q cepstral coefficient (CQCC) feature extraction and a back-end binary classifier, or a deep neural network (DNN) directly for further feature extraction and classification. Despite the rich literature on such a pipeline, we show in this paper that the pre-transform and hand-crafted features could simply be replaced by end-to-end DNNs. Specifically, we experimentally verify that by only using standard components, a light-weight neural network could outperform the state-of-the-art methods for the ASVspoof2019 challenge. The proposed model is termed Time-domain Synthetic Speech Detection Net (TSSDNet), having ResNet- or Inception-style structures. We further demonstrate that the proposed models also have attractive generalization capability. Trained on ASVspoof2019, they could achieve promising detection performance when tested on disjoint ASVspoof2015, significantly better than the existing cross-dataset results. This paper reveals the great potential of end-to-end DNNs for synthetic speech detection, without hand-crafted features.

I. INTRODUCTION

Neural speech synthesis has made spoofed speech harder to distinguish from human speech, while detection has largely relied on front-end features and back-end classifiers. The paper questions whether DNNs can learn directly from waveforms instead of using pre-transforms and hand-crafted features.

  • Neural TTS and related attacks such as voice conversion, impersonation, and cloning create new challenges for synthetic speech detection.
  • For nearly a decade, synthetic speech detection has predominantly combined a hand-crafted front-end feature extractor with a back-end binary classifier.
  • CQCC became the strongest hand-crafted feature choice and the ASVspoof2019 baseline, while later systems added subband features, feature fusion, or DNN classifiers.
  • Existing workflows commonly apply a time-frequency transform such as CQT before hand-crafted features or DNN processing.
  • The paper argues that pre-transforms and hand-crafted features can discard information and are not necessary for DNN-based synthetic speech detection.
  • TSSDNet uses lightweight end-to-end DNNs with ResNet-style or Inception-style CNN structures and raw speech waveforms as input.

II. THE PROPOSED MODELS

The proposed models are designed around the hypothesis that synthetic-speech artifacts are subtle and non-semantic, so relatively shallow networks may represent them better than deeper semantic features.

  • Synthetic speech detection targets artifacts left by forgery rather than semantic information.
  • The authors hypothesize that relatively shallow networks are better suited to representing subtle synthetic-speech artifacts.
  • TSSDNet incorporates ResNet skip connections and Inception-style structures into its end-to-end design.

A. Model Structure

Res-TSSDNet and Inc-TSSDNet share a common end-to-end architecture while differing in their repeated ResNet- or Inception-style modules. The Inception-style model additionally uses dilated convolutions to expand receptive fields and control complexity.

  • Res-TSSDNet and Inc-TSSDNet share the same first layer, three final fully connected linear layers, global max pooling, and batch normalization.
  • The models repeat their respective ResNet-style or Inception-style blocks M times, with channel counts CR and CI varying across layers.
  • The study excludes replay-only ASVspoof2017 data and the physical-access portion of ASVspoof2019 because their mechanisms differ from synthesis attacks.
  • Inc-TSSDNet incorporates dilated convolutions to increase receptive field and control model complexity.
  • All convolution layers use SAME padding and stride 1, while pooling stride equals the corresponding kernel size.

B. Training Strategy

Training uses fixed-length raw-waveform examples rather than CQT features, with each example retained or adjusted to six seconds before end-to-end training.

  • Training examples are truncated or repeated to six seconds and directly fed into the networks at a default 16 kHz sample rate.
  • Because convolution layers use SAME padding, pooling layers alone reduce the input feature-vector length.
  • The paper contrasts this fixed-duration treatment with CQCC methods, which are insensitive to recording length because all time slices contribute to classifier training.

2) Weighted Cross-Entropy Loss:

The training uses weighted cross-entropy to address class imbalance, with class weights defined from label frequencies and model selection based on development-set EER.

  • Weighted Cross-Entropy Loss: Weighted cross-entropy is applied because genuine examples are generally less numerous than fake examples.The loss is intended to cope with training-data imbalance.
  • Weighted Cross-Entropy Loss: The loss uses softmax probabilities for two classes and weights each label by its inverse training-set ratio.The class probabilities and inverse-ratio weights are defined explicitly for the loss.
  • Weighted Cross-Entropy Loss: The model with the lowest development-set equal error rate within 100 epochs is selected for evaluation.Training uses Adam with exponential learning-rate decay by a multiplicative factor of 0.95.

3) Mixup Regularization:

Mixup regularization trains the network on interpolated examples and labels, using a weighted combination of cross-entropy losses for the mixed samples.

  • Mixup Regularization: Mixup regularization replaces original training examples and labels with mixed examples and labels to improve generalization capability.The method is introduced as a booster for generalization to unseen attacks.
  • Mixup Regularization: Mixed inputs and labels are formed by interpolating two randomly selected training pairs with coefficient λ.λ is sampled from Beta(α, α), where α is a positive hyperparameter.
  • Mixup Regularization: The mixup loss is λCE(ẑ, y_i) + (1 − λ)CE(ẑ, y_j), combining losses for both original labels.ẑ contains softmax probabilities from the mixed examples.

III. RESULTS

The experiments compare the proposed TSSDNets with benchmark and state-of-the-art methods on ASVspoof2019, including ablation and cross-dataset evaluations. The main results emphasize strong performance from lightweight end-to-end models without feature engineering or fusion.

  • A. Main Results: The main comparison reports EER (%) on the ASVspoof2019 logical-access development and evaluation sets.The comparison includes the proposed networks, benchmarks, and state-of-the-art methods.
  • A. Main Results: The ablation study evaluates Res-TSSDNet and Inc-TSSDNet variants using ASVspoof2019 logical-access evaluation EER.The experiments vary network depth or width, and for Res-TSSDNet also examine use of the skip connection.
  • A. Main Results: The proposed Res-TSSDNet is a single end-to-end network without fusion or feature engineering and achieves the overall lowest evaluation EER by a clear margin.It has less than half the trainable weights of the model in and about one-tenth those of the model in.
  • A. Main Results: The Inc-TSSDNet has only 0.09M parameters while achieving an EER lower than those of the heavy models in –.This result highlights the performance of an extremely lightweight proposed model.

B. Ablation Study

The ablation study shows that moderate network depth or width is preferable, while skip connections and training choices materially affect EER. Model configuration and run-to-run variation both matter for performance.

  • B. Ablation Study: Moderate depth or width provides the best operating point for both Res-TSSDNet and Inc-TSSDNet.Going shallower or deeper raises EER, and the Inc-TSSDNet also favors moderate width.
  • B. Ablation Study: 1.05% EER reduction results from adding the ResNet skip connection.The comparison is against the corresponding network without the skip connection.
  • B. Ablation Study: Over 30 training runs, both models’ EERs remain within bounded ranges except one Res-TSSDNet evaluation outlier above 4%.Inc-TSSDNet has tighter development EERs, but higher evaluation EERs than Res-TSSDNet.
  • B. Ablation Study: Relatively lighter models offer the recommended trade-off between accuracy and efficiency.The recommendation follows from substantial intra-model variation alongside configuration differences.
  • B. Ablation Study: Training-example duration matters: 5-second truncation slightly degrades performance, whereas 2-second truncation drastically increases evaluation EER.Other tested choices include activation functions, first-layer kernel size, pooling, and cross-entropy weighting.

C. Cross-Dataset Testing

Cross-dataset testing trains models on ASVspoof2019 and evaluates them on ASVspoof2015, exposing weak transfer for several conventional systems. Mixup improves Res-TSSDNet transfer, while Inc-TSSDNet achieves strong generalization, including a 1.96% evaluation EER.

  • C. Cross-Dataset Testing: ASVspoof2019-trained systems are tested on the disjoint ASVspoof2015 development and evaluation sets.The experiments target transfer from newer synthesis methods to the older ASVspoof2015 data.
  • C. Cross-Dataset Testing: LFCC- and CQCC-based GMMs are generally inconsistent with ASVspoof2015, and the best ASVspoof2019 Res-TSSDNet also fails to generalize well.The reported cross-dataset EERs indicate nearly indistinguishable detection for these systems.
  • C. Cross-Dataset Testing: Mixup reduces Res-TSSDNet cross-dataset EERs to below 2% while slightly sacrificing performance on the original dataset.The reduction is observed as the mixup level α is increased.
  • C. Cross-Dataset Testing: 1.96% eval EER is achieved by the M = 5, 8-branch Inc-TSSDNet, the best reported cross-dataset result among the tested models.The result is reported for ASVspoof2019 training and ASVspoof2015 evaluation.
  • C. Cross-Dataset Testing: CQT-based features in prior work produced EERs greater than 20% under the same cross-dataset direction.The comparison is cited as evidence that the proposed cross-dataset performance is substantially better than existing results.

IV. CONCLUSION

The paper concludes that lightweight end-to-end neural networks can replace pre-transforms and hand-crafted features for synthetic speech detection. They achieve strong ASVspoof2019 performance and generalize to unseen datasets.

  • IV. CONCLUSION: Lightweight end-to-end networks achieve the paper’s best synthetic speech detection results without the conventional front-end and back-end pipeline.The approach uses raw speech waveform rather than pre-transform inputs and hand-crafted features.
  • IV. CONCLUSION: The proposed networks reduce ASVspoof2019 evaluation EER relative to heavier systems using pre-transforms, hand-crafted features, classifiers, or system fusion.The conclusion describes the reduction as a clear margin.
  • IV. CONCLUSION: Cross-dataset testing shows that the proposed networks can generalize to an unseen dataset.The conclusion relates this finding to evaluation beyond the training distribution.
Loading 2106.06341v1…