PaperlayerView exact revision on arXiv

Source-linked AI summary

CompSpoof: A Dataset and Joint Learning Framework for Component-Level Audio Anti-spoofing Countermeasures

Xueping Zhang, Yechen Wang, Linxi Li, Liwei Jin, Ming Li

arXiv:2509.15804v2cs.SDeess.AS

TL;DR

Component-level spoofing manipulates only speech or environmental sound, while other components remain genuine, a setting not covered by conventional whole-utterance or partial-time-span approaches. CompSpoof pairs a dataset spanning combinations of genuine and spoofed components with a separation-enhanced joint-learning framework. The authors report that the framework outperforms baselines.

  • Problem

    Existing anti-spoofing approaches generally assume an utterance is entirely genuine or spoofed, and partial-spoof settings address time spans rather than evaluating the genuineness of separate audio components.

    Source: §1 · p. 1

  • Method

    The paper constructs CompSpoof and uses a separation-enhanced joint framework that separates speech and environmental sound for dedicated anti-spoofing models trained to retain spoof-relevant information.

    Source: §1 · pp. 1–2

  • Results

    The proposed method outperforms the baseline, with experiments highlighting component separation and separate spoof detection for each component.

    Source: §1 · p. 2, §5 · p. 4

  • Takeaways & Limitations

    The paper’s results underscore the effectiveness of component separation and joint learning for component-level audio anti-spoofing.

    Source: §5 · p. 4

Abstract

from arXiv · show

Component-level audio Spoofing (Comp-Spoof) targets a new form of audio manipulation where only specific components of a signal, such as speech or environmental sound, are forged or substituted while other components remain genuine. Existing anti-spoofing datasets and methods treat an utterance or a segment as entirely bona fide or entirely spoofed, and thus cannot accurately detect component-level spoofing. To address this, we construct a new dataset, CompSpoof, covering multiple combinations of bona fide and spoofed speech and environmental sound. We further propose a separation-enhanced joint learning framework that separates audio components apart and applies anti-spoofing models to each one. Joint learning is employed, preserving information relevant for detection. Extensive experiments demonstrate that our method outperforms the baseline, highlighting the necessity of separate components and the importance of detecting spoofing for each component separately. Datasets and code are available at: https://github.com/XuepingZhang/CompSpoof.

1. INTRODUCTION

Component-level spoofing forges or substitutes speech or environmental sound while leaving other signal components genuine, a setting not addressed by whole-utterance or partial-time-span approaches. The paper introduces a component-spoofing dataset and a separation-enhanced joint framework, reporting that it outperforms the baseline.

  • Component-level spoofing can forge speech while retaining genuine environmental sound, or alter the environment while preserving speech, making manipulation harder to detect and more realistic to listeners.Source: §1 · p. 1
  • Existing systems commonly classify whole utterances as genuine or spoofed, while partial-spoof datasets address time spans rather than the genuineness of separate components.Source: §1 · p. 1
  • The framework couples separation with dedicated speech and environmental-sound anti-spoofing models, jointly trained to preserve spoof-relevant information in separated signals.Source: §1 · pp. 1–2
  • CompSpoof is introduced as a component-level dataset covering diverse combinations of genuine and spoofed speech and environmental audio.The dataset contains about 2,500 utterances across five categories.Source: §1 · p. 1
  • Experiments show the proposed method outperforms the baseline, underscoring the importance of separating components and detecting spoofing for each one.Source: §1 · p. 2

2. COMPSPOOF DATASET

CompSpoof provides a balanced five-class dataset for studying component-level anti-spoofing, assembled from mixed and original audio sources. The paper also specifies its signal processing and stratified data split.

  • CompSpoof contains 2,500 audio samples, evenly distributed across five classes with 500 samples per class.Source: §2 · p. 2
  • Mixed speech and environmental audio come from named bona fide and spoofed datasets, while the original-audio portion uses authentic VGGSound recordings with simultaneous speech and environmental sound.Audio durations range from 5 to 21 seconds; sampled environments include indoor, street, and natural settings.Source: §2 · p. 2
  • Files are resampled to 16 kHz, longer signals are truncated to match the shorter signal, and environmental sound is adjusted to a predefined SNR relative to speech.Source: §2 · p. 2
  • Stratified sampling maintains class balance across training, development, and evaluation sets at a 70%, 10%, and 20% ratio.Source: §2 · p. 2

3. SEPARATION-ENHANCED JOINT LEARNING FRAMEWORK

The framework extends a five-class baseline with explicit speech and environmental-sound separation, dedicated component detectors, and joint training. Its design aims to retain spoof-relevant information while separating components.

3.1. Baseline

  • The baseline extends XLSR-AASIST from binary bona fide/spoofed classification to five classes, but does not disentangle speech and environmental components.This may cause confusion when only one component is spoofed.Source: §3.1 · p. 2

3.2. Separation-Enhanced Joint Learning Framework

  • The framework uses a mixture detector, UNet-based separator, separate speech and environmental-sound detectors, and joint learning.The mixture detector makes a binary decision before component-wise detection.Source: §3.2 · p. 3
  • A UNet predicts a complex speech mask in the STFT domain, while an adaptive soft-mask estimates environmental sound from the residual.The environmental mask suppresses speech leakage to reduce the risk of residual speech being misclassified as environmental sound.Source: §3.2 · p. 3
  • Joint training combines separation, mixture-detection, component-classification, and consistency losses so separated signals retain spoof-relevant features.The consistency loss compares predictions on separated components with predictions on the reference waveform; inference combines mixture and component decisions into five-class predictions, with majority voting across chunks for a file-level label.Source: §3.2 · p. 3
  • SEF+JL consistently outperforms the baseline and SEF, especially for mixed-content classes.Table 2 reports classification performance on dev and eval sets.Source: Table 2 · p. 4

4. EXPERIMENTAL RESULTS

SEF+JL outperforms the baseline and SEF, particularly on mixed-content audio. Segment-level results also show higher anti-spoofing F1 with joint learning, while SEF without it is unstable on mixed audio.

4.1. Experimental Setup

  • Models use shared training and validation splits; joint training begins at epoch 5 after four independent epochs, and evaluation uses file-level Precision, Recall, and F1.Separation-based audio is chunked into 4-second windows with a 2-second hop; STFT uses 16 kHz audio, a 16 ms hop, and a 64 ms window.Source: §4.1 · p. 4

4.2. Experimental results and analysis

  • SEF+JL consistently outperforms baseline and SEF; for spoof-bonafide Class 2, F1 rises from 0.835 (baseline) to 0.916 (SEF+JL) on development.The reported advantage is especially pronounced when speech and environmental components differ.Source: §4.2 · p. 4
  • On separated segments, speech anti-spoofing F1 rises from 0.720 to 0.863 and environment anti-spoofing F1 from 0.718 to 0.849 with joint learning; environment anti-spoofing remains weaker.Table 3 reports segment-level Precision, Recall, and F1 on the CompSpoof eval set, before chunk-level predictions are aggregated.Source: Table 3 · p. 4, §4.2 · p. 4
  • Without joint learning, SEF reaches perfect performance on simple bona fide audio, but mixed-audio class F1 scores can fall to 0.508 or lower.The authors report that separation alone may distort downstream representations without joint-learning guidance.Source: §4.2 · p. 4

5. CONCLUSIONS

The paper presents CompSpoof, a dataset for component-level audio spoofing, and a separation-enhanced joint learning framework. Experiments show that the method outperforms baselines.

5. CONCLUSIONS

  • CompSpoof covers combinations of bona fide and spoofed speech and environmental sound at the component level.The paper describes it as the first dataset covering these combinations.Source: §5 · p. 4
  • The framework separates audio components and applies dedicated anti-spoofing models while preserving spoof-relevant information.Source: §5 · p. 4
  • Experiments show that the proposed method outperforms baselines, highlighting the effectiveness of component separation and joint learning.Source: §5 · p. 4
Loading 2509.15804v2…