PaperlayerView exact revision on arXiv

Source-linked AI summary

The PartialSpoof Database and Countermeasures for the Detection of Short Fake Speech Segments Embedded in an Utterance

Lin Zhang, Xin Wang, Erica Cooper, Nicholas Evans, Junichi Yamagishi

arXiv:2204.05177v3eess.AScs.CRcs.SD

TL;DR

Partial Spoof embeds generated speech segments in bona fide utterances, a setting that requires detection beyond fully spoofed-speech countermeasures. The paper combines self-supervised features, multi-resolution segment labels, and joint utterance- and segment-level countermeasures; it reports a 0.77% utterance-level EER on ASVspoof 2019 LA evaluation.

  • Problem

    Conventional spoofing scenarios focus on fully generated utterances, while PS embeds generated segments in bona fide speech and requires segment-level detection across variable-length regions.

    Source: §I · p. 3, B. Existing CM architectures for PS… · p. 6

  • Method

    The paper combines self-supervised features with multi-resolution segment labels and a countermeasure that jointly detects spoofing at utterance and segment levels.

    Source: §I · p. 3, §VII · p. 12

  • Results

    The proposed countermeasure achieved a best-reported utterance-level EER of 0.77% on the ASVspoof 2019 LA evaluation set; finer training information benefits utterance detection, while segment detection calls for matched resolution.

    Source: §VII · p. 12

  • Takeaways & Limitations

    Countermeasure training strategies should match the goal: finer-grained training information can benefit utterance detection, whereas segment detection calls for matched resolution.

    Source: §VII · p. 12

    PartialSpoof approximates rather than models deceptive messages, because its variable-length segments are replaced without considering sentence, word, or phoneme meaning and context.

    Source: D. Limitations · p. 5

Abstract

from arXiv · show

Automatic speaker verification is susceptible to various manipulations and spoofing, such as text-to-speech synthesis, voice conversion, replay, tampering, adversarial attacks, and so on. We consider a new spoofing scenario called "Partial Spoof" (PS) in which synthesized or transformed speech segments are embedded into a bona fide utterance. While existing countermeasures (CMs) can detect fully spoofed utterances, there is a need for their adaptation or extension to the PS scenario. We propose various improvements to construct a significantly more accurate CM that can detect and locate short-generated spoofed speech segments at finer temporal resolutions. First, we introduce newly developed self-supervised pre-trained models as enhanced feature extractors. Second, we extend our PartialSpoof database by adding segment labels for various temporal resolutions. Since the short spoofed speech segments to be embedded by attackers are of variable length, six different temporal resolutions are considered, ranging from as short as 20 ms to as large as 640 ms. Third, we propose a new CM that enables the simultaneous use of the segment-level labels at different temporal resolutions as well as utterance-level labels to execute utterance- and segment-level detection at the same time. We also show that the proposed CM is capable of detecting spoofing at the utterance level with low error rates in the PS scenario as well as in a related logical access (LA) scenario. The equal error rates of utterance-level detection on the PartialSpoof database and ASVspoof 2019 LA database were 0.77 and 0.90%, respectively.

DOI: 10.1109/TASLP.2022.3233236

The paper introduces Partial Spoof (PS), where generated speech segments are embedded in bona fide utterances, and argues that this threat calls for countermeasures that detect and locate spoofed segments. It extends earlier PS work with self-supervised features, multi-resolution labels, and a countermeasure for simultaneous utterance- and segment-level detection.

I. INTRODUCTION

  • PS embeds synthesized or transformed speech segments in bona fide utterances, potentially changing words or even smaller speech units.The paper describes substitutions ranging from words such as “not” to vowels and consonant transitions.Source: §I · p. 2
  • Short spoofed segments may evade conventional countermeasures because temporal aggregation can leave them with little effect on the final score.The paper calls for utterance-level detection and segment-level localization that does not assume the attacker’s unit length.Source: §I · p. 3
  • The paper extends earlier PS countermeasures with wav2vec 2.0 and WavLM features, six segment-label resolutions from 20 to 640 ms, and joint utterance- and segment-level detection.Earlier work used fixed-resolution segment labels and multi-task learning, but left room for improvement,.Source: §I · p. 3

A. Spoofing in the PS scenario and related scenarios

  • Unlike LA and DF, which generate entire audio signals, PS can mix generated and bona fide segments and can include segments from multiple TTS or VC methods.The paper also relates PS to splicing forgery, with generated rather than bona fide recordings supplying the inserted segments,.Source: §I · p. 2, A. Spoofing in the PS scenario and… · p. 3

B. New realistic threats

The paper motivates PS countermeasures by the possibility of targeted speech modification and develops a PartialSpoof database for variable-length generated segments. Its construction uses paired bona fide and generated utterances, assumptions about attacker methods, and a multi-stage process for substitution, labeling, and balancing.

B. New realistic threats

  • Speech-editing technologies can modify selected recording segments without changing the rest, increasing the possibility of impersonation and fraud.The paper notes potential threats to text-dependent ASV and the need for countermeasures that detect partially spoofed utterances –.Source: B. New realistic threats · p. 3

C. Databases for PS scenario

  • Unlike a database built around one TTS system replacing one word, PartialSpoof uses variable-length segments from multiple multi-speaker TTS and VC systems.The comparison database later became part of the ADD challenge database.Source: C. Databases for PS scenario · pp. 3–4

III. NEW PARTIALSPOOF DATABASE

  • PartialSpoof pairs fully generated and bona fide utterances, then randomly substitutes short segments to produce varied intra-speech generated segment ratios.The ratio is the duration of generated segments divided by total utterance duration.Source: §III · p. 4

A. Construction policies

  • The construction assumes evaluation spoofing methods mostly differ from training methods and uses VAD-detected variable-length speech regions regardless of linguistic content.It also assumes attackers use basic rather than advanced DSP, allowing some errors and artifacts to occur naturally.Source: A. Construction policies · p. 4

B. Construction procedure

  • The cited description states that the PartialSpoof database is constructed automatically through a five-stage processing pipeline, with the procedure shown in Figure 1.The figure caption identifies 0 as spoof and 1 as bona fide.Source: B. Construction procedure · p. 4, Figure 1 · p. 4
  • Three VADs and majority voting identify candidate regions; segments are matched across utterances from the same speaker by similar duration, then joined using cross-correlation and overlap-add.Labels cover multiple temporal resolutions and utterance level; post-processing balances ten generated-segment-ratio levels.Source: B. Construction procedure · pp. 4–5

C. Database statistics

The database statistics describe class balance across temporal resolutions, while the limitations section qualifies what the PartialSpoof speech represents. The overview defines countermeasure tasks for utterance- and segment-level detection.

C. Database statistics

  • The database analysis uses matched conditions across subsets to reduce mismatch in their intra-speech generated-segment ratios and simplify the analysis.Mismatched conditions are also possible.Source: C. Database statistics · p. 5
  • Finer segment resolutions contain more bona fide samples, reducing the extreme spoof-class bias seen per utterance; segment labels mark a segment spoof if it contains any generated waveform.The paper notes that differing class biases across resolutions may support more robust multi-task learning.Source: C. Database statistics · p. 5

D. Limitations

  • The PartialSpoof substitutions ignore sentence and word meaning and neighboring phonemes, so the database approximates rather than models attacks intended to change linguistic messages.Its variable-length segments still allow segment detection and evaluation at different temporal resolutions.Source: D. Limitations · p. 5

A. Task definition

  • A countermeasure for partial spoofing is tasked with detecting spoofing both for the whole utterance and for its constituent segments.The task definition describes utterance-level and segment-level scores and evaluates them using EER.Source: A. Task definition · pp. 5–6

B. Existing CM architectures for PS scenario

Partial-spoof countermeasures must produce segment scores as well as utterance decisions, unlike conventional utterance-only systems. Existing approaches add a segment-scoring branch or derive scores through a re-scoring step.

B. Existing CM architectures for PS scenario

  • Segment-level detection requires a sequence of scores for shorter-duration segments, so utterance-only countermeasures for LA, PA, or DF cannot be applied directly to partial spoofing.The utterance-level task itself is defined identically across these scenarios.Source: B. Existing CM architectures for PS… · p. 6

A conventional utterance-level CM

  • A conventional countermeasure extracts acoustic features, pools hidden features into a fixed-dimensional embedding, then linearly maps that embedding to one utterance score.The back-end can use machine-learning models and is typically trained with utterance labels.Source: A conventional utterance-level CM · p. 6

2) Adapting utterance-level CM for PS scenario:

  • Adapted LCNNs remove pooling and score hidden features per segment, optionally deriving the utterance score from the segment minimum; training can use segment and utterance cross-entropy losses.Segment scores align with the input sequence according to the network stride.Source: 2) Adapting utterance-level CM for PS… · p. 6

3) Attaching re-scoring step for PS scenario:

A non-trainable re-scoring step derives segment scores from an utterance-level LCNN without requiring segment labels or retraining. This lets an existing utterance-level countermeasure be used directly, but leaves it unadapted to partial-spoof segments.

3) Attaching re-scoring step for PS scenario:

C. CMs built for ADD challenge 2022

V. PROPOSED CM FOR PS SCENARIO

The proposed countermeasure combines a self-supervised front-end with a back-end that produces scores at multiple temporal resolutions. Its outputs support segment detection across varied durations and utterance-level detection.

A. Motivation

  • The proposed countermeasure addresses fixed-resolution segment scoring by pairing a self-supervised front-end with a back-end that supports multiple temporal resolutions.The motivation contrasts this flexibility with the LCNN configuration that scores once every 16 input frames.Source: A. Motivation · p. 7

B. Front-end

  • The front-end extracts features from wav2vec 2.0 or HuBERT Transformer blocks using a trainable weighted sum, and is fine-tuned jointly with the back-end.Source: B. Front-end · p. 7

C. Back-end

  • For utterance-level detection, the lowest-resolution hidden features are pooled into an embedding and transformed into an utterance score.Source: C. Back-end · p. 8
  • The back-end produces segment scores at six resolutions, from 20 to 640 ms, by repeatedly downsampling and scoring features in fine-to-coarse order.The score at each resolution indicates frame- or segment-level bona fide likelihood.Source: C. Back-end · p. 8

D. Training strategies

The proposed CM can be trained with strategies that use segment labels at different temporal resolutions.

D. Training strategies

  • Training can use either of two strategies when ground-truth labels are available at all temporal resolutions.The passage introduces the strategies but does not describe them further.Source: D. Training strategies · p. 8
  • For each temporal resolution, the CM computes a loss from that resolution’s scores and labels, alongside an utterance-level loss.The passage describes a gross loss formed from the resolution-specific losses and an utterance-level loss.Source: 1) Multiple temporal resolutions · p. 8

1) Multiple temporal resolutions:

The paper describes multi- and single-resolution training and reports the experimental choices used to select its CM components.

1) Multiple temporal resolutions:

  • The multiple-resolution strategy combines losses across temporal resolutions, including an utterance-level loss.The equation passage identifies the aggregate loss and its utterance-level term.Source: 1) Multiple temporal resolutions · p. 8

2) Single temporal resolution:

  • With labels at only one segment resolution, single-resolution training uses that resolution’s scoring layers and loss, and produces scores only at that resolution.It cannot produce scores for other segment or utterance levels after training.Source: 2) Single temporal resolution · p. 8

VI. EXPERIMENTS

  • The experiments include a PartialSpoof study and a cross-scenario evaluation on ASVspoof 2019 LA.The study compares training strategies and examines development/evaluation differences as well as application to LA.Source: §VI · p. 8

A. Experimental setup and model configurations

  • The study trained the CM at all temporal resolutions simultaneously after preliminary experiments found no obvious significant difference among alternative training orders and fine-tuning choices.The model was trained using the P2SGrad-based mean square error criterion.Source: A. Experimental setup and model… · p. 8

B. Comparing front- and back-ends for proposed CM

C. Comparing training strategies on the proposed CM

The training strategy’s performance depends on the detection level: multiple resolutions improve utterance-level results, while fine-grained segment detection remains difficult. Evaluation-set analyses point to attack-specific and boundary-related challenges.

C. Comparing training strategies on the proposed CM

  • Below 1% EER was achieved for utterance-level detection on the evaluation set with either training strategy, and multiple-resolution training performed better.For segment-level detection, multiple-resolution training was slightly better at 640 ms but worse than single-resolution training at 20–320 ms; at 20 ms, evaluation EER was around 10%.Source: C. Comparing training strategies on… · pp. 9–10, Figure 3 · p. 10
  • The authors recommend training at the target resolution for segment detection, while finer-grained information is expected to improve utterance-level detection.They also report that segment-level detection is difficult to generalize from development to evaluation.Source: C. Comparing training strategies on… · p. 10

D. EER gap between development and evaluation sets

  • Removing A15-containing trials significantly decreased evaluation-set EER; leaving out A10, A11, A12, or A17 also decreased it, but less.A15 was identified as the strongest spoofing system in the PS scenario.Source: D. EER gap between development and… · p. 10

EER (%)

The proposed CM supports multi-resolution segment detection and outperforms conventional CMs in the reported comparison. The evaluation analyses also identify boundary-related errors and show cross-scenario benefits from PartialSpoof training.

EER (%)

  • A15 was the strongest evaluation-set attack, but removing it did not eliminate the higher EER relative to development.The authors suggest A15 may account for part of the gap.Source: EER (%) · p. 11
  • On evaluation data, CM performance degraded as segments had fewer concatenated boundaries; zero-boundary segments had significantly higher EER at 20–160 ms.Development-set EER varied less with boundary count.Source: EER (%) · p. 11, Figure 5 · p. 11

E. Comparing proposed and conventional CMs

F. Cross-scenario study

  • PartialSpoof training improved performance on PS data and remained competitive on LA data; multiple-resolution training brought further improvement for both scenarios.LA-trained systems had EERs above 10% on PartialSpoof when trained with utterance-level labels only.Source: F. Cross-scenario study · pp. 11–12, Table VIII · p. 12

VII. CONCLUSION

The paper presents PartialSpoof as a scenario requiring utterance- and segment-level detection, and extends its database and countermeasures for multi-resolution training. Results highlight different resolution needs across detection tasks, challenges on the evaluation set, and scope for improvement.

VII. CONCLUSION

  • PartialSpoof provides segment labels at temporal resolutions from 20 to 640 ms, and the proposed countermeasures use self-supervised features and multi-resolution segment labels for simultaneous detection.The paper also introduces neural architectures and training strategies for this multi-resolution setting.Source: §VII · p. 12
  • Combining the two databases was left for future analysis because training with audio lacking segment labels would require substantial changes to the proposed countermeasure.The authors state this investigation is beyond the paper’s scope.Source: §VII · p. 12
  • Utterance-level detection can benefit from finer-grained training information, whereas the more challenging segment-level task calls for matched resolution.The authors conclude that countermeasure and training strategies should be adapted to the detection goal.Source: §VII · p. 12
  • Evaluation-set segment detection was much worse for segments with fewer concatenated boundaries, and unknown attack A15 was the strongest attack identified in PartialSpoof.The development-set performance was less affected by boundary count; the authors identify overcoming the evaluation-set issue as future work.Source: §VII · p. 12
  • 0.77% EER on the ASVspoof 2019 LA evaluation set was the best reported utterance-level result, while conventional LA utterance-level countermeasures largely failed on partially spoofed utterances.The authors note that results still show scope for improvement.Source: §VII · p. 12
  • Future work includes explicit linguistic information, robust training for imbalanced classes, and broader coverage of TTS and VC methods, including more up-to-date methods.The paper identifies increasing the number and recency of speech-generation methods as an important challenge.Source: §VII · pp. 12–13

APPENDIX A IMPACT ON ASV SYSTEM

The appendix evaluates how partially spoofed utterances affect an ASV system. Its equal error rates rise substantially when partial-spoof impostors are added, and vary with the generated-segment ratio.

APPENDIX A IMPACT ON ASV SYSTEM

  • The experiments used target and non-target trials with a Kaldi pre-trained x-vector speaker embedding extractor and a PLDA back-end adapted on bona fide training utterances.The development set had 10 target and 10 non-target speakers; the evaluation set had 48 target and 19 non-target speakers.Source: Appendix A · p. 15
  • EERs were below 3% on both development and evaluation sets with zero-effort impostors alone, but rose substantially when partially spoofed samples were added.The comparison considers zero-effort and partial-spoof impostors as non-target trials.Source: Appendix A · p. 15, Table IX · p. 15
  • ASV EERs increased as the intra-speech generated-segment ratio decreased, with groups formed by ratio and equalized to the same number of spoofed trials.Figure 6 reports EERs for quantized classes with equal trial counts and 5% significance-level confidence intervals.Source: Appendix A · p. 15, Figure 6 · p. 15
Loading 2204.05177v3…