Source-linked AI summary
Half-Truth: A Partially Fake Audio Detection Dataset
Jiangyan Yi, Ye Bai, Jianhua Tao, Haoxin Ma, Zhengkun Tian, Chenglong Wang, Tao Wang, Ruibo Fu
TL;DR
Prior fake-audio datasets largely omit short synthetic clips embedded in real speech, a case the paper says is difficult to detect. The paper develops HAD, a partially fake audio dataset, and reports that partially fake audio is harder to detect than fully fake audio.
Problem
Previous datasets focus on fully generated utterances and omit small fake clips hidden in real speech, where changed words are difficult to identify.
Source: §1 · p. 1Method
HAD creates partially fake audio by editing a few words, synthesizing the edited speech, and manipulating the original recording with that synthesis.
Source: §1 · p. 1, §2.1 · p. 2Results
Partially fake audio is more difficult to detect than fully fake audio, and experiments also report challenges in detecting and localizing partially fake regions.
Source: §5 · p. 4, §4.2 · p. 4, §4.3 · p. 4Takeaways & Limitations
Abstract
from arXiv · showhide
Diverse promising datasets have been designed to hold back the development of fake audio detection, such as ASVspoof databases. However, previous datasets ignore an attacking situation, in which the hacker hides some small fake clips in real speech audio. This poses a serious threat since that it is difficult to distinguish the small fake clip from the whole speech utterance. Therefore, this paper develops such a dataset for half-truth audio detection (HAD). Partially fake audio in the HAD dataset involves only changing a few words in an utterance.The audio of the words is generated with the very latest state-of-the-art speech synthesis technology. We can not only detect fake uttrances but also localize manipulated regions in a speech using this dataset. Some benchmark results are presented on this dataset. The results show that partially fake audio presents much more challenging than fully fake audio for fake audio detection. The HAD dataset is publicly available: https://zenodo.org/records/10377492.
1. Introduction
Earlier fake-audio datasets focused on utterance-level fakes, leaving hidden, short synthetic clips in real speech unaddressed. This paper introduces HAD, a partially fake corpus designed to support detection and localization of those edits.
1. Introduction
- Prior spoofing databases covered synthesis, voice conversion, replay, and other attacks, but did not address fake clips embedded in real utterances.The prior datasets described in the introduction include ASVspoof corpora and databases for particular spoofing methods.Source: §1 · p. 1
- Earlier datasets used fully generated utterances, leaving small fake clips hidden in real speech difficult to identify.The paper frames this overlooked attack as a serious threat when synthetic audio changes only a few words.Source: §1 · p. 1
- HAD introduces a partially fake corpus in which a few words are changed using state-of-the-art speech synthesis, including GST Tacotron.The paper describes benchmark evaluation for detecting fake utterances and localizing manipulated intervals.Source: §1 · pp. 1–2
- The authors describe HAD as the first partially fake dataset and provide fake-segment locations so models can be evaluated on localization.The introduction says the corpus will be publicly available.Source: §1 · p. 2
2. Dataset design
HAD is built on AISHELL-3 to evaluate detection and localization of partially fake audio, alongside fully real and fake utterances for comparison. Its partial examples replace selected words in real speech with synthesized audio, and the dataset includes speaker-disjoint splits and unseen-TTS test sets.
2. Dataset design
- HAD uses AISHELL-3, a multi-speaker Mandarin speech corpus, as its source corpus.AISHELL-3 contains 88,035 utterances from 218 speakers; the training set has 64,773 utterances from 174 speakers.Source: §2 · p. 2
2.1. Design policy
- The dataset provides Partial and Full subsets to compare detection and assess localization of manipulated regions.The Full subset contains fully real and fully fake audio; the Partial subset contains manipulated real speech.Source: §2.1 · p. 2
2.2. Editing strategies of content
- Edited text changes meaning by replacing named entities or substituting an attitude word with its antonym, with one replacement per sentence.The named-entity statistics distinguish entity counts in the speech from the size of the replacement pool; antonym editing uses 181 word/antonym pairs.Source: §2.2 · p. 2, Table 1 · p. 2
2.3. Fully fake audio generation
- The open-source TTS system uses GST and prosody transfer to model speaker style, then LPCNet to generate audio from spectrograms.The model is trained on AISHELL-3; fully fake audio is generated from original and edited text, and the fake segment is cut from a full synthetic utterance.Source: §2.3 · p. 2
2.4. Manipulation for partially fake audio generation
- Partial examples replace a selected keyword segment in original real audio with corresponding synthetic audio, using forced-alignment timestamps to locate characters.The synthetic segment is volume-normalized before replacement.Source: §2.4 · pp. 2–3
2.5. Dataset composition
- The Partial and Full subsets are partitioned into speaker-disjoint Train, Dev., and Test sets at 6:2:2, with equal utterance totals across parts.Regional accents are represented, and unseen test sets use another TTS model with an improved LPCNet vocoder to evaluate generalization.Source: §2.5 · p. 3, Table 2 · p. 3
3. Evaluation metric
The evaluation covers both utterance-level fake-audio classification and segment-level localization. It uses EER for utterance decisions and duration-based precision, recall, and F1 for segment detection.
3. Evaluation metric
- Utterance-level binary classification is evaluated using equal error rate (EER).The paper points to for further metric details.Source: §3 · p. 3
- Segment-level evaluation measures overall fake-audio detection, including localization, with precision, recall, and F1-score based on segment duration.Figure 3 illustrates the segment-level metric.Source: §3 · p. 3
- For segment scoring, TP is the duration where the detected fake hypothesis overlaps the reference fake segment; FP and FN are also defined by durations.The metrics use these duration-based overlaps to evaluate localized regions.Source: §3 · p. 3
4. Initial benchmarking experiments
The section benchmarks HAD with GMM and LCNN baselines for utterance-level detection and segment-level localization. Across both tasks, the reported results show that partially fake audio is challenging to detect and localize.
4.1. Experimental setup
- The baselines use GMM and LCNN classifiers with CQCC input features; both are trained on real and fake audio utterances from the training set, while development sets select models and hyperparameters.CQCC features use a 20-ms sliding window with a 10-ms shift; the LCNN has five convolution layers and ten Max-Feature-Map layers.Source: §4.1 · pp. 3–4
4.2. Baselines: utterance-level classification
- Utterance-level performance is evaluated by EER on seen and unseen test sets from both the Partial and Full sets.The section reports results for systems trained on fully fake and real data, partially fake data, and combinations of these data.Source: §4.2 · p. 4
- Detection trained on fully fake and real audio degrades on partially fake tests, while training on partially fake data still yields poor performance there.The authors conclude that partially fake audio is challenging to detect.Source: §4.2 · p. 4
4.3. Baselines: segment-level classification
- For localization, frame-level classifiers are converted to segment-level scores by averaging adjacent 5-second scores, then evaluated with precision, recall, and F1.The section reports results on seen and unseen test sets.Source: §4.3 · p. 4
- Segment-level experiments likewise show that localizing manipulated regions in partially fake audio is challenging.Table 7 reports precision, recall, and F1 for systems trained on fully fake, real, and partially fake data.Source: §4.3 · p. 4, Table 7 · p. 4
5. Conclusions
The paper presents HAD, a dataset for audio containing a fake segment of a few words, alongside fully real and fully fake audio. Its benchmarks find partially fake audio more difficult to detect than fully fake audio.
5. Conclusions
- HAD is presented as the first dataset focused on half-truth audio, where only a segment of a few words is fake.The paper describes its design policy, fake-data generation, manipulation, and metrics.Source: §5 · p. 4
- The benchmark results show that partially fake audio is more difficult to detect than fully fake audio.Source: §5 · p. 4
- HAD provides frame-level and utterance-level ground-truth labels, including the fake segment's location; future work includes more fake types and languages.Source: §5 · p. 4