Source-linked AI summary
DeepSonar: Towards Effective and Robust Detection of AI-Synthesized Fake Voices
Run Wang, Felix Juefei-Xu, Yihao Huang, Qing Guo, Xiaofei Xie, Lei Ma, Yang Liu
TL;DR
AI-synthesized fake voices can sound realistic and support fraud, while effective and robust detectors remain limited. DeepSonar monitors layer-wise neuron behaviors in a speaker-recognition network; across three datasets, it reports average accuracy above 98.1% and EER below 2%.
Problem
Realistic AI-synthesized voices can be used for fraud, while effective and robust detectors are still limited.
Source: §1 · p. 1Method
DeepSonar uses a binary classifier on layer-wise neuron behaviors from a DNN-based speaker-recognition system to distinguish real and AI-synthesized voices.
Source: §1 · p. 2Results
Across three datasets, DeepSonar achieves average accuracy >98.1% and EER <2%, and outperforms prior bispectral-artifact work in effectiveness and robustness.
Source: §1 · p. 2, §5.1 · p. 6Takeaways & Limitations
The results support monitoring neuron behaviors as an inside-out approach to detecting AI-aided multimedia fakes.
Source: §6 · p. 8High-magnitude mixtures of real-world noises can decrease DeepSonar’s performance to some extent.
Source: §5.3 · p. 8
Abstract
from arXiv · showhide
With the recent advances in voice synthesis, AI-synthesized fake voices are indistinguishable to human ears and widely are applied to produce realistic and natural DeepFakes, exhibiting real threats to our society. However, effective and robust detectors for synthesized fake voices are still in their infancy and are not ready to fully tackle this emerging threat. In this paper, we devise a novel approach, named \emph{DeepSonar}, based on monitoring neuron behaviors of speaker recognition (SR) system, \ie, a deep neural network (DNN), to discern AI-synthesized fake voices. Layer-wise neuron behaviors provide an important insight to meticulously catch the differences among inputs, which are widely employed for building safety, robust, and interpretable DNNs. In this work, we leverage the power of layer-wise neuron activation patterns with a conjecture that they can capture the subtle differences between real and AI-synthesized fake voices, in providing a cleaner signal to classifiers than raw inputs. Experiments are conducted on three datasets (including commercial products from Google, Baidu, \etc) containing both English and Chinese languages to corroborate the high detection rates (98.1\% average accuracy) and low false alarm rates (about 2\% error rate) of DeepSonar in discerning fake voices. Furthermore, extensive experimental results also demonstrate its robustness against manipulation attacks (\eg, voice conversion and additive real-world noises). Our work further poses a new insight into adopting neuron behaviors for effective and robust AI aided multimedia fakes forensics as an inside-out approach instead of being motivated and swayed by various artifacts introduced in synthesizing fakes.
1 INTRODUCTION
DeepSonar addresses the difficulty of detecting realistic AI-synthesized voices and the limited robustness of existing detectors. It monitors layer-wise behaviors in a speaker-recognition network and evaluates detection across datasets and manipulation attacks.
1 INTRODUCTION
- Realistic synthesized voices can be difficult for people to distinguish and can be misused for fraud, creating security and privacy concerns.The paper cites a reported case in which criminals used AI voice impersonation to swindle more than $243,000.Source: §1 · pp. 1–2
- Existing detectors had not fully handled TTS and voice-conversion fakes or thoroughly evaluated robustness against manipulation attacks.The paper identifies real-world noise and signal conversion, including resampling and pitch shifting, as manipulation attacks.Source: §1 · p. 2
- DeepSonar monitors layer-wise neuron behaviors in a DNN-based speaker-recognition system and feeds them to a binary classifier to distinguish real and fake voices.The authors conjecture these behaviors capture subtle features and provide cleaner classifier signals than raw voice inputs.Source: §1 · p. 2
- DeepSonar achieves average accuracy higher than 98.1% and EER lower than 2% across three datasets, and outperforms bispectral-artifact work in effectiveness and robustness.The evaluation includes voice conversion and additive real-world noise attacks.Source: §1 · p. 2
- The evaluation covers three datasets with varied synthesis techniques, English and Mandarin Chinese, and male and female speakers with different accents.The datasets include public data using commercial products and a self-built dataset using open-source tools.Source: §1 · p. 2
2 RELATED WORK
Prior voice synthesis ranges from earlier techniques with audible naturalness limitations to DNN-based systems producing natural speech. Fake-voice detection includes audio-forensics methods and bispectral-artifact analysis; DeepSonar instead monitors neuron behaviors.
2.1 Voice Synthesis
- Earlier synthesis techniques include concatenating prerecorded speech, acoustic or articulatory modeling, and HMM-based modulation, but suffer from naturalness issues audible to human listeners.DNN methods map linguistic features to acoustic features, with models including Boltzmann machines, deep belief networks, mixed density networks, and bidirectional LSTMs; WaveNet and Tacotron advanced commercial TTS and VC.Source: §2.1 · pp. 2–3
2.2 Fake Voice Detection
- Digital audio-forensics approaches examined metadata, file bytes, and acoustic-environment signatures, but did not address synthesized audio content..Source: §2.2 · p. 3
- The closest prior work detects AI-synthesized voices using unusual spectral correlations, termed bispectral artifacts, as higher-order polyspectral features.The paper notes artifact-based detectors may become invalid if artifacts are fixed or new synthesis techniques appear.Source: §2.2 · p. 3
- DeepSonar replaces raw-voice artifact analysis with neuron behaviors from a DNN-based speaker-recognition system and reports better effectiveness and robustness than bispectral-artifact detection.The layer-wise behaviors are proposed as more subtle features for distinguishing real and fake voices.Source: §2.2 · p. 3
3 METHOD
DeepSonar treats speaker-recognition network responses as features for detecting synthesized voices, based on the conjecture that layer-wise neuron behaviors expose subtle differences more cleanly than raw audio. It selects activated neurons using two coverage criteria and trains a shallow binary classifier on the resulting behaviors.
3.1 Insight
- DeepSonar uses layer-wise neuron behaviors from a speaker-recognition DNN as cleaner, subtler features for distinguishing human speech from AI-synthesized fake voices.The authors present this as a conjecture motivating their approach.Source: §3.1 · p. 3
3.2 Overview of DeepSonar Framework
- Real and fake voice clips pass through a DNN-based speaker-recognition system, and activated layer-wise behaviors become features for a supervised binary classifier.The inputs include TTS and voice-conversion fakes; real voices and fakes are collected with diversity in languages, accents, genders, and synthesis techniques.Source: §3.2 · pp. 3–4
3.3 Layer-wise Neuron Behaviors
- DeepSonar monitors a third-party speaker-recognition DNN’s learned layers, selecting convolutional and fully connected layers to capture voice representations.Activated neurons are defined by outputs above a threshold, and the method uses layer-specific neuron coverage criteria to determine that threshold.Source: §3.3 · p. 4
3.4 Neuron Coverage Criteria Design
- ACN counts activated neurons per layer, while TKAN selects the top-k neuron outputs in each layer to represent layer-wise behavior.Figure 3’s TKAN example shows activated-neuron features separating real and fake voices into independent clusters, while raw neuron features yield coarser clusters.Source: §3.4 · pp. 4–5
3.5 Fake Voice Detection
- A shallow binary classifier predicts whether a voice clip is human or synthesized from vectorized layer-wise neuron behaviors rather than raw audio.The authors state that these behavior inputs are insensitive to voice-conversion and additive real-world-noise manipulations.Source: §3.5 · p. 5
4 EXPERIMENTAL SETTING AND IMPLEMENTATION
The evaluation uses three datasets spanning TTS and voice conversion, English and Mandarin Chinese, and public and self-built collections. The setup compares DeepSonar with prior bispectral-artifact work, evaluates seven detection and false-alarm metrics, and specifies the classifier and robustness manipulations.
4.1 Dataset
- The three evaluation datasets comprise FoR TTS voices, the self-built Sprocket-VC dataset, and Mandarin Chinese MC-TTS voices synthesized with Baidu.FoR contains TTS fakes only; the datasets include English and Chinese voices and vary in real-voice sources and synthesis techniques.Source: Table 1 · p. 5, §4.1 · p. 5
4.2 Baseline
- DeepSonar is compared with prior work using bispectral artifacts to distinguish human speech from AI-synthesized fake voices.The authors identify it as, to their knowledge, a state-of-the-art work focused on AI-synthesized fake-voice detection.Source: §4.2 · p. 5
4.3 Evaluation Metrics
- Seven metrics assess detection and false alarms: accuracy, AUC, F1-score, and AP for detection, plus FPR, FNR, and EER for false alarms.The evaluation covers TTS and voice conversion across the three datasets.Source: §4.3 · p. 6
4.4 Implementation
- The detector uses a five-layer fully connected binary classifier, while neuron monitoring uses a thin-ResNet speaker-recognition backend; TKAN empirically sets k to 5, and Table 2 lists 15 manipulations.The 15 representative manipulations are classified as voice conversions and real-world noises formed by adding environmental noises.Source: §4.4 · p. 6
5 EXPERIMENTAL RESULTS
The experiments evaluate DeepSonar’s detection of TTS- and VC-generated fake voices across languages and its robustness to voice conversion and added environmental noise. Results show strong detection and comparative performance, while high-magnitude mixed noise and adversarial voices remain concerns.
5.1 Detection Results (RQ1)
- Across three datasets, DeepSonar achieves average accuracy >98.1% and EER <2% in discerning two typical fake voices in both English and Chinese.Source: §5.1 · p. 6
- On the more challenging FoR dataset, TKAN yields accuracy >99%, whereas ACN yields accuracy <90%.The authors therefore focus their baseline comparison on DeepSonar using TKAN.Source: §5.1 · p. 6
- On average across three datasets, TKAN-based DeepSonar significantly outperforms the bispectral-artifact baseline across seven metrics.Table 4 presents the comparison with the baseline on the three datasets.Source: §5.1 · p. 6, Table 4 · p. 7
5.2 Evaluation on Robustness (RQ2)
- Resampling leaves performance unaffected, while stretching decreases average performance by less than 5% and pitch shifting by 15% on average.The authors report greater susceptibility to pitch shifts, especially when the shift exceeds 2 steps and words become difficult to hear.Source: §5.2 · p. 7, Figure 6 · p. 8
- Across five indoor-noise magnitudes, average performance decreases less than 10%; adding footstep noise decreases it nearly 20%.The footstep recordings contain friction-related sizzle noise.Source: §5.2 · p. 7, Figure 5 · p. 7
- Across five outdoor-noise magnitudes, performance decreases less than 7% for engine and thunderstorm noise, less than 18% for fireworks and trains, and nearly 25% for wind and rain.The authors attribute the greater wind and rain degradation to those noises mixing with other sounds, such as raindrops.Source: §5.2 · p. 7, Figure 6 · p. 8
- DeepSonar outperforms the baseline on both manipulation attacks, and performs well on single-source additive noise and mixed noise at SNR above 35.The paper notes reduced robustness when pitch shifts seriously damage voices and when mixed environmental noises are intense.Source: §5.2 · pp. 7–8
5.3 Discussion
- Adversaries could model neuron behaviors to generate evasion voices, and high-magnitude mixtures of real-world noises can reduce DeepSonar’s performance.The authors identify voice denoising as future work for high-intensity mixed noise.Source: §5.3 · p. 8
6 CONCLUSIONS
DeepSonar detects AI-synthesized fake voices by monitoring learned neuron behaviors. The authors report effectiveness and robustness on three datasets and identify multimodal detection as a direction for future work.
6 CONCLUSIONS
- DeepSonar detects AI-synthesized fake voices by monitoring learned neuron behaviors.The conclusion frames this as an inside-out approach to detecting AI-aided multimedia fakes.Source: §6 · p. 8
- The authors propose combining voice detection with detectors for generated or forged images and DeepFake videos as future research.They identify audio-visual inconsistency in video DeepFakes as a forensic clue.Source: §6 · p. 8