Source-linked AI summary
Privacy against Real-Time Speech Emotion Detection via Acoustic Adversarial Evasion of Machine Learning
Brian Testa, Yi Xiao, Harshit Sharma, Avery Gump, Asif Salekin
TL;DR
Smart-speaker speech may expose users’ emotional information to unauthorized inference. DARE-GP uses constrained genetic programming to create additive noise that masks emotion while preserving transcription. The authors report robustness to defenses and favorable survey responses to the noise.
Problem
Smart-speaker speech can expose users’ emotional information to unauthorized inference, raising privacy concerns about its use.
Source: §1 · pp. 1–2Method
Results
DARE-GP was reported robust to multiple defenses, and 89.5% of survey respondents rated its noise Noninvasive or Completely Inaudible.
Source: §1 · p. 3Takeaways & Limitations
The authors argue that satisfying the stated attributes makes DARE-GP deployable in real-world settings, with acoustic evaluations using two off-the-shelf smart speakers.
Source: §1 · p. 4The threat model assumes a population-trained SER classifier and a defender without users’ unmodified speech, ground-truth labels, or exact EON parameters.
Source: • Proposed System’s Access Level · p. 5, §3 · p. 6
Abstract
from arXiv · showhide
Smart speaker voice assistants (VAs) such as Amazon Echo and Google Home have been widely adopted due to their seamless integration with smart home devices and the Internet of Things (IoT) technologies. These VA services raise privacy concerns, especially due to their access to our speech. This work considers one such use case: the unaccountable and unauthorized surveillance of a user's emotion via speech emotion recognition (SER). This paper presents DARE-GP, a solution that creates additive noise to mask users' emotional information while preserving the transcription-relevant portions of their speech. DARE-GP does this by using a constrained genetic programming approach to learn the spectral frequency traits that depict target users' emotional content, and then generating a universal adversarial audio perturbation that provides this privacy protection. Unlike existing works, DARE-GP provides: a) real-time protection of previously unheard utterances, b) against previously unseen black-box SER classifiers, c) while protecting speech transcription, and d) does so in a realistic, acoustic environment. Further, this evasion is robust against defenses employed by a knowledgeable adversary. The evaluations in this work culminate with acoustic evaluations against two off-the-shelf commercial smart speakers using a small-form-factor (raspberry pi) integrated with a wake-word system to evaluate the efficacy of its real-world, real-time deployment.
1 INTRODUCTION
DARE-GP targets unauthorized inference of users’ emotions from smart-speaker speech, while preserving command transcription. The paper presents a real-time acoustic approach and reports robustness to defenses, real-world evaluation, and favorable user-survey responses.
1 INTRODUCTION
- DARE-GP addresses concern that smart-speaker recordings may expose emotional information after users lose control of uploaded command speech.The paper also raises commercial exploitation and government use, including concerns about bias and fairness in affect recognition.Source: §1 · pp. 1–2
- Companies’ investment in speech emotion recognition, alongside the rarity of conversational smart-speaker use (< 10%), motivates concern that emotion may be inferred from acoustic speech rather than transcripts.The paper notes that common smart-speaker interactions are primarily task commands.Source: §1 · p. 2
- The central question is whether users can limit disclosure of emotion through speech acoustics while retaining smart-speaker utility, including on unseen classifiers and utterances.The stated questions also cover comparison with existing audio-evasion methods, knowledgeable defenses, and acoustic deployment under variable user locations and SWAP constraints.Source: §1 · p. 2
- Constrained genetic programming learns users’ emotion-related spectral traits and generates one additive Emotion Obfuscating Noise that masks them while preserving transcription utility.The same precomputed perturbation is intended for real-time use on previously unheard commands and to transfer to unseen black-box SER classifiers.Source: §1 · p. 3
- 89.5% of surveyed respondents described DARE-GP’s additive noise as Noninvasive or Completely Inaudible.The paper identifies minimizing disruption to users’ daily lives as a design requirement.Source: §1 · p. 3
- Evaluations report superior performance against state-of-the-art SER evasion, robustness to a knowledgeable adversary, and over-air testing with two off-the-shelf smart speakers.The authors say satisfying the required attributes makes DARE-GP the first approach deployable in a real-world environment.Source: §1 · p. 4
2 ATTACK MODEL
The attack targets a provider-operated black-box SER classifier while preserving voice-assistant usability. DARE-GP is restricted to the user-facing audio channel and assumes a knowledgeable provider may try to defend the classifier.
• Target System:
- The target is a provider-operated black-box SER classifier that infers emotion from speech’s spectral components rather than its transcript.The attack goal is to degrade emotion inference while minimizing transcription errors and supporting real-time, on-demand use.Source: §2 · p. 4, • Target System · p. 5
• Proposed System’s Access Level:
- DARE-GP assumes no privileged speaker access, pre-upload code, side channels, or classifier access for training a surrogate or learning its features, topology, or labels.Interactions with the classifier are limited to the same audio channel used for voice-assistant commands.Source: • Proposed System’s Access Level · p. 5
- The knowledgeable-provider threat model gives the defender EON-mixed audio but not the user’s unmodified speech or ground-truth labels, and assumes EON parameters remain unknown.The paper evaluates robustness against several state-of-the-art audio-evasion defenses.Source: • Proposed System’s Access Level · p. 5
• Assumptions:
DARE-GP assumes emotion can be inferred from speech acoustics rather than command transcripts and uses a transcription service expected to be less noise-tolerant than commercial systems. Its goal is to reduce SER accuracy without sacrificing transcription accuracy.
• Assumptions:
- The approach assumes smart-speaker command transcripts typically lack emotion information, because transcript-based emotion protection would otherwise require interfering with command execution.The paper says Appendix A reports an experiment validating this assumption.Source: • Assumptions · p. 5
- DARE-GP was developed with Vosk/Kaldi transcription because smart speakers may not expose transcript APIs; the authors assume this service is less noise-tolerant than commercial speech-to-text.They report that DARE-GP was more successful on real smart speakers, consistent with that assumption.Source: • Assumptions · p. 5
• Outcome of the Attack:
- The desired attack outcome is lower SER prediction accuracy with transcription accuracy preserved, including robustness to defenses from a knowledgeable SER provider.Source: • Outcome of the Attack · p. 5
• Non-Invasivenes:
DARE-GP constrains its additive perturbations to limit invasiveness and runtime effects.
• Non-Invasivenes:
- Perturbations are restricted to typical speech-frequency ranges and bounded in amplitude to avoid negative runtime impacts.The paper motivates these constraints by noting potential invasiveness from excessive loudness or frequencies outside human hearing.Source: • Non-Invasivenes · p. 5
3 PROBLEM STATEMENT
The problem is to generate one Emotion Obfuscating Noise (EON) for users in U that increases a black-box SER classifier’s misclassification rate within environment E without reducing transcription effectiveness. The evaluation also considers transfer from a digital environment, knowledgeable defenses, and real-world acoustic use.
3 PROBLEM STATEMENT
- The target is a single EON that increases classifier C’s misclassification rate on utterances from U within E without impacting speech-to-text system S.C is trained to improve classification performance on U [99].Source: §3 · p. 6
- The approach constrains transcription utility while optimizing a surrogate classifier C* for misclassification that transfers to black-box classifiers C.The EON is developed in a digital environment before being assessed against C in the original environment.Source: §3 · p. 6
- The problem statement also calls for testing whether a knowledgeable SER provider can infer true classes and whether a digitally developed EON works in the original acoustic environment.The latter assessment concerns the transition from digitally developed EON P-hat to acoustic EON P.Source: §3 · p. 6
4 TECHNICAL APPROACH
DARE-GP uses constrained genetic programming to create a universal additive audio perturbation that masks users’ emotion-relevant spectral traits while limiting transcription errors. It evolves tone parameters against a surrogate classifier, then uses Kernel SHAP to relate perturbation-affected frequency bands to misclassification.
EON Description
- Each EON is an additive mixture of tones with learnable frequency, amplitude, start time, and duration, shared across a target user set.The number of tones is an empirically selected hyperparameter.Source: §4 · p. 6
How Does an EON Cause Misclassification?
- Kernel SHAP masks frequency bands to estimate their importance; an Echo Dot example links perturbation-affected bands to a shift from angry to happy classification.The illustrated perturbation uses three tones, whose bands correspond to frequencies important to the classifier’s correct angry prediction.Source: §4 · pp. 6–7, Figure 2 · p. 7
Use of Genetic Programming (GP)
- DARE-GP evolves universal perturbations that mask users’ emotion-relevant spectral traits, supporting transfer to unseen black-box classifiers, previously unheard utterances, and acoustic operation.The approach targets underlying spectral attributes rather than a target classifier’s feature representation.Source: §4 · p. 7
Genetic Programming (GP) Workflow
- GP initializes EON-parameterized individuals, scores them on training samples, and iteratively selects, crosses over, and mutates candidates across generations.The final EON is selected by its evasion success rate on validation data; selection, crossover, and mutation settings are chosen empirically.Source: §4 · p. 8
Fitness
- Candidate fitness multiplies deception and transcription scores, rewarding surrogate-classifier evasion while harshly penalizing transcription errors.The transcription score is zeroed when a sample is transcribed incorrectly or its confidence falls below a specified threshold.Source: §4 · pp. 9–10
5 DATASETS
The experiments use RAVDESS and TESS, with samples screened for transcription and baseline-classification correctness; IEMOCAP and SAVEE appear only in training data for some evaluated black-box models.
5 DATASETS
- RAVDESS contains 1,440 spoken samples from 24 actors, while the study's TESS subset contains 1,800 samples from two actresses.TESS was also used to assess whether an EON extends to multiple previously unheard utterances.Source: §5 · pp. 10–11
- Samples were excluded when transcription failed or the surrogate SER classifier misclassified them before EON application.This screening avoids penalizing EONs for transcription errors and avoids inflating evasion scores through natural classifier errors.Source: §5 · p. 11
- IEMOCAP and SAVEE were included in training data for several evaluated black-box SER models, but not in EON development or evaluation.Source: §5 · p. 11
6 EVALUATION
The evaluation tests transfer to unseen black-box classifiers and utterances, compares DARE-GP with other attacks and defenses, and assesses acoustic performance through wake-word-triggered deployment. EON development, black-box classifier training, and evaluation use disjoint data.
6.1 Evasion Success and Transferability
- DARE-GP transferred to previously unseen classifiers and unheard utterances; ESR was especially significant for SEC, TDNN, wav2vec, and RESNET.The latter classifiers used Mel-scaled spectrogram features and differed most from the surrogate in topology and feature representation; ESR was computed on samples the classifiers originally classified correctly.Source: §6.1 · p. 13, Table 5 · p. 14
6.2 Comparison with Other Evasion Attacks
- DARE-GP outperformed spectral-envelope attacks, while FGSM and PGD had ESR of 0 because their audio was too garbled to transcribe correctly.DARE-GP adds noise to specific emotion-related spectral aspects rather than broadly distorting spectral attributes and transcription-relevant information.Source: §6.2 · pp. 13–14, Table 5 · p. 14
6.3.1 Active Defense.
- Resampling and Mel-spectrogram extraction and inversion were ineffective; removing frequencies within +/- 10Hz of EON frequencies severely impacted SER classifiers.Mel preprocessing improved ESR against Data Flair and wav2vec in some cases by degrading classifier performance.Source: §6.3.1 · pp. 14–15, Table 6 · p. 15
6.3.2 Passive Defense - True Class Inference.
- EONs weakened true-to-predicted emotion correlations across classifiers, and the observed forced-label patterns did not enable providers to reverse-engineer true classes.Misclassification distributions varied: some classifiers favored particular labels, while others spread errors more evenly.Source: Table 7 · p. 15, §6.3.2 · p. 15
6.4.1 Acoustic Evaluation on Commercial Smart Speakers.
- At one foot, commercial smart speakers had evasion rates around 0.9 with low WER; the loudest EON was 50dB in a room averaging 46dB background noise.The authors report that low WER indicates EONs were not overpowering user utterances, and 50dB is below normal conversation range [1].Source: §6.4.1 · p. 16, Table 8 · p. 17
6.4.2 Evaluation of EON Robustness from Different Directions.
- User direction caused slight ESR degradation when aligned closely with the DARE-GP speaker, but EONs remained robust across directions, including with Echo Dot beamforming.The authors attribute robustness to physical contact enabling EON propagation through smart-speaker materials.Source: Table 9 · p. 17, §6.4.2 · p. 17
6.4.3 Automated EON Invocation.
- Automated playback on a Raspberry Pi achieved an ESR of 0.738 versus 0.76 for the original trial, with 0.15 seconds between wake-word detection and playback.The setup used a wake-word engine and consumed less than 3.7 W at peak usage.Source: §6.4.3 · p. 18
7 DISCUSSION
The discussion situates DARE-GP’s results in its user-specific adaptation, deployment constraints, and evaluation scope. It also considers how EONs affect emotion inference and how users view the system.
7 DISCUSSION
- The evaluation used recorded audio and off-the-shelf SER classifiers to benchmark DARE-GP on known datasets and demonstrate generalization.The authors chose these rather than potentially introducing bias through other evaluation choices.Source: §7 · p. 18
- The surrogate classifier differed from several black-box classifiers in accuracy, emotion classes, and features, yet DARE-GP remained effective across those differences.The surrogate’s reported accuracy was .877.Source: §7 · p. 18
- The final EONs are untargeted: actual and false classes were sufficiently independent to prevent providers from inferring the utterance’s true emotion.The authors say this would prevent successful targeted advertising using utterance emotion.Source: §7 · p. 18
- EONs tailored to different user groups masked different spectral regions, including distinct formants and higher-frequency bands.The two examples were trained on groups with different demographic compositions.Source: §7 · p. 18, Figure 7 · p. 19
- Cross-dataset tests found that RAVDESS-trained EONs needed tailoring for TESS users, requiring labeled audio samples from end users.The authors suggest pretrained models to label unlabeled user data as a possible way to reduce this issue; investigation remains future work.Source: §7 · pp. 18–19
- In a survey of 19 students, concern about companies using emotion data for targeted ads averaged 3.16 out of 4, while 36.8% were unwilling to use a simulated record-modify-replay system.Only one respondent found the evaluated EON somewhat distracting.Source: §7 · p. 19
- The beamforming countermeasure relies on physical contact between the DARE-GP speaker and smart speaker; effects of future vibration-limiting speakers remain unevaluated.The authors leave that defense for future work because no commercial smart speakers advertised the feature.Source: §7 · p. 19
- Evaluations covered laboratories and two homes, not exhaustive in-the-wild conditions or long-term EON invasiveness.The authors identify broader in-the-wild sampling as future work.Source: §7 · p. 19
8 RELATED WORKS
Related work spans protection from unintended recording, genetic-programming evasion, and attacks on audio classifiers. The paper distinguishes DARE-GP by its universal, real-time perturbation and transcription constraint.
8.1 Protection of Audio against Unanticipated Recording
- DARE-GP complements recording-prevention systems by protecting emotional content when a device is intended to hear a command.Patronus and MicShield instead prevent access to audio from unanticipated devices or at unplanned times.Source: §8.1 · pp. 19–20
8.2 Evasion of Black-Box Classifiers Using Genetic Programming
- Unlike prior GP evasion work that generated sample-specific perturbations, DARE-GP precomputes a universal perturbation for real-time use on unheard utterances.Constrained GP also protects transcription utility, unlike prior GP-based audio evasion approaches.Source: §8.2 · p. 20
8.3 Adversarial Evasion of Audio Classifiers
- Audio-evasion approaches differ in access requirements, real-time use, acoustic deployment, and transcription effects, which the paper uses to distinguish DARE-GP.The comparison includes speaker recognition, ASR, and SER attacks.Source: §8.3 · p. 20
- Advpulse shares real-time acoustic use of universal additive noise, but unlike DARE-GP it lacks transcription protection and requires target-model weights and gradients.Advpulse used variable delays to address alignment; DARE-GP relies on wake-word alignment in its use case.Source: §8.3 · pp. 20–21
- EDGY disentangles linguistic and paralinguistic embeddings without target-classifier access, but requires processing and vocoder-generated speech rather than real-time additive noise.The cited work reports degraded classification of attributes including emotion, age, accent, and gender.Source: §8.3 · p. 21
- Other compared attacks include non-real-time spectral filters that introduce transcription errors, and GP-based ASR methods using sample-specific perturbations or hard-label access.These include Gao et al.’s spectral-envelope attacks and GP-based approaches by Wu, Yi, et al., Alzantot et al., and Taori et al.Source: §8.3 · p. 21
9 CONCLUSION
DARE-GP aims to reduce emotion-related privacy loss in smart-speaker use. It precomputes additive EONs that disrupt SER in real time, and the evaluation reports effectiveness across classifiers, defenses, and commercial speakers.
9 CONCLUSION
- DARE-GP’s EON achieved an ESR of 0.738 against an Amazon Dot in a real-time, real-world demonstration using a wake-word system.The system used a small-form-factor device to deploy the EON when a user interacted with the speaker.Source: §9 · p. 22
A SENTIMENT ANALYSIS RESULTS
The sentiment-analysis evaluation found that short command transcripts produced inconsistent emotional inferences. The appendix also describes the SER classifiers and recording procedures used in the broader evaluations.
A SENTIMENT ANALYSIS RESULTS
- Google Cloud Sentiment Analysis found 8 of 106 utterances not neutral at confidence > 0.5; the listed examples had scores from -0.7 to 0.7.Source: A SENTIMENT ANALYSIS RESULTS · p. 27
- The authors describe sentiment inferences from short transcripts as inconsistent, including a nominally neutral TESS phrase classified as negative.They state that it was unclear why some commands received positive or negative labels.Source: A SENTIMENT ANALYSIS RESULTS · p. 27
B SPEECH-EMOTION-RECOGNITION (SER) CLASSIFIER DETAILS
- The evaluations used a broad set of SER classifiers with heterogeneous features, topologies, and classes.The passage points to further classifier details but does not enumerate them.Source: B SPEECH-EMOTION-RECOGNITION (SER)… · p. 27
C DETAILS REGARDING ACOUSTIC EVALUATION RECORDINGS
- Fine-tuning audio was captured with an external microphone touching the speaker at an empirically determined 100-degree angle.The setup was intended to capture room acoustics and sound conduction through the speaker without privileged access.Source: C DETAILS REGARDING ACOUSTIC… · p. 28
- For acoustic evaluation, EON-modified audio was presented to the smart speaker, then recordings were downloaded from Alexa’s voice-history page for evaluation.Fine-tuning and evaluation involved separate recording actions.Source: C DETAILS REGARDING ACOUSTIC… · pp. 27–28
D USABILITY SURVEY
A survey of 19 college students assessed concern about emotion-targeted advertising and willingness to use DARE-GP versus a record/playback system. Nearly all respondents found DARE-GP’s EON noninvasive or inaudible, while nearly half were unwilling to use the alternative.
D USABILITY SURVEY
- Nineteen college students assessed concern about emotion-targeted ads, distraction from the first system, and willingness to use both systems.The survey questions covered concern about companies using emotion data to target ads, distraction from the first system, and willingness to use each system.Source: D USABILITY SURVEY · pp. 28–29
- The first system played an EON while respondents spoke a smart-speaker command; the other recorded, modified, and replayed speech with a wake word.Respondents were introduced to two exemplar systems intended to protect emotional information in speech.Source: D USABILITY SURVEY · pp. 28–29
- Nearly all respondents found DARE-GP’s EON noninvasive or inaudible, none were unwilling to use it, and nearly half were unwilling to use record/playback.Survey responses are summarized in Table 12.Source: D USABILITY SURVEY · p. 29, Table 12 · p. 30