Source-linked AI summary
Audio Jailbreak: An Open Comprehensive Benchmark for Jailbreaking Large Audio-Language Models
Zirui Song, Qian Jiang, Mingxuan Cui, Mingzhe Li, Lang Gao, Zeyu Zhang, Zixiang Xu, Yanbo Wang, Chenxi Wang, Guangxian Ouyang, Zhenhao Chen, Xiuying Chen
TL;DR
LAM jailbreak safety lacks systematic evaluation that captures speech’s temporal and semantic characteristics. AJailBench introduces a broad adversarial-audio benchmark and semantically constrained audio perturbation toolkit, finding that leading LAMs remain vulnerable and that optimized perturbations further reduce safety performance.
Problem
Research lacks systematic, quantitative evaluation of LAM safety against jailbreak attacks despite the distinctive temporal and semantic challenges of speech.
Method
AJailBench combines AJailBench-Base, containing 1,495 adversarial audio prompts across 10 policy-violating categories, with APT-generated variants optimized under semantic consistency constraints.
Results
Evaluations of leading LAMs find no single model robust across all safety dimensions, while semantically preserved perturbations further degrade performance.
Takeaways & Limitations
AJailBench provides a practical testbed showing the need for more robust, semantically aware defenses against audio jailbreaks.
Takeaways & Limitations
The study focuses primarily on English audio and does not investigate defenses against audio adversarial attacks.
Abstract
from arXiv · showhide
The rise of Large Audio Language Models (LAMs) brings both potential and risks, as their audio outputs may contain harmful or unethical content. However, current research lacks a systematic, quantitative evaluation of LAM safety especially against jailbreak attacks, which are challenging due to the temporal and semantic nature of speech. To bridge this gap, we introduce AJailBench, the first benchmark specifically designed to evaluate jailbreak vulnerabilities in LAMs. We begin by constructing AJailBench-Base, a dataset of 1,495 adversarial audio prompts spanning 10 policy-violating categories, converted from textual jailbreak attacks using realistic text to speech synthesis. Using this dataset, we evaluate several state-of-the-art LAMs and reveal that none exhibit consistent robustness across attacks. To further strengthen jailbreak testing and simulate more realistic attack conditions, we propose a method to generate dynamic adversarial variants. Our Audio Perturbation Toolkit (APT) applies targeted distortions across time, frequency, and amplitude domains. To preserve the original jailbreak intent, we enforce a semantic consistency constraint and employ Bayesian optimization to efficiently search for perturbations that are both subtle and highly effective. This results in AJailBench-APT, an extended dataset of optimized adversarial audio samples. Our findings demonstrate that even small, semantically preserved perturbations can significantly reduce the safety performance of leading LAMs, underscoring the need for more robust and semantically aware defense mechanisms.
1 Introduction
AJailBench addresses limited systematic evaluation of LAM jailbreak robustness by combining a broad adversarial-audio benchmark with semantically preserved signal-level perturbations. Evaluations show that no single model is robust across all safety dimensions, while optimized perturbations further degrade performance.
- 1 Introduction: The benchmark targets a gap left by prior LAM jailbreak studies that mainly convert textual attacks into speech and overlook audio-specific characteristics and perturbation spaces.These limitations make existing approaches insufficient for comprehensive safety-robustness evaluation.
- 1 Introduction: AJailBench-Base contains 1,495 adversarial audio prompts spanning 10 policy-violating categories and converted from textual attacks with realistic text-to-speech synthesis.The benchmark is designed for automated and systematic evaluation of LAM jailbreak vulnerabilities.
- 1 Introduction: Audio Perturbation Toolkit (APT) generates dynamic adversarial variants through time-, frequency-, and mixing-based perturbations across seven methods.AJailBench-APT+ uses semantic consistency constraints and Bayesian optimization to identify effective perturbation configurations that preserve jailbreak intent.
- 1 Introduction: AJailBench compares its signal-level, semantically constrained, combinable perturbation benchmark with recent audio jailbreak studies and releases it openly.The comparison emphasizes broader coverage of audio perturbations and attack construction.
- 1 Introduction: AJailBench evaluates seven leading LAMs and finds no single model robust across all safety dimensions.Models exhibit different safety strategies, ranging from strict denial to permissiveness, reflecting robustness–usability trade-offs.
2 Related Work
Prior audio jailbreak research remains limited and generally transfers textual attacks into speech, with newer methods beginning to optimize audio-specific perturbations. These approaches still leave gaps in breadth and robustness evaluation.
- 2 Related Work: Existing audio jailbreak studies are limited, with early work transferring textual benchmarks such as AdvBench into speech using text-to-speech systems.This approach does not fully address the effects of audio characteristics such as pitch and frequency on audio encoders.
- 2 Related Work: ADVWAVE introduces a white-box dual-phase optimization attack for open-source models, but its scope does not provide broad evaluation across LAMs.The passage identifies this as a limitation of prior audio jailbreak work.
- 2 Related Work: Prior work motivates audio-specific benchmarks because semantic-only attacks overlook the temporal and acoustic perturbation space of speech.This gap supports evaluating signal-level manipulations beyond straightforward text-to-speech conversion.
3 AJailBench
AJailBench builds adversarial audio jailbreak data and tests dynamic perturbations while preserving the original jailbreak semantics. Its toolkit spans waveform, frequency, and hybrid transformations, with Bayesian optimization targeting reduced refusal responses.
- AJailBench construction: Text jailbreak samples are collected from published research, online examples, and automated generation tools, then converted into diverse spoken audio using 118 timbres across four English accents.The synthesis process reduces bias from individual voices while retaining automatically generated samples, including disordered vocabulary.
- AJailBench-APT+: AJailBench-APT+ addresses clean-audio limitations by combining stronger audio-specific perturbations and multiple perturbation types to increase attack diversity and effectiveness.The motivation includes temporal variation, acoustic ambiguity, and combinatorial effects across perturbations.
- Semantic consistency constraint: The unified framework represents perturbation as T(x; θ), producing x′ while restricting valid variants to S(x, x′) ≥τ so the jailbreak intent remains semantically preserved.Here x is the original waveform, x′ is the perturbed audio, S measures Similarity, and τ is the preservation threshold.
- Audio Perturbation Toolkit: APT groups parameterized editing operations into waveform-domain, frequency-domain, and hybrid perturbations that modify audio in controlled ways.Waveform operations include gain scaling, windowing, and local deletion; frequency operations use STFT-based transformations; hybrid operations combine the signal with external sounds.
- Audio Perturbation Toolkit: Energy scaling amplifies or attenuates the waveform, trimming removes a bounded interval, fade ramps alter signal boundaries, and temporal or pitch transformations independently modify timing or perceived pitch.The framework also includes hybrid perturbations such as extra-auditory priming and natural-noise injection.
- AJailBench-APT+ via Bayesian Optimization: Bayesian optimization searches a two-parameter perturbation space, where one parameter selects perturbation types and the other controls their intensity or characteristics, to reduce refusal similarity.The optimized perturbation minimizes similarity between model outputs and reference refusal phrases, exposing potential jailbreaks or unintended behaviors.
4 Benchmark Results
Across AJailBench, LAMs show heterogeneous safety behavior, while semantically consistent audio perturbations significantly degrade safety metrics. The results expose a trade-off between strict refusal and benign-query usability.
- 4.3 Performance on JailBench-base: Across the evaluated LAMs, Qwen2 Audio, DiVA, LLaMA Omni, and GPT-4o show highly consistent safety behavior between text and audio inputs.The benchmark evaluates seven open- and closed-source LAMs using five metrics: ASR, TS, PV, Relevance, and Similarity.
- 4.3 Performance on JailBench-base: SALMONN enforces extremely strict safety constraints but frequently fails to provide helpful responses to benign queries, reducing usability.This contrasts with SpeechGPT, which shows the weakest safety alignment against jailbreak attacks.
- 4.3 Performance on JailBench-base: GPT-4o often deflects potential jailbreak requests with vague, neutral language, balancing safety alignment with less restrictive responses.Its audio modality also has lower ASR, TS, and PV scores than its text counterpart, according to the benchmark results.
- 4.4 Semantic Safety Thresholds Experiment: Semantic-consistency thresholds constrain perturbation intensity so adversarial audio preserves jailbreak intent while remaining effective.The threshold process uses semantic-similarity experiments and human evaluation; Bayesian optimization selects perturbations within the predefined safe range, with ten overlay rounds chosen conservatively.
- 4.5 Performance on AJailBench-APT+: Models exhibit significantly degraded safety metrics on AJailBench-APT+ compared with AJailBench-Base, showing that semantically consistent perturbations increase attack difficulty.The comparison uses paired AJailBench-Base and AJailBench-APT+ evaluations; Table 3 marks statistically significant differences at p < 0.01.
- 4.5 Performance on AJailBench-APT+: Time-stretch and fade perturbations are most frequently selected by Bayesian optimization and have the strongest effect on degrading robustness across varied inputs.The selected techniques form the seven-tool distribution reported for AJailBench-APT+.
5 Conclusion
AJailBench systematically evaluates jailbreak vulnerabilities in LAMs using adversarial audio prompts and semantically preserved perturbations. Experiments find that current state-of-the-art LAMs remain vulnerable, motivating more robust, semantically aware defenses.
- AJailBench combines adversarial audio prompts with an Audio Perturbation Toolkit for realistic, semantically preserved attack variants.
- State-of-the-art LAMs are highly vulnerable to both static and perturbed inputs.
- AJailBench provides a practical testbed for LAM safety and highlights the need for more robust, semantically aware defenses.
A Detail implementation of Bayesian Optimization
The Bayesian Optimization procedure searches a two-dimensional audio-perturbation space by modeling good and bad parameter regions with TPE. Iterative acquisition and full-pipeline evaluation prioritize perturbations that minimize refusal similarity under a limited evaluation budget.
- Bayesian Optimization searches X = [0, 1]^2 for perturbations minimizing the refusal similarity score S(M(E(aorig; x))).
- TPE separates observations into good and bad sets using a quantile threshold, then estimates l(x) and g(x) for their parameter distributions.
- Maximizing l(x)/g(x) guides candidate selection toward parameter regions expected to produce low refusal similarity.
- The configured search uses 10 startup trials, γ = 0.10, and 24 acquisition candidates per step.
- Each iteration samples candidate parameters, executes perturbation, model inference, and refusal-similarity evaluation, then updates the search history.
- The resulting optimum identifies transformation parameters that most effectively induce targeted model behavior under limited function evaluations.
B Semantic Safety Threshold Experiment via Human Evaluation
The semantic safety threshold experiment uses human intelligibility ratings on progressively noise-perturbed audio, followed by transcription similarity analysis. Thirteen perturbation rounds yield low intelligibility for most samples, and an average similarity of 0.638 becomes the safety threshold.
- Three domain-relevant undergraduate evaluators rated each sample using a 0–10 intelligibility scale, where higher scores indicate clearer speech.
- The evaluation corpus contains 150 audio samples generated across 15 rounds, with 10 distinct samples selected per round.
- After 13 cumulative noise-addition rounds, most samples scored below 4 for intelligibility, indicating difficult-to-understand audio.
- Cohen’s Kappa was 0.72, indicating substantial agreement among annotators.
- The average textual similarity between original and 13-round perturbed transcriptions was 0.638, establishing the semantic safety threshold.
C Limitation
AJailBench does not evaluate defenses against audio adversarial attacks and focuses primarily on English inputs. Cross-lingual robustness under adversarial perturbations therefore remains unexplored.
- The study does not investigate defenses against audio adversarial attacks because established audio-specific defense methods remain limited.
- The evaluation focuses primarily on English audio, leaving cross-lingual robustness under adversarial perturbations unexplored.