JALMBench: Benchmarking Jailbreak Vulnerabilities in Audio Language Models

• 21.5% average ASR for harmful audio queries exceeded 17.0% for text, while AdvWave reached 96.2% ASR on jailbreak attacks.Source: §1 · p. 2
• LALMs more readily reject explicit hate content than misinformation, and non-US accents tend to increase attack success rates.Source: §1 · p. 2
• Discrete tokenization may preserve textual safety properties better than continuous feature extraction, while interleaved modality choices affect cross-modal robustness.Source: §1 · p. 2
• Prompt- and response-level defenses reduced average ASR by 19.6 and 18.0 percentage points, respectively, but prompt mitigation reduced utility.Source: §1 · p. 2
• 245,355 audio samples, over 1,000 hours of audio, and 11,316 text samples form the benchmark’s dataset.Source: §1 · pp. 1–2
• JALMBench evaluates 12 LALMs against 8 text-transferred and audio-originated attacks, with analyses spanning efficiency, topics, voices, and architecture.Source: §1 · p. 2

Large Audio Language Models

• LALMs combine speech encoders with text processing, using either continuous acoustic features or token-based audio encoding strategies.Source: §2 · p. 3

Jailbreak Attacks

• LLM jailbreaks include white-box gradient-based methods and black-box template completion, prompt rewriting, and LLM-generation approaches.Source: §2 · p. 3
• LALM attacks include audio editing, SSJ’s cross-modal separation, and adversarial audio optimization such as AdvWave.Source: §2 · p. 3

Benchmark Gap

• Existing audio-jailbreak benchmarks focus mainly on perturbation-based, multilingual, or accent attacks, limiting their scope.Source: §2 · p. 3

Jailbreak Defenses

• Jailbreak defenses include prompt-level detection, perturbation, and additional defense prompts that avoid fine-tuning or architectural changes.Source: §2 · p. 3

• JALMBench supports 12 LALMs, 8 jailbreak attacks, and 5 defense methods through an extensible modular framework.Source: §3 · p. 3
• The benchmark contains 245,355 audio samples spanning over 1,000 hours and 11,316 text samples.Source: §3 · p. 4

Harmful Query Category

• Its harmful-query category contains 246 curated textual queries, synthesized audio counterparts, diverse voice variants, and some human recordings.Source: §3 · p. 4

Text-Transferred Jailbreak Category

• The text-transferred category applies ICA, DAN, DI, and PAP to harmful queries and converts the resulting attacks into audio counterparts.Source: §3 · p. 4

Audio-Originated Jailbreak Category

• The audio-originated category contains adversarial audio generated by SSJ, AMSE, BoN, and AdvWave specifically for LALMs.Source: §3 · p. 4

Text-Transferred Attacks

• Audio inputs generally produce higher ASR than text, but SpeechGPT and Spirit LM are more vulnerable in text, while LLaMA-Omni and VITA-1.0 are more vulnerable in audio.The latter pattern is attributed to insufficient audio-specific safety alignment.Source: §4.1 · p. 5
• PAP exceeds 90% ASR across most models and modalities, whereas ICA performance degrades with three in-context examples because inputs average 330.4 seconds and often exceed model context limits.GPT-4o-Audio and DiVA are comparatively robust, while VITA-1.0 and LLaMA-Omni are notably vulnerable.Source: §4.1 · pp. 5–6

Audio-Originated Attacks

• AdvWave achieves near-perfect effectiveness, increasing average ASR by up to 97% and compromising even strongly aligned models such as GPT-4o-Audio.Audio-originated attacks generally outperform text-transferred attacks, exposing weak robustness to direct audio manipulation.Source: §4.1 · p. 6

Attack Efficiency

• Attacks exceeding 60% ASR typically require at least 100 seconds, whereas SSJ and AMSE reach around 40% ASR within approximately 10 seconds.The results indicate a trade-off between attack efficiency and effectiveness, with stronger attacks requiring more processing time.Source: §4.2 · pp. 6–7

Effect of Topics and Voice Diversity

• Hate and Harassment queries have a relatively low average ASR of 41%, while language switching causes substantially greater ASR variability than changes in speaker gender, accent, TTS system, or human voice.One overt hate-speech example averages 18% ASR across models and attacks.Source: §4.2 · pp. 7–8

Effect of Architecture

• LLaMA-Omni shows a stark modality gap, with 9.6% text ASR versus 58.9% audio ASR, whereas GLM-4-Voice has nearly identical ASRs of 18.7% and 19.5%.The findings associate discrete audio tokenization and interleaved audio-text training with tighter cross-modal alignment; continuous encoders can require additional alignment objectives.Source: §4.2 · p. 8

Defense comparison

• Response-level defenses typically provide stronger safety effectiveness, while prompt-level methods trade greater ASR reduction for greater utility loss.Across 12 models, 8 attack types, and 5 defenses, the study compares both defense levels; AdaShield reduces average ASR by 19.6 percentage points but decreases accuracy by up to 6.3%.Source: §5 · p. 9, Table 2 · p. 9

Efficiency in Mitigation

• 118.6% higher average attack cost shows that the best defenses can substantially increase the rounds needed for successful PAP, BoN, and AdvWave attacks.These defenses remain insufficient to fully prevent the listed sophisticated attacks.Source: Prompt-Level Mitigation · p. 9, Table 3 · p. 9

Utility in Mitigation

• Response-level moderation has minimal impact on QA utility, whereas prompt-level mitigation incurs a non-negligible utility drop.Utility is measured on 455 OpenBookQA questions converted to audio, with average audio duration of 18.9 seconds per question.Source: Prompt-Level Mitigation · pp. 9–10, §5 · p. 9

Takeaways

• The authors characterize response-level moderation as the preferred deployable black-box defense because it combines strong safety gains with minimal utility loss.They nevertheless argue that layered defense-in-depth remains necessary because sophisticated attacks are not eliminated.Source: Prompt-Level Mitigation · pp. 9–10

Safety–utility trade-offs

• Figure 8 frames mitigation as a safety–utility trade-off by comparing ASR reduction with QA accuracy and refusal rate with accuracy.The figure compares defense mechanisms across models and LALMs without defenses.Source: Figure 8 · p. 10

Limitations

• The benchmark does not exhaustively cover speaker identity, emotional prosody, or fine-grained accent variation, which may influence attack success.These voice-related factors are identified as a scope limitation for the current evaluation.Source: §6 · p. 10
• Multi-turn jailbreak attacks remain underexplored, and some models give minimal acknowledgments that could conceal more effective interaction-based behaviors.The discussion specifically mentions Gemini-2.0 and SALMONN as examples.Source: §6 · p. 10
• The study leaves quantization and reasoning-mode effects for future work and has limited audio samples for some attacks, including DAN.The authors suggest that more diverse audio prompts could provide stronger empirical insights.Source: §6 · p. 10

Conclusion

• Across 12 LALMs, 8 attack methods, and 5 defenses, current models remain vulnerable, particularly to audio-originated attacks, while adapted vision-language defenses are largely ineffective.The conclusion positions JALMBench as a systematic benchmark for harmful-query and jailbreak safety evaluation.Source: §6 · p. 10

Ethics

• Six PhD students recorded harmful-query utterances with informed consent, and the private recordings will not be disclosed or published.The protocol was submitted in advance for institutional review, and the authors state that the study did not involve direct experimentation with human subjects or participants.Source: ETHIC STATEMENT · p. 11

Implementation

• LLMs were used for grammar and spelling checks, adversarial-prompt generation, and judging whether LALMs were successfully jailbroken.Their use is described as central to the attacks-and-defenses framework.Source: USAGE OF LLMS · p. 11

Benchmark use

• JALMBench supports text or audio inputs without additional preprocessing and allows users to test their own datasets.The framework is presented as a comprehensive tool for LALM safety risk assessment.Source: A USING JALMBENCH · p. 16

Input module

• The benchmark accepts configurable TTS languages, accents, and gendered voices, and users may replace Google TTS with their own tools.Its input module supports text, audio, and prompts with or without a system prompt.Source: A USING JALMBENCH · p. 16

Extensibility

• The framework implements eight attacks and five defenses, supports three judge models, and can be extended with new LALMs through an abstract class.It also provides post-processing for transcribing audio outputs and a CUDA-compatible Docker image.Source: A USING JALMBENCH · p. 16

Data preprocessing

• Queries are manually filtered to remove non-harmful examples and support clear topic categories for downstream analysis.Examples removed include legitimate requests about commodity trading and police-community relations.Source: §B.1 · p. 16

Harmful-query categories

• The harmful-content framework covers categories including illegal activity, child exploitation, hateful or violent content, malware, physical or economic harm, fraud, disinformation, and adult content.These categories are drawn from OpenAI’s Usage Policy.Source: Evaluation prompt · p. 17

Policy scope

• The preprocessing and evaluation materials also include restrictions concerning privacy, legal advice, financial advice, medical advice, and high-risk government decisions.These restrictions are specified as disallowed or unsafe uses in the policy text.Source: Evaluation prompt · p. 18

Evaluation setup

• All benchmark experiments use greedy decoding with top_k=1 for models and judge models to ensure deterministic outputs.The experiments required approximately 6,000 GPU-hours on eight NVIDIA L20 GPUs.Source: C ATTACK EVALUATION · p. 22
• The primary judge model is GPT-4o-2024-11-20, and jailbreak success is defined using ratings of at least 4.Ratings of 4 or 5 are categorized as successful jailbreaks, while ratings of 3 or below are unsuccessful.Source: §C.1 · p. 22

Judge repeatability

• 0.46% disagreement separates greedy GPT-4o judging from majority-vote sampled evaluations, indicating high consistency between decoding strategies.Three sampling evaluations had 0.83% overall repeat inconsistency.Source: §C.1.1 · pp. 22–23

Cross-model consistency

• α = 0.913 indicates strong agreement among GPT-4o-2024-11-20, LLaMA-3.3-70B-Instruct, and Qwen3-80B-A3B-Instruct.The analysis covers 1,080 evaluated samples, and α ≥0.80 is treated as strong reliability.Source: §C.1.2 · p. 23

Human verification

• Cohen’s κ reaches 0.96 between the two primary human annotators and 0.97 between final human labels and GPT-4o judgments.Human evaluation covered sampled successful and failed jailbreaks across attack types and models.Source: §C.1.3 · p. 23

Overall reliability

• Together, the checks support evaluator stability across repeated runs, independent judge models, and human verification.The paper reports only rare discrepancies across these dimensions.Source: §C.1.3 · p. 23

ASR reporting

• Lower ASR indicates better safety, with text and text-transferred attacks reported in Table 8 and audio-originated attacks in Table 9.Subscripts denote changes relative to AHarm.Source: §C.5 · p. 25, Table 8 · p. 25, Table 9 · p. 25

Text and transferred attacks

• Table 8 provides detailed ASR (%) results for text and text-transferred attack methods.The table is part of the benchmark’s separated modality-specific reporting.Source: Table 8 · p. 25

Audio-originated attacks

• Table 9 provides detailed ASR (%) results for audio-originated attack methods.These results are reported separately from text and text-transferred attacks.Source: Table 9 · p. 25

D.1 RESULTS OF VOICE DIVERSITY

• Voice-diversity experiments synthesize harmful queries across English accents, gendered voices, additional TTS engines, and nine target languages.Google TTS generates British, Indian, and Australian accents plus male and female US voices; F5-TTS, MMS-TTS, and SpeechT5 provide additional TTS variants.Source: §D.1 · p. 25, Table 10 · p. 26
• The benchmark reports average ASR for human-recorded samples across six speakers to evaluate robustness to speaker variation.The reported average aggregates model responses across all six speakers.Source: §D.1 · p. 26
• The harmful queries are short, averaging 12.32 words, and manual checks with multilingual speakers were used to assess translation quality.Queries range from 3 to 29 words, reducing their exposure to translation errors.Source: §D.1 · p. 26
• Voice-diversity results are summarized in Table 1, while multilingual effects are reported separately in Table 10.AHarm uses English text with a US accent and neutral gender as the default configuration.Source: §D.1 · p. 26, Table 10 · p. 26

E.3 MITIGATION RESULTS

• Figure 10 reports ASR for eight attacks with different defenses, while Figures 11 and 12 show average score distributions across models, attacks, and defenses.These visuals provide the detailed attack-level and model-level comparisons for mitigation performance.Source: §E.3 · p. 28, Figure 10 · p. 29, Figure 11 · p. 30
• Mitigation evaluation jointly measures utility on OpenBookQA and average ASR under different techniques, enabling safety–utility comparison.Table 12 reports accuracy (%) and ASR (%) together, and Figure 8 plots their Pareto-optimal combinations using architecture-specific colors.Source: Table 12 · p. 31, §F.2 · p. 31, Figure 13 · p. 31
• VITA-1.5 and GPT-4o-Audio are Pareto-optimal for utility and latency, respectively, with VITA-1.5 faster and GPT-4o-Audio more accurate.The comparison uses per-model latency and utility measurements.Source: §F.3 · p. 31
• JALMBench frames mitigation as necessary because harmful LALM outputs can enable illegal activity and existing evaluation coverage is inconsistent.The benchmark aims to provide a unified framework spanning attacks, defenses, datasets, and LALMs.Source: G SOCIAL IMPACTS · p. 31