Source-linked AI summary
ARENA: Automated Red-Teaming for Large Audio Language Models
Jiaming He, Zhicong Huang, Tian Jin, Zhen Sun, Cheng Hong, Yi Yu, Wenbo Jiang, Xudong Jiang
TL;DR
Text-only red-teaming can miss harmful behavior that emerges when safe-looking queries are combined with audio. ARENA addresses this with closed-loop audio-grounded red-teaming and finds substantially more failures across four LALMs while maintaining a near-perfect prompt-pass rate.
Problem
Text-only red-teaming may miss unsafe assistance induced by jointly interpreting a safe text query and audio input.
Method
ARENA trains and refines a controller for text-safe, audio-grounded test cases using adaptive feedback, while reserving Llama Guard 3 for final evaluation.
Results
Across four target LALMs, ARENA finds substantially more failures than static audio jailbreak baselines while maintaining a near-perfect prompt-pass rate.
Takeaways & Limitations
Safety evaluation of LALMs should test the full generation-and-execution loop rather than only fixed prompts.
Takeaways & Limitations
Remaining failures mainly involve refusals, incorrect audio-event granularity, or non-actionable descriptions despite risky situations.
Abstract
from arXiv · showhide
Large audio-language models (LALMs) make it possible to interact with language models through speech, music, and environmental sound, but they also introduce a safety surface that is difficult to expose with text-only red-teaming. We study automated audio-grounded red-teaming, where a text query must remain safe in isolation while the joint text-audio input induces harmful target behavior. We propose ARENA, a closed-loop framework that trains a controller on an independent 2,000case text-audio dataset. MD-Judge supplies training rewards and adaptive search feedback, while a separate, non-adaptive Llama Guard 3 evaluator alone labels final outcomes. On 520 held-out AdvBench objectives, ARENA achieves FDR/PSR of 87.9/100.0%, 71.5/96.3%, 68.1/100.0%, and 75.4/98.5% on Audio Flamingo 3, Qwen2-Audio, MiMo-Audio, and GPTAudio, respectively. Ablations show that feedback-based refinement and audio-variant search substantially improve attack discovery.
1 Introduction
Audio inputs create safety vulnerabilities that text-only red-teaming can miss because benign text may become harmful when combined with speech or environmental sound. ARENA addresses this gap with a closed-loop controller that jointly generates text-safe queries and audio prompts, using adaptive feedback to discover unsafe LALM behavior.
- Motivation: Benign-looking text can become harmful when combined with spoken content, speaker cues, or non-speech acoustic evidence.The joint text-audio input may cause the target LALM to produce unsafe assistance.
- Method: ARENA formulates audio-grounded red-teaming as a closed-loop text-audio generation problem.The controller uses failed responses to identify bottlenecks and refine subsequent attempts.
- Evaluation: 2,000 text-audio specifications train the ARENA controller disjoint from the evaluation set.MD-Judge supplies reward labels and adaptive refinement feedback, while Llama Guard 3 independently scores final outcomes and is never exposed to the controller.
- Evaluation: 520 held-out AdvBench objectives are used to evaluate ARENA on Audio Flamingo 3, Qwen2-Audio, MiMo-Audio, and GPT-Audio.The evaluation includes open-source and API-based LALMs, along with baseline comparison, transferability analysis, and ablations.
2 Related Work
Prior work established large audio-language models by combining acoustic representations with language reasoning, while multimodal red-teaming developed automated searches and feedback-guided attack refinement. HarmBench further standardized behaviors, attacks, and evaluation for this setting.
- Audio-language models: LALMs connect acoustic representations with language reasoning across speech, music, and environmental sound.Whisper, CLAP, and AudioLM established scalable speech recognition, audio-text alignment, and neural audio token modeling.
- Audio-language models: SpeechGPT and AudioPaLM integrated spoken interaction with large language models.The related work also identifies general-purpose LALMs including Pengi, LTU, SALMONN, and Qwen-Audio.
- Multimodal red-teaming: Multimodal red-teaming searches for unsafe text-to-image inputs and uses visual or system feedback to guide subsequent attacks.SneakyPrompt, MMA-Diffusion, and Groot search for unsafe inputs, while ART, FGPI, and RPG-RT use feedback for attack guidance.
3 Methodology
ARENA formulates audio-grounded red-teaming as finding text-safe queries whose joint audio-text inputs induce harmful, audio-grounded compliance in black-box LALMs. Its closed-loop methodology combines modality-aware candidate generation, staged target querying, reward-based controller training, adaptive refinement, and independent final evaluation.
- Problem formulation: ARENA seeks test cases where qp is text-safe alone, but (qp, a) induces harmful compliance grounded in recognized audio content.The target response is evaluated jointly with the objective, audio, recognition response, and candidate prompts.
- Candidate generation: The controller maps each objective to (qp, pa, m), choosing a text-safe query, content prompt, and speech-or-sound rendering modality.Speech uses text-to-speech for linguistic or social cues, while sound uses TangoFlux for environmental acoustic events.
- Seed pool: The training-only seed pool contains 2,000 independent specifications: 1,200 speech cases and 800 environmental-audio cases, disjoint from AdvBench.Speech seeds cover spoken-content risks, whereas environmental-audio seeds cover non-speech events such as explosions, collisions, and forced entry.
- Target interaction: ARENA first rejects text-unsafe qp, then renders remaining prompts and queries the target with recognition and red-teaming requests using the same audio.The recognition response helps determine whether the final harmful response is grounded in the intended audio content.
- Training and evaluation: Reward-weighted supervised fine-tuning uses shaped MD-Judge rewards, upweighting higher-reward near misses and adding unit weight to detected-fault examples.MD-Judge supplies reward labeling and search feedback, while final reported scoring uses independent Llama Guard 3 evaluation after search.
4 Experiments
ARENA is evaluated on 520 held-out AdvBench objectives across four target LALMs, using frozen-controller testing and non-adaptive Llama Guard 3 evaluation. It outperforms static baselines, while feedback refinement and sound-variant search substantially improve attack success, with transfer remaining substantial but imperfect.
- Evaluation setup: 520 held-out AdvBench objectives evaluate four targets: Audio Flamingo 3, Qwen2-Audio-7B-Instruct, MiMo-Audio-7B-Instruct, and GPT-Audio.AdvBench does not overlap with the 2,000 controller-training specifications, and objectives span six harmful categories.
- Main comparison: ARENA achieves the highest FDR on every target while maintaining near-perfect PSR, unlike static baselines that respectively struggle with unsafe elicitation or input-side moderation.AJailBench uses fixed speech-rendered templates, whereas JALMBench provides diverse audio-language prompts; neither uses reward labeling or feedback refinement.
- Transferability: 59.7%, 60.0%, and 50.4% ASR transfer AF3-found cases to Qwen2-Audio, MiMo-Audio, and GPT-Audio, respectively, showing substantial but incomplete cross-target transfer.Replay omits controller generation and refinement, so auditing a new target still benefits from closed-loop refinement.
- Refinement budget: 88%, 74%, and 68% ASR at K = 30 on AF3, Qwen2-Audio, and MiMo-Audio rises from only 23–30% at K = 0, with most gains in ten rounds.Judge feedback corrects recognition failures and overly direct text framing, while later rounds address specific acoustic events or less refusal-triggering queries.
- Sound variants: 95%, 96%, and 85% ASR at M = 16 on AF3, Qwen2-Audio, and MiMo-Audio improve from 70%, 59%, and 49% at M = 1.Multiple TangoFlux realizations keep the first unsafe response found, showing waveform realization matters even when the semantic audio prompt is fixed.
- Category and failure analysis: ARENA remains effective across six harmful categories and preserves nontrivial, distributed test-case diversity, but failures include refusals, misrecognized acoustic granularity, and near misses.Violence/weapons and fraud are especially successful; self-harm/substance abuse is harder on AF3 and MiMo-Audio, while cybersecurity depends more on the target.
5 Conclusion
ARENA is a closed-loop framework for automated red-teaming of large audio-language models in a text-safe, audio-grounded threat setting. It combines MD-Judge-guided prompt refinement with Llama Guard 3 final labeling, finding more failures than static audio jailbreak baselines while maintaining a near-perfect prompt-pass rate.
- 5 Conclusion: ARENA automates audio-grounded red-teaming by refining modality-aware prompts with MD-Judge feedback under a text-safe threat setting.Llama Guard 3 is used only for final labels.
- 5 Conclusion: Across four target LALMs, ARENA finds substantially more failures than static audio jailbreak baselines while maintaining a near-perfect prompt-pass rate.The transfer and ablation results indicate shared audio-grounded weaknesses across current LALMs.