Source-linked AI summary
Intent Laundering: AI Safety Datasets Are Not What They Seem
Shahriar Golchin, Marc Wetter
TL;DR
Existing adversarial safety datasets may not faithfully represent real-world attacks because they overuse overt triggering cues. The paper evaluates this gap with intent laundering, which removes those cues while preserving malicious intent, and reports sharply higher attack success rates and effective black-box jailbreaking across studied models.
Problem
The paper asks whether widely used adversarial safety datasets reflect real-world attacks and genuinely measure safety risks, given the importance of dataset quality and data-point diversity.
Method
The paper analyzes AdvBench and HarmBench in isolation and practice, then uses intent laundering to remove triggering cues while preserving malicious intent and relevant details.
Results
Removing triggering cues raises mean ASR from 3.86% to 87.54% on AdvBench, 10.55% to 79.10% on HarmBench, and 3.54% to 88.06% on StrongREJECT; iterative laundering reaches 90.00%–100.00% ASR across models and datasets.
Takeaways & Limitations
The findings indicate that existing dataset-based evaluations can overestimate model safety and expose a gap between dataset evaluation and real-world adversarial behavior.
Takeaways & Limitations
The analysis excludes topic diversity because dataset creators generally account for it, while focusing on diversity at the data-point level.
Abstract
from arXiv · showhide
We systematically evaluate the quality of widely used adversarial safety datasets from two perspectives: in isolation and in practice. In isolation, we examine how well these datasets reflect real-world adversarial attacks based on three defining properties: being driven by ulterior intent, well-crafted, and out-of-distribution. We find that these datasets overrely on "triggering cues": words or phrases with overt negative/sensitive connotations that are intended to trigger safety mechanisms explicitly, which is unrealistic compared to real-world attacks. In practice, we evaluate whether these datasets genuinely measure safety risks or merely provoke refusals through triggering cues. To explore this, we introduce "intent laundering": a procedure that abstracts away triggering cues from adversarial attacks (data points) while strictly preserving their malicious intent and all relevant details. Our results show that current adversarial safety datasets fail to faithfully represent real-world adversarial behavior due to their overreliance on triggering cues. Once these cues are removed, all previously evaluated "reasonably safe" models become unsafe, including Gemini 3 Pro and Claude Sonnet 3.7/4. Moreover, when intent laundering is adapted as a jailbreaking technique, it consistently achieves high attack success rates, ranging from 90.00% to 100.00%, under fully black-box access. Overall, our findings expose a significant disconnect between how existing datasets evaluate model safety and how real-world adversaries behave.
1 Introduction
The paper evaluates whether widely used adversarial safety datasets represent real-world attacks and measure genuine safety risks. It identifies triggering-cue overuse and shows that intent laundering exposes substantially higher attack success rates, including against frontier models.
- Safety-dataset credibility depends on representing adversarial attacks that are ulterior-intent-driven, well-crafted, and out-of-distribution.
- AdvBench and HarmBench are studied because of their outsized influence on AI safety evaluation and their role as seed data for derivative datasets.
- N-gram analysis finds recurring inherent and contextual triggering cues with overt negative or sensitive connotations, producing duplication and undermining all three attack properties.
- Intent laundering abstracts away triggering cues while preserving malicious intent and relevant details through connotation neutralization and context transposition.
- 3.86% to 87.54% on AdvBench, 10.55% to 79.10% on HarmBench, and 3.54% to 88.06% on StrongREJECT after triggering cues are removed.
- 90.00%–100.00% ASR is achieved across studied models after adding iterative revision–regeneration, including Gemini 3 Pro and Claude Sonnet 3.7/4.
2 Empirical Motivation
AdvBench and HarmBench contain overt triggering cues and substantial duplication, making them poor approximations of real-world adversarial attacks and potentially inflating safety evaluations.
- Word Clouds: N-gram analysis reveals unusual overrepresentation of overtly negative or sensitive triggering words and phrases in AdvBench and HarmBench.Triggering cues intensify from unigrams to higher-order n-grams.
- Word Clouds: Triggering cues directly contradict real-world attacks, which are typically well-crafted and conceal harmful intent behind less overt language.The datasets therefore fail to capture two defining properties of real-world adversarial behavior.
- Data Duplication: Over 45% of AdvBench data points are near-identical at a 0.95 similarity threshold, while over 11% are almost exact copies at 0.99.These rates are unusually high for a 520-point dataset intended to represent out-of-distribution, well-crafted attacks.
- Data Duplication: At a 0.85 threshold, only about 11% of AdvBench points are unique, compared with nearly 94% in its size-matched GSM8K subset.HarmBench has 16% duplicated points versus 3.5% in its GSM8K counterpart at the same threshold.
- Data Duplication: Triggering-cue overuse and duplication undermine the datasets’ representation of real-world attacks and can inflate safety evaluations.Near-duplicate groups often encode the same malicious intent in nearly identical contexts.
3 Approach
The paper introduces intent laundering to remove overt triggering language while preserving malicious intent and relevant details, then evaluates safety and practicality before extending the procedure into iterative jailbreaking.
- Intent Laundering: Intent laundering abstracts away overt triggering language while preserving the original attack’s malicious intent and relevant details.The procedure targets whether safety evaluations rely on cues that aligned models are trained to detect and refuse.
- Intent Laundering: Connotation neutralization replaces negative or sensitive cues with neutral, positive, or descriptive alternatives.This component changes how harmful requests are expressed without removing their underlying intent.
- Intent Laundering: Context transposition replaces triggering real-world scenarios and referents with non-real-world alternatives while retaining transferable details.Examples include fictional narratives or game worlds with stand-ins for people and institutions.
- Intent Laundering: An LLM automates laundering through few-shot in-context demonstrations pairing original data points with manually crafted revisions.Generated revisions are evaluated against the original data points for safety and practicality.
- Evaluation: Attack success requires a response to be both unsafe and practical under independently assessed five-point Likert scales.Unsafe means rated extremely unsafe, highly unsafe, or unsafe; practical means rated extremely applicable, highly applicable, or applicable.
- Jailbreaking: The jailbreaking extension iteratively regenerates failed revisions, using prior failures to improve subsequent revisions until a regeneration limit or target ASR is reached.The mechanism seeks both unsafe responses and responses with practical, transferable details.
4 Experimental Setup
Experiments compare safety datasets with size-matched GSM8K subsets, evaluate unique points across three datasets, and use diverse models, automated criteria generation, and LLM judging under specified inference settings.
- Dataset Analysis: Word clouds use the 40 most frequent unigrams, bigrams, and trigrams from AdvBench and HarmBench after limited text normalization and filtering.Stopwords, punctuation, special characters, and selected instruction words are removed for visualization.
- Evaluation Models: GPT-4o generates evaluation criteria with 8-shot prompting, while GPT-5.1 performs intent laundering and LLM-based judging with separate inference caps.Model-specific reasoning settings and default inference hyperparameters are used as specified.
- Evaluation Datasets: A conservative 0.9 similarity threshold yields 207 AdvBench points and all 200 standard HarmBench points after selecting one representative per duplicate group.Points above the threshold are treated as duplicates sharing similar sentence structures and identical malicious intent.
- Evaluation Datasets: StrongREJECT contributes 288 non-overlapping points after excluding 25 items taken directly from AdvBench.The extension assesses whether the findings generalize beyond the two primary datasets.
- Evaluation Models: The evaluation uses 10 models, including Gemini 3 Pro, Claude Sonnet 3.7, GPT-4o, Llama 3.3 70B, and Gemini 2.5 Flash.Claude Sonnet 4 replaces Sonnet 3.7 for StrongREJECT because the former was deprecated.
5 Results and Discussion
Across AdvBench, HarmBench, and StrongREJECT, removing triggering cues sharply increases attack success, while iterative intent laundering further strengthens jailbreak performance across models. These results suggest that current evaluations and some alignment techniques may rely on shared triggering cues.
- Removing triggering cues: StrongREJECT ASR rises from a mean of 3.54% to 88.06% after triggering cues are removed.The same pattern generalizes beyond AdvBench and HarmBench.
- Iterative jailbreaking: Intent laundering achieves 90.00%–100.00% ASR across all studied models and datasets within only a few iterations.The result includes Gemini 3 Pro and Claude Sonnet 3.7.
- Iterative jailbreaking: Mean ASR increases by 9.00% on AdvBench, 11.81% on HarmBench, and 9.89% on StrongREJECT by the final iteration.The first iteration produces the largest leap, but subsequent iterations continue raising ASR.
- Generalization and implications: Intent laundering remains effective on StrongREJECT despite being designed from AdvBench and HarmBench, suggesting it exploits common safety-dataset weaknesses.The authors interpret this as evidence that the method is not specific to its source datasets.
6 Related Work
Related work frames safety alignment as preventing unsafe behavior and safety datasets as evaluating those defenses. Prior adversarial attacks include invasive weight-modification methods and non-invasive prompt-engineering methods, while newer alignment work often uses reasoning to improve robustness.
- Safety Alignment: Safety alignment balances helpfulness and harmlessness while avoiding both underrefusal and overrefusal.Adversarial attacks can disrupt this balance and elicit harmful outputs from aligned models.
- Adversarial attacks: Invasive attacks modify model weights through specialized training or fine-tuning, whereas non-invasive attacks operate through input prompt engineering.The passage distinguishes the categories by whether model parameters are altered.
- Adversarial attacks: Non-invasive attacks include ciphers, many-shot in-context learning, membership inference, and engineered red-teaming prompts.These methods can be human-crafted or generated by language models.
- Recent alignment methods: Recent safety-alignment work primarily leverages reasoning to improve robustness, while the same reasoning capability can be exploited adversarially to jailbreak models.The cited literature describes this dual role in large reasoning models.
- Safety Datasets: Safety datasets aim to measure alignment effectiveness by capturing real-world scenarios, with earlier benchmarks targeting risks such as bias, toxicity, and ethical judgment.HarmBench is identified as a prominent benchmark for broader safety alignment.
7 Conclusion
The conclusion argues that AdvBench and HarmBench overrely on triggering cues and therefore do not faithfully approximate real-world adversarial behavior. Across 10 models, removing those cues undermines prior safety conclusions, while intent laundering produces high attack success.
- Conclusion: AdvBench and HarmBench do not faithfully approximate real-world adversarial behavior because they overuse triggering cues designed to artificially activate safety mechanisms.The cues have overt negative or sensitive connotations.
- Conclusion: Intent laundering removes triggering cues while preserving malicious intent and all relevant details.The procedure tests whether safety conclusions persist without overt cues.
- Conclusion: Across 10 models, prior safety conclusions do not hold once triggering cues are removed, including for Gemini 3 Pro and Claude Sonnet 3.7/4.The observed safety performance is described as largely driven by triggering cues rather than underlying safety risks.
Ethics Statement
The authors acknowledge that their findings may affect public and institutional trust in current AI safety claims and evaluations. They frame the work as intended to strengthen scientific rigor and practical relevance, alongside responsible disclosure.
- Ethics Statement: The authors acknowledge that their findings may affect public and institutional trust in current AI safety claims and evaluations.They distinguish this possible effect from their stated intent to undermine trust in AI safety research.
- Ethics Statement: The authors state that they proactively shared their findings with affected model providers and collaborated to improve product safety.This disclosure approach accompanies their discussion of potential societal risks and benefits.
A Agreement Analysis Between LLM-Based and Human Evaluations
The study compares an LLM judge with consensus ratings from three human experts on binary safety and practicality evaluations. The judge shows higher agreement with human consensus than mean human–human agreement in both categories.
- Human consensus used three experts, while safety and practicality responses were reduced to binary categories.Safety was categorized as unsafe or safe, and practicality as practical or impractical.
- 90.00% agreement was achieved between the LLM judge and human consensus for binary safety evaluation.The 95% CI was 83.00%–95.00%.
- 94.00% agreement was achieved between the LLM judge and human consensus for binary practicality evaluation.The 95% CI was 89.00%–98.00%.
- The LLM judge’s agreement exceeded mean pairwise human agreement for both safety and practicality.Mean human–human agreement was 82.00% for safety and 77.33% for practicality.
B Examples of Intent-Laundered Revisions and Model Responses
The paper presents unsafe responses from Gemini 3 Pro and Claude Sonnet 3.7 to intent-laundered revisions based on AdvBench and HarmBench data points. These responses were detailed and transferable to real-world contexts despite abstract framing.
- Gemini 3 Pro produced an unsafe, detailed response to an intent-laundered AdvBench revision.The response was evaluated as unsafe and applicable to real-world contexts.
- Claude Sonnet 3.7 produced an unsafe, detailed response to an intent-laundered HarmBench revision.The response was evaluated as unsafe and applicable to real-world contexts.
- Both responses remained transferable to real-world contexts despite being framed as abstract scenarios.
C Input Prompts
The study uses prompts for intent laundering, iterative revision and regeneration, evaluation-criteria generation, and safety and practicality assessment. Figures 7–12 present these prompts in sequence, while Figures 5 and 6 illustrate resulting model responses.
- Figures 5 and 6 provide examples of model responses to intent-laundered revisions based on AdvBench and HarmBench data points.
- Intent laundering begins with a prompt that generates a revision from the data point being transformed.
- The revision–regeneration loop uses failed revisions together with the original data point to produce further revisions.
- An evaluation-criteria prompt is populated with the original data point before assessing generated responses.
- Safety and practicality evaluation prompts place the evaluation criterion before the model response.
- Regular safety evaluation assesses responses to original attacks without applying intent laundering.