Source-linked AI summary
SALAD-Bench: A Hierarchical and Comprehensive Safety Benchmark for Large Language Models
Lijun Li, Bowen Dong, Ruohui Wang, Xuhao Hu, Wangmeng Zuo, Dahua Lin, Yu Qiao, Jing Shao
TL;DR
Existing LLM safety benchmarks provide narrow coverage, limited challenge, and costly evaluation. SALAD-Bench addresses these gaps with a hierarchical benchmark containing standard, attack-enhanced, defense-enhanced, and multiple-choice questions, evaluated by MD-Judge and MCQ-Judge. It supports safety, attack, and defense evaluation while revealing varied safety performance across models and domains.
Problem
Existing safety benchmarks often cover narrow threat perspectives, traditional harmful questions are handled with about 99% safety rates, and evaluation can be time-consuming or expensive.
Method
SALAD-Bench combines a three-level taxonomy with standard, attack-enhanced, defense-enhanced, and multiple-choice subsets, using MD-Judge for QA pairs and MCQ-Judge for multiple-choice evaluation.
Results
SALAD-Bench provides 21k test samples spanning 6 domains, 16 tasks, and 66 categories, with 5k attack-enhanced, 200 defense-enhanced, and 4k multiple-choice questions.
Takeaways & Limitations
The benchmark extends LLM safety assessment to joint evaluation of models, attack methods, and defense methods, while identifying varied safety performance across models and domains.
Takeaways & Limitations
The taxonomy may become outdated as safety threats evolve, and evaluation depends on automated data filtering, MD-Judge performance, or candidate-model instruction following.
Abstract
from arXiv · showhide
In the rapidly evolving landscape of Large Language Models (LLMs), ensuring robust safety measures is paramount. To meet this crucial need, we propose \emph{SALAD-Bench}, a safety benchmark specifically designed for evaluating LLMs, attack, and defense methods. Distinguished by its breadth, SALAD-Bench transcends conventional benchmarks through its large scale, rich diversity, intricate taxonomy spanning three levels, and versatile functionalities.SALAD-Bench is crafted with a meticulous array of questions, from standard queries to complex ones enriched with attack, defense modifications and multiple-choice. To effectively manage the inherent complexity, we introduce an innovative evaluators: the LLM-based MD-Judge for QA pairs with a particular focus on attack-enhanced queries, ensuring a seamless, and reliable evaluation. Above components extend SALAD-Bench from standard LLM safety evaluation to both LLM attack and defense methods evaluation, ensuring the joint-purpose utility. Our extensive experiments shed light on the resilience of LLMs against emerging threats and the efficacy of contemporary defense tactics. Data and evaluator are released under https://github.com/OpenSafetyLab/SALAD-BENCH.
1 Introduction
SALAD-Bench addresses narrow coverage, insufficient challenge, and costly evaluation in existing LLM safety benchmarks. It introduces hierarchical coverage, enhanced questions, specialized evaluators, and joint support for evaluating models, attacks, and defenses.
- Motivation: Prior safety benchmarks often cover only narrow threat perspectives, while traditional harmful questions can reach about 99% safety rates on modern LLMs.The paper also identifies human or GPT-based evaluation as time-consuming or expensive.
- Contributions: SALAD-Bench organizes safety evaluation into 6 domains, 16 tasks, and 66 categories across three hierarchical levels.This structure supports analysis of specific safety dimensions rather than only overall safety.
- Contributions: Attack-infused questions increase evaluation difficulty, while the multiple-choice subset broadens safety inquiry formats.The enhanced questions provide a more stringent test of safety responses.
- Contributions: MD-Judge evaluates standard and attack-enhanced question-answer pairs, while MCQ-Judge evaluates multiple-choice responses through instruction following and regex parsing.The evaluators are designed for different benchmark subsets.
- Dataset and utility: SALAD-Bench contains 21k test samples, including 5k attack-enhanced, 200 defense-enhanced, and 4k multiple-choice questions.The benchmark supports evaluation of both LLM safety and attack or defense methods.
- Dataset construction: The base set contains over 200 questions per category and undergoes human verification after automated collection and labeling.The construction combines public and self-instructed data with automated taxonomy labeling.
3 Question Enhancement
SALAD-Bench expands its base questions into attack-enhanced, defense-enhanced, and multiple-choice subsets. These constructions target model vulnerabilities, attack robustness, defense robustness, and format diversity.
- Subset design: Three subsets—attack-enhanced, defense-enhanced, and multiple-choice—expand the base set to deepen challenge, broaden perspectives, and support automatic testing.Together they assess safety and robustness against attack methods.
- Attack enhancement: The attack-enhanced subset filters questions with low rejection rates, applies multiple attack methods, and retains questions with the lowest safety rates.The process includes human-designed prompts, red-teaming LLMs, and gradient-based methods.
- Attack enhancement: The final attack-enhanced subset contains 5000 questions selected using the top unsafe score p_unsafe.The score aggregates unsafe responses across selected models.
- Defense enhancement: The defense-enhanced subset targets questions less likely to elicit harmful responses and retains questions with the lowest attack success rates.Prompting-based defense methods are then used to enhance the selected questions.
- Multiple-choice construction: The multiple-choice subset samples harmful questions by category, generates safe and unsafe candidate responses, and constructs questions requiring selection of safe or unsafe choices.Selection parsing and human re-checking precede final question construction.
4 Evaluator
SALAD-Bench uses MD-Judge for question-answer pairs and MCQ-Judge for multiple-choice questions. The evaluators align outputs with the safety taxonomy while targeting reliable and efficient assessment.
- MD-Judge: MD-Judge is an LLM-based evaluator for plain and attack-enhanced question-answer pairs, fine-tuned with taxonomy-aligned safety labels.Its training combines open-sourced safety data with specially generated data.
- MD-Judge: The MD-Judge template reformats question-answer pairs with structured safety-taxonomy information for prediction.The template uses domain-level areas as taxonomy guidance.
- Evaluation results: Claude2 leads across the base and attack-enhanced sets, while Gemini declines to approximately 20% on the attack-enhanced subset.The figure reports safety rates at the domain level for black-box LLMs.
- MCQ-Judge: MCQ-Judge uses in-context learning, explicit answer delimiters, and regex parsing to extract selections from open-ended responses.This design addresses variation in instruction-following capability across LLMs.
5 Experiments
Experiments evaluate SALAD-Bench’s judges, model safety across base and attack-enhanced sets, attack and defense methods, and multiple-choice performance. Results show strong evaluator performance, substantial attack-enhancement effects, model-specific vulnerabilities, and trade-offs between safety restrictions and answer accuracy.
- Evaluator evaluation: MD-Judge surpasses competing evaluators on proprietary and public safety test sets and remains comparable to GPT-4 on Harmbench and Lifetox.It largely surpasses LlamaGuard by at least 20% on the out-of-distribution benchmarks.
- Evaluator evaluation: MCQ-Judge matches human evaluation more closely than keyword- and GPT-based evaluators with 0.43s latency.The comparison supports both its efficiency and effectiveness for multiple-choice evaluation.
- Model safety evaluation: Claude2 reaches 99.77% safety on the overall evaluation, while GPT-3.5 reaches 88.62% among black-box models; Llama-2 models exceed 96% among open-source models.Safety rates drop substantially from the base set to the attack-enhanced set, with GPT-4 and Claude2 leading there.
- Model safety evaluation: Safety challenges vary by model and dimension: GPT-4 performs best in Information & Safety Harms, while Claude2 performs best in Malicious Use.Attack-enhanced questions shift several models’ hardest domains toward Human Autonomy & Integrity Harms, Socioeconomic Harms, or Malicious Use.
- Attack and defense evaluation: Most attack methods achieve lower ASR on the defense-enhanced set than on AdvBench-50 and the base question set.For Llama-2-13B, self-reminder prompts reduce ASR to 12.68%, whereas some perturbation defenses raise ASR to 51.16% and 54.94%.
- Multiple-choice evaluation: GPT-4 achieves 88.96% on both Acc-O and Acc-V for the MCQ subset, while strict safety restrictions can reduce Gemini Pro’s Acc-O from 78.71% Acc-V to 44.19% Acc-O.InternLM-20B and Yi-34B also show large gaps between safety rates and valid accuracy, reflecting instruction-following effects.
6 Conclusion
The paper presents SALAD-Bench as a hierarchical benchmark that evaluates LLM safety alongside attack and defense methods. Its experiments reveal varied model performance and identify safety areas requiring further attention.
- Conclusion: SALAD-Bench combines hierarchical taxonomies with MD-Judge and MCQ-Judge to evaluate LLM safety, attacks, and defenses.The benchmark is designed as a source for joint evaluation beyond standard model safety assessment.
- Conclusion: Experiments show varied performance across models and highlight areas requiring further attention to improve LLM safety and reliability.
7 Limitations
The paper identifies limitations involving taxonomy currency, base-set data cleaning, and dependence on evaluator or model instruction-following quality.
- Limitations: The taxonomy may become outdated as safety threats emerge and evolve.The paper suggests regularly updating the taxonomy and evaluation data.
- Limitations: Base-set quality depends substantially on filtration algorithms and reward models because data cleaning did not rely on intensive human labor.
- Limitations: Question-answer evaluation precision depends on MD-Judge, while MCQ evaluation may depend on candidate LLM instruction-following ability.
8 Broader Impact and Ethics Statement
The paper motivates comprehensive LLM safety evaluation, develops a broad taxonomy and dataset, and surveys evaluation approaches for harmful outputs. SALAD-Bench includes large-scale generated and public data spanning multiple safety domains.
- Motivation: Rising LLM safety concerns motivate comprehensive evaluation because human-like text generation can enable potential misuse.
- Related work: Earlier safety datasets often cover narrow threat perspectives, such as toxicity or bias, rather than the wider spectrum of safety concerns.
- Evaluation methods: Existing harmfulness evaluators include moderation classifiers, keyword methods, GPT-based referees, and LLM-based safety guards with different coverage or robustness limitations.Keyword methods can produce false positives and false negatives, while LlamaGuard is reported to lack strong generalization on enhanced question-answer pairs.
- Taxonomy: SALAD-Bench uses a three-level taxonomy covering six domain-level harmfulness areas, including toxicity, misinformation, information safety, malicious use, autonomy, and socioeconomic harms.
- Dataset: The benchmark contains 15k questions generated with fine-tuned GPT-3.5 and 6k questions sourced from open datasets.
E Human verification of Dataset Quality
Human verification supports the quality of SALAD-Bench’s taxonomy labels and confirms that its questions are predominantly unsafe. The dataset is constructed through public and self-instructed questions, automated taxonomy labeling, attack augmentation, and safety-focused judge training.
- Human Verification: 94.3% taxonomy-label consistency with human annotations supports the reliability of SALAD-Bench’s category assignments.Annotators verified 458 questions sampled across all 65 categories, with cross-validation by four authors.
- Human Verification: 96.7% of sampled questions were unsafe, confirming that the dataset questions guide unsafe LLM outputs.
- Dataset Construction: Attack enhancement combines human jailbreak prompts, LLM red teaming, and gradient-based suffix methods to create candidate augmented questions.
- Evaluator Construction: MD-Judge training data combine public question-answer pairs with specially generated attack-enhanced pairs containing safe and unsafe answers.
I Detailed Evaluation Results of Evaluators
MD-Judge outperforms GPT-4-based and other evaluators on standard and attack-enhanced QA evaluation, while MCQ-Judge achieves accuracy close to human evaluation. Evaluator variants show that Mistral-7B-v0.1 with a task-level template is selected for final use.
- QA Evaluators: MD-Judge exceeds the GPT-4-based evaluator by 3% on SALAD-Base-Test and 5% on SALAD-Enhance-Test.
- QA Evaluators: MD-Judge improves ToxicChat performance by 15% over the next-best method and also outperforms other evaluators on Beavertails and SafeRLHF.
- Evaluator Variants: Mistral-7B-v0.1 with task-level template training is selected because it significantly improves ToxicChat results while remaining competitive on other datasets.
- Evaluator Variants: Task-level templates improve Mistral-7B performance over domain-level templates, but this enhancement is not replicated for Llama-2-7B.The paper offers context length as a plausible explanation for the model difference.
- MCQ Evaluator: MCQ-Judge achieves similar Acc-V results to human evaluation for GPT-4 and Vicuna and falls behind by about 3.4% for Qwen.
M Details in Evaluating Attack and Defense Methods
The attack evaluation targets Llama2-7B-Chat and combines jailbreak, red-teaming, gradient-based, and chain-of-utterances methods. Defense evaluation compares paraphrasing, perturbation, and prompting approaches on attack-enhanced questions.
- Attack Methods: Llama2-7B-Chat is used as the target for applicable attacks because it is among the safest models, with transferability hypothesized as a motivation.
- Attack Methods: TAP retains attack-enhanced questions with evaluation scores ≥9, while GPTFuzz and AutoDAN retain questions from successful attacks.
- Attack Methods: Human jailbreak templates are selected using the top 10 JB-score rankings and top 10 upvote rankings from jailbreakchat.com.
- Attack Methods: GCG appends 20 previously searched suffixes to all questions, while Chain-of-Utterances applies the chat template when generating from open-source models.
- Defense Methods: Defense evaluation includes GPT-paraphrasing, four perturbation methods, and Safe/XSafe and Self-Reminder prompting methods.
O Quantitive Results and Analysis
Safety performance is generally high on the base set but drops substantially on attack-enhanced questions. Difficulty varies by domain, task, and category, with attack-enhanced evaluations exposing weaknesses in human autonomy and financial data leaks.
- Overall Results: Attack-enhanced questions produce a significant drop in safety rates compared with the base set across evaluated models.
- Domain Analysis: Base-set difficulty is highest for Malicious Use and Socioeconomic Harms, whereas attack-enhanced difficulty shifts toward Information & Safety Harms and Human Autonomy & Integrity Harms.
- Task Analysis: Adult content tasks have lower base-set safety rates, while Unfair representation tends to have higher rates across most models.
- Defense Results: GPT-Paraphrasing and Self-Reminder prompt perform best among the evaluated defense methods on the attack-enhanced subset.
- Category Analysis: Pornography and Erotic chat have lower base-set safety rates, while attack-enhanced questions reveal pronounced weaknesses in Financial data leaks.
P More Analysis for MCQ Subset
The MCQ analysis examines whether models consistently distinguish safe from unsafe selections and how rejection behavior affects accuracy. Strict safety alignment can improve rejection of unsafe choices while also causing rejection of safe answers and lower Acc-V.
- GPT-4 achieves an 86.93% selection consistency rate, the best performance among evaluated LLMs.
- LLMs with higher selection consistency generally obtain higher accuracy on the MCQ subset.
- InternLM answers all multiple-choice questions but achieves only 3.85% Acc-V, illustrating weak rejection performance alongside low accuracy.
- Llama-2 and Claude2 reject unsafe-selection questions at 84.48% and 87.71%, respectively, but also reject safe-selection questions at 63.39% and 36.04%.
- Gemini has a 43.85% overall rejection rate, balanced rejection across safe and unsafe questions, and 78.71% Acc-V.
Q Qualitative Results
Qualitative examples show that attack-enhanced prompts can elicit unsafe responses from models and that MCQ errors arise from incomplete selection or incorrect formatting. The examples illustrate both attack transferability and the difficulty of preserving safety and helpfulness together.
- Attack-enhanced examples contrast with base-question responses and include a GPT-paraphrasing defense condition for qualitative safety inspection.
- GCG-generated attack prompts cause both Llama-2 and Mixtral-8x7B to generate unsafe responses, indicating transferable attack capability.
- MCQ errors include failing to select all correct options when questions have multiple correct selections.
- Some models choose the right options but use an inaccurate response format that the MCQ-Judge evaluator cannot detect.
- The qualitative examples cover base questions, attack-enhanced questions, defense-modified questions, and safe- versus unsafe-selection MCQs.