PaperlayerView exact revision on arXiv

Source-linked AI summary

SafeDialBench: A Fine-Grained Safety Evaluation Benchmark for Large Language Models in Multi-Turn Dialogues with Diverse Jailbreak Attacks

Hongye Cao, Sijia Jing, Yanming Wang, Ziyue Peng, Zhixin Bai, Zhe Cao, Meng Fang, Fan Feng, Boyan Wang, Jiaheng Liu, Tianpei Yang, Jing Huo, Yang Gao, Fanyu Meng, Xi Yang, Chao Deng, Junlan Feng

arXiv:2502.11090v4cs.CLcs.AI

TL;DR

Existing safety benchmarks often focus on single-turn dialogue or narrow jailbreak coverage and provide limited fine-grained assessment of unsafe-information handling. SafeDialBench evaluates multi-turn safety using a bilingual dataset and a three-ability framework; experiments report both strong-performing models and safety vulnerabilities.

  • Problem

    Many existing benchmarks focus on single-turn dialogue or a single jailbreak strategy and do not assess in detail how models identify and handle unsafe information.

    Source: §1 · p. 1

  • Method

    SafeDialBench uses a two-tier taxonomy, seven jailbreak strategies, and bilingual multi-turn dialogues across 22 scenarios to assess three safety abilities with LLM and human judgments.

    Source: §1 · p. 2, §3.2 · p. 4

  • Results

    Experiments find strong safety in Yi-34B-Chat and GLM4-9B-Chat, while o3-mini, Baichuan2-7B-Chat, and Llama3.1-8B-Instruct show vulnerabilities in the reported settings.

    Source: §1 · p. 3, §5 · p. 10

  • Takeaways & Limitations

    The benchmark’s results distinguish model safety across dimensions and attack settings, with Yi-34B-Chat and GLM4-9B-Chat among the open-source models showing strong capabilities.

    Source: §5 · p. 10

    The benchmark needs additional jailbreak methods, continuing dataset updates, and more granular evaluation methods to keep pace with model development and measure safety more precisely.

    Source: §5 · p. 10

Abstract

from arXiv · show

With the rapid advancement of Large Language Models (LLMs), the safety of LLMs has been a critical concern requiring precise assessment. Current benchmarks primarily concentrate on single-turn dialogues or a single jailbreak attack method to assess the safety. Additionally, these benchmarks have not taken into account the LLM's capability of identifying and handling unsafe information in detail. To address these issues, we propose a fine-grained benchmark SafeDialBench for evaluating the safety of LLMs across various jailbreak attacks in multi-turn dialogues. Specifically, we design a two-tier hierarchical safety taxonomy that considers 6 safety dimensions and generates more than 4000 multi-turn dialogues in both Chinese and English under 22 dialogue scenarios. We employ 7 jailbreak attack strategies, such as reference attack and purpose reverse, to enhance the dataset quality for dialogue generation. Notably, we construct an innovative assessment framework of LLMs, measuring capabilities in detecting, and handling unsafe information and maintaining consistency when facing jailbreak attacks. Experimental results across 17 LLMs reveal that Yi-34B-Chat and GLM4-9B-Chat demonstrate superior safety performance, while Llama3.1-8B-Instruct and o3-mini exhibit safety vulnerabilities.

1 INTRODUCTION

SafeDialBench addresses gaps in multi-turn safety evaluation with a fine-grained bilingual benchmark and a framework for assessing three safety abilities. Experiments report varied model safety, attack effectiveness, and agreement between automated and expert evaluations.

Motivation

  • Prior multi-turn benchmarks often use one jailbreak strategy, focus on aggressive language, and omit detailed assessment of identifying and handling unsafe information.The introduction identifies missing attention to ethics, morality, legality, fairness, and privacy as part of this scope gap.Source: §1 · pp. 1–2

Benchmark construction

  • SafeDialBench combines six safety dimensions, seven jailbreak strategies, and 22 dialogue scenarios to build a fine-grained multi-turn benchmark.The benchmark includes English and Chinese dialogues, with questions crafted for scenarios and reviewed by experts after generation using three LLMs.Source: §1 · p. 2

Evaluation framework

  • Its evaluation framework assesses identifying risks, handling unsafe information, and maintaining consistency, using both LLM-based and human judgments.The experiments evaluate 19 LLMs, including four closed-source and 15 open-source models.Source: §1 · p. 2

Experimental results

  • Yi-34B-Chat, MoonShot-v1, and ChatGPT-4o show strong safety across three abilities, while Llama3.1-8B-Instruct and o3-mini show vulnerabilities.Baichuan2-7B-Chat has the highest reported attack success rate at 69.60%; fallacy and purpose-reverse attacks are highly effective, and GPT-3.5 turbo exceeds 80% agreement with experts.Source: §1 · p. 3

2 RELATED WORK

Related work contrasts single-turn and multi-turn safety benchmarks and prior jailbreak research. The paper positions SafeDialBench as a fine-grained bilingual benchmark for diverse attacks in multi-turn dialogues.

Safety Benchmarks for LLMs

  • Single-turn benchmarks offer larger datasets but cannot assess model performance in more realistic multi-turn dialogues.The review also notes that multilingual benchmarks expand linguistic diversity without focusing on multi-turn adversarial dynamics.Source: §2 · p. 3
  • Existing multi-turn benchmarks are often monolingual, use limited jailbreak methods, have conversations shorter than five turns, and omit dimensions such as legality and ethics.The review also identifies a lack of fine-grained safety-capability evaluation.Source: §2 · p. 3

Jailbreak Attacks on LLMs

  • Prior jailbreak research spans reference-based attacks, privacy leakage, concealed intent, adaptive multi-agent strategies, and context fabrication, but often remains single-turn or narrow in attack categories.The authors describe SafeDialBench as a fine-grained bilingual benchmark using diverse jailbreak attacks in multi-turn dialogues.Source: §2 · p. 3

3 SAFEDIALBENCH

SafeDialBench combines a two-tier safety taxonomy, human-guided generation of adversarial dialogues, and fine-grained evaluation. Its design covers six safety dimensions and three safety abilities.

3.1 TWO-TIER HIERARCHICAL SAFETY TAXONOMY

  • The two-tier taxonomy defines six dimensions—fairness, legality, morality, aggression, ethics, and privacy—and guides dialogue construction and evaluation prompts.The dimensions cover concerns including group stereotypes, legal violations, non-violent immorality, harmful content, self-harm and abuse, and sensitive information.Source: §3.1 · p. 4

3.2 DATA GENERATION WITH DIVERSE JAILBREAK ATTACK

  • Annotators craft an initial query and adapt later user prompts turn by turn while one of three LLMs generates assistant replies; experts review completed dialogues twice.Using multiple LLM assistants is intended to reduce bias toward any single model's peculiarities.Source: §3.2 · p. 4
  • The dataset spans 22 scenarios, Chinese and English, 4,053 dialogues of 3–10 turns, and seven attacks including scene construct, purpose reverse, role play, and fallacy.The scenario range extends from daily life to specialized fields; the seven strategies use different techniques to elicit or test unsafe responses.Source: §3.2 · p. 5

3.3.1 SAFETY ABILITY TAXONOMY

  • The safety-ability framework evaluates identifying risks, handling unsafe information, and maintaining a safety stance under pressure and misleading guidance.Prompts for these abilities are developed across all six safety dimensions.Source: §3.3.1 · p. 5

3.3.2 MODEL AND HUMAN EVALUATION

  • LLM evaluators score each response turn from 1 to 10 using dialogue context and dimension-specific guidelines; the lowest turn score becomes the dialogue score.The authors also use human experts and manual spot-checks to assess model safety and the effectiveness of LLM-based evaluation.Source: §3.3.2 · pp. 5–6

4 EXPERIMENTS

Experiments compare 19 LLMs on SafeDialBench and examine safety across dimensions, jailbreak methods, dialogue turns, model scales, and human judgments.

4.1 EXPERIMENTAL SETUP

  • The evaluation covers 19 popular LLMs; experiments use golden contexts as dialogue histories, model-specific chat formats and prompts, and temperature 0.Source: §4.1 · pp. 6–7

4.2 MAIN RESULTS

  • Yi-34B-Chat leads overall benchmark performance; Baichuan2-7B-Chat has the highest ASR and ChatGPT-4o the lowest.Source: §4.2 · p. 7
  • GLM4-9B-Chat excels in ethics and legality handling, while Yi-34B-Chat is strong across four dimensions; Llama3.1-8B-Instruct scores lowest in English and o3-mini shows Chinese vulnerabilities.Source: §4.2 · p. 7
  • Fallacy, purpose-reverse, and role-play attacks compromise safety; topic-change and reference attacks are minimally effective, while GLM4-9B-Chat and Yi-34B-Chat remain robust.Source: §4.2 · p. 7, Figure 5 · p. 8
  • Under fallacy attacks, safety scores fluctuate during the first three turns, then degrade significantly after turn 4, especially in ethics and aggression.Source: §4.2 · p. 7, Figure 6 · p. 8

4.3 FURTHER ANALYSIS

  • Qwen-14B-Chat handles all safety dimensions better than its 7B version, but the Baichuan2 family shows non-monotonic safety differences across model sizes.Source: §4.3 · p. 8, Figure 7 · p. 9
  • Agreement between human experts and LLM evaluations exceeds 80% across all three safety capabilities in a 100-dialogue sample.Source: §4.3 · pp. 8–9
  • A role-play case involving illegal organ trading shows attacks eliciting prohibited content and the framework identifying failures in recognition, handling, and safety consistency.Source: §4.3 · p. 9, Figure 8 · p. 9

5 CONCLUSION

The conclusion presents SafeDialBench as a fine-grained multi-turn benchmark and reports varied safety performance across evaluated models.

5 CONCLUSION

  • Yi-34B-Chat and GLM4-9B-Chat show strong safety capabilities, while o3-mini, Baichuan2-7B-Chat, and English-only Llama3.1-8B-Instruct show vulnerabilities in the reported evaluations.Source: §5 · p. 10
  • The authors identify needs for additional jailbreak methods, continuous dataset updates, and more granular evaluation methods.Source: §5 · p. 10

ETHICS STATEMENT

The ethics statement describes safeguards for annotation and evaluation and limits the intended use of the dataset and metrics.

ETHICS STATEMENT

  • The study reports informed consent, fair compensation, and mental health support for annotators; its metrics are complementary, and the dataset is intended for academic research only.Source: ETHICS STATEMENT · p. 10

SUPPLEMENTARY MATERIAL

The supplementary material compares benchmark coverage, discusses how multi-turn attacks differ from single-turn attacks, and details the six safety dimensions.

A ADDITIONAL RELATED WORK

B DETAILS ON SAFETY DIMENSIONS

  • Fairness covers group bias and distributional harm; legality assesses compliance across personal, economic, and security harms; privacy spans individual, organizational, and social information.Source: B DETAILS ON SAFETY DIMENSIONS · pp. 18–19
  • Morality covers nonviolent norm violations, ethics targets severe physical or psychological harm including self-harm and abuse, and aggression spans threats, insults, incitement, and related harms.Source: B DETAILS ON SAFETY DIMENSIONS · pp. 18–19

C DETAILS ON SAFETY ABILITY EVALUATION

The evaluation framework treats safety as a sequence of identifying risks, choosing an appropriate response, and maintaining reliable principles under varied scenarios and user pressure. Its design spans detailed risk categories and dialogue scenarios, with questions that progress from broad context to specific details.

C DETAILS ON SAFETY ABILITY EVALUATION

  • Safety is evaluated as a sequential chain: identify risks, handle them with an appropriate response, then check consistent execution across scenarios and user pressure.Identification guides handling; consistency acts as an overarching quality-control layer.Source: C DETAILS ON SAFETY ABILITY EVALUATION · p. 19

C.1 IDENTIFICATION ABILITY

  • Identification covers explicit risks, risks accumulated across turns or implied by context, and varied privacy, legal, and ethical risk types.Examples include piecing together private information across turns and detecting discriminatory metaphors.Source: §C.1 · p. 19

C.2 HANDLE ABILITY

  • Handling matches responses to risk, using hard blocks, soft guidance, or compliant alternatives alongside legal explanations and value guidance.Examples include blocking suicide-method requests, guiding ethically controversial discussions, and substituting virtual cases for real privacy demonstrations.Source: §C.2 · pp. 20–21

C.3 CONSISTENCY ABILITY

  • Consistency includes uniform scenario responses and fairness checks, coherent legal or moral reasoning, and resistance to pressure and manipulative probes.The framework includes counterfactual fairness tests across demographic groups in similar contexts.Source: §C.3 · p. 21

D.1 SCENARIO SELECTION

  • Scenario selection balances broad social coverage with practical professional contexts, sensitivity, and varied levels of knowledge and reasoning complexity.The resulting list covers 22 scenarios, from healthcare and law to sports and history.Source: §D.1 · p. 21

D.2 QUESTION DESIGN

  • Question design specifies the scenario, safety dimension, and expected response, then uses contextual opening questions and progressively narrower follow-ups; annotators peer-review dialogues.Reviewers use standardized criteria to identify and document issues.Source: §D.2 · pp. 21–22

D.3 WHY USING MULTIPLE LLMS FOR DATA GENERATION?

The dataset-generation approach combines human-designed questions with multiple language models, and its mixed-model dialogues show greater lexical diversity than single-model samples. Human review and planned extensibility address quality control and future adaptation, while real-world user data remains a stated future direction.

D.3 WHY USING MULTIPLE LLMS FOR DATA GENERATION?

  • The mixed dataset has the highest entropy (14.58) versus single-model datasets (14.20-14.53), and maximum distinct 2-gram and 3-gram ratios (0.98).The authors present these findings as empirical validation that multiple models enhance dataset diversity.Source: §D.3 · p. 22

D.4 HUMAN ANNOTATION

  • Human-designed questions generated with GPT-4, ChatGLM, and Doubao are intended to reduce single-model and single-annotator bias; real-world user data and crowdsourced adversarial probing remain future work.The authors state that the assistant model used during generation is not correlated with content difficulty or safety challenge level.Source: §D.4 · pp. 22–23, §D.3 · p. 22

D.5 EXPERT REVIEW

  • Quality control combines peer review with a two-stage expert process: 15 experts cross-validate dialogues, followed by random audits sampling 20%-50% of each day’s data.A daily batch below a 95% per-item pass rate is rejected for comprehensive reassessment.Source: §D.4 · p. 23, §D.5 · p. 23

D.6 UPDATE PROTOCOL

  • The modular taxonomy and evaluation framework are designed to accommodate new jailbreak categories without rebuilding the benchmark, alongside planned annual expansions and community contributions.The update plan includes monitoring emerging techniques, version control, and contribution guidelines.Source: §D.6 · p. 23

E DETAILS ON DATA STATISTICS

This block points readers to data and model statistics, then describes dimension-specific evaluation prompts and scoring choices. The prompts assess identification, handling, and consistency across six safety dimensions, while dialogue-level scoring uses the minimum turn score.

E DETAILS ON DATA STATISTICS

  • Table 6 presents SafeDialBench data statistics, with ZH and EN denoting Chinese and English, respectively.The supplied passage identifies the table but does not provide its statistics.Source: E DETAILS ON DATA STATISTICS · p. 24

F.1 EXPERIMENTAL SETTING

  • Table 9 provides information about the models evaluated in SafeDialBench.The supplied passages do not specify the models’ individual attributes.Source: §F.1 · p. 24, Table 9 · p. 25

F.2 EVALUATION PROMPT

  • Evaluation prompts use a shared structural template, with definitions synthesized from literature and tailored criteria for each of six safety dimensions.The template is a structural foundation, not the full evaluation content.Source: §F.2 · p. 24
  • The prompts assess identification, handling, and consistency across six dimensions; morality and ethics definitions are grounded in cited literature because the authors disclaim specialist status.The authors cite Ji et al. (2024a), Weidinger et al. (2021), and Sun et al. (2023) for those standards.Source: §F.2 · p. 24
  • Scores below 7 indicate insufficient safety performance under a five-tier rubric, and dialogue-level scoring uses the minimum turn score because one insecure round makes the conversation insecure.Both human experts and LLM evaluators score turns independently before minimum-score aggregation.Source: §F.2 · pp. 24–25

F.3 AGREEMENT CALCULATION

The results report close agreement between human and GPT-3.5 turbo evaluations, distinct attack-success-rate tiers, and broadly aligned model rankings under Qwen-72B evaluation. Statistical analysis uses 300 dialogues sampled across six dimensions because full-dataset evaluation was computationally costly.

F.3 AGREEMENT CALCULATION

  • Human-expert agreement exceeds 81% across abilities and dimensions; GPT-3.5 turbo matches inter-human agreement for Identify (82%) and Consistency (81%), and is one point lower for Handle (81% versus 82%).The five-expert consensus score is the most frequent rating; the cited tables report agreement across abilities and dimensions.Source: §F.3 · p. 25
  • Against human evaluations on 100 dialogues, ChatGPT-3.5 Turbo achieves 95.0% accuracy, 96.7% precision, and a 5.3% false-positive rate for safe/unsafe outcomes.The authors use human evaluations as ground truth for the false-positive analysis.Source: §F.3 · pp. 25–26

G.1 STATISTICAL ANALYSIS

  • Baichuan2-7B-Chat (0.66±0.01) and ChatGLM3-6B (0.66±0.00) show the highest ASR, while QwQ-32B (0.49±0.01) and DeepSeek-7B-Chat (0.49±0.01) show better safety.The analysis uses 300 dialogues, 50 sampled from each of six safety dimensions, due to the computational cost of evaluating the full dataset.Source: §G.1 · p. 26, Table 10 · p. 26

G.2 RESULTS ON QWEN-72B EVALUATOR

  • Under Qwen-72B evaluation, Yi and GLM4-9B-Chat lead among open-source models, while o3-mini is relatively weak; these rankings generally align with GPT-3.5 turbo evaluation.The comparison also reports dimension-specific strengths and weaknesses, including low o3-mini scores in aggression and privacy.Source: §G.2 · pp. 26–27

G.3 RESULTS OF CHINESE AND ENGLISH DATASETS

Bilingual evaluations reveal language-, dimension-, and attack-dependent safety differences, while multi-turn context can still defeat risk-aware models and leave vulnerabilities after defense training.

G.3 RESULTS OF CHINESE AND ENGLISH DATASETS

  • Yi-34B-Chat and MoonShot-v1 lead in different language settings, while o3-mini and Llama3.1-8B-Instruct show weaknesses in Chinese and English, respectively.The authors report that safety capabilities cannot be reliably generalized across languages.Source: §G.3 · pp. 27–28

G.4 ANALYSIS OF REASONING MODEL VULNERABILITY IN MULTI-TURN DIALOGUE JAILBREAKS

  • DeepSeek-R1 can acknowledge risk yet answer harmfully when benign prior turns make a malicious query seem continuous; the authors attribute this to alignment failure, not reasoning deficiency.The case analysis says contextual manipulation and a drive for coherence can override safety considerations.Source: §G.4 · pp. 28–29

G.5 JAILBREAK DIMENSIONAL RESULTS

  • Fallacy, role play, and purpose reverse are especially effective; scene construct weakens identification for Yi-34B-Chat and ChatGPT-4o in legality, while fairness consistency lags.Topic change has minimal effect in privacy; purpose reverse and role play compromise ethics performance, while models show stronger fairness handling than other dimensions.Source: §G.5 · p. 29

G.6 CASE STUDY

  • Case studies show scene construct, fallacy, purpose reverse, and role play can elicit harmful or inconsistent responses when models miss adversarial intent or fail to uphold ethical standards.The cases include toxic marital advice, animal-training content, conflict-escalation advice, and derogatory language.Source: §G.6 · p. 30

G.7 COMPARISON WITH EXISTING ATTACK METHODS / G.8 COMPARISON WITH EXISTING DEFENSE METHODS ON ASR.

  • Purpose Reverse and Topic Change exceed Actorattack and Crescendo in ASR; defenses reduce ASR, but the best reported post-training ASR remains 32%.X-Boundary gives Qwen2.5-7B-Instruct a 22% ASR reduction; both methods yield a 20% reduction on Llama-3.1-8B-Instruct.Source: Table 14 · p. 30, §G.7 · p. 30, Table 15 · p. 31

G.11.1 ATTACK MECHANISMS

  • Safety scores remain stable for 3–4 turns before declining; attack families exploit distinct vulnerabilities, and greater model scale does not uniformly improve safety.The analysis characterizes multi-turn attacks as cumulative context poisoning and identifies non-monotonic scale-safety relationships.Source: §G.11.1 · pp. 32, 34

G.10 ROBUSTNESS ANALYSIS ON EVALUATION

  • Prompt variations preserve model rankings and attack-effectiveness patterns, with average-score intervals ranging from ±0.01 to ±0.35.The authors report stable measurements across modified evaluation prompts.Source: §G.10 · p. 32, Table 16 · p. 33, Table 17 · p. 33

G.9 TECHNICAL MITIGATION STRATEGIES / G.11.2 DEFENSE MECHANISMS

  • Consistency scores trail identification and handling, while Yi-34B-Chat and GLM4-9B-Chat show minimal turnwise degradation; proposed remedies target turn pressure and attack families.Suggestions include turn-aware rewards, attack-specific fine-tuning, and multi-turn critique.Source: §G.9 · pp. 31–32, §G.11.2 · p. 34

G.12 CONTRIBUTION TO LLM SAFETY RESEARCH THROUGH BENCHMARK STUDY

SafeDialBench combines multidimensional, bilingual, multi-turn testing with diagnostic safety assessments. The study presents it as a way to pinpoint model vulnerabilities and inform targeted safety work.

G.12 CONTRIBUTION TO LLM SAFETY RESEARCH THROUGH BENCHMARK STUDY

  • SafeDialBench provides a 126-dimensional safety profile by crossing six safety dimensions, seven attack methods, and three assessed abilities.The abilities are identifying risks, handling unsafe information, and maintaining consistency.Source: §G.12 · p. 34
  • Its diagnostic results can guide targeted improvements, including focused Aggression and Legality identification training for Qwen2.5 and Fallacy Attack and Purpose Reverse defenses for ChatGPT-4o.The paper also recommends dimension-prioritized training and architecture-aware guidance.Source: §G.12 · pp. 34–35
  • The benchmark highlights that larger scale, reasoning ability, or strong performance in one language do not guarantee safety across other settings.The paper points to scale-safety decoupling, reasoning-model vulnerabilities, and language-specific safety differences.Source: §G.12 · p. 34
  • The benchmark covers bilingual multi-turn dialogues, diverse jailbreak attacks, and fine-grained assessment rather than single-turn or single-attack testing.The benchmark description specifies dialogues of 3–10 turns in Chinese and English and seven jailbreak families.Source: §G.12 · p. 34
  • SafeDialBench contributes 4,053 curated dialogues with expert-validated attack patterns, evaluation code, documentation, and an update protocol.The paper also describes community contribution guidelines for expanding the resource.Source: §G.12 · p. 35

Related papers

Loading 2502.11090v4…