PaperlayerView exact revision on arXiv

Source-linked AI summary

ThinkGuard: Deliberative Slow Thinking Leads to Cautious Guardrails

Xiaofei Wen, Wenxuan Zhou, Wenjie Jacky Mo, Muhao Chen

arXiv:2502.13458v2cs.CLcs.AIcs.CRcs.LG

TL;DR

Existing guardrails often rely on rules or single-pass labels, leaving nuanced safety judgments and explanations limited. THINKGUARD distills structured critiques from high-capacity LLMs into guardrail fine-tuning, and leads multiple benchmark averages while outperforming LLaMA Guard 3 on reported accuracy and macro F1.

  • Problem

    Many guardrails classify with labels alone rather than reasoning about intent, context, and risk, limiting nuanced distinctions and explanations.

    Source: §1 · p. 1

  • Method

    THINKGUARD prompts existing LLMs for step-by-step safety reasoning, then fine-tunes guardrails on datasets with labels and critiques using two conversational rounds.

    Source: §1 · p. 2

  • Results

    Across multiple safety benchmarks, THINKGUARD achieves the highest average F1 and AUPRC, with accuracy improving by 16.1% and macro F1 by 27.0% over LLaMA Guard 3.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Structured critiques enhance classification precision and nuanced safety reasoning, while THINKGUARD’s efficiency supports its suitability for real-world deployment.

    Source: §1 · p. 2

    Its effectiveness depends on critique quality and alignment with safety guidelines, while generating structured critiques adds computational overhead.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Ensuring the safety of large language models (LLMs) is critical as they are deployed in real-world applications. Existing guardrails rely on rule-based filtering or single-pass classification, limiting their ability to handle nuanced safety violations. To address this, we propose ThinkGuard, a critique-augmented guardrail model that distills knowledge from high-capacity LLMs by generating structured critiques alongside safety labels. Fine-tuned on critique-augmented data, the captured deliberative thinking ability drastically enhances the guardrail's cautiousness and interpretability. Evaluated on multiple safety benchmarks, ThinkGuard achieves the highest average F1 and AUPRC, outperforming all baselines. Compared to LLaMA Guard 3, ThinkGuard improves accuracy by 16.1% and macro F1 by 27.0%. Moreover, it surpasses label-only fine-tuned models, confirming that structured critiques enhance both classification precision and nuanced safety reasoning while maintaining computational efficiency.

1 Introduction

Existing guardrails often reduce safety judgments to labels, limiting nuanced risk assessment and transparency. THINKGUARD adds deliberative critiques and achieves leading results across safety benchmarks.

1 Introduction

  • Many guardrails predict labels without deliberating over intent, context, or risk, making benign sensitive statements harder to distinguish and decisions less interpretable.Source: §1 · p. 1
  • The illustration shows THINKGUARD identifying risks that Llama Guard initially misclassified after fine-tuning on expert-generated reasoning.Source: Figure 1 · p. 2
  • THINKGUARD distills step-by-step reasoning from existing LLMs into datasets containing labels and reasoning, then fine-tunes guardrails in two conversational rounds.Source: §1 · p. 2
  • The two-round format provides reasoning when needed while allowing users to receive only the final prediction, preserving efficiency.Source: §1 · p. 2
  • Across multiple safety benchmarks, THINKGUARD achieves the highest average F1 and AUPRC, and improves accuracy by 16.1% and macro F1 by 27.0% over LLaMA Guard 3.Source: §1 · p. 2

2 Related Work

Prior guardrails span rigid rule-based filters and more adaptable LLM classifiers, while slow-thinking research explores deliberate reasoning in model decisions.

2 Related Work

3 THINKGUARD

ThinkGuard adds critique-based deliberative reasoning to guardrail safety classification, aiming to improve safety decisions and their explanations. It trains a smaller guardrail model with critiques generated by a high-quality language model.

3.1 Overview

  • Single-pass guardrails lack reasoning and explanatory capabilities, limiting their ability to detect implicit risks and provide transparent moderation.Source: §3.1 · p. 3

3 THINKGUARD

  • ThinkGuard incorporates critiques generated by a high-quality language model to train a smaller guardrail model for safety classification and justification.The framework uses multi-step safety assessments rather than relying solely on direct classification.Source: §3.1 · p. 3

3.2 Problem Definition

  • A guardrail assesses whether text follows safety guidelines, categorizes unsafe content, and explains why assigned categories were violated.Source: §3.2 · pp. 3–4

3.3 Critique-Augmented Fine-Tuning

  • Training data pair prompts and responses with safety labels and LLM-generated critiques, while fine-tuning jointly optimizes classification and critique-generation objectives.Source: §3.3 · pp. 4–5

3.4 Inference and Decision Making

  • At inference, ThinkGuard predicts a safety label, adds violation categories for unsafe responses, and generates a critique explaining its decision.Source: §3.4 · p. 5

4 Experiments

THINKGUARD is evaluated on multiple safety benchmarks using critique-augmented training, with comparisons to guardrail models and larger LLMs. It leads on average F1 and AUPRC, with additional results on BeaverTails, model capabilities, and data scaling.

4.1 Experiment Setup

  • The evaluation covers BeaverTails, Toxic Chat, OpenAI Moderation, and WildGuardMix, including datasets with adversarial dialogues and ambiguous cases.BeaverTails is the primary training source and contains human-labeled prompt-response pairs annotated with one or more harm categories.Source: §4.1 · pp. 5–6
  • The study compares THINKGUARD with prior guardrails, in-context and label-only LLaMA Guard 3 variants, and large-scale LLMs; its training data pairs safety labels with synthetic critiques.GPT-4o and DeepSeek models generate critiques that explain classifications and identify violated safety categories when applicable.Source: §4.1 · pp. 5–6

4.2 Results

  • THINKGUARD achieves the highest average F1 and AUPRC across four safety benchmarks.Table 1 compares guardrail models using F1 and AUPRC scores.Source: Table 1 · p. 6, §4.2 · p. 7
  • THINKGUARD improves F1 over LLaMA Guard 3 by 14.1% on Toxic Chat and 6.1% on WildGuardMix, and maintains more stable performance across four datasets than the label-only model.It has the highest AUPRC on BeaverTails and best F1 on WildGuardMix; GPT-4o+CoT has the best F1 on BeaverTails but fluctuates across datasets.Source: §4.2 · p. 7
  • THINKGUARD performs well across output stability, guideline adaptability, concision, detailed safety categorization, and explanations; other guardrails differ in classification or response consistency.WildGuard predicts only safe or unsafe, while other models must also identify risk categories for unsafe cases, making their evaluation more stringent.Source: §4.2 · p. 8
  • On BeaverTails, THINKGUARD has the highest accuracy and improves accuracy by 16.1% and macro F1 by 27.0% over LLaMA Guard 3.It trails GPT-4o+CoT in macro F1; THINKGUARD is an efficient 8B model, while GPT-4o is computationally expensive.Source: Table 3 · p. 7, §4.2 · p. 8
  • As training data grows beyond 9k examples, THINKGUARD’s performance gap over the label-only model becomes more pronounced, especially for macro F1.At 15k examples, THINKGUARD scores 82.3% F1 and 76.4% macro F1; the label-only baseline scores 83.7% F1 but lower macro F1.Source: Figure 3 · p. 7, §4.2 · p. 8

5 Conclusion

THINKGUARD integrates structured critiques to support safety classification and interpretability, and achieves leading average F1 and AUPRC across multiple safety benchmarks. The authors identify broader safety tasks and efficiency optimization as future work.

5 Conclusion

  • THINKGUARD integrates structured critiques to improve safety classification and interpretability.Source: §5 · p. 8
  • THINKGUARD achieves the highest average F1 and AUPRC across multiple safety benchmarks, outperforming existing baselines.The conclusion describes stronger generalization than label-only fine-tuned models, particularly on datasets with diverse harm categories.Source: §5 · p. 8
  • Compared with LLaMA Guard 3, THINKGUARD improves F1 by 14.1% on Toxic Chat and 6.1% on WildGuardMix.The paper links these results to robustness against adversarial and ambiguous cases.Source: §5 · p. 8
  • Future work will extend critique-based fine-tuning to broader safety tasks and optimize efficiency for real-world deployment.Source: §5 · p. 9

Limitations

THINKGUARD’s effectiveness depends on critique-augmented training data quality, and generating critiques adds computational overhead relative to simpler classification models. The authors identify data refinement, parameter-efficient adaptation, and broader safety domains as future work.

Limitations

  • Incomplete or guideline-misaligned critiques may cause THINKGUARD to inherit biases or inconsistencies.Source: Limitations · p. 9
  • Generating structured critiques adds computational overhead compared with simpler classification models.The authors identify a trade-off between reasoning depth and efficiency that requires further optimization.Source: Limitations · p. 9
  • Future work will refine critique generation, explore parameter-efficient adaptations, and extend the approach to broader safety domains.Source: Limitations · p. 9

Ethics Statement

The ethics statement calls for ethically sourced training data and transparency in collection and annotation, while noting risks of over-restriction and unfair enforcement. It also identifies fairness, accountability, and transparency as continuing research concerns.

Ethics Statement

  • Ethically sourced training data and transparent collection and annotation are needed to ensure fairness and minimize bias.Source: Ethics Statement · p. 9
  • Biased safety policies may over-restrict benign inputs, while cultural and contextual differences complicate fair moderation.Source: Ethics Statement · p. 9
  • The statement calls for appropriate oversight because relying on automated guardrails without human review can lead to unfair enforcement or misalignment with societal norms.Source: Ethics Statement · p. 9
  • Future research should continue addressing fairness, accountability, and transparency in safety-enhancing AI.Source: Ethics Statement · p. 9

A Different Finetuning strategies results

The fine-tuning comparison evaluates Explanation-Then-Classification and Classification-Then-Explanation at two training-set sizes. Classification before explanation performs best among the reported strategies, with the 15k Cla-Exp model corresponding to THINKGUARD.

A Different Finetuning strategies results

B Guardrail Model Outputs

The BeaverTails examples expose failure modes and inconsistencies across guardrails, supporting the findings in Tab. 2 and §4.2.

Guardrail Model Outputs

  • Aegis Guard often produces excessive, irrelevant text, including unrelated political content, reducing the clarity of its safety assessments.Its behavior closely resembles LLaMA Guard because it is built on LLaMA Guard and fine-tuned using PEFT.Source: B Guardrail Model Outputs · p. 13
  • LLaMA Guard 2 can append conflicting, repetitive, erroneous labels, while WildGuard’s binary-only classification makes its F1 difficult to compare directly with multi-label models.WildGuard cannot specify violated categories.Source: B Guardrail Model Outputs · p. 13
  • LLaMA Guard 3-ICL sometimes produces ambiguous repeated “unsafe” outputs, while the label-only fine-tuned model gives category labels without detailed explanations.The label-only model’s lack of detailed explanations reduces interpretability.Source: B Guardrail Model Outputs · p. 13
  • DeepSeek-LLaMA-70B produces overly detailed explanations that increase reasoning time and can make security principles harder to explain.GPT-4o also sometimes assigns unrelated categories, introducing noise into safety assessments.Source: B Guardrail Model Outputs · p. 13

C Inference Latency Discussion

ThinkGuard and LlamaGuard3 are compared for inference latency across four datasets. Disabling explanations yields comparable latency, while generating explanations adds reasoning time and supports interpretability and transparency.

Inference Latency Discussion

  • The latency comparison covers ThinkGuard and LlamaGuard3 across four datasets.The cited passages identify the comparison and say it is shown in Table 5, but provide no latency values.Source: C Inference Latency Discussion · p. 13
  • With explanation generation disabled, ThinkGuard’s latency is comparable to LlamaGuard3; enabling explanations increases computational time through additional reasoning steps.ThinkGuard offers inference with or without explanation generation.Source: C Inference Latency Discussion · p. 13
  • The added explanation-generation overhead provides insights into the model’s reasoning, supporting interpretability and transparency for safety-critical applications.Understanding the rationale behind a decision is described as essential in those applications.Source: C Inference Latency Discussion · p. 13
Loading 2502.13458v2…