Source-linked AI summary
BARRED: Synthetic Training of Custom Policy Guardrails via Asymmetric Debate
Arnon Mazza, Elad Levi
TL;DR
Custom guardrails need task-specific accuracy and low latency, but existing approaches require costly retraining, larger runtime models, or extensive labeled data. BARRED generates diverse, faithful synthetic training data from a task description and unlabeled examples using dimension decomposition and multi-agent debate. Across diverse tasks, compact fine-tuned models outperform or match frontier LLMs and dedicated guardrails, while both core components prove essential.
Problem
Existing guardrails struggle to adapt accurately and efficiently to custom policies, while fully supervised task-specific training requires costly labeled data.
Method
BARRED decomposes the task domain into dimensions, generates boundary-challenging synthetic samples, and validates their labels through multi-agent debate.
Results
Small models, including 3B-parameter models, consistently outperform or match frontier LLMs and dedicated guardrail models across four diverse tasks.
Takeaways & Limitations
BARRED offers a practical path to accurate, efficient custom guardrails when labeled data is scarce but task specifications are available.
Takeaways & Limitations
Data generation requires multiple LLM calls, although this cost is amortized over the resulting compact deployable model.
Abstract
from arXiv · showhide
Deploying guardrails for custom policies remains challenging, as generic safety models fail to capture task-specific requirements, while prompting LLMs suffers from inconsistent boundary-case performance and high inference costs. Training custom classifiers achieves both accuracy and efficiency, yet demands substantial labeled data that is costly to obtain. We present BARRED (Boundary Alignment Refinement through REflection and Debate), a framework for generating faithful and diverse synthetic training data using only a task description and a small set of unlabeled examples. Our approach decomposes the domain space into dimensions to ensure comprehensive coverage, and employs multi-agent debate to verify label correctness, yielding a high-fidelity training corpus. Experiments across diverse custom policies demonstrate that small language models finetuned on our synthetic data consistently outperform state-of-the-art proprietary LLMs (including reasoning models) and dedicated guardrail models. Ablation studies confirm that both dimension decomposition and debate-based verification are critical for ensuring the diversity and label fidelity required for effective fine-tuning. The BARRED framework eliminates the reliance on extensive human annotation, offering a scalable solution for accurate custom guardrails.
1. Introduction
Custom guardrails must handle context-dependent policies while balancing accuracy, latency, flexibility, and data requirements. BARRED addresses this challenge by generating and verifying task-specific synthetic data from minimal inputs, with strong results across diverse tasks.
- Context-dependent safety policies make a response benign in one deployment but a critical failure in another.
- Static guardrails achieve strong predefined-category accuracy but require costly retraining for novel policies, whereas dynamic models trade flexibility for latency and accuracy.
- BARRED trains fully customized guardrails using only a policy definition and few-shot unlabeled examples.
- Dimension decomposition expands domain coverage, while multi-agent debate resolves ambiguous labels to improve synthetic-data diversity and faithfulness.
- Across three task types, compact BARRED-trained models outperform specialized guardrails and frontier-class LLMs despite relying entirely on synthetic data.
2. Related Work
Prior work includes static and dynamic guardrails, synthetic-data bootstrapping, boundary-case generation, and multi-agent debate. BARRED combines these directions for custom guardrail training.
- Static guardrails fine-tune compact models on curated safety taxonomies, but they do not generalize to novel user-defined policies without retraining.
- Dynamic guardrails condition on arbitrary policies at inference time to address static models’ policy limitations.
- Constitutional AI, Self-Instruct, and Alpaca demonstrate synthetic-data generation through critique, revision, or minimal seed data.
- Boundary-case generation targets ambiguous examples near decision boundaries to improve prompt optimization and classifier robustness.
- Multi-agent debate uses multiple model instances and cross-verification to improve reasoning and factuality across tasks.
3. Methods
BARRED constructs synthetic guardrail data by decomposing the task, sampling diverse configurations, generating boundary cases, and validating labels through asymmetric debate and iterative refinement.
- Dimension decomposition: BARRED identifies diverse task-relevant dimensions from a task description and unlabeled seeds, then verbalized sampling generates multiple instantiations per dimension.
- Sample generation: The generator uniformly samples dimensions, instantiations, and labels to create balanced boundary-challenging samples with label reasoning.
- Debate-based validation: An Advocate argues for the generated label while multiple Judges independently assess whether the sample and reasoning support it.
- Debate-based validation: Validation accepts a sample when Judges reach consensus on the target label within the allowed rounds.
- Iterative refinement: If validation fails, Judge feedback guides regeneration with the same configuration until the sample passes or reaches the iteration limit.
4. Experimental Setup
The evaluation spans conversational, agentic, and healthcare-regulatory guardrail tasks, using human and synthetic test sets and comparisons against LLM and generic-guardrail baselines.
- Datasets: BARRED is evaluated across customer-service policy enforcement, structured-task plan verification, and healthcare health-advice detection.
- Datasets: Evaluation combines human-curated and independently generated synthetic test samples, with synthetic samples verified by human annotators.
- Datasets: The conversational benchmark includes repetition-handling and employee-location privacy policies with human-annotated training and test data.
- Evaluation: BARRED experiments compare fine-tuned models on accuracy and analyze debate verification and dimension decomposition through ablations.
- Baselines: Baselines include prompted GPT models, GPT-5-mini, Qwen2.5-14B, and generic guardrail models.
- Scaling: Qwen2.5 model sizes from 1.5B to 14B are examined to measure how parameter scaling affects accuracy across tasks.
5. Results and Analysis
BARRED’s evaluation compares fine-tuned models with strong baselines and analyzes dimension decomposition and debate verification. Across tasks, synthetic training benefits from broader dimension coverage and adversarial debate, including nontrivial disagreement and persuasion patterns.
- Comparative Performance: BARRED’s fine-tuned models consistently outperform all baselines across four tasks and test-set types.Qwen2.5-14B surpasses frontier LLMs, while the 3B model outperforms generic guardrails across benchmarks.
- Scaling and Efficiency: Even the smallest Qwen2.5 variants achieve competitive performance, while scaling gains depend on task complexity.Repetition handling saturates earlier; privacy protection and health advice continue improving with larger models.
- Dimension Decomposition Analysis: Additional dimension instantiations increase test coverage and accuracy, with logarithmic accuracy gains and diminishing returns at higher counts.Coverage is measured by LLM-judged relevance, with a sample covered when relevance exceeds 0.5 for at least one instantiation.
- Debate Verification Analysis: Removing debate verification causes a 27% accuracy drop on human-annotated data.The comparison includes raw generation without verification and single-model self-refinement.
- Debate Verification Analysis: A health-advice example shows Judge-1 revising its prediction after considering another agent’s arguments and reaching consensus.The example illustrates persuasion during the second debate round.
- Debate Verification Analysis: Over 30% of plan-verification debates follow nontrivial paths involving persistent disagreement, persuasion, or consensus breaking.Green paths indicate acceptance when the debate outcome matches the target label; red paths indicate rejection.
6. Conclusion
BARRED generates custom guardrail training data from task descriptions and minimal unlabeled examples by combining dimension decomposition with asymmetric multi-agent debate. Across four tasks, 3B models trained on this synthetic data match or outperform much larger frontier and dedicated guardrail models, while both components are essential for data quality.
- BARRED generates high-quality synthetic training data for custom guardrail models.
- Dimension decomposition provides diverse domain coverage, while asymmetric debate improves label faithfulness in boundary samples.
- 3B models consistently outperform or match larger frontier LLMs and dedicated guardrail models across four diverse tasks.
- The approach offers a practical path for accurate, efficient custom guardrails when labeled data is scarce and task specifications are available.Generation costs from multiple LLM calls are amortized over a compact model with lower latency and expense.
Impact Statement
BARRED is intended to help organizations train custom content-moderation guardrails efficiently. Its synthetic-data process reduces reliance on expensive human annotation and supports smaller models for policy enforcement.
- BARRED targets efficient training of custom content-moderation guardrails for deployed language models.
- The framework reduces reliance on expensive human annotation while enabling smaller models to achieve strong policy enforcement.
- The intended benefit is broader access to robust safety measures beyond well-resourced organizations.
A.1. Dimension decomposition
Dimension decomposition extracts task-relevant dimensions and then enumerates their reasonable instantiations. Each instantiation is associated with label relevance and a sampling probability to support test-case generation.
- Dimension decomposition has two phases: extracting dimensions and finding instantiations for each dimension.
- The extraction prompt asks for self-contained dimensions that affect the criterion and reflect relevant input positions or computation values.
- The criterion and example input block are supplied as the inputs for dimension extraction.
- The instantiation phase creates all reasonable values for a dimension that can construct criterion test cases.
- Each instantiation is labeled as relevant to True, False, or Both and receives a 0–1 probability score for sampling.
A.2. Generator
The Generator uses initial and refinement prompts to create classifier test inputs aligned with a target dimension and verdict. It targets challenging boundary cases and revises failed generations using dissenting debate arguments before final classification.
- Initial generation: The Generator creates an input block and proposed label for a classifier criterion, aligning the input with a specified dimension and target verdict.The output includes reasoning and is intended to match the domain and style of an example input block.
- Initial generation: Generated inputs should match the example’s domain and style while avoiding references to test cases, models, dimensions, or labels.The generator may diverge from the example topic when necessary to satisfy the generation requirements.
- Initial generation: Boundary-case generation is intended to stress-test a capable classifier rather than produce trivial examples.The prompt explicitly directs the Generator to create challenging cases for the criterion.
- Refinement: Failed generations are revised by examining the failed input and dissenting debaters’ arguments, then regenerating for dimension alignment and the target verdict.The revised output includes the input block, believed label, and reasoning without mentioning the debate process.
- Debate verification: Debate agents use complementary strategies: one prioritizes recall, while another favors strict precision without interpretive flexibility.Agents reconsider their prior classifications in light of other responses but are not instructed to switch labels automatically.
- Final classification: The final classification prompt asks an impartial judge to output only 1 or 0 according to whether the criterion applies to the input.The same prompt is used for fine-tuned models and comparison language models.
B. Qualitative Analysis
The qualitative analysis examines whether generated plans satisfy the available-tool and formatting requirements. One analyzed plan fails because it invokes an unavailable API, despite ending with the required tag.
- Plan verification: The analyzed task requires counting all edits to Wikipedia’s Photosynthesis page through Dec. 31, 2022 and verifying the result with official revision history.The requested response must include short and detailed outcomes plus additional context.
- Plan verification: The plan proposes inspecting revision history, selecting API parameters, retrieving paginated revisions, cross-checking counts, and compiling the result.Its steps describe both primary retrieval and alternate verification procedures.
- Plan verification: The plan fails because MediaWiki API access is not among the explicitly listed tools, even though its final characters are exactly "\n<end_plan>".The available tools include web search, page visiting, scrolling, page search, archive lookup, and final answer submission.
- Plan verification: A valid plan uses only specified tools, refers to tools abstractly rather than writing explicit calls, and ends with "\n<end_plan>".These criteria are stated as the basis for qualitative analysis of plan-verification samples.
- Other qualitative analyses: The qualitative-analysis material also includes training-sample analysis for privacy protection and health advice tasks.This passage identifies the task areas but provides no further findings here.