SafePyramid: A Hierarchical Benchmark for In-context Policy Guardrailing

Motivation

• In-context policy guardrailing requires guardrails to execute application-specific safety policies at inference time instead of relying on predefined risk taxonomies.This enables safety criteria to remain transparent, inspectable, traceable, and adaptable by editing policy specifications rather than redesigning taxonomies and retraining guardrails.Source: §1 · p. 2

Benchmark design

• SafePyramid evaluates individual-rule understanding, rule-dependency resolution, and adaptation to novel policy frameworks through a three-level hierarchy.L0 uses decisive and distractor rules, L1 adds exception and conditional dependencies, and L2 rewrites rules under fictional policy frameworks.Source: §1 · pp. 2–3

Benchmark scope

• The benchmark contains 1,000 multi-turn conversations across 10 safety domains, 3,000 policies, and 61,699 distinct natural-language rules.It requires predicting the complete set of violated rules rather than a binary safe/unsafe label.Source: §1 · p. 3

Evaluation protocols

• SafePyramid supports both frontier LLMs and fine-tuned guard models through per-policy and per-rule evaluation protocols.Per-policy evaluation predicts the complete violated-rule set, whereas per-rule evaluation judges one target rule at a time and aggregates the decisions.Source: §1 · p. 3

Evaluation results

• GPT-5.5 exactly matches the full violated-rule set in 54.0%, 35.3%, and 12.9% of cases on L0, L1, and L2, respectively.The monotonic drop across levels indicates that current models struggle to reliably execute in-context safety policies.Source: §1 · p. 3

LLM-based guardrails

• Conventional guardrails, including Llama Guard and ShieldGemma [44], classify inputs or outputs using fixed risk taxonomies, limiting their suitability for in-context policies.Recent work has begun exploring policy-configurable guardrails guided by user-provided policies.Source: §2 · p. 3

Benchmark distinction

• SafePyramid is the only compared safety benchmark that formulates violated-rule set prediction while separately testing individual-rule understanding, rule-dependency resolution, and novel context adaptation.Table 1 compares benchmark characteristics rather than dataset quality or overall usefulness.Source: Table 1 · p. 4, §2 · pp. 4–5

Policy-configurable guardrails

• Policy-configurable guardrails such as ShieldLM, GPT-OSS-Safeguard, DynaGuard [20], GSPR, and FlexGuard [9] differ in policy handling, outputs, and evaluated capabilities.This diversity leaves in-context policy guardrailing under-defined and motivates a unified benchmark.Source: §2 · p. 4

LLM safety benchmarks

• Safety benchmarks commonly measure harmful-request refusal [5] [38] [39], toxic or biased generations [8] [13], or jailbreak resistance [4] [6] [19] [30] using fixed criteria.FlexBench [9] varies enforcement regimes but remains taxonomy-centered, while DynaBench [20] mainly tests explicit response constraints rather than violated safety-rule identification.Source: §2 · p. 4

Adjacent capability benchmarks

• Instruction-following benchmarks test lexical, formatting, or system constraints [23] [42], rule-reasoning benchmarks apply professional rules [18] [26], and CL-bench [12] and CL-bench Life test context learning.These capabilities relate to SafePyramid’s L2 setting, which requires learning policy-specific concepts from a novel framework before identifying violations.Source: §2 · p. 4

3.1 Problem Setting

• Guardrails must predict the subset of in-context natural-language rules violated by a multi-turn conversation, rather than outputting only a binary safety label.This makes the task a rule-level set-prediction problem requiring recovery of the ground-truth violated-rule set.Source: §3.1 · p. 5

3.2 Hierarchical Design

• SafePyramid increases difficulty from L0 individual-rule understanding to L1 rule-dependency resolution and L2 adaptation to novel policy frameworks.Each level retains violated-rule identification while adding one distinct source of complexity, enabling diagnosis of where models fail.Source: §3.2 · p. 5

3.2.1 Level 0: Understanding Individual Rules

• L0 tests evidence-grounded application of decisive rules while using topic-related distractors to detect surface-level overmatching.Decisive rules are directly supported or unsupported by conversational evidence, whereas distractors are relevant to the topic but not behaviorally triggered.Source: §3.2.1 · p. 5

3.2.2 Level 1: Resolving Rule Dependencies

• L1 introduces exception and conditional rules that can waive, reinterpret, or strengthen base rules, requiring models to revise isolated rule judgments.Exceptions may override base classifications, while conditionals impose additional safeguards when contextual requirements are met.Source: §3.2.2 · pp. 5–6, Figure 3 · p. 6

3.2.3 Level 2: Adapting to Novel Policy Frameworks

• L2 rewrites L1 policies in a fictional framework with new concepts, terminology, and semantics while preserving rule structure to isolate contextual policy learning.The design prevents reliance on pretrained safety priors or familiar taxonomies and treats the fictional framework as a diagnostic abstraction.Source: §3.2.3 · p. 6

3.3 Domains and Statistics

• The benchmark spans 10 safety domains and 1,000 multi-turn conversations, pairing each with three policies totaling 3,000 policies and 61,699 distinct rules.Average policy size grows from 16.1 rules in L0 to 28.7 in L1 and 33.0 in L2, while average input length grows from 1.6K to 4.3K tokens.Source: §3.3 · p. 7

3.4 Benchmark Construction / 3.5 Quality Control

• SafePyramid combines schema-guided frontier-LLM generation with cross-model validation and two rounds of expert review, removing cases with disputed labels or ambiguity.Validation uses majority votes, cited evidence, and reasoning; domain experts show over 90% consistency with LLM validation on reviewed samples.Source: §3.4 · p. 7, §3.5 · pp. 7–8

3.6 Evaluation Protocols / 3.7 Evaluation Metrics

• Per-policy evaluation requires complete violated-rule-set prediction, while per-rule evaluation supports fixed-category interfaces at higher inference cost; RMR measures matching and RDR measures rule-level disagreement.RMR averages thresholds 0.7, 0.8, 0.9, and 1.0, with RMR@1.0 as exact match; RDR is lower-is-better and equals zero only for exact agreement across scored cases.Source: §3.6 · p. 8, §3.7 · pp. 8–9

4.1 Experimental Setup

• The experiments evaluated 10 frontier LLMs and 5 policy-configurable guard models, with GPT-OSS-Safeguard models tested under both per-policy and per-rule protocols.The study also evaluated Codex and Claude Code harnesses with GPT-5.4 or Claude-Opus-4.7 on 900 sampled cases.Source: §4.1 · pp. 9–10

4.2 Main Results

• 52.4% average RMR versus 23.6% was achieved by GPT-OSS-Safeguard-120B under per-rule versus per-policy evaluation, while GPT-5.5 improved only from 54.2% to 55.5%.The contrast indicates that smaller guard models benefit substantially from pre-organized relevant policy context, whereas frontier LLMs are less bottlenecked by full-policy processing.Source: Table 4 · p. 10, §4.2 · p. 10
• 54.2% average RMR and 34.9% average RMR@1.0 were achieved by GPT-5.5, while average RDR remained 19.7%; performance generally declined from L0 to L2.The L0-to-L1 drop reflects difficulty with exceptions and conditionals, while L2 exposes a major bottleneck in adapting to novel policy frameworks.Source: §4.2 · p. 10

4.3 Ablation Study

• 5.5 and 14.5 RMR points were gained by xhigh reasoning effort on L1 and L2, while RDR decreased by 3.9 and 15.7 points, respectively.The gains were small on L0, indicating that additional reasoning primarily helps resolve rule dependencies and adapt to novel frameworks.Source: §4.3 · p. 10, Figure 4 · p. 11
• 60.4% average RMR and 40.4% RMR@1.0 were reached by Claude-Opus-4.7 with Claude Code, while average RDR fell from 23.5% to 17.4%.For GPT-5.4, Codex and Claude Code raised average RMR from 56.0% to 58.0% and 57.9%, respectively, with gains especially useful for tracking requirements and verifying rule classifications.Source: Table 5 · p. 11, §4.3 · p. 11

4.4 Cost-Performance Trade-off

• Above 52% average RMR was achieved by DeepSeek-V4-Pro and Kimi-K2.6 at substantially lower cost than the most expensive models, while Gemini-3.5-Flash delivered near-50% average RMR at the lowest evaluated cost.GPT-5.5 had the best overall RMR but substantially higher evaluation cost, making model selection and policy-context organization relevant for large-scale evaluation.Source: §4.4 · pp. 11–12, Figure 5 · p. 12

4.5 Error Attribution

• Decisive rules dominated L0 errors, exception rules became dominant at L1, and conditional-rule errors grew at L2, especially for weaker models.Common failures included surface-cue matching, over-triggering exceptions, and treating conditional rules as violations rather than using them to evaluate corresponding base rules under novel frameworks.Source: §4.5 · p. 12

Lack of human baseline

• The benchmark lacks a trained human-auditor baseline, so results compare models with validated ground truth rather than expert human-level policy auditing.Future work can establish expert baselines, measure inter-annotator agreement, and study how auditors resolve rule dependencies using explicit policies.Source: §5 · p. 13

Limited modality coverage

• SafePyramid covers only text conversations, leaving multimodal violations involving images, audio, video, or documents for future evaluation.Such violations may depend on evidence distributed across modalities.Source: §5 · p. 13

Finite policy coverage

• Despite covering 10 safety domains and many rules, SafePyramid has finite policy coverage and may omit regional, legal, operational, and evolving organizational requirements.Future expansions can add domains, languages, jurisdictions, and real-world policy variants.Source: §5 · p. 13

Contribution

• SafePyramid decomposes in-context policy guardrailing into understanding individual rules, resolving rule dependencies, and adapting to novel policy frameworks.The benchmark evaluates whether guardrails can reliably execute safety policies provided in context.Source: §6 · p. 13

Findings

• Evaluations of 10 frontier LLMs and 5 policy-configurable guard models show that current systems remain far from reliable, especially under rule dependencies and newly defined policy concepts.Source: §6 · p. 13

Implications

• Per-rule evaluation and agentic harnesses suggest that stronger guardrails may require better policy decomposition, context organization, and rule-level verification.Source: §6 · p. 13
• SafePyramid aims to move guardrail research beyond fixed risk classification toward explicit, auditable, and adaptable policy execution.Source: §6 · p. 13

Ethics Statement

• The benchmark covers sensitive domains but fictionalizes conversations and policies under controlled constraints, avoiding identifiable people, organizations, and real-world incidents.It is intended for safety evaluation and guardrail research, not harmful-content generation or safeguard bypassing.Source: Ethics Statement · p. 13
• LLM-assisted construction may introduce model-derived biases, so SafePyramid applies cross-model validation, two human-review rounds, and conservative filtering of ambiguous or disputed cases.The authors encourage safer, more transparent, and more auditable guardrail systems.Source: Ethics Statement · p. 13

A Benchmark Example

• The benchmark example reuses one conversation across L0, L1, and L2 cases, with corresponding policy samples illustrating each evaluation level.The L0 sample is presented separately, L1 spans 30 rules across two tables, and L2 uses a novel framework with 32 rules across three tables.Source: Table 6 · p. 17, Table 7 · p. 18, Table 8 · p. 19
• The L2 example defines a novel framework whose 32 policy rules reference definitions by section number.The full L2 policy sample continues across Tables 11–13.Source: Table 10 · p. 21, Table 11 · p. 22, Table 12 · p. 23

B Domains and Scenarios

• Conversation scenario seeds provide the basis for constructing SafePyramid’s domain-specific evaluation conversations.The supplied material identifies Table 14 as the first of five scenario-seed tables.Source: Table 14 · p. 25

C Validation Checklists

• Separate validation checklists cover conversations, L0 policies, L1 policies, and L2 frameworks and policies.These checklists are presented in Tables 19–22.Source: Table 19 · p. 30, Table 20 · p. 30, Table 21 · p. 30

D System Prompt for Policy-configurable Guards

• FlexGuard-Qwen3-8B and DynaGuard-8B use their official system prompts for policy-configurable guard evaluation.Their prompts are documented separately in Tables 23 and 25.Source: Table 23 · p. 32, Table 25 · p. 33
• GPT-OSS-Safeguard-120B/20B receives policy and output schema on the Harmony developer channel and conversation on the user channel.This channel allocation is specified in its system prompt.Source: Table 24 · p. 33
• ShieldLM-14B-Qwen is evaluated with its official instruction prompt, documented alongside the other policy-configurable guard prompts.The prompt appears in Table 26.Source: Table 26 · p. 34

E Instruction Prompt for Agent Harness Experiment

• Every agentic harness receives a general instruction prompt through the harness CLI.The prompt is documented in Table 27.Source: Table 27 · p. 34

• The per-policy evaluation is summarized in an RMR breakdown table.Source: Table 28 · p. 35
• The per-rule evaluation is summarized in a separate RMR breakdown table covering the same reported RMR thresholds.Source: Table 29 · p. 35

G Per-domain Performance Breakdown

• Per-policy domain performance is compared for GPT-5.5 and Claude-Opus-4.7.Figure 6 reports their performance across domains.Source: Figure 6 · p. 36
• The domain breakdown includes both policy-level and rule-level evaluations across the benchmark’s model set.The figures distinguish per-policy results from per-rule results while organizing performance by domain.Source: Figure 6 · p. 36, Figure 7 · p. 37, Figure 8 · p. 38
• Additional per-policy domain comparisons cover Kimi-K2.6, DeepSeek-V4-Pro, Doubao-Seed-2.0-Pro, and Gemini-3.5-Flash.Figures 7 and 8 report these model comparisons across domains.Source: Figure 7 · p. 37, Figure 8 · p. 38
• Per-policy domain performance is also reported for Hunyuan-HY3-Preview, Qwen-3.6-Max-Preview, GLM-5.1, and Grok-4.3.Figures 9 and 10 provide the corresponding cross-domain breakdowns.Source: Figure 9 · p. 39, Figure 10 · p. 40
• Per-rule domain performance is compared for GPT-5.5 and GPT-OSS-Safeguard-120B.Figure 11 presents their per-rule results across domains.Source: Figure 11 · p. 41
• Further per-rule domain comparisons cover GPT-OSS-Safeguard-20B, FlexGuard-Qwen3-8B, DynaGuard-8B, and ShieldLM-14B-Qwen.Figures 12 and 13 report these models’ per-rule performance across domains.Source: Figure 12 · p. 42, Figure 13 · p. 43

H Per-domain Effect of Agent Harness

• The effect of an agent harness is evaluated per domain for per-policy performance on a SafePyramid subset.Figure 14 reports this domain-specific harness analysis.Source: Figure 14 · p. 44

• Figure 15 reports the paper’s error attribution analysis.Source: Figure 15 · p. 45

Decisive-rule failures

• 54% of decisive failures stem from not applying trigger or scope predicates, producing surface-cue matches.This is the dominant illustrated mechanism in the decisive-rule analysis.Source: Table 30 · p. 46

Distractor-rule failures

• 58% of distractor-rule failures arise from over-reading specificity after dropping qualifiers.The analysis identifies dropped-qualifier specificity over-read as the dominant illustrated mechanism.Source: Table 31 · p. 47

Exception-rule failures

• 43% of exception-rule failures occur when an exception fires even though its activation condition is unmet.This is the dominant illustrated mechanism in the exception-rule analysis.Source: Table 32 · p. 48

Conditional-rule escalation

• 26% at L2 versus 2% at L1: GPT-5.5 incorrectly treats conditional rules as violations when identical templates are expressed through novel §-clause concepts.Under L1, the model correctly treats the conditional as a modifier and lists only the base rule; under L2, it mishandles the same conversation and policy template.Source: Table 33 · p. 49, Table 34 · p. 50