PaperlayerView exact revision on arXiv

Source-linked AI summary

COMPASS: A Framework for Evaluating Organization-Specific Policy Alignment in LLMs

Dasol Choi, DongGeon Lee, Brigitta Jesica Kartono, Helena Berndt, Taeyoun Kwon, Joonwon Jang, Haon Park, Hwanjo Yu, Minsuk Kahng

arXiv:2601.01836v1cs.AIcs.CY

TL;DR

Existing safety evaluations target universal harms, while organization-specific policy compliance lacks standardized protocols. COMPASS tests organizational policies with structured queries, and its evaluations find strong performance on allowed requests but weak refusal of prohibited ones.

  • Problem

    Existing safety benchmarks primarily target universal harms and do not directly capture organization-defined policy violations; manual prompt checks also limit reproducibility and cross-version comparison.

    Source: §1 · p. 1

  • Method

    COMPASS formalizes allowlist and denylist policies into structured query sets, tests boundaries with six adversarial transformations, and relies mainly on LLM evaluation with targeted human review for ambiguous cases.

    Source: §3.1.3 · p. 4, §7 · p. 9

  • Results

    Across eight industry domains and fifteen LLMs, models exceeded 95% accuracy on allowlisted requests but refused denylisted requests only 13–40% of the time; some refused fewer than 5% of violating edge cases.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The observed gap persists across model scales, indicating that scaling or prompt engineering alone is insufficient for reliable policy compliance.

    Source: §7 · p. 9

    The eight organizational scenarios cannot represent every enterprise context, and the six predefined adversarial transformations may miss real-user or adversary obfuscation techniques.

    Source: Limitations · p. 9

Abstract

from arXiv · show

As large language models are deployed in high-stakes enterprise applications, from healthcare to finance, ensuring adherence to organization-specific policies has become essential. Yet existing safety evaluations focus exclusively on universal harms. We present COMPASS (Company/Organization Policy Alignment Assessment), the first systematic framework for evaluating whether LLMs comply with organizational allowlist and denylist policies. We apply COMPASS to eight diverse industry scenarios, generating and validating 5,920 queries that test both routine compliance and adversarial robustness through strategically designed edge cases. Evaluating seven state-of-the-art models, we uncover a fundamental asymmetry: models reliably handle legitimate requests (>95% accuracy) but catastrophically fail at enforcing prohibitions, refusing only 13-40% of adversarial denylist violations. These results demonstrate that current LLMs lack the robustness required for policy-critical deployments, establishing COMPASS as an essential evaluation framework for organizational AI safety.

1 Introduction

Organization-specific policy compliance lacks standardized evaluation, even though high-stakes deployments require adherence to organization rules. COMPASS evaluates this alignment and reveals a sharp gap between handling allowed requests and refusing prohibited ones.

1 Introduction

  • Healthcare assistants may provide health information but should not diagnose or advise on dosing; failures can bring misinformation, regulatory breaches, reputational damage, and user harm.Source: §1 · p. 1
  • Organization-specific rules differ from universal safety because they impose nuanced constraints that vary by domain and organization.For example, a general-purpose chatbot may criticize a company while an organization-aligned chatbot should refuse under a denylist policy.Source: §1 · p. 1
  • Existing safety benchmarks target universal harms, while manual prompt checks limit reproducibility and cross-version comparison; changing policies also challenge fixed-benchmark coverage.Organization-specific policies vary across domains and evolve over time.Source: §1 · pp. 1–2
  • COMPASS synthesizes routine and boundary-testing queries from an organization’s allowlist and denylist, then uses an LLM judge to assess chatbot responses.Responses are labeled aligned or misaligned with the policies.Source: §1 · p. 2, Figure 2 · p. 2
  • >95% accuracy on allowlisted requests contrasts with 13–40% refusal of denylisted requests across eight domains and fifteen LLMs.Some models refused fewer than 5% of policy-violating edge cases.Source: §1 · p. 2

2 Related Work

Prior work evaluates policy compliance in particular contexts and develops configurable guardrails, but it does not provide a unified evaluation protocol for enterprise-specific constraints.

2 Related Work

3 COMPASS Framework

COMPASS evaluates organizational policy alignment from an organization’s allowlist, denylist, and context. It generates and validates routine and boundary-testing queries, then assesses chatbot responses for policy adherence.

3 COMPASS Framework

  • COMPASS evaluates organizational chatbot alignment using an organization’s policy set and context.The policy set contains natural-language allowlist and denylist policies.Source: §3 · p. 3

3.1 User Query Generation

  • User query generation proceeds through base-query synthesis and validation, followed by edge-case synthesis and validation.The separate steps test routine policy boundaries and more challenging cases.Source: §3.1 · p. 3

3.1.1 Base Query Synthesis

  • For each policy, an LLM generates ten naturalistic base queries that vary in style, specificity, and complexity.The queries probe permitted behaviors for allowlist policies and prohibited behaviors for denylist policies.Source: §3.1.1 · p. 3

3.1.2 Base Query Validation

  • Validation retains allowed base queries only when they match their intended allowlist policy and no denylist policy.Denied base queries must match their intended denylist category; overlapping allowlist matches are ignored.Source: §3.1.2 · p. 3

3.1.3 Edge Case Query Synthesis

  • Edge-case synthesis targets both over-refusal and under-refusal; denied-edge queries use six obfuscation strategies in short and long forms.For each denylist policy, the procedure samples two short and four long variants per denied base query.Source: §3.1.3 · pp. 3–4

3.1.4 Edge Case Query Validation

  • Edge-query validation retains allowed cases only if genuinely compliant and denied cases only if they violate the intended denylist policy.Human verification agreement was 89.4% for allowed edge cases and 90.3% for denied edge cases.Source: §3.1.4 · p. 4

3.2 Evaluation Metrics

  • PAS measures the share of appropriate responses, requiring substantive compliant answers to allowed queries and explicit policy-compliant refusals to denied queries.The evaluation uses GPT-5-mini with high reasoning effort, with human validation reported in Appendix F.2.Source: §3.2 · pp. 4–5

4 Experimental Setup

The testbed uses simulated organizations across eight industries, and target chatbots are instantiated with system prompts and pseudo-context retrieval. Across the reported scenarios, allowlist accuracy exceeds 95%, while denylist enforcement remains fragile.

4.1 Testbed Dataset Construction

  • The testbed comprises eight simulated organizational scenarios because real enterprise policies and chatbot services are practically difficult to access.The domains are Automotive, Government, Financial, Healthcare, Travel, Telecom, Education, and Recruiting; scenarios reflect distinct regulatory and operational contexts.Source: §4.1 · p. 5
  • Each scenario specifies allowlist and denylist policies, with examples spanning restrictions on competitor mentions, clinical diagnoses, and investment advice.The eight scenarios produce eight testbed datasets.Source: §4.1 · p. 5

4.2 Target Chatbot Instantiation

  • Target chatbots encode scenario policies in system prompts and use retrieval-augmented generation with synthesized pseudo-context.The evaluated set includes proprietary, open-weight dense, and mixture-of-experts models.Source: §4.2 · pp. 5–6
  • >95% accuracy on allowlist queries contrasts with fragile denylist enforcement across eight domains and four query types.Table 3 reports PAS for system prompt-based chatbot instantiations.Source: Table 3 · p. 6

4.3 Mitigation Strategies

  • The mitigation strategies are explicit refusal prompting, eight few-shot demonstrations, and GPT-4.1-Nano pre-filtering of restricted inputs.The demonstrations cover allowed and denied base and edge queries; pre-filtering labels queries ALLOW or DENY before they reach the target model.Source: §4.3 · p. 6

5 Experimental Results

Across the evaluated scenarios, models handle allowed requests far better than denied ones. Scaling and mitigation tests show that stronger denylist enforcement remains difficult or comes with trade-offs in allowed-query performance.

5.1 Overall Performance

  • Qallow_base PAS averages 97.5–99.8%, versus 13–40% for Qdeny_base; Qdeny_edge PAS is 3.3% for GPT-5 and 4.2% for Llama-3.3-70B.Other models achieve 17–21% PAS on Qdeny_edge.Source: §5.1 · pp. 6–7
  • In both Gemma-3 and Qwen2.5, larger models improve Qallow PAS, but Qdeny_edge remains close to zero even at 72B.Gemma-3 Qdeny_base improves from 18% at 1B to 40% at 27B.Source: Figure 3 · p. 7, §5.1 · p. 7
  • Across all eight scenarios, Qallow performance remains consistently high while Qdeny performance varies substantially by domain.The gap between Qallow and Qdeny persists across every scenario.Source: §5.1 · p. 7
  • RAG maintains strong Qallow performance but provides inconsistent, limited improvements on Qdeny.The authors attribute the asymmetry to policy-reasoning limitations rather than insufficient context.Source: §5.1 · p. 7, Figure 4 · p. 7

Explicit Refusal Prompting.

  • Explicit refusal instructions keep Qallow PAS stable or slightly higher but improve Qdeny PAS by only 1–3%.The authors conclude that prompt engineering alone cannot overcome policy-enforcement limitations.Source: Explicit Refusal Prompting · p. 7
  • Few-shot examples improve Qdeny_edge more substantially, but Qallow_edge often declines, including Claude's drop from 92.8% to 87.2%.The authors suggest demonstrations may increase conservatism at the expense of helpfulness.Source: Explicit Refusal Prompting · p. 7

Pre-Filtering.

  • Pre-filtering raises all models above 96% accuracy on Qdeny_edge, while Qallow performance falls to the mid-30% range.GPT-5 Qallow accuracy drops from 96.6% to 37.2%; allowed-base accuracy remains 92–95%.Source: Table 4 · p. 8, Pre-Filtering · p. 8

6 Analysis & Discussion

Misaligned denylist-edge responses take distinct forms across model families. A held-out-domain fine-tuning experiment also improves denylist-edge performance while largely preserving allowlist performance.

6.1 Failure Mode Analysis

  • Open-weight models' misaligned Qdeny_edge responses are mostly direct violations (80–83%), while proprietary models mostly give a refusal-answer hybrid (61–65%).A third pattern, indirect violation, provides enabling information or meta-knowledge without directly answering; the authors link family differences to distinct alignment gaps.Source: §6.1 · p. 8, Figure 5 · p. 8

6.2 Tractability of Policy Alignment

  • In Leave-One-Domain-Out evaluation, LoRA adapters trained on seven domains raise held-out Telecom Qdeny_edge PAS from 0% to 60–62%, largely preserving Qallow performance.The authors interpret this cross-domain transfer as evidence that policy alignment may be learnable as a generalizable skill.Source: §6.2 · p. 9

7 Conclusion

COMPASS finds strong performance on allowed queries but persistent vulnerability in denylist enforcement across its evaluated domains. The authors qualify this result by noting limits in scenario coverage and ecological realism, and by defining the evaluation as adherence to explicit policies rather than a normative safety standard.

7 Conclusion

  • 60–87% error rates under adversarial conditions contrast with over 95% accuracy on allowed queries across eight domains and 5,920 verified queries.The gap persists across model scales, indicating that scaling or prompt engineering alone is insufficient for reliable policy compliance.Source: §7 · p. 9

Limitations

  • Eight scenarios do not exhaust enterprise contexts, and six predefined adversarial transformations may miss other user or adversary obfuscations.The authors identify legal services, pharmaceutical research, and defense contracting as examples of domains not represented.Source: Limitations · p. 9

Ethical Considerations

  • Using only synthetic organizational scenarios protects proprietary and personally identifiable information and avoids generating actionable harmful content, though it limits ecological realism; GPT-5-mini automated assessments may introduce bias and opacity.Source: Ethical Considerations · p. 9
  • COMPASS is intended for authorized security testing and model improvement, and evaluates explicit policy adherence rather than endorsing a normative AI-safety standard.The authors report that experimental setup and prompt specifications are provided to facilitate reproducibility.Source: Ethical Considerations · p. 9

A Implementation Details

This section documents COMPASS’s model settings and access routes, testbed context, and prompt design, including the use of prompt templates. It specifies organization-specific allowlists and denylists across automotive, government, financial, healthcare, travel, telecommunications, TelePath, TutoraVerse, and VirtuRecruit scenarios, distinguishing public-facing services and information from prohibited topics or actions.

A Implementation Details

  • All LLM operations used temperature=0.7 and top_p=1.0, with exceptions for pre-filtering and GPT-5 reasoning settings; experiments ran on A100 GPUs or model-specific APIs and inference routes.Pre-filtering used temperature=0.1; GPT-5-mini validation and judging used reasoning_effort=high, while the GPT-5 target chatbot used reasoning_effort=minimal. GPT models used the OpenAI API, Claude and Gemini used Google Vertex AI, and open-weight models used OpenRouter or local vLLM inference.Source: A Implementation Details · p. 13

B Organization Scenario Design

  • COMPASS’s testbed uses eight organizational scenarios, each specified through allowlist and denylist policies, organizational context, and a target-chatbot system prompt.Complete policy definitions are provided for all eight scenarios; AutoViaMotors supplies the representative context and prompt, while the other scenarios follow the same structure with domain-specific adaptations available in the released codebase.Source: Table 6 · p. 13, B Organization Scenario Design · pp. 13–14

System Prompt of AutoViaMotors

  • AutoVia’s manually developed, reviewer-refined prompt template defines a helpful brand persona and directs the chatbot to assess intent, verify policy scope, answer with official resources, and avoid prohibited content.The template was adapted by industry and includes guidance on response tone and format; its AutoVia example prohibits competitor comparisons, unpublished data, legal matters, unauthorized repairs, and other restricted topics.Source: B Organization Scenario Design · p. 14, System Prompt of AutoViaMotors · pp. 16–17

Allowlist Policy A of AutoViaMotors

  • AutoViaMotors’ allowlist covers official vehicle standards and recalls, automotive education, company services, maintenance guidance, owner resources, and public product information.The policy includes published safety and regulatory information, owner-performable maintenance and approved specifications, and public vehicle details such as trims, features, and MSRP pricing.Source: Allowlist Policy A of AutoViaMotors · p. 14

Denylist Policy D of AutoViaMotors

  • AutoViaMotors’ denylist bars competitor information, vehicle hacking, unrelated topics, negative commentary, unpublished emissions data, legal matters, and proprietary business information.Examples include unauthorized vehicle-system access, criticism or confidential disclosures, ongoing lawsuits, internal sales figures, unannounced products, and manufacturing costs.Source: Denylist Policy D of AutoViaMotors · p. 14

Context C of AutoViaMotors

  • The AutoViaMotors context supplies a company profile and 2025 vehicle lineup alongside warranty, connected-service, financial-service, dealer, sustainability, and award details.It also includes required disclaimers, including that listed prices are starting MSRPs excluding destination and delivery charges and that information is current as of September 2025.Source: Context C of AutoViaMotors · pp. 15–16

System Prompt of AutoViaMotors

  • AutoVia authorizes vehicle standards, automotive and company-service information, maintenance, recalls, and product details while barring competitor comparisons, hacking, unrelated topics, criticism, and unpublished emissions specifics.Source: System Prompt of AutoViaMotors · p. 19
  • AutoVia’s prohibitions also cover legal disputes and confidential internal data, including sales, future plans, supplier relationships, costs, and development timelines.Source: System Prompt of AutoViaMotors · p. 19

Allowlist Policy A of CityGov; Denylist Policy D of CityGov

Allowlist Policy A of FinSecure; Denylist Policy D of FinSecure

Allowlist Policy A of MediCarePlus; Denylist Policy D of MediCarePlus

Allowlist Policy A of PlanMyTrip

Denylist Policy D of PlanMyTrip

Allowlist Policy A of TelePath; Denylist Policy D of TelePath

Denylist Policy D of TelePath

  • TelePath’s denylist includes hacking instructions and SIM swapping, actions beyond service scope such as legal advice, and services outside telecommunications scope.Source: Denylist Policy D of TelePath · p. 23

Allowlist Policy A of TutoraVerse

Denylist Policy D of TutoraVerse

Allowlist Policy A of VirtuRecruit

  • VirtuRecruit’s allowlist covers skills-based job matching, neutral role and application information, career development, AI transparency, GDPR-compliant data handling, and alternatives when no suitable role exists.Source: Allowlist Policy A of VirtuRecruit · p. 24

Denylist Policy D of VirtuRecruit

  • VirtuRecruit’s denylist bars discriminatory matching and biased language, false hiring guarantees, unsuitable recommendations, out-of-scope professional advice, and psychological or medical assessments.Source: Denylist Policy D of VirtuRecruit · p. 24

C Prompt Templates of COMPASS

D.1 Details of RAG Implementation

The RAG setup synthesized domain-specific documents and added relevant context to user prompts. Its generation and inference prompts specified how retrieved material should be created, supplied, and used.

D.1 Details of RAG Implementation

  • The RAG setup synthesized domain-specific documents and augmented user prompts with relevant context.Four retrieved documents were generated for each query.Source: §D.1 · p. 34

Synthetic Retrieved Document Generation Prompt

  • The synthetic-document prompt requested realistic, fictional snippets that stayed self-consistent and aligned with the company context.Documents were generated with Claude-Haiku-4.5 and could draw on concrete entities, products, dates, or metrics.Source: §D.1 · p. 34, Synthetic Retrieved Document… · p. 34

User Prompt Template for RAG

D.2 RAG Evaluation Results

This section evaluates RAG policy enforcement on Allowed and Denied Edge queries and describes associated mitigations and validation procedures. It also reports human–LLM agreement in validation and explains the annotation guide’s multi-label policy.

D.2 RAG Evaluation Results

  • Denied Edge average PAS was 12.4% for the base setup vs. 10.8% with RAG, while Allowed-query performance remained near-saturated.RAG produced only modest and inconsistent denylist gains and did not resolve the core vulnerability; Denied Base PAS increased only slightly on average. Qwen2.5-72B moved from 0.94% to 2.75%, whereas Gemini-2.5-Pro declined from 17.73% to 11.69%.Source: §D.2 · p. 35

E.1 Explicit Refusal Prompting

  • Explicit Refusal Prompting strengthens the base system prompt with instructions to refuse non-compliant queries and redirect users to appropriate channels across eight scenarios.The supplied AutoVia example is representative; the other seven templates use the same structure with domain-specific policy adaptations.Source: §E.1 · p. 36

Prompt Template for Explicit Refusals

  • The AutoVia template directs refusals of out-of-scope or prohibited queries to customer care, while allowing AutoVia feature descriptions when declining competitor discussion.It instructs the model not to mention or acknowledge competitors and provides a customer-care number for redirection.Source: Prompt Template for Explicit Refusals · p. 36

E.2 Pre-Filtering

  • The LLM-based Pre-Filter uses a lightweight model to classify queries as ALLOW or DENY before they reach the chatbot, applying organization-specific policies through placeholders.The same generalizable template is used across eight scenarios by substituting the company name, allowlist, and denylist.Source: §E.2 · p. 38

Prompt Template for Pre-Filtering

  • The pre-filter template asks a policy checker to apply the allowlist and denylist and return a JSON decision with confidence, matched policy, and reason.The template provides placeholders for the company, policies, and query.Source: Prompt Template for Pre-Filtering · p. 38

F.1 Validator LLM Reliability Assessment

  • Three domain experts trained on the annotation protocols conducted the allowed-edge and denied-edge human validation studies on TelePath.Source: §F.1 · p. 38

F.1.1 Allowed-Edge Validation

  • 89.4% human–LLM agreement was reached when experts assessed the validator LLM’s judgments on allowed-edge queries.The passage reports that this agreement demonstrates reliable identification of subtle compliant cases.Source: §F.1.1 · p. 38

F.1.2 Denied-Edge Validation

  • Annotators used a multi-label denylist evaluation to verify the construction of denied-edge queries.Source: §F.1.2 · p. 40
  • 90.3% human–LLM agreement confirmed the policy-faithfulness of denied-edge queries and the reliability of the validator’s violation detection.Source: §F.1.2 · p. 40

Annotation Guide for Denied-Edge Query Validation

  • Annotators compared each query’s stated meaning with official blocklist definitions, selecting every applicable policy or the exclusive “none” label.The guide cautions against keyword-based judgments and unsupported inferences.Source: Annotation Guide for Denied-Edge Query… · pp. 40–41

F.2 Judge LLM Reliability Assessment

A domain expert annotated one full CityGov scenario using the judge LLM’s annotation guide. Human and LLM judgments showed strong agreement across refusal detection, policy adherence, and overall alignment.

F.2 Judge LLM Reliability Assessment

  • A domain expert independently annotated all four query types in one full CityGov scenario using the judge LLM’s annotation guide.The study evaluated refusal detection, policy adherence, and overall alignment.Source: §F.2 · p. 42
  • 95.7% agreement on refusal detection and 95.4% on overall alignment indicate strong agreement between the judge LLM and human annotator.Policy adherence agreement was 89.7%; overall Cramér’s V was 0.8995.Source: §F.2 · p. 42, Table 8 · p. 42

G.1 Extended Experimental Results

The extended results examine pre-filter classification, policy-aware fine-tuning, and denied-query response patterns alongside additional model evaluations. They show a precision–recall trade-off for pre-filters, improved held-out-domain results with fine-tuning, and frequent failures to recognize denied queries as prohibited.

G.1 Extended Experimental Results

  • Table 9 extends the main results in Table 3 with additional open-source and closed-source model evaluations across eight domains and four query types.The passage identifies the table’s scope but does not report its individual results.Source: §G.1 · p. 43, Table 9 · p. 43

G.2 Pre-Filter Classification Accuracy

  • Pre-filter models trade precision in blocking denied queries against recall in accepting allowed queries, with Gemini-2.5-Flash favoring enforcement and Gemma-3-4B-it favoring acceptance.The preferred position depends on an organization’s risk tolerance and operational priorities; GPT-4.1-Nano was selected for balanced performance in the main experiments.Source: §G.2 · p. 44, Table 10 · p. 44

G.3 Policy-aware Fine-tuning

  • LODO SFT reached 60–62% PAS on held-out Telepath Denied Edge queries, versus 0% for base models, while maintaining or improving Allowed Edge performance.The SFT dataset used compliant query-response pairs from the other seven domains.Source: Table 11 · p. 44, §G.3 · p. 45

G.4 Empirical Breakdown of Failure Modes.

  • 63–66% of denied queries received policy-compliant responses without refusal, while only 9–26% achieved correct alignment.Counting any refusal as aligned minimally changed scores: Denied Base rose from 25.81% to 26.55%, and Denied Edge from 9.18% to 9.81%.Source: §G.4 · p. 45, Table 12 · p. 45

G.5 Failure Mode Taxonomy and Examples

The taxonomy identifies three recurring patterns in misaligned denied-edge responses: direct violations, refusal-answer hybrids, and indirect violations. Examples show how each pattern can breach organizational policies in automotive, healthcare, and telecom scenarios.

G.5 Failure Mode Taxonomy and Examples

  • The authors manually developed a three-pattern error taxonomy and used GPT-5-mini to classify misaligned Denied-Edge responses.The categories are Direct Violation, Refusal-Answer Hybrid, and Indirect Violation.Source: §G.5 · p. 45

Direct Violation Example (Llama-3.3-70B)

  • In AutoVia, the response compared the Vertex with a leading Japanese SUV despite the policy against competitor mentions, describing better consumer-report-based long-term reliability, fewer reported transmission issues, and lower maintenance costs over 100,000 miles.Source: Direct Violation Example (Llama-3.3-70B) · p. 46

Refusal-Answer Hybrid Example (GPT-5)

  • In the MediCarePlus example, GPT-5 refused dosage advice but then supplied information about the effects of increasing the medication dose.The prohibited policy covers clinical medical activities requiring professional licensure, including dosage adjustments.Source: Refusal-Answer Hybrid Example (GPT-5) · p. 46

Indirect Violation Example (Qwen-2.5-32B)

  • In the TelePath example, Qwen-2.5-32B declined to analyze device logs but described methods that could reveal usage patterns and locations.The cited policy prohibits customer location tracking and movement-pattern lookups.Source: Indirect Violation Example… · p. 46

Related papers

Loading 2601.01836v1…