Source-linked AI summary
Granite.Trust Policy Tools: Shareable, Actionable Policies for Generative AI Applications
Nathalie Baracaldo, Nicolas Mello, Kush R. Varshney, Heiko Ludwig, Kate Soule, David Cox
TL;DR
GenAI applications require policies that address generative, content-based risks and satisfy varied stakeholder expectations, but existing approaches do not provide suitable specification. Granite.trust Policy Tools introduces a human-readable, machine-enforceable schema and a synthetic-data pipeline for model alignment.
Problem
GenAI applications create novel risks and require policies that satisfy safety, security, regulatory, and business expectations, while existing policy specification lacks a suitable approach for these applications.
Method
Granite.trust Policy Tools combines a policy schema for human-readable, machine-enforceable specification with a synthetic data pipeline that translates policies into training data for model alignment.
Results
The policy schema enables human-readable yet machine-enforceable policy specification, and the synthetic data pipeline translates policies into training data for model alignment.
Takeaways & Limitations
The tools address policy specification, testing, and enforcement across different stages of the GenAI lifecycle.
Takeaways & Limitations
The participant set was limited to six people in the first experiment and three in the second, requiring further large-scale study.
Abstract
from arXiv · showhide
When it comes to safety policies for generative AI, one size does not fit all. Each organization and use case needs to mitigate different risks depending on the application context, regulatory environment, organizational values, and user personas. Yet, existing policy specification approaches are designed for traditional access control and fail to capture the nuances of GenAI application: the enforcement of content-based constraints. We present two contributions to address this gap: (1) the Actionable Policy schema, a YAML-based format for specifying what model responses can and cannot contain. The schema enables exception-based policy governance, proposing exceptions to track policy violations; (2) synthetic data generation pipeline that produces policy-aligned training data for model alignment and testing, and a set of tools to help define the schema and enforce policy. Together, these enable organizations to specify policies once and enforce them throughout the GenAI application lifecycle: from model alignment to runtime monitoring. The Actionable Policy schema, example policies, and tools are available as open source: https://github.com/ibm-granite/granite.trust.policy-tools We welcome new ideas, contributions and feedback.
1 Introduction
GenAI applications require policies that capture content-based behavior, exceptions, and organization-specific risks beyond traditional access control. Granite.Trust Policy Tools provide a YAML schema, policy-driven synthetic data generation, exception governance, and complementary tools spanning model development through runtime enforcement.
- Policies defined through risk management can support model alignment, testing, red teaming, pre-deployment compliance assessment, monitoring, and runtime enforcement.
- GenAI policies must specify response content, harmful-request declines, logged exceptions, and communication of policy violations.
- The tools address organization-specific policies because pre-existing guardrails and online datasets may not reflect narrowly defined use cases.
- The YAML-based policy schema is human-readable and machine-enforceable, defining what responses can and cannot contain, how to decline requests, and what exceptions to raise.
- The synthetic-data pipeline generates policy-aligned adversarial prompts and safe responses for model fine-tuning, guardrail generation, and compliance testing.
- Exception-based governance tracks policy violations across application boundaries and supports handling them within the application or upstream in another agent.
2 Requirements and Related Work
The paper frames GenAI policy specification as a need for human-readable, machine-actionable, versioned, shareable, and exception-aware controls. It situates this approach among risk taxonomies, default model safeguards, alignment methods, datasets, and synthetic-data techniques that do not fully encode organization-specific policies.
- A clear policy definition is necessary for effective evaluation and governance of GenAI applications.
- The policy format must be readable by non-programmers so technical teams, legal counsel, and compliance officers can collaborate on risk controls.
- Machine-actionable policies must be precise enough to support synthetic data generation, automated compliance testing, and runtime policy enforcement.
- Versioning supports audit trails, compliance certification, and tracking which datasets were generated under particular policy definitions.
- Shareable policies enable comparison, conflict detection, and collaborative development across organizations, partners, regulators, and agents.
- Typed exceptions can propagate through the application stack to support consistent handling when a policy boundary is crossed.
- Existing risk taxonomies identify risks but do not complete the process of deciding whether and how each risk should be mitigated.
- Pre-baked model safeguards and guardrails provide general defenses but offer limited organizational control and may cause over-refusal or lose alignment after fine-tuning.
3 Policy Schema
The Policy schema defines versioned, hierarchical risks with declarative content constraints and typed responses, while exceptions support tracking policy violations. Its structure accommodates deployment-specific policies for the same underlying risk.
- The schema was designed with IBM governance collaborators and informed by developing actual policies.
- Four principles guide the schema: evolving risk tracking, hierarchical risk organization, declarative constraints, and typed responses.Risks are grouped into scenarios, constraints specify permitted or prohibited content, and response types standardize handling.
- Each risk item defines a specific scenario, while policy fields specify identifiers, versions, denial reasons, response types, exceptions, and content boundaries.The schema documents field semantics in Table 2 and distinguishes hard prohibitions from explicitly permitted content.
- Standard enumeration values support interoperability, including explicit refusal, informative response, disclaimer, polite redirect, partial response, human escalation, and silent logging.
- The alcohol example uses the same schema structure to express jurisdiction-specific restrictions and permitted alternatives, with an exception code for violations.The example prohibits alcohol information and recommendations while allowing polite explanations and non-alcoholic alternatives.
- The same underlying alcohol-related harm can produce different policies depending on deployment context.
4 Generate Synthetic Data According to Policy
The DGT safety sdg module turns policy specifications into synthetic adversarial prompts paired with policy-following answers for model training, assessment, and red teaming.
- The DGT safety sdg module generates prompt-answer pairs in which prompts aim to violate policy and answers follow the policy.It is implemented within the open-source DGT framework, which supports different data-generation algorithms and models.
4.1 The Challenge
Policy-aligned safety-data generation must address alignment mismatch, adversarial weakness, and licensing constraints.
- Prompting an LLM to generate unsafe examples may reflect the generating model’s alignment rather than the target policy.
- Naive generation can produce benign examples that fail to stress-test the target model and may lead to over-refusal during training.
- The pipeline must use appropriately licensed models because some models prohibit using their outputs to train other models.
4.2 Pipeline Architecture
The safety sdg pipeline uses a policy, example seeds, and a requested sample count to generate diverse adversarial prompts and safe responses through staged generation and filtering.
- Pipeline Architecture: The pipeline takes a Granite.trust Policy, example data seeds, and a requested number of samples as inputs.It produces adversarial prompt and safe response pairs through five stages.
- Pipeline Architecture: Policy fields populate a prompt template that guides an LLM to generate adversarial questions, while in-context seeds promote diversity.
- Pipeline Architecture: ROUGE scoring removes near-duplicates against both seed data and newly generated instructions.
- Pipeline Architecture: Granite Guardian retains only instructions classified as unsafe, defining adversariality according to the policy.
- Pipeline Architecture: For each adversarial instruction, the pipeline generates a safe refusal guided by the policy’s reply may contain fields.
- Pipeline Architecture: Granite Guardian filters generated responses and retains only those classified as safe.
4.3 Setup and Output Format
The DGT safety sdg pipeline iteratively generates policy-aligned synthetic training data, using randomized in-context examples and Guardian filtering to promote diversity and output safety.
- The five-stage YAML-configured process repeats until it reaches the desired number of synthetic samples.Each iteration selects new random ICLs, while generated training pairs include policy metadata.
- Each generated training pair records an instruction, response, risk, risk identifier, and policy version.The example addresses obtaining alcohol for a minor and provides a refusal with non-alcoholic alternatives.
- The pipeline’s dual-stage Guardian filtering checks input adversariality and output safety against the defined safety policy.Policy-driven generation supports extension to new risk categories, while ICL prompting produces diverse adversarial examples.
5 Evaluation
The evaluation examined stakeholder usability and synthetic-data quality through policy review and authoring studies, followed by policy-driven data generation and compliance training.
- The evaluation measured stakeholder usability and synthetic-data generation quality across separate experiments.The studies included reviewing pre-written policies, writing policies from scratch, and generating policies for 102 risks.
- Experiment 1: Six stakeholders reviewed and refined policies through debate, with an organizationally authoritative approver making final deployment-oriented decisions.Five debaters and one approver participated; debaters could revise vocabulary, content, risk groupings, and policies.
- Experiment 1: The initial pool of 102 policies was reduced to 80 after deduplication, regrouping, and changes to the risk hierarchy.Participants largely ignored metadata fields and focused on the main policy definitions, suggesting some fields could be hidden during specification.
- Lessons Learned: Participants found policy contribution intuitive, created policies for diverse risks, and valued the format’s explainability.Legal experts supplied supporting documents, suggesting automation could generate YAML files for later expert review.
- Lessons Learned: Debaters found conflicts, fixed them, and improved the risk taxonomy, while approvers accepted, revised, or rejected policies based on the scenario.Runtime metadata was largely overlooked and could be added later by software developers.
- Lessons Learned: The resulting policies successfully generated synthetic data and trained LoRA adapters for policy compliance.The policy-aware approach filtered some benign samples syntactically close to malicious content and exposed risks when models failed to generate malicious samples.
- Limitations: The participant set was limited to six people in the first experiment and three in the second, requiring larger studies to assess policy-specification experience.
6 Complementary Tools
The complementary tools support policy collaboration and lifecycle use by reporting coverage, detecting conflicts, and enabling synthetic-data generation or direct compliance verification.
- Stakeholder policy-generation experience led to complementary tools that facilitate policy definition.
- Policy reporting: Policy reports summarize covered risks, risk hierarchies, and exception types in shareable text, Markdown, and HTML formats.
- Conflict detection: A topic-similarity tool detects conflicting or duplicate policies, whose automated output requires verification.
- Compliance verification: Synthetic data can support fine-tuning or steering, while Granite Guardian 4.1 verifies compliance through bring your own criteria.
7 Conclusions
Granite.trust Policy Tools address the lack of standardized, human- and machine-suitable GenAI policy specification with schema, data, exception tracking, and supporting tools.
- The policy schema provides human-readable yet machine-enforceable specification for GenAI policies.
- The synthetic-data pipeline translates policies into training data for model alignment.
- Exception-based tracking supports governance in single- and multi-agent systems by recording policy violations.
- Complementary tools facilitate policy definition, conflict detection, and compliance verification.
- Together, the contributions let organizations define safety requirements once and enforce them throughout the GenAI stack.The tools and example policies are available as open source.