PaperlayerView exact revision on arXiv

Source-linked AI summary

Scaling Policy Compliance Assessment in Language Models with Policy Reasoning Traces

Joseph Marvin Imperial, Harish Tayyar Madabushi

arXiv:2509.23291v1cs.CLcs.LG

TL;DR

Policy-compliance assessment requires expert-level interpretation of rules, but documenting gold-standard reasoning is costly. The paper introduces generated Policy Reasoning Traces (PRTs) as a reasoning bridge for in-context learning and finetuning, and reports HIPAA gains and new GDPR state-of-the-art baselines.

  • Problem

    Policy-compliance assessment requires identifying applicable policy constraints, while documenting expert-level reasoning traces is expensive and time-consuming.

    Source: §1 · pp. 1–2

  • Method

    The paper generates pseudo-expert Policy Reasoning Traces (PRTs) from case information and gold-standard verdicts for few-shot in-context use or supervised finetuning.

    Source: §1 · p. 2

  • Results

    PRTs produce 50-100% significant performance increases for open-weight LLMs on HIPAA and new state-of-the-art GDPR baselines through finetuning.

    Source: §1 · p. 2

  • Takeaways & Limitations

    PRTs also support cross-policy transfer and improve models’ ability to cite correct policy clauses when reasoning toward verdicts.

    Source: §1 · p. 2

    On ModelSpec, using PRTs with OpenAI GPT-OSS-20B and GPT-5-MINI is associated with an average 4.7 accuracy decline in models described as doubly optimized.

    Source: §4.2 · p. 7

Abstract

from arXiv · show

Policy compliance assessment is a fundamental task of evaluating whether an input case strictly complies with a set of human-defined rules, more generally known as policies. In practice, human experts follow a systematic, step-by-step process to identify violations with respect to specific stipulations outlined in the policy. However, such documentation of gold-standard, expert-level reasoning processes is costly to acquire. In this paper, we introduce Policy Reasoning Traces (PRT), a form of specialized generated reasoning chains that serve as a reasoning bridge to improve an LLM's policy compliance assessment capabilities. Our empirical evaluations demonstrate that the use of PRTs for both inference-time and training-time scenarios significantly enhances the performance of open-weight and commercial models, setting a new state-of-the-art for HIPAA and GDPR policies. Beyond accuracy gains, we also highlight how PRTs can improve an LLM's ability to accurately cite policy clauses, as well as influence compliance decisions through their high utilization from the raw chains of thought.

1 INTRODUCTION

Policy compliance assessment requires applying policy provisions to case facts, but documenting expert-level reasoning is costly. The paper introduces generated Policy Reasoning Traces (PRTs) and reports gains across policy-compliance evaluations, including HIPAA and GDPR.

1 INTRODUCTION

  • Documenting detailed expert reasoning for each policy stipulation is costly and time-consuming, despite the need for expert-level knowledge to identify applicable provisions.Source: §1 · p. 2
  • PRTs are pseudo-expert reasoning demonstrations generated by querying frontier reasoning LLMs with cases and gold-standard verdicts, for few-shot use or supervised finetuning.They connect policy-specific constraints and nuances to compliance judgments.Source: §1 · p. 2
  • PRTs yield 50-100% significant performance increases for open-weight LLMs on HIPAA and new state-of-the-art GDPR baselines through finetuning.The paper also reports cross-policy transfer and improved citation of policy clauses when reasoning toward verdicts.Source: §1 · p. 2
  • The evaluation covers HIPAA, GDPR, and OpenAI’s ModelSpec, comparing few-shot in-context learning and self-feedback across open-weight and commercial reasoning LLMs.Source: §1 · p. 2

2 POLICY REASONING TRACES (PRT)

Policy Reasoning Traces (PRTs) are reference examples of policy-based reasoning intended to help resolve nuanced cases. The paper formalizes compliance assessment and describes how PRTs are generated and used at inference and training time.

2.1 MOTIVATION

  • PRTs provide reference examples of policy-based reasoning to help resolve nuanced cases in policy-dependent tasks.The paper motivates this approach by analogy to courts using reasoning from precedents and identifying applicable policy provisions before issuing verdicts.Source: §2.1 · p. 2

2.2 TASK FORMALIZATION

  • Policy compliance assessment predicts one COMPLIANT or NON-COMPLIANT verdict for an input case under a policy, using a model that can reason before answering.A policy comprises written constraints or rules, optionally associated with gold-standard examples and verdicts.Source: §2.2 · pp. 2–3

2.3 PRT GENERATION

  • For each gold-standard case–verdict pair, an expert reasoning model generates a corresponding PRT, grounding the trace in information tied to the verdict.The resulting examples augment the training data with case, verdict, and PRT triples.Source: §2.3 · p. 3

2.4 INFERENCE AND FINETUNING WITH PRTS

  • PRT-augmented examples support in-context learning and supervised finetuning, with finetuning using only relevant policy clauses to limit context length.Both paradigms condition assessment of unseen cases on the case, policy, and corresponding PRTs; the paper also mentions feedback learning and imitation learning as possible techniques.Source: §2.4 · pp. 3–4

3 EXPERIMENT SETUP

The experiments assess compliance across healthcare, data privacy, and model-safety policies using datasets with policy text, case demonstrations, and expert labels. They compare open-weight and commercial reasoning models across standard prompting, self-feedback, and PRT-augmented approaches.

Policies and Test Datasets

  • The evaluation covers HIPAA, GDPR, and ModelSpec, using datasets with expert compliance verdicts and publicly available policy text.The HIPAA sets contain 309 training and 107 test cases; GDPR contains 764 training and 326 test cases; ModelSpec uses 64 training examples and 450 XSTest cases for testing.Source: §3 · p. 4
  • The policy datasets have binary COMPLIANT or NONCOMPLIANT labels, no train–test overlap, and policy texts reformatted and lightly summarized for the experiments.The cited passage says GPT-5-MINI was used for the reformatting and summarization.Source: §3 · p. 4

Expert Models for PRT Generation

  • PRTs are generated from gold-verdict training examples by generalist and specialist reasoning models, including DEEPSEEK-R1 and SAULLM-INSTRUCT.The specialist model was selected for domain-overlapping legal pretraining and instruction tuning; manual inspection found its traces more frequently cite policies, while generalist traces are more conversational and thinking-like.Source: §3 · pp. 4–5, Figure 2 · p. 5

Learner Models and Assessment Methods

  • The selected learner models are a representative set of open-weight and commercial reasoning LLMs, including DEEPSEEK-R1, GEMINI-2.5, and QWEN2.5 models.The reported model list also names GPT-5-MINI and GPT-OSS.Source: §3 · p. 5
  • The prompting comparisons include base and few-shot standard prompts, one-round self-feedback, and PRT demonstrations selected randomly or by similarity.For PRT selection by similarity, GPT-5-MINI chooses three similar case instances; models are required to handle at least 8192 tokens of context.Source: §3 · pp. 5–6

4 RESULTS

PRTs improve policy-compliance results across inference-time and training-time settings, including stronger benchmark performance and policy-clause recall. Results also show cross-policy generalization and frequent use of PRT examples in model reasoning, with gains varying for models already optimized for a policy.

4.1 INFERENCE-TIME AND TRAINING TIME POLICY COMPLIANCE ASSESSMENT

  • For HIPAA, adding PRTs as few-shot prompts significantly improves reasoning-model accuracy by 16–30 points for open-weight models and 5–16 raw points for commercial models versus no PRTs.One-sided paired t-tests (random and relevant PRTs vs. no PRTs) gave HIPAA corrected p-values of 0.0005 and 0.002 (Bonferroni), and 0.0005 and 0.0012 (Holm).Source: §4.1 · p. 6
  • Generalist PRTs outperform Specialist PRTs in most comparisons, so the authors use Generalist traces in subsequent closer-look experiments.Figure 4 also compares baseline, few-shot PRT, finetuned, and state-of-the-art methods; its bars are ordered HIPAA, GDPR, then ModelSpec.Source: §4.1 · p. 7, Figure 4 · p. 7
  • Finetuning on PRTs reaches 80–81% HIPAA accuracy and sets a GDPR state of the art at 81.0%; few-shot PRTs bring DEEPSEEK-R1-LLAMA-8B to 77.7% versus GOLDCOIN’s 79.9%.Figure 4 reports that PRTs beat previous HIPAA work by up to +1.4% and bring open-weight model performance within <10% of optimized commercial models.Source: §4.1 · p. 7, Figure 4 · p. 7

4.2 INTERACTIONS ON SAFETY OPTIMIZATIONS AND DOMAIN GENERALIZATION

  • For ModelSpec, OpenAI’s GPT-OSS-20B and GPT-5-MINI lose an average of 4.7 accuracy with PRTs, while Gemini-2.5-Flash gains an average of 13.0 accuracy.The authors posit that models optimized using the same policy may be doubly optimized; Gemini is described as not specifically using ModelSpec.Source: §4.2 · pp. 7–8
  • Cross-policy finetuning generalizes strongly: HIPAA-trained QWEN2.7-7B-INSTRUCT reaches 78.5 on GDPR and 86.6 on ModelSpec, while within- and cross-policy performance differences are nonsignificant.The paired test reports p = 0.886, and Cohen’s d is 0.04; the reported cross-policy mean accuracy is 0.695 versus 0.668 within-policy.Source: §4.2 · p. 8, §4.3 · p. 8

4.3 POLICY CLAUSE RELEVANCE AND REASONING PERSISTENCE OF PRTS

  • PRTs raise policy-clause recall by +2.0–9.0, while exact-match scores can deteriorate slightly because examples encourage citing more clauses.Table 1 compares finetuned QWEN2.5-7B-INSTRUCT models with GPT-5-MINI; its Δ values are changes from No-PRT to PRT.Source: §4.3 · p. 8, Table 1 · p. 8
  • Few-shot PRTs achieve utility rates above 80% across HIPAA, GDPR, and ModelSpec; reference frequency is higher for ModelSpec (6.0–7.2) than for HIPAA and GDPR (1.2–1.9).The paper analyzes DeepSeek-R1’s raw chain-of-thought for references to the PRT demonstrations.Source: §4.3 · pp. 8–9

5 RELATED WORK

Related work ranges from formal policy representations and justification to machine-learning systems and reasoning-trace methods. PRTs are distinguished as regulatory-grounded traces derived from prior cases and judgments, usable without added reward modeling or preference optimization.

AI for Constitutional and Policy Compliance

  • Early policy-AI work formalized rules and added justifications, while later approaches used deep learning, retrieval, task decomposition, and policy-framework anchors.The paper traces formalization to Sergot et al. (1986) and Kowalski (1992), and justifications to Bench-Capon et al. (1991) and Bench-Capon et al. (1993).Source: §5 · p. 9

Improving LLM Performance using Reasoning Traces

  • Table 2 analyzes DeepSeek-R1 hidden chains of thought for PRT references and reports more than 80% compliance-assessment setups across all policies.The related-work passages do not contextualize this table result as prior work.Source: Table 2 · p. 9
  • Reasoning-trace research includes chain-of-thought prompting, self-generated feedback, and preference optimization for safety and symbolic-reasoning benchmarks.The cited examples include BUDGET FORCING (Muennighoff et al., 2025), RISE (Qu et al., 2024), and DELIBERATIVE ALIGNMENT (Guan et al., 2024).Source: §5 · p. 9

6 CONCLUSION

The conclusion presents Policy Reasoning Traces (PRT) as a reasoning bridge that improves LLM policy-compliance assessment, while identifying expert annotation as a possible cost of future preference tuning.

6 CONCLUSION

  • PRTs use a reasoning LLM’s chain-of-thought as a bridge to policy-compliance assessment, with reported accuracy gains for open-weight and commercial models across HIPAA and GDPR.The conclusion names QWEN2.5-7B, DEEPSEEK-R1, GEMINI-2.5-FLASH, and GPT-5-MINI.Source: §6 · pp. 9–10
  • The conclusion reports significant accuracy gains from both few-shot PRT demonstrations and finetuning on PRT collections.These approaches are described for open-weight and commercial models across HIPAA and GDPR.Source: §6 · pp. 9–10
  • Future work could use preference tuning on higher-quality PRTs to help LLMs learn preferred reasoning angles for nuanced cases.The authors note that this may require extensive annotation by domain experts.Source: §6 · p. 10

ETHICS STATEMENT

The ethics statement describes the data used and emphasizes that policy-compliance applications should retain human oversight and expert validation.

ETHICS STATEMENT

  • The GDPR-related cases came from publicized real-world cases made available by Data Protection Authorities for public awareness and noncommercial research.The authors state that no personally identifiable data beyond what was already public was used.Source: ETHICS STATEMENT · p. 10
  • The datasets used in the work are described as publicly available.The statement specifically identifies GDPRHub data as containing cases publicized by Data Protection Authorities.Source: ETHICS STATEMENT · p. 10
  • The authors do not intend the work to justify replacing human components in compliance assessment in high-stakes domains or contexts.They emphasize human oversight and expert validation for AI applications in policy compliance.Source: ETHICS STATEMENT · p. 10

DISCLOSURE OF LLM USE

The disclosure lists tools used in preparing the work and identifies tables documenting inference-time and training-time compliance results.

DISCLOSURE OF LLM USE

  • Grammarly, Cursor, and ChatGPT were used for grammar and spelling, prototyping and scripts, and table, figure, and Matplotlib troubleshooting, respectively.The authors say Cursor code completions were examined and validated, and no LLM was used for brainstorming, content generation, idea conception, or related-literature writing.Source: DISCLOSURE OF LLM USE · p. 10

A FULL TABLES FOR INFERENCE AND TRAINING-TIME COMPLIANCE ASSESSMENT

  • Table 3 reports inference-time accuracy for prompting-based compliance assessment across HIPAA, GDPR, and ModelSpec.Its caption covers standard and Self-Refine ICL using Generalist and Specialist PRTs across open-weight and commercial models.Source: Table 3 · p. 17, A FULL TABLES FOR INFERENCE AND… · p. 17
  • Table 4 reports training-time accuracy after supervised finetuning on combined Generalist PRTs, evaluated on each policy’s corresponding test set.Source: Table 4 · p. 18

B ADDITIONAL INFORMATION ON EXPERIMENTS

The appendix supplies experimental settings, generated PRT examples, and reproducibility information, including libraries and hyperparameter configurations.

B ADDITIONAL INFORMATION ON EXPERIMENTS

  • Table 5 summarizes generated PRTs by mean word and sentence counts, including standard deviations, across source models and policies.Source: Table 5 · p. 18
  • Prompting experiments used a temperature of 0.7 to encourage reasoning before the final verdict, and no temperature or sampling ablations were run because compute was limited.Source: B ADDITIONAL INFORMATION ON EXPERIMENTS · p. 18
  • Finetuning used policy-specific PRT training subsets with associated policy clauses, while full policy text was supplied at inference.The authors say this avoids requiring models to memorize whole policy chunks; the models used support at least 8192 tokens of context.Source: B ADDITIONAL INFORMATION ON EXPERIMENTS · p. 19

C EXAMPLES OF PRTS FROM EXPERT MODELS

  • The examples show Generalist PRTs as more explanatory or interpretive, while Specialist PRTs are shorter and more rule-focused across the illustrated policies.The GDPR Generalist example also considers Articles 9, 25, and 32 beyond Article 5; the ModelSpec Specialist example mentions fairness that may not apply.Source: Figure 5 · p. 19, Figure 6 · p. 20, Figure 7 · p. 21

D LIBRARIES, HYPERPARAMETERS AND CONFIGURATIONS

E ADDITIONAL RESULTS

Additional experiments examine how the number of few-shot PRTs and model scale relate to performance across HIPAA, GDPR, and ModelSpec.

Effects of Fewshot PRT Quantity

  • Three PRTs gave the best reported HIPAA accuracy for finetuned QWEN2.5 (80.4) and GPT-5-MINI (75.7), and GDPR accuracy for GPT-5-MINI (80.1) and QWEN2.5 (74.0).For ModelSpec, GPT-5-MINI scored 92.8 with two PRTs versus 92.1 with three, a 0.7-point difference.Source: E ADDITIONAL RESULTS · p. 23

Effects of Model Scale

  • Increasing model scale tended to improve performance across all three policies; total gains were +13.7 for HIPAA, +14.4 for GDPR, and +12.2 for ModelSpec.DEEPSEEK-LLAMA had the largest HIPAA gain, rising from 66.3 to 75.7 when scaled from 8B to 70B; GPT-OSS gained most on ModelSpec.Source: E ADDITIONAL RESULTS · p. 23, Figure 9 · p. 23

Effects of Fewshot PRT Quantity

  • The quantity experiment compared one to five few-shot PRTs using QWEN2.5-7B, finetuned QWEN2.5, and GPT-5-MINI.The paper reports no substantial difference from its default of three PRTs.Source: Figure 8 · p. 23, E ADDITIONAL RESULTS · p. 23

Effects of Model Scale

  • The scale experiment compared GPT-OSS at 20B and 120B, DEEPSEEK-LLAMA at 8B and 70B, and QWEN2.5 at 7B and 70B.The authors recommend larger open-weight models when inference or compute budget allows and higher accuracy is prioritized.Source: Figure 9 · p. 23, E ADDITIONAL RESULTS · p. 23

F COST AND EFFICIENCY ANALYSIS OF PRTS

The analysis weighs inference cost against accuracy for PRT and non-PRT methods across HIPAA, GDPR, and ModelSpec. Open-weight models often offer cost-efficient options, while higher-cost models may be justified when accuracy is prioritized.

Motivation

  • The cost analysis frames longer policy and case contexts, with added PRT demonstrations, as requiring further inference budget.It examines the balance between accuracy and inference costs across the three policies.Source: F COST AND EFFICIENCY ANALYSIS OF PRTS · p. 24

Setup

Results

  • On HIPAA, QWEN2.5-7B and DEEPSEEK-LLAMA are the most cost-efficient options and are relatively comparable to GPT-5-MINI; PRTs move models toward higher accuracy.The HIPAA Pareto frontier compares log-scale inference cost with accuracy.Source: F COST AND EFFICIENCY ANALYSIS OF PRTS · p. 24, Figure 10 · p. 25
  • GDPR's longer policy text is associated with higher inference costs than HIPAA; DEEPSEEK-LLAMA without few-shot PRTs is at par with GPT-OSS-20B.The authors say GPT-5-MINI or DEEPSEEK-R1 may justify higher costs in regulatory applications prioritizing interpretability and high-bar accuracy.Source: F COST AND EFFICIENCY ANALYSIS OF PRTS · p. 24, Figure 11 · p. 25
  • PRT-enhanced QWEN2.5-7B and DEEPSEEK-LLAMA improve accuracy without substantial cost-efficiency loss on ModelSpec, while GPT-OSS-20B without PRTs is most efficient.The caption notes that GPT-5-MINI improves with PRTs but is not efficient because of higher inference costs.Source: F COST AND EFFICIENCY ANALYSIS OF PRTS · p. 24, Figure 12 · p. 26

G LIMITATIONS

The limitations section characterizes PRTs as imperfect weak supervision and notes a performance deterioration for some safety-optimized OpenAI models. The appendices document the inference and utility prompts used in the experiments and supporting analyses.

G LIMITATIONS

  • PRTs are scaffolds rather than gold-standard references, since expert generators may produce inconsistencies or hallucinations.The paper points to clause-relevance analysis and raw DEEPSEEK-R1 chain-of-thought analysis as validation of PRT use.Source: G LIMITATIONS · p. 27
  • With ModelSpec, adding PRTs slightly deteriorated performance for doubly policy-optimized OpenAI models such as GPT-5-MINI.The authors describe further study as dependent on comprehensive public safety policies and compliance datasets beyond ModelSpec.Source: G LIMITATIONS · p. 27

H PROMPTS

  • The prompt appendix distinguishes inference prompts for recorded performance experiments from utility prompts for supporting analyses.Utility prompts cover clause relevance, PRT utilization, and PRT generation.Source: H PROMPTS · p. 28

H.1 INFERENCE PROMPTS

H.2 UTILITY PROMPTS

  • Utility prompts cover policy-text formatting and summarization, PRT generation, relevant-case extraction, clause relevance, and PRT-utilization counting.The appendix identifies GPT-5-MINI for case extraction, clause relevance, and utilization analysis.Source: §H.2 · pp. 32–34, Figure 24 · p. 34

Related papers

Loading 2509.23291v1…