PaperlayerView exact revision on arXiv

Source-linked AI summary

GoldCoin: Grounding Large Language Models in Privacy Laws via Contextual Integrity Theory

Wei Fan, Haoran Li, Zheye Deng, Weiqi Wang, Yangqiu Song

arXiv:2406.11149v2cs.CLcs.CR

TL;DR

Narrowly predefined privacy patterns and scarce open-source cases complicate analysis across social contexts and alignment to specific laws. GOLDCOIN grounds LLMs in privacy laws through contextual-integrity-based synthetic cases, and tuned models outperform baselines on real-case HIPAA applicability and privacy-risk detection.

  • Problem

    Prior privacy research focuses on narrow predefined patterns, while scarce open-source court cases hinder LLM alignment with specific privacy laws.

    Source: §1 · pp. 1–2

  • Method

    GOLDCOIN uses contextual integrity to represent privacy-law clauses as information-flow norms and combines seed norms with GPT-4-generated case backgrounds to create synthetic cases.

    Source: §1 · p. 2

  • Results

    On real cases, GOLDCOIN-tuned models surpass baselines by 8% to 23% in HIPAA applicability and by 8% to 18% in privacy-risk detection.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The HIPAA experiments show improved model capability to assess legal relevance and pinpoint privacy risks using contextual-integrity-grounded synthetic cases.

    Source: §6 · p. 9

    The experiments cover HIPAA, omit interconnections among legal norms, and do not evaluate few-shot settings.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Privacy issues arise prominently during the inappropriate transmission of information between entities. Existing research primarily studies privacy by exploring various privacy attacks, defenses, and evaluations within narrowly predefined patterns, while neglecting that privacy is not an isolated, context-free concept limited to traditionally sensitive data (e.g., social security numbers), but intertwined with intricate social contexts that complicate the identification and analysis of potential privacy violations. The advent of Large Language Models (LLMs) offers unprecedented opportunities for incorporating the nuanced scenarios outlined in privacy laws to tackle these complex privacy issues. However, the scarcity of open-source relevant case studies restricts the efficiency of LLMs in aligning with specific legal statutes. To address this challenge, we introduce a novel framework, GoldCoin, designed to efficiently ground LLMs in privacy laws for judicial assessing privacy violations. Our framework leverages the theory of contextual integrity as a bridge, creating numerous synthetic scenarios grounded in relevant privacy statutes (e.g., HIPAA), to assist LLMs in comprehending the complex contexts for identifying privacy risks in the real world. Extensive experimental results demonstrate that GoldCoin markedly enhances LLMs' capabilities in recognizing privacy risks across real court cases, surpassing the baselines on different judicial tasks.

1 Introduction

GOLDCOIN uses contextual integrity to connect privacy-law norms with synthetic case backgrounds, addressing the scarcity of relevant open-source cases. On real cases, tuned models outperform baselines in HIPAA applicability and privacy-risk detection.

1 Introduction

  • Prior work focuses on narrow predefined patterns or manually annotated rules, limiting privacy-risk detection across diverse social contexts.Privacy violations include improper information transmission that contradicts societal expectations and legal statutes.Source: §1 · p. 1
  • Scarce open-source court cases make it difficult to align LLMs with specific privacy laws, while statutory-law training yields unstable, limited improvements.Court cases provide practice-oriented factual backgrounds, judicial analyses, and judge opinions.Source: §1 · p. 2
  • GOLDCOIN uses contextual integrity to represent legal clauses as norms governing information flows among entities in specific roles and contexts.Norms specify transmitted information and principles such as purpose or consent, and can permit or forbid transmission.Source: §1 · p. 2
  • GPT-4 generates synthetic case backgrounds from formalized contextual integrity and seed norms; filters check essential features and norm consistency, while diversity ranking broadens backgrounds.The framework combines background contexts and seed norms into synthetic court cases tailored to privacy laws.Source: §1 · p. 2
  • GOLDCOIN-tuned models surpass baselines by 8% to 23% in HIPAA applicability and by 8% to 18% in privacy-risk detection on real cases.The evaluation uses a ground-truth benchmark sourced from real U.S. court cases in the Caselaw Access Project.Source: §1 · p. 2

2 Related Work

Related work frames contextual integrity as a way to assess information flows through social roles and norms, reviews legal LLMs and their privacy-data constraints, and situates GOLDCOIN's case generation among instruction-generation research.

2.1 Privacy and Contextual Integrity

  • Contextual integrity models information flows through sender, recipient, and subject roles, information type, and transmission principles; legal clauses become norms that permit or forbid flows.Roles vary by context, including healthcare and commerce.Source: §2.1 · pp. 2–3

2.2 LLMs in Law

  • Legal LLMs support services such as judgment prediction and question answering, but often underperform in privacy, where training and evaluation datasets are limited and generally closed-source.The cited examples include LawGPT, Lawyer LLaMA, SaulLm, and ChatLaw.Source: §2.2 · p. 3

2.3 LLMs for Instruction Generation

  • Instruction-generation research targets zero-shot and few-shot learning, reasoning, and instruction following; GOLDCOIN uses GPT-4 to generate datasets and align LLMs with privacy laws for judicial use.The approach is motivated by addressing case scarcity through contextual-integrity-grounded generation.Source: §2.3 · p. 3
  • A statutory norm is formed by concatenating content along the path from a leaf clause to the HIPAA root node.The example path runs from clause 164.502(a)(1)(ii) to HIPAA.Source: Figure 2 · p. 3

3 Method

GOLDCOIN turns privacy-law norms into contextual-integrity-based synthetic cases, postprocesses them, and instruction-tunes LLMs for applicability and compliance judgments.

3 Method

  • The framework evaluates whether a privacy law applies to a case and whether its described transmission complies with that law.It generates synthetic cases, postprocesses them, instruction-tunes LLMs, and evaluates both judicial tasks.Source: §3 · p. 3

3.1 Legal Statute Preprocessing

  • HIPAA text is structured as a graph, whose aggregated leaf-to-root paths become norms classified as permit, forbid, or other.Permit and forbid norms describe allowed and prohibited transmissions; other norms contain definitions, exceptions, and requirements.Source: §3.1 · pp. 3–4

3.2 Law-Grounded Case Generation

  • Each selected seed norm grounds a synthetic case with a background, contextual features, related norm, and applicability and compliance conclusions.Prompts specify contextual-integrity features, including entities, roles, information type, and transmission principles.Source: §3.2 · p. 4

3.3 Case Postprocessing

  • Postprocessing removes cases missing key contextual features, mismatched to their seed norm, or inconsistent in applicability or compliance, then ranks backgrounds for diversity.The diversity ranking uses ROUGE-L similarity and selects the highest-scoring case for each norm.Source: §3.3 · pp. 4–5

3.4 Real Court Case Collection

  • Real HIPAA cases are retrieved from CAP and processed with GPT-4 and manual annotation, alongside privacy-related cases that do not apply to HIPAA.The non-applicable cases provide negative examples for the applicability task.Source: §3.4 · p. 5

3.5 Instruction and Response Compilation

  • Applicability responses first extract context-specific features and then judge HIPAA applicability; compliance responses extract features, retrieve a norm, and judge whether it permits transmission.Both tasks use manually constructed multi-step instructions based on case backgrounds.Source: §3.5 · p. 5

4 GOLDCOIN-HIPAA Dataset Overview

GOLDCOIN-HIPAA combines synthetic cases grounded in HIPAA norms with real court cases from CAP. The synthetic cases were reviewed by experts, and the dataset also includes HIPAA-irrelevant cases for applicability evaluation.

4.1 GOLDCOIN Synthetic Cases

  • 309 synthetic cases were selected across 269 permit and 40 forbid norms, using the highest-ROUGE-L candidate for each norm.Figure 5 presents ROUGE-L score distributions for original and filtered cases; the text says the comparison indicates diversity-ranking efficacy.Source: §4.1 · p. 5, Figure 5 · p. 6
  • Experts confirmed that all synthetic cases apply to HIPAA, with most related to the seed norm.Two experts assessed case quality using the criteria in Table 1.Source: §4.1 · pp. 5–6, Table 1 · p. 6

4.2 CAP Real Court Cases

  • The CAP collection includes 107 HIPAA-relevant real cases for compliance evaluation, plus an equivalent number of HIPAA-irrelevant cases for applicability testing.The dataset also uses 309 non-applicable CAP cases as negative training examples.Source: §4.2 · p. 6

5 Experiment

The experiments evaluate GOLDCOIN on HIPAA applicability and compliance judgments, then test its synthetic-case filters and contextual-integrity instructions. Across these analyses, results highlight the value of law-grounded cases and their contextual features.

5.1 Experimental Settings

  • The evaluation uses HIPAA synthetic cases, real court cases, four instruction-tuned open-source LLMs, and zero-shot and few-shot GPT models.Accuracy and Macro F1 measure applicability and compliance judgments; comparisons include zero-shot, law recitation, and direct-prompt baselines.Source: §5.1 · p. 6

5.2 Overall Performance

  • GOLDCOIN exceeds baseline methods on HIPAA applicability and improves compliance Macro F1 over zero-shot across four open-source models.For Mistral-7B, compliance Macro F1 increases 17.96% compared with zero-shot.Source: §5.2 · pp. 6–7, Table 2 · p. 7

5.3 Ablation Study

  • Removing contextual feature filters lowers Llama2-13B performance by 3.26% on applicability and 2.36% on compliance.The ablation retains cases lacking key contextual-integrity features such as sender or recipient.Source: Table 3 · p. 7, §5.3 · p. 8
  • Removing the norm-consistency filter lowers Mistral-7B compliance by 3.62%, while removing conclusion checks lowers Llama2-13B applicability by 4.99%.The authors link norm checks to mitigating generation hallucinations and incorrect conclusions to increased perplexity during training.Source: §5.3 · p. 8
  • Removing diversity ranking lowers Llama2-13B applicability Macro F1 by 3.86%; removing all filters and ranking lowers Mistral-7B scores by 5.89% and 5.05% across the two tasks.Random sampling without ranking can produce similar cases in entity roles or information categories.Source: §5.3 · p. 8

5.4 Discussion of GOLDCOIN Instruction

  • Omitting contextual-integrity feature extraction lowers norm retrieval accuracy across tuned open-source LLMs and reduces Llama2-13B conclusion performance by 5.14%.The comparison reports norm retrieval accuracy and conclusion Macro F1 for judgments of permit or forbid.Source: Table 4 · p. 8, §5.4 · p. 8
  • Multi-step instructions alone yield mixed applicability changes: MPT-7B and Mistral-7B average a 1.70% Macro F1 improvement, while Llama2 models decline 2.17%.The authors conclude detailed instructions alone are insufficient for effective contextual-integrity judgments.Source: §5.4 · p. 8

6 Conclusion

The conclusion presents GOLDCOIN as a contextual-integrity framework for applying privacy laws to violation detection, evaluated with HIPAA synthetic cases. It reports improved legal-relevance and privacy-risk assessment and identifies other laws as possible future extensions.

6 Conclusion

  • GOLDCOIN uses contextual integrity to apply privacy laws to privacy-violation detection.The paper builds synthetic cases for aligning LLMs with the HIPAA Privacy Rule.Source: §6 · p. 9
  • Experiments report improved model capability to assess legal relevance and identify privacy risks.Source: §6 · p. 9
  • The authors identify GDPR, COPPA, and general legal domains as possible extensions of the generation and alignment method.Source: §6 · p. 9

Limitations

GoldCoin's experiments are limited to HIPAA, omit cross-references among norms, and do not evaluate few-shot prompting or norm retrieval.

Limitations

  • The framework models permit and forbid norms but not their interconnections, although adjudication may hinge on multiple norms simultaneously.The authors suggest constructing cases from multiple norms as future work.Source: Limitations · p. 9
  • Evaluation covers HIPAA, selected for its prominence and relatively abundant open-source cases, leaving extension to laws such as COPPA and GDPR for future work.Source: Limitations · p. 9
  • The study omits few-shot prompting because multiple examples can exceed LLM input-length limits, and does not use retrieval methods for relevant norms.Dynamic indexing and retrieval of norms is identified as a promising direction.Source: Limitations · p. 9

Ethics Statement

The appendix describes the ethical handling of data and annotations, then outlines the contextual-integrity concepts and HIPAA norm categories used in GoldCoin.

Ethics Statement

  • The study follows CAP access rules, removes potentially privacy-leaking synthetic cases, and has legal experts review cases and remove explicit content.Annotations were compensated at 15 USD per hour, above the local minimum wage.Source: Ethics Statement · p. 9

A Contextual Integrity: Theory and Framework

A.2 Roles and Contexts

  • Entities' roles vary by context, and identifying those roles helps determine which privacy law applies to a potential privacy risk.The appendix contrasts a doctor's healthcare role, a supermarket consumer role, and a family role.Source: §A.2 · p. 12

A.3 Transmission Principles

  • GoldCoin selects Purpose, In Reply To, Consented By, and Belief as transmission principles that constrain information flows.Source: §A.3 · p. 12

A.4 Informational Norms

  • GoldCoin represents information-flow norms as (P, R)∧T ∧Ω and relies on standardized legal frameworks to extract them.The appendix describes permit norms as allowing transmissions that satisfy conditions and forbid norms as prohibiting transmissions matching specified features.Source: §A.4 · p. 13, §A.5 · p. 13

A.5 Example

  • Because the doctor is a HIPAA covered entity and blood-test results are health information, this information transmission aligns with HIPAA norm 164.502(a)(1)(ii) and is permitted.Source: §A.5 · p. 13

B.1 Brief Introduction

  • HIPAA's Privacy Rule protects personal health information, limits disclosure without consent, and gives patients rights to access and amend records.Source: §B.1 · p. 13

B.2 Requirement, Exception, and General Definition

  • HIPAA norms can include requirements, exceptions, and general definitions, but this work focuses only on permit and forbid types.A requirement makes permission conditional; an exception waives a rule in specified circumstances; a general definition explains a term.Source: §B.2 · p. 13

B.3 Details of Norm Classification

The appendix describes GPT-4-based norm classification and reports the distribution of permit and forbid norms in HIPAA, alongside expert-identified misclassifications.

B.3 Details of Norm Classification

  • 269 of 691 HIPAA norms allow certain information transmissions, while 40 of 691 prohibit specific transmissions.The authors report these counts from their analysis of 45 CFR Part 164; Table 6 presents the norm-category statistics.Source: §B.3 · p. 14, Table 6 · p. 14
  • GPT-4 classifies each compiled norm using a classification instruction, but the authors report that norm-type classification is not always accurate.Expert annotation identified one permit norm and two forbid norms that were misclassified, with three compliance results conflicting with specified norm types.Source: §B.3 · p. 14

C Details of GOLDCOIN-HIPAA Dataset

The appendix details how GoldCoin generates and describes GOLDCOIN-HIPAA cases, illustrates generated examples, and summarizes the Caselaw Access Project source.

C.1 Prompt of Background Generation

  • GPT-4 generates a case background from a provided HIPAA norm and norm type, with prompts incorporating contextual-integrity features.Source: §C.1 · p. 14

C.2 Statistics

  • The dataset statistics distinguish GOLDCOIN-generated HIPAA cases labeled “Synthetic” from CAP cases processed through the pipeline and labeled “Rea.”Table 5 contains the dataset statistics, but the supplied passage does not give its values.Source: §C.2 · p. 14

C.3 Case Study

  • The case study presents one permitted and one forbidden case, each comprising a seed norm, background story, contextual-integrity features, and a conclusion.Source: §C.3 · p. 14

D Details of the Caselaw Access Project

D.1 Dataset Collection

The appendix describes collecting and processing HIPAA-related cases, preparing prompts for privacy-law tasks, and tuning models on the resulting examples. It also documents case examples and supplementary experiments.

D.1 Dataset Collection

  • HIPAA-related cases were retrieved using the “HIPAA Privacy Rule” keyword, filtered by length, and supplemented with 2,000 general privacy-violation cases for applicability.Cases longer than 30,000 words or shorter than 100 words were excluded.Source: §D.1 · p. 15

D.2 Prompt / D.3 Human Annotation

  • GPT-4 prompts extract HIPAA-relevant case facts, contextual features, norms, and conclusions; two human experts then remove irrelevant cases, separate background from analysis, and check extracted conclusions.The human annotators had studied privacy protection and privacy laws for over a year.Source: §D.2 · p. 15, §D.3 · p. 15

D.4 Case Study

  • Three processed court cases illustrate transmissions permitted by HIPAA, forbidden by HIPAA, or outside HIPAA’s applicability.The examples correspond to Figures 11, 12, and 10, respectively.Source: §D.4 · p. 15

E.1 Instruction Template

  • The instruction template concatenates instructions and optional example inputs using separate templates for inputs that are present or empty.The placeholders “instruction” and “input” are replaced with the actual content.Source: Table 7 · p. 15, §E.1 · p. 15

E Implement Details

  • The implementation instruction-tunes four open-source models, including MPT-7B, Mistral-7B, Llama2-7B, and Llama2-13B, and uses dated Azure API versions of ChatGPT and GPT-4.The open-source models use LoRA; API generation and evaluation cost approximately $100 and $20, respectively.Source: E Implement Details · p. 15

E.2 Vanilla and Multi-step Prompts for the Applicability Task

  • Applicability prompts use “Applicable” or “Not Applicable” outputs, with GOLDCOIN also using a multi-step prompt.The appendix contrasts the vanilla and multi-step prompt formats.Source: §E.2 · pp. 15–16, Table 8 · p. 16

E.3 Vanilla and Multi-step Prompts for the Compliance Task

  • Compliance prompts use “Permit” or “Forbid” outputs, while the multi-step version analyzes contextual features and retrieves relevant norms.The vanilla and multi-step prompt formats are shown in Table 9.Source: §E.3 · p. 16, Table 9 · p. 16

E.4 Prompt of Law Recitation

  • The law-recitation baseline tunes models with an instruction containing all norms from HIPAA Part 164.The prompt is provided in Table 10.Source: §E.4 · p. 16, Table 10 · p. 16

F.1 Full Results of Ablation Study

Supplementary experiments compare category-level performance, ablations, multi-step prompting, and sensitivity to norm-category imbalance. They report both strengths and limitations across models and settings.

F.1 Full Results of Ablation Study

  • Conclusion inaccuracies cause the largest ablation degradation; MPT-7B’s applicability accuracy falls 6.25%, while norm inconsistencies expose GPT-4 hallucinations and random behavior.Table 11 reports Macro F1-scores and score changes across four models.Source: §F.1 · p. 16, Table 11 · p. 17

F.3 Baselines under Multi-step Instruction

  • Instruction-tuning on GOLDCOIN-HIPAA degrades performance; Llama-2 13B exhibits a 3.33% decrease in the “Zero-shot” setting.The decline is attributed to the model’s inability to comprehend and apply contextual integrity without direct reference to legal knowledge.Source: §F.3 · p. 17

F.4 Robustness and Sensitivity Analysis

  • Oversampling 269 forbid cases to match permit cases has negligible impact on final performance for Mistral-7B and Llama2-13B.The comparison tests sensitivity to the imbalance in HIPAA norm and case categories.Source: §F.4 · p. 17, Table 12 · p. 17

F.2 Overall Performance across Categories

  • GOLDCOIN results vary across categories and models: GPT-4 performs best with Direct Prompt on applicability, while Mistral-7B and Llama2-13B reach 100% positive-category precision and negative-category recall with GOLDCOIN.For compliance under Direct Prompt, Mistral-7B has 97.44% permit precision but 27.94% forbid precision.Source: §F.2 · p. 17, Table 13 · p. 18, Table 14 · p. 18

F.4 Robustness and Sensitivity Analysis

Loading 2406.11149v2…