PaperlayerView exact revision on arXiv

Source-linked AI summary

Privacy Washing: Detecting Internal Contradictions in Privacy Policies

Thomas Brackin

arXiv:2609.02055v1cs.CYcs.CL

TL;DR

Privacy policies can contain commitments undermined by practices elsewhere, but this within-document phenomenon has lacked a computational operationalization. The paper defines privacy washing and detects it with a four-stage pipeline across two corpora collected 11 years apart, finding recurring category patterns while showing that some prevalence and category claims depend on panel and filter configuration.

  • Problem

    Prior privacy-policy contradiction systems target logical negation, leaving commitment-versus-practice pragmatic contradictions within a single policy insufficiently operationalized.

  • Method

    A four-stage pipeline extracts statements, filters compatible commitment-practice pairs, screens them with NLI, verifies them using a multi-model judge panel, and analyzes themes.

  • Results

    The same category patterns recur across OPPT and OPP-115, while stability results reproduce OPPT prevalence but show third-party-sharing majority claims are panel-sensitive.

  • Takeaways & Limitations

    The robust paper-level conclusion is that recurring within-policy contradiction patterns can be operationally detected across corpora and eras, without establishing authorial intent.

  • Takeaways & Limitations

    No pipeline stage has been validated against human labels, so reported quantities are properties of the pipeline rather than validated prevalence estimates.

Abstract

from arXiv · show

Privacy policies may contain internal contradictions in which commitments are undermined by practices documented elsewhere in the same policy. We operationalize this phenomenon, privacy washing, through a four-stage pipeline: statement extraction, compatibility filtering and natural language inference screening, multi-model judge verification, and thematic analysis, with contradictions confirmed by majority vote of a three-model LLM panel. Applied to two corpora of website privacy policies, 123 collected in 2026 (OPPT) and 115 collected in 2015 (OPP-115), the pipeline finds the same category patterns recurring across the 11-year gap, with third-party sharing contradictions the majority of confirmed cases in each primary run, consistent with structural factors in policy composition rather than necessarily intentional deception. At least one panel-confirmed contradiction appears in 12.2% of OPPT companies (15/123; 9.8% excluding legacy pairs) and 36.5% of OPP-115 companies (42/115). A stability re-run seven months later, with a fully separated configuration (new extraction models, judges from three Chinese providers absent from both corpora, matched filters, no judge-submission similarity threshold), reproduces the OPPT prevalence under the original protocol (13.0% vs. 12.2%), finds sub-threshold pairs confirm at rates of the same order as those above (raising prevalence to 20.3% and 40.9%), and shows the third-party majority is panel-sensitive while the recurrence of the same category pairs is not. Two caveats govern all figures: panel verdicts are not validated against human expert judgment, so precision is unknown and prevalence figures are lower bounds; and the two primary runs used different filter configurations, so their prevalence difference is not interpretable as a corpus or era effect (the matched re-run reduces the gap to roughly twofold but does not eliminate it).

1 Introduction

Privacy washing describes privacy-policy commitments undermined by practices documented elsewhere in the same policy. The paper operationalizes and detects this previously unmeasured pragmatic contradiction through an automated pipeline, while distinguishing it from logical contradiction and code-versus-policy inconsistency.

  • Motivation: Privacy washing is a within-policy pattern in which reassuring commitments are undermined by documented practices elsewhere in the same document.The paper adapts the term to provide its first computational operationalization of this commitment-practice contradiction.
  • Research gap: The paper targets pragmatic tension that symbolic systems for logical tuple negation are not designed to capture.A broad reassurance and a specific practice may be individually defensible yet jointly create a misleading impression.
  • Approach: The four-stage pipeline extracts typed statements, filters compatible pairs, screens with NLI, verifies candidates through multi-model judging, and analyzes themes.The paper presents this approach as a computational operationalization of privacy washing across two policy corpora.
  • Data and scope: The study analyzes 238 companies across OPPT and OPP-115, two corpora collected 11 years apart.The larger literature includes substantial policy resources, but this method currently relies on per-segment annotations available in the purpose-annotated corpora.
  • Contribution: Existing privacy-policy contradiction systems focus on logical inconsistencies, while this paper formalizes commitment-versus-practice contradiction as a distinct computational phenomenon.The paper positions this gap within privacy-policy NLP, where mismatches between stated policies and actual practices remain comparatively understudied.

3 Defining Privacy Washing

The paper defines privacy washing as a directional, within-document pragmatic contradiction between commitments and practices, without attributing authorial motive. Its analysis identifies recurring category patterns, especially third-party sharing and collection-scope tensions, while distinguishing the phenomenon from related inconsistency types.

  • Definition: Privacy washing is the presence of commitments in one privacy policy that are contradicted by documented practices elsewhere in that same policy.The definition covers promises, reassurances, or stated limits on data practices and does not attribute motive to authors.
  • Conceptual basis: Privacy washing is pragmatic rather than purely logical: both statements may be individually true while jointly undermining the impression created by the commitment.This distinguishes the target from direct negation and from contradictions between policy and external system behavior.
  • Operationalization: The pipeline evaluates only directional COMMITMENT-to-PRACTICE pairs within one policy, excluding unrelated, differently scoped, hedged, or separately contextualized practices.These exclusions operationalize pragmatic contradiction through the judge prompt and its six criteria.
  • Empirical patterns: Third-party sharing patterns are most prevalent, including THIRD_PARTY-to-THIRD_PARTY and SALE_SHARING-to-THIRD_PARTY contradictions.The SALE_SHARING pattern is linked to blanket “we do not sell” statements conflicting with disclosures involving affiliates, service providers, and advertising partners.
  • Empirical patterns: FIRST_PARTY commitments are also contradicted by extensive collection, while cross-category cases connect collection limitations with sharing practices.A smaller set concerns tracking and targeted advertising; no SENSITIVE_DATA contradictions were panel-confirmed in either corpus.
  • Structural interpretation: Recurring structural patterns include multi-division policies and regulatory accretion, while acquisition and transfer clauses remain hypothesized rather than confirmed.The paper also distinguishes privacy washing from commitment avoidance and from interface dark patterns.

4 Methodology

The methodology decomposes privacy policies into typed atomic statements, progressively filters candidate commitment-practice pairs, and verifies contradictions through multi-model judging. The study documents important limitations involving model dependence, category typing, corpus-specific processing, filtering, and validation.

  • Four stages extract statements, filter and screen compatible pairs with NLI, verify candidates with multiple judges, and analyze themes.
  • Atomic statement decomposition isolates approximately 10–30-word commitments and practices from long policy segments, producing more focused NLI inputs.
  • Only COMMITMENT statements are paired with PRACTICE statements, focusing detection on the directional pattern defining privacy washing.
  • Progressive filtering narrowed tens of thousands of raw pairs to 956 judged candidates and 111 panel-confirmed contradictions.
  • Primary extraction and judging used the same three models, allowing shared model biases to propagate across stages.
  • OPP-115 annotation guidance failed because of a segment-identifier mismatch, leaving all 4,975 statements typed by extraction models alone.
  • SALE_SHARING introduces a temporal category mismatch because CCPA sale disclosures were absent from the original OPP-115 scheme.
  • Exact qualifier matching prioritizes precision over recall, so paraphrased equivalents may be missed.

5 Datasets

The datasets combine a 2026 OPPT corpus of 123 companies with the established 2015 OPP-115 corpus, while annotation guidance and dataset composition constrain comparisons. The OPPT sample spans diverse industries and policy styles.

  • The OPPT corpus contains privacy policies from 123 companies collected January 11–13, 2026, in a three-day snapshot window.
  • Table 1 presents OPPT corpus composition after statement extraction and shows the top five companies by statement count.
  • Annotation guidance was minimal for OPPT and inoperative for OPP-115 because a segment-identifier mismatch caused the OPP-115 join to fail.
  • The source OPP-115 Does/Does Not attribute had substantial disagreement, with values set to Unspecified in approximately a third of mergers.
  • OPPT companies span social media, telecommunications, financial services, healthcare, travel, retail, data brokers, and AI companies.

6 Results

The pipeline identifies panel-confirmed contradictions across both privacy-policy corpora, with recurring category patterns and substantial sensitivity to filtering, base rates, and judge-panel composition. Results are lower bounds because unjudged candidates and classification error remain unresolved.

  • Prevalence: 15 of 123 OPPT companies contain at least one panel-confirmed contradiction (12.2%), falling to 9.8% when legacy pairs are excluded.The judged-company rate is 15 of 60 (25%).
  • Limitations: The full-corpus prevalence is a lower bound because 18 companies had below-threshold NLI-flagged pairs, while other contradictions may have been missed by filtering or vocabulary differences.Precision against human judgment is unknown, and the reported intervals capture sampling variation but not classification error.
  • Contradiction patterns: Three category pairs tie as most frequent at 6 of 32 (18.8%) each, while third-party sharing appears on at least one side in 18 of 32 OPPT cases (56.3%).The three tied pairs are THIRD_PARTY →THIRD_PARTY, FIRST_PARTY →FIRST_PARTY, and SALE_SHARING →THIRD_PARTY.
  • Contradiction patterns: Third-party or sale-sharing pairs were confirmed at 13.2% versus 8.4% for other OPPT pairs, but the difference was not statistically significant (p = 0.26).For OPP-115, the corresponding rates were 15.7% versus 13.6% (p = 0.50).
  • Filtering and selection: Similarity-bin confirmation rates were flat to noisy for OPPT, with 10.4%, 13.9%, and 7.3% across the 0.50–0.55, 0.55–0.60, and 0.60–0.65 bins.The concentration of confirmations at low similarity reflects the judged sample’s base rates rather than evidence that low-similarity pairs are intrinsically more confirmable.

7 Cross-Temporal Comparison: OPP-115 Analysis

The OPP-115 comparison shows recurring contradiction categories across the 11-year gap, but its higher prevalence cannot be interpreted as an era effect because the primary runs used different filters. Third-party sharing contradictions dominate, while domain-specific and regulatory patterns shape some cases.

  • Scale results: 11.9% and 10.9% were the comparable confirmation rates despite differing filter configurations.Disentangling substantive factors would require controlled experiments; a matched-configuration rerun could specifically address filtering asymmetry.
  • Scale results: Kaleida Health accounted for 11% of all OPP-115 contradictions, yet excluding it left 50.6% prevalence among the remaining judged companies.This indicates that the OPP-115 pattern was not driven solely by that outlier, although its HIPAA-related contradictions remain a corpus-level confound.
  • Pattern consistency: The same dominant contradiction categories appear in OPP-115 and OPPT, consistent with structural rather than era-specific causes.Third-party-to-third-party contradictions comprise 40.5% of OPP-115 confirmations and are joint-largest in OPPT at 18.8%.
  • Pattern consistency: 40.5% of OPP-115 confirmed contradictions were third-party-to-third-party, while first-party-to-first-party contradictions occurred at 24.1%.The corresponding OPPT shares were 18.8% for both listed patterns, making recurrence more stable than exact category proportions.
  • Scale results: The similarity distributions look similar across corpora, but confirmation rates by similarity bin diverge, so similarity is not evidence about privacy-washing nature.OPP-115 confirmation rates rise with similarity, whereas OPPT rates are flat with sparse high-similarity cells.
  • Pattern consistency: 18.8% of OPPT contradictions versus 11.4% of OPP-115 contradictions were SALE_SHARING-to-THIRD_PARTY, a difference that was not statistically significant.The direction is consistent with post-2020 CCPA-mandated “do not sell” disclosures, but the mechanism argument does not rest on this comparison.
  • Scale results: 51% of judged OPP-115 companies versus 25% of judged OPPT companies had a confirmed contradiction, but the runs were configured differently.The comparison is not interpretable as a corpus or era difference because OPPT used compatibility filters that OPP-115 lacked.
  • Prevalence across corpora: 36.5% of OPP-115 companies had at least one confirmed contradiction, compared with 12.2% of OPPT companies in the full-corpus primary results.These figures are lower bounds, and the higher OPP-115 prevalence may reflect corpus composition, substantive practices, or filtering asymmetry.

8 Stability Experiment: Separated Panels and Matched Configuration

The stability experiment separates extraction and judging panels, matches filters across corpora, and tests threshold sensitivity. It preserves OPPT prevalence and recurring category pairs while showing that category ordering and some company-level findings are panel-sensitive.

  • Interpretation: The stability comparison tests the pipeline as a whole because extraction and judging changed together, preventing attribution of differences to either stage alone.The fully separated configuration removes extractor-judge overlap but confounds extractor and judge effects.
  • Stability findings: 13.0% of OPPT companies (16/123) were confirmed under the stability run versus 12.2% (15/123) in the primary run.The matched-protocol comparison used different extraction and judge panels and an empty cache.
  • Stability findings: 34 of 46 pre-reclassification OPP-115 companies recurred in stability-run confirmations, including 27 at similarity ≥0.5 and 7 below it.Of the 12 losses, 9 were judged and rejected, while 3 lost pairs during matched filtering.
  • Threshold sensitivity: 20.3% OPPT and 40.9% OPP-115 prevalence results after audit show that sub-threshold pairs confirm at rates of the same order as above-threshold pairs.Before audit, removing the threshold yielded 20.3% for OPPT and 42.6% for OPP-115; the audit corrected the latter to 40.9%.
  • Matched configuration: 27.8% matched-threshold OPP-115 prevalence remained roughly twice 13.0% OPPT prevalence, reducing but not eliminating the primary gap.The filtering asymmetry explains part, but not all, of the difference; corpus selection and pre-regulation drafting remain unseparated explanations.
  • Category stability: The same category pairs recur, but third-party sharing falls to 43% in each corpus under the stability panel instead of primary-run majorities of 56.3% and 69.6%.FIRST_PARTY → FIRST_PARTY becomes modal in both corpora, so category recurrence is more stable than category ordering.

9 Discussion

The discussion interprets privacy washing as a pragmatic, structurally arising tension between reassuring commitments and documented practices. It distinguishes this target from logical contradiction systems and outlines cautious implications for review and remediation.

  • Structural interpretation: Privacy washing may arise from regulatory accretion, multi-division policies, acquisition boilerplate, template evolution, and commitment asymmetry rather than intentional deception.These are hypothesized mechanisms grounded in qualitative examination, not quantified findings.
  • Structural interpretation: CCPA-related “we do not sell” claims can conflict with extensive sharing practices that may fall outside the legal definition of sale.The original CCPA’s ambiguity around valuable consideration is presented as a structural condition for the SALE_SHARING pattern.
  • Structural interpretation: Multi-division policies can place product-specific commitments beside enterprise-wide practices, creating apparent tensions across scopes.The Microsoft example contrasts a Phone Link storage commitment with enterprise-wide collection of app information and activity data.
  • Relation to prior work: The pipeline targets pragmatic contradictions that symbolic tuple-based systems do not detect, such as “we do not sell” paired with sharing identifiers with advertising partners.The contradiction concerns the impression created by the commitment and the practice documented elsewhere, not direct logical negation.
  • Relation to prior work: The reported rates are not directly comparable with PolicyLint’s 14.2% because the studies differ in contradiction definition, unit, corpus scale, and policy type.This comparison is between website-company rates and mobile-app-policy rates using different contradiction concepts.
  • Implications: Pipeline outputs are proposed as candidates for expert review and possible self-audit, while precision validation remains necessary before enforcement use.Metadata filtering reduces review volume, but its effect on precision is untested.

10 Limitations

The principal limitation is that the pipeline’s extraction, typing, and judging have not been validated against human labels, so reported quantities describe pipeline outputs rather than established prevalence. Additional limits concern recall, model and prompt sensitivity, scope, and interpretation.

  • Validation and scope: No pipeline stage has been validated against human labels, so every reported quantity is a property of the pipeline rather than a measurement of contradiction prevalence.The study covers 238 companies across two corpora, limiting generalizability beyond these settings.
  • Statement typing: The binary COMMITMENT label conflates company commitments with user capabilities, and 15.9% of original confirmations were removed by a refined three-class audit.The audit removed 21 affected confirmations, disproportionately from OPP-115, which lacked metadata filters.
  • Model dependence: The primary models served as both extractors and judges, raising possible self-preference concerns; separated panels left aggregate prevalence essentially unchanged but shifted category composition.The stability run does not establish invariance for individual verdicts.
  • Model dependence: Results may shift as models improve, while identical-configuration run-to-run variation remains unquantified and intervals omit that variation.Panel-confirmed means LLM majority agreement, not expert validation.
  • Prompt sensitivity: Prompt examples may prime judges toward third-party sharing or sale patterns, and the analysis cannot fully separate prompt effects from corpus-specific framing.The observed third-party confirmation lift was small and not statistically significant, but prompt effects cannot be ruled out.
  • Recall: Extraction recall is unmeasured, and progressive filtering may discard true contradictions, especially those expressed with substantially different vocabulary.The pipeline prioritizes precision over recall, beginning with consensus extraction and continuing through downstream filters.
  • Interpretation: The pipeline detects textual contradictions without testing user impact, intentionality, or legal defensibility.Behavioral studies are needed to establish whether reading commitments produces expectations contradicted by documented practices.
  • Validation and scope: The study is limited to English-language policies and selected jurisdictions, and policies may have changed since collection.It does not analyze policies drafted primarily for frameworks such as the GDPR or LGPD.

11 Future Work

Future work prioritizes human validation, repeated judging, broader longitudinal and cross-domain evaluation, and remediation-oriented tools. These steps address uncertainty in accuracy, stability, scale, and practical use.

  • Method development: Future experiments should disable the qualifier-coverage branch and judge discarded pairs to assess legacy-pair losses.The stability experiment already tested sub-threshold judging, panel separation, and matched cross-corpus filtering.
  • Validation: Repeated judging of identical pairs would measure judge-stage verdict stability under nominally deterministic decoding.This would isolate judge variation that the stability run cannot separate because extraction and judging changed together.
  • Validation: Human expert validation is the priority, using blinded independent privacy-law annotation and precision and recall against a human gold standard.A minimum design would involve two trained annotators labeling a stratified sample of roughly 100 pairs.
  • Scaling and generalization: Temporal analysis and larger contemporary or longitudinal corpora would test whether contradictions resolve, persist, accumulate, or generalize.Suggested targets include Fortune 500 policies and million-document historical resources.
  • Applications: Remediation tools could suggest edits that resolve detected contradictions and extend analysis to sector-specific patterns.Healthcare examples motivate domain-specific study, but OPP-115 contains only two dedicated healthcare facilities.

12 Responsible Disclosure and Ethical Considerations

The paper reports its findings as unvalidated textual conditions, not allegations of legal violation or intentional deception. It also discloses model-provider conflicts, public-document-only analysis, and the absence of personal-data processing.

  • LLM verdicts were not validated by human experts, so precision is unknown and individual findings may be false positives.
  • The findings identify commitments and practices in pragmatic tension, without attributing legal violations or intentional deception.
  • The primary judge models came from Anthropic, OpenAI, and Google, whose companies all appear in the OPPT corpus, creating a provider conflict.A stability run used models from providers absent from both corpora; none of the Google confirmations recurred, partly because legacy pairs were excluded by design.
  • The term “privacy washing” is retained for continuity with greenwashing literature, although the structural findings do not establish deliberate misconduct.
  • The analysis used only publicly available policy documents and processed no personal data.

13 Conclusion

The paper defines and measures privacy washing as internal contradictions in privacy policies across two corpora collected 11 years apart. Its prevalence results and recurring category patterns are qualified by differing filters and the absence of expert validation.

  • 12.2% of OPPT companies and 36.5% of OPP-115 companies had panel-confirmed contradictions, but differing filter configurations prevent interpreting the gap as an era effect.The OPPT reproducible subset was 9.8% after excluding carried-over pairs.
  • Third-party-sharing contradictions formed the majority of confirmed cases in both primary runs, while recurring category patterns were consistent with structural policy composition factors.
  • Expert validation is the measurement that would most change the findings because reported quantities currently have no known relationship to human judgment.The pipeline is therefore characterized as a candidate-generation tool rather than a measurement instrument or enforcement tool.

Competing Interests

The author discloses a professional interest connected to the paper’s subject while reporting no external funding.

  • The author founded Varitas, which develops privacy policy analysis technology related to the described pipeline.
  • No external funding supported the work.

Data and Code Availability

The paper makes its computational materials and primary-run configuration publicly available through a GitHub repository. It identifies the models and execution dates used in the primary experiments.

  • The detection pipeline, intermediate outputs, and analysis scripts are available at the paper’s GitHub repository.
  • The primary extraction and judging panel comprised Claude Haiku 4.5, GPT-5 mini, and Gemini 3 Flash Preview via OpenRouter.
  • The primary OPPT and OPP-115 experiment runs were executed on January 31, 2026, and February 3, 2026, respectively.

A LLM Judge Prompt

The judge prompt frames contradiction detection as testing whether a practice undermines a company’s commitment. It distinguishes genuine conflicts from unrelated, compatible, or differently scoped statements and uses a three-model JSON verdict.

  • All three judges received the same unmodified prompt with company names and statement texts substituted, returning a contradiction verdict and brief reasoning in JSON.
  • Statement A is a company commitment about data handling, while Statement B describes what the company actually does with data.
  • The judge determines whether Statement B violates or undermines the commitment made in Statement A.
  • A contradiction includes a no-selling commitment paired with sharing data with advertising partners for targeted marketing.
  • The prompt excludes practices that implement a commitment, restate it, concern unrelated topics, differ in data scope, or describe a hedged possibility.

B Statement Extraction Prompt

The extraction prompt converts privacy-policy segments into self-contained atomic commitments or practices with structured metadata. It preserves source qualifiers and meaning while retaining statements supported by at least two extraction models.

  • Statements are retained when at least two of three extraction models produce matches at cosine similarity threshold 0.7.
  • The extractor identifies each distinct claim or practice as a separate atomic statement with enhanced metadata.
  • Statements are classified as COMMITMENT promises or limitations, or PRACTICE descriptions of what the company does with data.
  • Each statement includes text, type, subject, aspect, scope, qualifiers, and category, as illustrated by sharing and tracking examples.
  • The rules require self-contained wording, preserve original meaning and qualifiers, split combined commitments and practices, and ignore procedural text.
  • A no-selling example preserves legal and merger-related exceptions, while an advertiser example represents third-party browsing-data collection as a practice.
Loading 2609.02055v1…