Model Card for OpenAI Privacy Filter

1 Introduction

• Privacy Filter converts an autoregressively pretrained checkpoint into a bidirectional token classifier that labels sequences in one pass and decodes coherent privacy spans.The classifier predicts across eight privacy categories and uses constrained Viterbi decoding.Source: §1 · p. 4
• With 1.5 billion total parameters and 50 million active parameters, Privacy Filter can run in a browser or on a laptop.It is released under Apache 2.0, which permits experimentation, customization, and commercial deployment.Source: §1 · p. 4
• Users can fine-tune Privacy Filter to specific data distributions using a simple, data-efficient process.The model also provides configurable span decoding to control precision-recall tradeoffs and detected span lengths.Source: §1 · p. 4
• A 128,000-token context window supports processing long text with high throughput and no chunking.Source: §1 · p. 4
• Privacy Filter is presented as a lightweight privacy-preserving infrastructure direction that the authors are releasing for community feedback and further iteration.The stated goal is for models to learn about the world rather than private individuals.Source: §1 · p. 4

2 Model Details

• The model converts autoregressive pretraining into a bidirectional banded-attention classifier, then uses constrained BIOES decoding for coherent spans.Its band size is 128, corresponding to an effective attention window of 257 tokens including self.Source: §2 · p. 5

2.1 Model Architecture

• Eight transformer blocks combine grouped-query attention and sparse top-4-of-128 experts; one-pass labeling supports throughput-sensitive use.The model has 14 query heads and 2 key-value heads, with a residual-stream width of 640.Source: §2.1 · p. 5

2.2 Output Shape

• The output head emits 33 classes: background plus four BIOES boundary tags for each of eight privacy categories.The categories include account number, address, email, person, phone, URL, date, and secret.Source: §2.2 · pp. 5–6

2.3.1 Rationale

• Constrained Viterbi decoding scores complete label paths and enforces allowed BIOES transitions to improve span coherence and boundary stability.The decoder uses start, transition, and end terms, plus six transition-bias parameters.Source: §2.3.1 · p. 6

2.3.2 Operating-Point Calibration

• Users can tune sequence-decoding parameters toward broader, more contiguous spans for recall or toward greater precision.Runtime parameter changes control the precision-recall tradeoff.Source: §2.3.2 · p. 6

3.1 Intended Uses

• The model identifies candidate sensitive spans and classes for downstream handling, including ingestion sanitization, corpus preparation, enterprise text redaction, and domain adaptation.A surrounding system can mask, remove, replace, pseudonymize, alert, or route spans for review.Source: §3.1 · pp. 6–7

3.2 Out-of-Scope and Misuse

• It is not an anonymization, compliance, or safety guarantee; validate default operating points on the target distribution and adapt when local policy differs.Unredacted output should not be treated as safe for release.Source: §3.2 · p. 7

4.1 Over-reliance Risk

• Blanket anonymization claims risk missing privacy objectives; use the model as one layer in a holistic privacy-by-design approach.The paper characterizes Privacy Filter as a redaction and data-minimization aid, not a guarantee.Source: §4.1 · p. 7

4.2 Static Label Policy

• Detection is limited to the trained taxonomy and static label policy; defaults may miss governance needs, and policy changes require fine-tuning.The native policy prioritizes personal identifiers and may preserve context that is not strongly person-linked.Source: §4.2 · p. 7

4.3 Failure Modes

• Performance may drop out of distribution, with missed names or identifiers, excess redaction, shifted boundaries, and missed or falsely flagged secrets.Names underrepresented in training or following different conventions may be more likely to be missed or inconsistently bounded.Source: §4.2 · p. 7, §4.3 · pp. 7–8

4.4 High-Risk Deployment Caution

• In high-sensitivity workflows, false negatives can expose information while false positives can remove context needed for review, auditing, or decisions.The paper names medical, legal, financial, human-resources, education, and government settings.Source: §4.4 · p. 8

4.5 Recommendations

• Before production use, evaluate in-domain against local policy; fine-tune when policy differs, and retain human review for high-sensitivity workflows.The recommended approach is layered privacy-by-design rather than a blanket anonymization claim.Source: §4.5 · p. 8

5 How to Get Started with the Model

• The CLI can predict typed spans and return redacted text with person and address placeholders for an example sentence.The example identifies “Ben Morgan” as a private person and “12 3rd St.” as a private address.Source: §5 · pp. 8–9
• For file input, -f obtains span predictions; --format json produces verbose JSON with a summary and identified-span records.The JSON example includes output mode, span count, and counts by label.Source: §5 · pp. 8–9
• The redacted output mode collapses detected categories to a single redaction label.The example replaces both the person and address spans with <REDACTED>.Source: §5 · p. 10
• Fine-tuning is available through opf train with an output-directory option.The documentation points users to the train help command for more information.Source: §5 · p. 10

6 Training Details

• The model combines publicly available and synthetic data, with synthetic examples augmented for privacy-pattern diversity and filtered by automated quality controls.For public data without ground truth, annotations used two prompt formats crossed with two reasoning settings.Source: §6 · p. 10

6.1 Label Taxonomy

• The shipped taxonomy has eight categories covering identifiers and secrets, while placeholders are not intended to be classified.The secret class excludes example API keys and other placeholder values.Source: §6.1 · p. 11

6.2 Training Procedure

• An autoregressively pretrained checkpoint is converted to a bidirectional banded-attention token classifier and post-trained on BIOES-style labels.The output layer is replaced with a classification head, and attention uses both left and right context while preserving local attention.Source: §6.2 · p. 11

6.2.1 Decoding and Calibration

• Constrained Viterbi decoding enforces valid BIOES transitions, and held-out span metrics guide operating points for different precision–recall preferences.Users can adjust decoding behavior to favor conservatism or coverage.Source: §6.2.1 · pp. 11–12

7.1 Evaluation Goals

• Evaluation targets detection quality across domain, format, and language variation, label-policy definitions, and standard external benchmarks.The stated goals include characterizing strengths and weaknesses across evaluation settings.Source: §7.1 · p. 12

7.2 Evaluation Datasets

• PII-Masking-300k covers multilingual text, CredData tests codebase credentials, and SPY covers synthetic medical consultations and legal questions.Together, the datasets represent general multilingual text, software-security data, and specialized professional domains.Source: §7.2 · p. 12

7.2.1 PII-Masking-300k

• PII-Masking-300k labels were mapped to supported categories, then corrected metrics excluded spans adjudicated as missing labels.The adjudication marked about 10k spans in 8k examples as missing labels; 100 human-reviewed spans agreed with the reasoning model’s decisions.Source: §7.2.1 · pp. 12–13

7.2.2 Credential Detection in Codebases

• CredData’s true-positive labels map to the secret category, while false-positive and unknown labels are discarded; evaluation counts token or exact-span matches on a test split.Both baseline and corrected dataset versions are evaluated.Source: §7.2.2 · p. 14

7.3 Metrics and Matching Rules

• Recall is reported at token and span levels alongside precision and F1.The supplied passage also gives recall = TP / (TP + FN).Source: §7.3 · p. 14

7.4.1 Benchmark Summary

• Privacy Filter achieves strong overall PII detection and strong recall on secret-credential detection tasks.The benchmark summary covers both datasets as provided and corrected; it notes that CredData span scores are affected by exact-boundary discrepancies for partially detected spans.Source: §7.4.1 · p. 14, Table 1 · p. 14

7.4.2 Fine-tuning Efficiency

• Fine-tuning on 10% of SPY training data raises F1 above 96% and nearly saturates the benchmark.SPY labels only the question author’s PII and includes idiosyncratic account-number types; the evaluation mapping excludes secret and private_date.Source: §7.4.2 · p. 15
• The CLI supports fine-tuning to target distributions and custom decision criteria, including a label set that distinguishes private from public information of a given type.The example separates private_address from public_address.Source: §7.4.2 · p. 15

7.5 Stress-Testing OPF Capabilities on Synthetic Data

• Synthetic toy evaluations were generated by prompting frontier GPT models, and the model was not trained on data generated in the same manner.The tests were intended to clarify Privacy Filter’s capabilities and limitations.Source: §7.5 · p. 15

7.5.1 Influence of the Category Clue Position

• The clue-position evaluations compare 1,181 examples with clues before PII, 1,179 with clues after PII, and 1,200 PII-only examples without clues.The examples use simple sentence structures, and any provided clue unambiguously identifies the PII category.Source: §7.5.1 · pp. 15–16, Table 5 · p. 16, Table 4 · p. 16

7.5.2 Multilingual Evals

• The multilingual analyses distinguish performance across languages in PII-Masking-300K from generalization to less-represented or absent languages in synthetic evaluations.A further evaluation examines category-clue-before-PII data in languages absent from PII-Masking-300K, including English as a baseline.Source: §7.5.2 · p. 16, Table 6 · p. 17, Table 7 · p. 17

7.5.3 One-Hop Reasoning Evals

• Recall declines as an alias definition grows farther from its sensitive value, indicating difficulty applying deferred references across long contexts.The authors suggest that long-range alias resolution may be underrepresented in current training data.Source: Figure 2 · p. 18, §7.5.3 · p. 18

7.5.4 Adversarial Evals

• Adversarial formatting tests identify line-broken URLs as a source of fragmented false-positive private_person predictions instead of one private_url span.Name-like tokens in phonetic spellings can also explain precision for the phonetic-alphabet format.Source: §7.5.4 · pp. 18–19, Figure 3 · p. 19

• Privacy Filter combines compact, efficient PII detection with practical inspectability, adaptability, and deployment.Source: §8 · p. 20
• The authors expect it to serve as a flexible component in larger privacy workflows, with long context, configurable decoding, and fine-tuning supporting varied deployment needs.The preview is released to invite community feedback and guide improvements in quality, multilingual coverage, and practical deployment.Source: §8 · p. 20