PaperlayerView exact revision on arXiv

Source-linked AI summary

GLiNER2-PII: A Multilingual Model for Personally Identifiable Information Extraction

Urchade Zaratiana, Ash Lewis, George Hurn-Maloney

arXiv:2605.09973v1cs.CLcs.AI

TL;DR

PII detection must handle heterogeneous, context-sensitive spans in noisy documents, while shareable annotated data are difficult to obtain because real PII is sensitive. GLiNER2-PII combines a broad label inventory with constraint-driven synthetic training data. It achieves the highest F1 on both SPY evaluation sets, though evaluation scope and human validation remain limitations.

  • Problem

    PII spans vary by locale and context, and sensitive real-world examples are difficult to collect and share for training.

    Source: §1 · p. 1, §3 · p. 3

  • Method

    GLiNER2-PII fine-tunes GLiNER2 for 42 PII types using a constraint-driven pipeline to generate diverse synthetic annotated texts.

    Source: §1 · pp. 1–2, §3 · p. 3

  • Results

    GLiNER2-PII achieves the highest F1 on both SPY evaluation sets despite training entirely on synthetic data.

    Source: §6 · p. 5

  • Takeaways & Limitations

    The results indicate that diverse synthetic training data combined with a broad label inventory can produce effective detectors on naturally occurring text from unseen domains.

    Source: §6 · p. 5

    Evaluation covers only legal and medical documents, and the entirely synthetic training data have not been validated by human annotators.

    Source: §6 · p. 5

Abstract

from arXiv · show

Reliable detection of personally identifiable information (PII) is increasingly important across modern data-processing systems, yet the task remains difficult: PII spans are heterogeneous, locale-dependent, context-sensitive, and often embedded in noisy or semi-structured documents. We present GLiNER2-PII, a small 0.3B-parameter model adapted from GLiNER2 and designed to recognize a broad taxonomy of 42 PII entity types at character-span resolution. Training such systems, however, is constrained by the scarcity of shareable annotated data and the privacy risks associated with collecting real PII at scale. To address this challenge, we construct a multilingual synthetic corpus of 4,910 annotated texts using a constraint-driven generation pipeline that produces diverse, realistic examples across languages, domains, formats, and entity distributions. On the challenging SPY benchmark, GLiNER2-PII achieves the highest span-level F1 among five compared systems, including OpenAI Privacy Filter and three GLiNER-based detectors. We publicly release the model on Hugging Face to support further research and practical deployment of open PII detection systems.

1 Introduction

PII detection must handle varied, context-sensitive information in noisy text while returning precise spans. GLiNER2-PII is presented as a fine-grained detector evaluated on SPY.

1 Introduction

  • PII appears across digital systems and noisy, unstructured documents, where precise character spans support downstream masking, auditing, or routing.Regulatory frameworks such as GDPR and CCPA make reliable detection and removal a prerequisite in these settings.Source: §1 · p. 1
  • Locale-dependent formats, ambiguous values, and mixed or semi-structured documents complicate detection, while missed spans and excessive masking carry different costs.Missed spans create privacy and compliance risk; aggressive masking can degrade data utility.Source: §1 · p. 1
  • Label-conditioned span extractors accept target labels as inputs, unlike token classifiers designed around predefined schemas, supporting flexible PII categories and policies.Closely related entity types may need different downstream treatment.Source: §1 · p. 1
  • Figure 1 compares span-level F1 on SPY and reports GLiNER2-PII outperforming three baselines across both domains.The caption identifies the benchmark as SPY (Savkin et al., 2025).Source: Figure 1 · p. 2
  • GLiNER2-PII is fine-tuned from GLiNER2 and covers 42 PII types; on SPY it achieves the highest span-level F1 among five compared systems.The compared systems include OpenAI Privacy Filter and three GLiNER-based detectors.Source: §1 · pp. 1–2

2 Method

The method treats PII detection as schema-based entity extraction: a text and target schema yield typed spans for downstream redaction. GLiNER2-PII uses a 42-label schema with exact character spans.

2 Method

  • PII detection is framed as schema-based entity extraction, returning typed spans that a downstream redaction system can mask.The extraction receives an input text and a schema of target entity types, optionally with natural-language descriptions.Source: §2 · p. 2
  • GLiNER2 supports multiple information-extraction tasks through a common schema interface, and target labels can be changed at inference without modifying the architecture.GLiNER2-PII is fine-tuned for entity extraction over 42 PII labels and produces exact character spans for masking.Source: §2 · p. 2
  • The label inventory groups 42 fine-grained PII types into seven semantic categories, spanning identity, contact, financial, credential, and temporal attributes.The inventory also permits nested spans, such as first- and last-name spans inside a full-name span.Source: Table 1 · p. 3
  • The inference example loads the fine-tuned checkpoint, defines target labels, and returns entities with confidence scores and character spans.The example is described as a minimal inference workflow.Source: §2 · pp. 2–3

3 Data Generation

The data-generation framework uses label inventories and task descriptions to synthesize annotated PII examples under constraints. The resulting corpus spans multiple document formats and languages.

3 Data Generation

  • Because realistic PII is sensitive and difficult to share, the paper constructs a synthetic corpus with a constraint-driven generation framework.The framework derives sampling constraints from a natural-language task description and conditions a large frontier decoder to produce annotated examples.Source: §3 · p. 3
  • The 42-type inventory allows nested entities and exact-value spans, supporting broad masking or selective redaction of inner credentials.Annotations exclude surrounding words, punctuation, and field labels unless they are part of the value.Source: §3 · p. 3
  • The pipeline combines programmatic label-composition constraints with diversity constraints inferred from the task description, including document type, locale, register, and tone.For each example, it samples constraints, prompts a large frontier decoder at temperature 0.01, and generates text with span-level annotations.Source: §3 · p. 3
  • The generated corpus covers chat logs, support tickets, CRM notes, KYC forms, invoices, medical records, and credential files across seven languages.The languages are English, French, Spanish, German, Italian, Portuguese, and Dutch, with occasional mixed-language passages.Source: §3 · pp. 3–4

4 Evaluation Setup

The evaluation uses SPY's legal and medical subsets to assess out-of-distribution performance, compares four public detectors, and scores exact-match spans after a shared label mapping.

4 Evaluation Setup

  • SPY (Synthetic PII Yesterday) comprises two domain-specific subsets: 100 legal-Q&A forum documents and 100 medical-transcript documents, annotated with seven PII types.The labels are name, address, email, phone_num, id_num, url, and username.Source: §4 · p. 4
  • The study excludes ai4privacy/pii-masking-300k because public PII models trained on its training split could make true out-of-distribution performance difficult to isolate.This exclusion is stated as a benchmark-selection rationale.Source: §4 · p. 4
  • Four public detectors represent token classification and label-conditioned extraction, including OpenAI Privacy Filter and three GLiNER-based models.The GLiNER-based comparisons include NVIDIA GLiNER PII, urchade/gliner_multi_pii-v1, and knowledgator/gliner-pii-base-v1.0.Source: §4 · p. 4
  • A deterministic mapping aligns each model's labels to SPY's seven categories, and the same procedure is applied across systems for fair comparison.Metrics are exact-match span-level precision, recall, and F1; both label type and character boundaries must match.Source: §4 · p. 4

5 Results

GLiNER2-PII achieves the highest exact-match F1 on both SPY subsets and overall. Its results favor recall while retaining competitive precision, a trade-off relevant to redaction.

  • GLiNER2-PII achieves the highest exact-match F1 on both legal and medical subsets and the best overall average.Source: §5 · p. 4
  • Recall is 0.722 on legal and 0.681 on medical, versus 0.308–0.431 for other GLiNER variants; OpenAI Privacy Filter reaches 0.640–0.671 recall but only 0.250–0.271 precision.The passage identifies missed PII spans as particularly costly in redaction settings.Source: §5 · p. 4
  • urchade/gliner_multi_pii-v1 has the highest precision (0.483–0.522) but the lowest recall (0.308–0.314), while GLiNER2-PII favors recall with competitive precision.Performance is generally consistent across the legal and medical domains for all evaluated systems.Source: §5 · p. 5

6 Discussion and Limitations

The discussion reports transfer from synthetic training data to naturally occurring legal and medical text, while identifying name over-prediction and limited evaluation as constraints. It concludes that diverse synthetic data and a broad label inventory can yield effective detection, with broader validation and evaluation left for future work.

6 Discussion and Limitations

  • GLiNER2-PII, trained entirely on generated synthetic data, achieves the highest F1 on both naturally occurring evaluation sets.The discussion suggests that the 42-label inventory may strengthen representations of broader categories; the largest gains over other GLiNER variants appear on names.Source: §6 · p. 5
  • Name over-prediction is the main precision issue, with personal names sometimes confused with common nouns, organization names, or product names.Label-specific thresholds, lightweight filtering, or small-set calibration could likely improve precision without substantially reducing recall.Source: §6 · p. 5
  • Evaluation covers only legal and medical documents, and the entirely synthetic training data have not been validated by human annotators.Strong performance on naturally occurring text is reported as evidence that the generation pipeline can transfer to real-world PII detection tasks.Source: §6 · p. 5
  • The authors conclude that diverse synthetic data and a broad label inventory can produce effective detectors on naturally occurring text from unseen domains.Future work includes human-validated fine-tuning, more locales and languages, broader multilingual evaluation, and end-to-end redaction benchmarking for accuracy and efficiency.Source: §6 · p. 5

Related papers

Loading 2605.09973v1…