Source-linked AI summary
Privacy- and Utility-Preserving Textual Analysis via Calibrated Multivariate Perturbations
Oluwaseyi Feyisetan, Borja Balle, Thomas Drake, Tom Diethe
TL;DR
The paper addresses how to learn from user text while providing quantifiable privacy and preserving the text’s intended semantics. It perturbs word embeddings with calibrated noise under dχ-privacy, calibrates ε using embedding-based plausible-deniability statistics, and evaluates privacy and utility across tasks. The approach provides stronger tunable privacy than baselines while retaining practical utility for binary-classifier training.
Problem
The paper asks how to privatize user word choices that can fingerprint users while preserving the original intent and supporting useful analysis.
Method
The mechanism adds calibrated noise to word-embedding vectors, maps the perturbed vectors back to words, and calibrates ε using statistics of the resulting distributions and embedding geometry.
Results
The approach provides tunable privacy guarantees over 3 times greater than baselines while incurring < 2% utility loss on training binary classifiers across a range of ε values.
Takeaways & Limitations
Embedding geometry supports text perturbation that preserves semantics more effectively than unconstrained word-level privatization while retaining measurable privacy–utility trade-offs.
Takeaways & Limitations
Highly concentrated embedding neighborhoods can increase Pr[M(w) = w] and shrink the effective output support, weakening plausible deniability.
Abstract
from arXiv · showhide
Accurately learning from user data while providing quantifiable privacy guarantees provides an opportunity to build better ML models while maintaining user trust. This paper presents a formal approach to carrying out privacy preserving text perturbation using the notion of dx-privacy designed to achieve geo-indistinguishability in location data. Our approach applies carefully calibrated noise to vector representation of words in a high dimension space as defined by word embedding models. We present a privacy proof that satisfies dx-privacy where the privacy parameter epsilon provides guarantees with respect to a distance metric defined by the word embedding space. We demonstrate how epsilon can be selected by analyzing plausible deniability statistics backed up by large scale analysis on GloVe and fastText embeddings. We conduct privacy audit experiments against 2 baseline models and utility experiments on 3 datasets to demonstrate the tradeoff between privacy and utility for varying values of epsilon on different task types. Our results demonstrate practical utility (< 2% utility loss for training binary classifiers) while providing better privacy guarantees than baseline models.
1 INTRODUCTION
The paper develops a scalable, formally private mechanism for perturbing text while preserving semantics and supporting deployment in existing text-processing pipelines. It calibrates privacy using word-embedding geometry and evaluates privacy–utility trade-offs across tasks.
- Motivation: Privacy-preserving analysis is needed because insufficient guarantees limit the use of sensitive user-generated text, while disclosure can impose reputational and financial costs.The paper distinguishes privacy breaches from security breaches and frames privacy as a prerequisite for useful computation over sensitive data.
- Motivation: Differential Privacy protects against reconstruction and arbitrary attacker side knowledge by making outputs similar across neighboring databases, with ε controlling privacy strength.ε = 0 represents absolute privacy, whereas ε = ∞ represents null privacy.
- Approach: The mechanism is designed to map text inputs to text outputs and scale to large datasets, enabling integration into existing text-processing pipelines.These deployment requirements exclude some hash-based private data structures.
- Approach: The proposed mechanism maps each input word into an embedding space, adds calibrated noise, and projects back to a privatized word satisfying dχ-privacy.The resulting guarantee is interpreted as plausible deniability for the original word.
- Evaluation: The paper analyzes embedding geometry and evaluates privacy–utility trade-offs across binary sentiment analysis, multi-class classification, and question answering.The contributions include analysis of two embedding models and experiments at different ε values.
2 PRIVACY PRESERVING MECHANISM
The mechanism privatizes words by perturbing their embedding representations, using dχ-privacy calibrated to embedding-space distance while retaining semantic similarity. Theorem 1 establishes εdχ-privacy for strings of any length under the proposed metric.
- Motivation: Word choices can fingerprint users, even when they are not traditional personally identifiable information.The threat model concerns one word submitted by each user and aims to hide identity-related information while preserving intent.
- Why dχ-privacy: Local differential privacy is poorly suited to semantic text perturbation because every input word must have nonnegligible probability of becoming any other vocabulary word.As vocabulary size grows, semantically related words receive vanishingly small probability under LDP.
- Why dχ-privacy: dχ-privacy scales indistinguishability with the distance between inputs, allowing nearby words to receive higher probability than semantically unrelated words.Its guarantee bounds output log-likelihood ratios by εd(x,x′), while ε controls privacy relative to the chosen metric.
- Mechanism: The embedding-derived string metric sums Euclidean distances between corresponding word vectors and is a valid metric when the embedding is injective.The embedding is assumed independent of the data being privatized.
- Mechanism: The mechanism maps each word into an embedding space, adds noise with density pN(z) ∝ exp(−ε∥z∥), and returns the nearest dictionary word.For strings, it applies this process independently to every word and preserves the input length.
3 STATISTICS FOR PRIVACY CALIBRATION
The paper calibrates ε for dχ-privacy using embedding geometry and plausible-deniability statistics, which quantify unchanged outputs and effective output support. Privacy improves as outputs become less concentrated, but this trades off against semantic utility.
- Calibration strategy: ε calibration must account for the word-embedding metric because its meaning is not transferable across different metrics.The calibration strategy uses geometric properties of the embedding space rather than traditional Hamming-distance rules of thumb.
- Qualitative behavior: As ε → 0, outputs become independent of the input word, providing absolute privacy but effectively randomizing semantics.As ε → ∞, the mechanism returns the input word and provides null privacy while fully preserving semantics.
- Output distribution: For any finite ε, every vocabulary word has non-zero output probability, although probabilities for distant or incorrect words vanish as ε increases.Words close to the input receive comparatively similar probabilities because embedding distance reflects semantic similarity.
- Privacy–utility trade-off: Increasing concentration raises the probability of releasing the original word and shrinks effective support, weakening plausible deniability.Nw and Sw also act as proxies for max- and min-entropy, respectively.
- Plausible deniability: The statistics Nw and Sw measure plausible deniability through the probability of leaving the input unchanged and the effective support of outputs.A privacy-preserving setting should make Nw small and Sw large for almost all words.
4 ANALYSIS OF WORD EMBEDDINGS
The paper analyzes how embedding geometry affects perturbation behavior and uses worst-case plausible-deniability statistics to select ε. GloVe and fastText exhibit different distance distributions, while embedding dimensionality changes the ε scale at comparable guarantees.
- Embedding models: Word embeddings map vocabulary words to vectors, with skip-gram training shaping the geometry that defines perturbation distances.The mechanism's output distribution depends directly on this embedding geometry.
- Embedding analysis: 319,000 words shared by GloVe and fastText were analyzed using Euclidean distances to neighbors at k values from 1 to 1000.Distances were summarized at the 5th, 20th, 50th, 80th, and 95th percentiles.
- Embedding analysis: fastText has a smoother distance distribution with a wider spread across percentiles than GloVe.This shows that embedding models with the same vector dimension can induce different distance distributions.
- Privacy calibration: At ε = 5, every word produced at least 300 distinct new words and no word was returned more than 500 times in 1,000 trials.These worst-case Sw and Nw values illustrate how ε can be selected for a desired guarantee.
- Embedding selection: The same embedding model at different dimensionalities can provide similar average-case guarantees at different ε values.Accordingly, model selection depends on downstream utility, and ε must be interpreted within its metric space.
- Embedding selection: Average Sw and Nw values cannot conclusively compare embedding models because different distributions can yield the same entropy.The paper therefore recommends worst-case guarantees instead of relying only on averages.
5 ML UTILITY EXPERIMENTS
The paper evaluates privacy–utility tradeoffs for word perturbation across three NLP task types and datasets, using training-time and test-time privatization settings. Binary classification remains robust, while question answering is most sensitive to injected noise.
- Experimental setup: Three downstream tasks use IMDb binary classification, Enron multi-class classification, and InsuranceQA question answering datasets.The experiments evaluate privacy versus utility across common ML and NLP task types.
- Experimental setup: Two evaluation settings privatize either the training data while testing remains plain or the test data after training on the available training set.These settings simulate different levels of access to private data.
- Experimental setup: 300d GloVe embeddings and biLSTM models are used throughout the experiments.
- Results: Binary classification remains robust to injected perturbations at both training and test time.
- Results: Question answering is the most sensitive task to noise, while performance also degrades on the other non-binary task.
6 ML PRIVACY EXPERIMENTS
The privacy experiments compare the proposed dχ mechanism with query-scrambling baselines using a privacy auditor. Existing baselines are perfectly identifiable, whereas dχ provides tunable guarantees that outperform baselines for ε ≤23 on AUC and remain better than chance across metrics.
- Baselines and evaluation: The evaluation compares dχ-privacy with Versatile and Incognito query-scrambling baselines using search-log data.The baselines use semantic or statistical scrambling and perturbations with or without noise.
- Baselines and evaluation: The privacy auditor is trained and tested on samples from 8,670 users, using Precision, Recall, Accuracy, and AUC to measure identification ability.
- Results: Existing baselines fail to prevent privacy attacks: the auditor perfectly identifies perturbed queries regardless of whether they were used for training.Table 3 scores measure privacy loss, with lower values being better.
- Results: Over 3x greater privacy than baselines is achieved on AUC scores for ε ≤23, while all metrics remain better than chance at ε ≤23.
7 DISCUSSION
The paper concludes that word perturbation can provide formal privacy guarantees while retaining useful performance across multiple textual datasets and task types. The experiments expose a tunable tradeoff between privacy guarantees and task utility.
- Experiments across different datasets and task types provide empirical evidence for the feasibility of adopting formal privacy-preserving word perturbation.
- The findings demonstrate a tradeoff between desired privacy guarantees and achieved task utility.
8 RELATED WORK
Related work treats text redaction as either plausible deniability for transactional data or confidentiality for plain text. The paper differs by perturbing words through embeddings while avoiding fixed sensitive-data classifications and assumed attacker knowledge.
- Transactional redaction seeks anonymity or plausible deniability, whereas plain-text redaction seeks confidentiality against association with an entity.
- The general literature pipeline detects sensitive terms and then obfuscates the identified entities.
- Some approaches define private queries or sensitive terms using uniformity, uniqueness, linkability, or recognized PII classes.
- The paper uses word embeddings for entity replacement and avoids relying on a priori sensitive-data classifications that assume bounded attacker knowledge.
9 CONCLUSION
The approach combines formal dχ-privacy guarantees with calibrated word-embedding perturbations to balance privacy and utility. Experiments report stronger tunable privacy than baselines while keeping binary-classifier utility loss below 2%.
- < 2% utility loss was incurred when training binary classifiers across a range of ε values.
- Privacy guarantees were tunable and over 3 times greater than those of the baselines.
- Worst-case plausible-deniability statistics and privacy-utility experiments help data holders choose ε for their tasks.