Source-linked AI summary
Generalised Differential Privacy for Text Document Processing
Natasha Fernandes, Mark Dras, Annabelle McIver
TL;DR
The paper asks how to obfuscate authorship in text while preserving content. It combines generalized differential privacy with a bag-of-words mechanism using semantic similarity, and reports a trade-off in which author identification drops while topic classification holds roughly constant in the evaluated snippets.
Problem
Method
The paper applies a generalized differential-privacy mechanism to bag-of-words documents, using Earth Mover’s distance to measure semantic similarity.
Source: §1 · p. 2, §4 · p. 10Results
Experiments demonstrate a privacy–utility trade-off: author identification drops while topic classification holds roughly constant on the evaluated snippets.
Source: §6 · p. 18, §8 · p. 20Takeaways & Limitations
The mechanism provides a differential-privacy-style guarantee for text processing and experimentally demonstrates a trade-off between privacy and utility.
Source: §8 · p. 20Obfuscation can disguise identity only when several authors write on similar topics, because topic itself can characterize author identity.
Source: §5 · p. 15
Abstract
from arXiv · showhide
We address the problem of how to "obfuscate" texts by removing stylistic clues which can identify authorship, whilst preserving (as much as possible) the content of the text. In this paper we combine ideas from "generalised differential privacy" and machine learning techniques for text processing to model privacy for text documents. We define a privacy mechanism that operates at the level of text documents represented as "bags-of-words" - these representations are typical in machine learning and contain sufficient information to carry out many kinds of classification tasks including topic identification and authorship attribution (of the original documents). We show that our mechanism satisfies privacy with respect to a metric for semantic similarity, thereby providing a balance between utility, defined by the semantic content of texts, with the obfuscation of stylistic clues. We demonstrate our implementation on a "fan fiction" dataset, confirming that it is indeed possible to disguise writing style effectively whilst preserving enough information and variation for accurate content classification tasks.
1 Introduction
The paper frames author obfuscation as changing a document while preserving its topic, despite bags-of-words retaining signals useful for both authorship and topic classification. It proposes a differential-privacy-inspired mechanism whose utility is measured by semantic similarity.
1 Introduction
- Writing style can reveal author identity and other personal attributes, motivating automated obfuscation of stylistic clues.The paper cites author identification, native language, gender, and age as attributes inferable from writing style.Source: §1 · pp. 1–2
- The target is to change a document’s bag-of-words without disturbing topic classification while making its author unidentifiable.The paper notes that bag-of-words retains word frequencies but not word order, yet can support both author identification and topic classification.Source: §1 · p. 2
- The proposed mechanism draws on generalized differential privacy and uses semantic similarity to balance privacy against preservation of document content.The paper describes noisy bag-of-words outputs and measures semantic similarity using the Earth Mover’s distance.Source: §1 · p. 2
2 Documents, topic classification and Earth Moving
The paper represents documents as bags of words and compares their semantic content using word embeddings and Earth Mover’s distance. For equal-sized bags, the distance is a metric, and a worked example ranks two documents as closer to each other than to a third.
2 Documents, topic classification and Earth Moving
- A document is a finite bag of words, retaining word identities and frequencies without representing their order.The paper uses this representation for machine-learning classification tasks.Source: §2 · p. 3
2.1 Word embeddings
- Word embeddings map words to real-valued vectors, with distances between vectors serving as distances between word meanings.The paper uses the Word Mover’s Distance, based on Earth Mover’s Distance, to compare documents for semantic similarity.Source: §2.1 · pp. 3–4
- Earth Mover’s Distance compares documents by minimizing the cost of transporting word occurrences between them; for equal-sized bags, it gives the Earth Mover’s metric.The paper illustrates the distance with an optimal flow between two sample documents.Source: §2.1 · pp. 4–5
- For equal-sized bags, the Earth Mover’s distances are 3.941 between b2 and b3 and 4.121 between b1 and b3, so b1 and b2 are closer.The example interprets this ordering as matching the linguistic interpretation of the original texts.Source: §2.1 · p. 5
3 Differential Privacy and the Earth Mover’s Metric
The section defines privacy in terms of Earth Mover’s distance between documents and shows how to extend word-level mechanisms to bags. It then applies the guarantee to text and establishes properties for changing metrics and post-processing.
3 Differential Privacy and the Earth Mover’s Metric
- For equal-size bags, independently applying a word-level ϵdX-private mechanism yields ϵN EdX-privacy, a stronger guarantee than follows directly from vector composition.The theorem applies to bags of fixed size N.Source: §3 · p. 7
3.1 Application to Text Documents
- With EdS(b1, b2)=2.816, ϵ∼1/32 yields a 1.42 multiplier between the example’s output distributions, which the authors describe as almost indistinguishable.The mechanism satisfies 4ϵEdS-privacy.Source: §3.1 · p. 9
3.2 Properties of Earth Mover’s Privacy
- If one word metric is pointwise no greater than another, privacy under its Earth Mover’s metric implies privacy under the larger metric.The paper gives Euclidean distance as an example: Euclidean-based privacy implies privacy for any distance measure exceeding Euclidean distance.Source: §3.2 · p. 9
- Post-processing preserves the same ϵEdX-privacy guarantee when the underlying word distance is a pseudometric.The result applies to composition with another mechanism K′.Source: §3.2 · p. 9
4 Earth Mover’s Privacy for bags of vectors in Rn
The section constructs an n-dimensional Laplacian mechanism for vectors and uses it to obtain Earth Mover’s privacy for bags of vectors. It also establishes a utility lower bound for high-dimensional representations.
4 Earth Mover’s Privacy for bags of vectors in Rn
- The n-dimensional Laplacian samples a Gamma-distributed radius and an independent uniform direction, yielding ϵ||·||-privacy for vectors.The section uses this mechanism to implement privacy for bags of vectors.Source: §4 · pp. 11–12
4.1 Earth Mover’s Privacy in BRn
- Applying the n-dimensional Laplacian to each vector gives Algorithm 1, which satisfies ϵN||·||-privacy for equal-sized bags.The result applies to any two bags in BRn of size N.Source: §4.1 · p. 13
4.2 Utility Bounds
- The section proves a lower bound on the probability that Algorithm 1 outputs a bag within Earth Mover’s distance ∆ of its input.The supplied theorem passage does not include the bound’s value.Source: §4.2 · pp. 13–14
5 Text Document Privacy
The document mechanism adds privacy noise to word embeddings and reconstructs a bag of words. Its guarantee depends on embedding-space similarity: distant-topic documents may require noise that disrupts topic content.
5 Text Document Privacy
- The procedure embeds each word, generates a private noisy bag of vectors, then maps each vector back to its nearest word embedding.The inverse mapping chooses the word minimizing Euclidean distance to the vector.Source: §5 · pp. 14–15
- Algorithm 2 satisfies ϵN E||·||-privacy relative to the embedding distance, bounding output probabilities for equal-sized input bags.The theorem defines the distance between the embedded bags as k.Source: §5 · p. 15
- For documents far apart in topic, the small ϵ needed to make outputs indistinguishable makes outputs unlikely to remain close to either topic.The authors conclude that obfuscation is feasible when documents have similar topics.Source: §5 · p. 15
- Author attribution can use character n-grams after the mechanism, and the paper states this post-processing preserves its privacy guarantee.Character n-grams capture stylistic clues such as systematic spelling errors.Source: §5 · p. 16
6 Experimental Results
Experiments on fan-fiction snippets evaluate author identification and topic classification under several noise levels. Author identification declines while topic classification is broadly maintained with different-representation methods; same-representation results show a different pattern.
6 Experimental Results
- The evaluation uses held-out sets of 20 and 50 authors from Harry Potter and Twilight fan fiction, evenly split between the two topics.The authors selected the domain to provide multiple authors per topic and multiple topics.Source: §6 · p. 16
- The different-representation author attacker uses character n-gram counts and cosine similarity, while fastText classifies topics.Same-representation analyses use 1-NN for authors and 5-NN for topics.Source: §6 · pp. 17–18
- With different-representation inference, author identification drops as ϵ varies while topic classification holds roughly constant.On unmodified snippets, author identification scored 15/20 and 27/50; topic classification scored 18/20 and 43/50 for the 20- and 50-author sets, respectively.Source: §6 · p. 18, Table 1 · p. 18
- Same-representation author identification falls below chance, while same-representation topic classification is unstable in high-dimensional Euclidean space.The authors describe the topic-classification instability as probably an artefact of high-dimensional space.Source: §6 · pp. 18–19
7 Related Work
Prior text-sanitisation and author-obfuscation methods use varied strategies, but the paper distinguishes its approach by requiring a differential-privacy guarantee. It situates that choice within generalised differential privacy, which had not yet been applied to text privacy.
Author Obfuscation
- Prior work also modifies stylometric features or perturbs sensitive terms, including approaches based on semantic noise over an ontology.The cited semantic-noise method perturbs sensitive terms using distances in a predefined directed graph.Source: §7 · pp. 19–20
- Other author-obfuscation approaches targeting semantic sensibility use ad hoc privacy guarantees designed to increase author-attribution misclassification.The paper contrasts this with its formal privacy guarantee.Source: §7 · pp. 19–20
Generalised Differential Privacy
- Generalised differential privacy, also called dX-privacy, had not yet been applied to text privacy in the paper’s account.The definition was originally motivated by geo-location privacy.Source: §7 · pp. 19–20
Author Obfuscation
- The paper’s primary distinction from approaches with strong utility is its differential-privacy guarantee, under which every output document could arise from any input with some probability.The authors characterize this as the strongest possible notion of plausible deniability.Source: §7 · p. 20
8 Conclusions
The paper combines text-document representations with generalised differential privacy and experimentally demonstrates a utility–privacy trade-off. It identifies readable summaries as a future direction requiring reconstruction from the mechanism’s bag-of-words output.
8 Conclusions
- The paper combines text-document representations with generalised differential privacy to implement a privacy mechanism for documents.It also demonstrates experimentally a trade-off between utility and privacy.Source: §8 · p. 20
- Producing readable summaries with a privacy guarantee remains a future direction that could require reconstructing text from the mechanism’s bag-of-words output.The authors propose exploring reconstruction techniques as a post-processing step.Source: §8 · p. 20
A Appendix A
- Appendix A presents proofs omitted from the main body.The passage identifies the appendix’s role but gives no further proof details.Source: A Appendix A · p. 26
A.1 Proofs Omitted from §3
The appendix proves that independently applying a metric-private mechanism to each element and collecting outputs as bags yields a privacy guarantee scaled by bag size. It also establishes post-processing preservation of privacy.
A.1 Proofs Omitted from §3
- The construction forms a valid bag-output distribution by collecting the probabilities of vector outputs across distinct orderings.The proof first applies K independently to vector elements, then aggregates probabilities for output permutations.Source: §A.1 · p. 27
- Applying K independently to elements of a fixed-size N bag and collecting outputs as bags yields ϵN E_dX-privacy.The theorem assumes K satisfies ϵdX-privacy on the underlying space.Source: §A.1 · p. 28
- The resulting mechanism satisfies the bag-level privacy inequality, first shown for singleton outputs and then extended to finite output sets.The proof compares the probabilities for corresponding bag outputs under b and b′.Source: §A.1 · p. 28
- Post-processing preserves ϵE_dX-privacy for the mechanism described in the lemma.The lemma states this for a mechanism K followed by K′.Source: §A.1 · p. 28
A.2 Proofs Omitted from §4
The appendix develops the n-dimensional Laplacian’s sampling representation and uses its radial distribution in probability calculations. It then outlines simplifications used in the proof of Theorem 3.
A.2 Proofs Omitted from §4
- The n-dimensional Laplacian is represented using an independent radial component and a direction uniform on the n-sphere.The radial component follows a Gamma distribution, while the direction is uniform.Source: §A.2 · pp. 29–30
- Probabilities for a ball of radius R under the n-dimensional Laplacian are calculated from its radial Gamma distribution.The appendix notes that the radial and angular distributions are independent.Source: §A.2 · p. 31
- The proof relates vector distances to bag Earth Mover’s distance to bound the probability of outputs within a specified region.It uses an inclusion between regions defined by the vector-distance and bag-distance constraints.Source: §A.2 · p. 32
- The proof of Theorem 3 uses further data constraints and approximations, including the stated word2vec dimension condition n ≥ 30.The appendix also invokes exact sphere formulas and Stirling’s approximation.Source: §A.2 · pp. 32–33
B Experimental Details
The experimental details describe the fan-fiction document construction and word-embedding vocabulary restrictions used to make the experiments feasible.
B Experimental Details
- The appendix provides further dataset and inference-mechanism details to support replicability, along with additional analysis.Source: B Experimental Details · p. 34
B.1 Dataset Construction
- Known-author texts use each story’s first 2,000 words, while unknown-author snippets use its final 1,000 words, followed by normalization and truncation to fixed lengths.The resulting lengths are 420 words for the 20-author set and 402 for the 50-author set.Source: §B.1 · p. 34
- The experiments use 300-dimensional Google News word2vec embeddings and restrict vocabulary to the most frequent words for computational feasibility.The vocabulary caps are 100,000 words for the 20-document dataset and 30,000 for the 50-document dataset.Source: §B.1 · p. 34
B.2 Inference Mechanism Implementation Details
The inference implementations use separate procedures for author identification and topic classification. Table 2 reports distances to nearest known-author texts and their authors.
B.2 Inference Mechanism Implementation Details
- Author identification uses the algorithm of [27] without training, with a 400-character minimum training text length and mostly inherited hyperparameter settings.The settings follow a replication used as a starting point, based on the empirical analysis of.Source: §B.2 · p. 34
- Topic classification uses pretrained embeddings and a fastText model trained for 25 epochs, reaching 0.937 validation accuracy.Training used 448 known-author texts of 2,000 words each, split evenly between two topics; validation used 111 texts.Source: §B.2 · p. 34
- Table 2 lists each snippet’s distances to its nearest known-author texts and the corresponding authors’ identities.d_i is the distance to the ith-closest known-author text; a_i is the identity of the ith-closest author.Source: Table 2 · p. 35
B.3 Analysis of Examples
The examples show that the privacy mechanism changes snippet wording to different degrees as ε varies. Some substitutions are semantically close, while others can produce unlikely terms that may look out of place in-domain.
B.3 Analysis of Examples
- For a sample of unknown-author snippets, the three closest distances are relatively close, which the passage attributes to high-dimensional Euclidean space.Table 2 shows these distances and their authors.Source: §B.3 · p. 35
- At ε = 30, the example snippet has mostly minor, semantically close substitutions, while lower ε introduces more randomness and unlikely words.Examples include felt → feels and uncle → grandma at ε = 30, and ENERGY STAR at lower ε.Source: Table 3 · p. 35, §B.3 · p. 36
- Unlikely phrases such as ENERGY STAR could look out of place in-domain, but a practical application could tailor the vocabulary by removing entries absent from training data.Table 3 shows the beginnings of one snippet under different ε levels.Source: Table 3 · p. 35, §B.3 · p. 36