PaperlayerView exact revision on arXiv

Source-linked AI summary

Private Retrieval Augmented Generation via Random Projection

Dixi Yao, Tahseen Rabbani, Tian Li

arXiv:2610.22297v1cs.AIcs.CR

TL;DR

RAG can expose sensitive information in its external datastore, while DP defenses may harm output quality. This paper applies the same random projection to datastore and query embeddings; it reports over 50% average utility improvement over baseline methods without degrading empirical privacy.

  • Problem

    Carefully crafted prompts can extract sensitive personal information from external RAG datastores, while DP defenses may degrade output quality and complicate implementation.

    Source: §1 · p. 1

  • Method

    The method applies the same random projection to datastore and query embeddings, aiming to preserve pairwise similarities while altering datastore representations.

    Source: §1 · p. 1

  • Results

    Over 50% average utility improvement across datasets over baseline methods, without degrading empirical privacy performance.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Random projection is presented as a competitive practical baseline for privacy-preserving RAG, with empirical protection comparable to a parametric model without a RAG datastore and a 7% overall utility decrease versus vanilla RAG.

    Source: §5 · p. 8

    Large-scale evaluation in real-world LLM-based RAG remains future work; at about 8 seconds per query, one million queries would take 92 days without system-level optimization.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Retrieval-Augmented Generation (RAG) enhances the capabilities of large language models (LLMs) by querying external structured knowledge. However, it can also introduce privacy risks by leaking sensitive information from the retrieval datastore. We propose a simple method to preserve datastore privacy in RAG systems via random projection. By applying the same projection to both datastore embeddings and query embeddings, our method provably preserves semantic similarity between queries and retrieved items while substantially mitigating data extraction attacks. Across multiple RAG architectures and datasets, we show that this lightweight approach achieves superior retrieval and generation performance compared to prior methods which leverage formal differential privacy (DP) guarantees, while exhibiting comparable empirical privacy under strong attack models. Our results for the first time suggest that random projection can serve as a competitive and practical baseline for privacy-preserving RAG systems.

1 Introduction

RAG can expose sensitive datastore information through crafted prompts, while differential privacy defenses may reduce output quality and complicate deployment. The paper applies shared random projections to query and datastore embeddings and reports over 50% average utility improvement over baseline methods without degrading empirical privacy.

1 Introduction

  • Carefully crafted prompts can extract sensitive personal information from external RAG datastores, and simple access control may block ordinary users.Attack queries can closely resemble legitimate user queries [3] [4].Source: §1 · p. 1
  • The method applies one random projection to datastore embeddings offline and the same matrix to user queries online, preserving pairwise similarities in lower dimensions.The added randomness alters datastore embeddings, with the aim of hindering recovery of original text.Source: §1 · p. 1
  • The evaluation considers KNN-LM and Standard RAG, measuring privacy leakage through success rates of strong existing attacks rather than formal guarantees.The paper focuses on settings where formal privacy guarantees are not desired.Source: §1 · p. 1
  • Over 50% average utility improvement across datasets over baseline methods, without degrading empirical privacy performance.The paper reports this as the overall result of its random-projection technique.Source: §1 · p. 2

2 Related Works

Prior defenses for private RAG include differential privacy, synthetic data, local privacy, and document sanitization, each with practical or coverage limitations. The paper situates random projection among methods that reduce dimensionality while preserving similarity measures with high probability.

Private RAG

  • Existing defenses include DP sampling and aggregation, synthetic data, local differential privacy, and document mixing, but can reduce efficiency or require extra validation, training, or private-token identification.Synthetic data alone may not fully prevent memorization or leakage, while token masking can miss sensitive information spanning sentences or paragraphs [15].Source: §2 · p. 2

Privacy-Preserving Random Projection

  • Random projection commonly reduces dimensionality while preserving some similarity measures with high probability, and prior projection-based methods have achieved DP in specific applications.The paper applies random projection to a variety of RAG scenarios [16, 17, 18, 19, 20, 21, 22, 23, 24, 25].Source: §2 · p. 2

3 Private RAG via Random Projection

The method is evaluated under a black-box API threat model with the projection matrix undisclosed and white-box and system-level attacks out of scope. It projects datastore embeddings offline and queries online, then retrieves in projected space for RAG generation.

3.1 Threat Model

  • The black-box attacker sees API responses but not model parameters, the projection matrix, logits, embeddings, or latent representations; white-box and network/system-level attacks are out of scope.Attackers may submit repeated or multi-round adaptive queries, but cannot access embeddings or the projection matrix directly.Source: §3.1 · p. 2

3.2 Algorithm

  • The corpus contains natural-language documents embedded by a fixed pretrained encoder, and the attacker interacts through API queries constructed to extract datastore content.The paper assumes a fixed encoder for simplicity.Source: §3.2 · p. 2
  • A Gaussian projection matrix is applied once to datastore embeddings offline and to each query online, enabling nearest-neighbor retrieval in the lower-dimensional space.Datastore projections are precomputed; each query requires one matrix-vector multiplication.Source: §3.2 · p. 3
  • Standard RAG concatenates the retrieved text with the original query before generation, while KNN-LM combines neighbor outputs with language-model outputs.For Standard RAG, projected-query similarity determines the retrieved text [28, 29, 31].Source: §3.2 · p. 3
  • Random projection can select embeddings near the original top match, allowing alternative tokens to replace the token associated with the original retrieval.The paper attributes this privacy effect to changes in the retrieval process.Source: §3.2 · p. 3

4 Evaluation

Across repeated-query and multi-round attacks, random projection (RP) provides strong empirical privacy while preserving utility. The evaluation also examines projection-dimension choices, efficiency and scalability, and why differential-privacy constraints can reduce utility.

4.2 Repeated-Query Attacks

  • RP maintains utility close to Standard RAG while leaking almost no private information under repeated-query attacks; DP-RP-G has strong privacy but unsatisfactory utility, and DPSparseVoteRAG lacks a balanced trade-off.Standard RAG leaks over 200 personal items in Enron; increasing DPSparseVoteRAG’s privacy budget reduces perplexity but increases leakage.Source: §4.2 · p. 5
  • RP keeps leakage below 3% for Enron email addresses, URLs, and phone numbers, and at 2.19–2.27% on ECHR legal narratives; URLs are easiest to leak.The reported Enron rates cover both KNN-LM and Standard RAG.Source: §4.2 · p. 5, Table 3 · p. 5

4.3 Multi-Round Adaptive Attacks

  • Under multi-round attacks, the three privacy-preserving methods have similar utility, RP attains the highest accuracy, and differential-privacy methods degrade retrieval performance.The evaluation uses chunk recovery rate and extended edit distance to assess extraction; high recovery and low EED indicate successful extraction.Source: §4.3 · p. 6, Table 5 · p. 6, Table 6 · p. 6
  • In a multi-round case study, RP avoids revealing sensitive identity details while preserving answers, although it returns incorrect birthdates that may lead an attacker to stop.Standard RAG reveals a large portion of the original content; the cited comparison also reports extraction by Jiang et al. [9] and the knowledge-graph method.Source: §4.3 · p. 6

4.4 Effects of k

  • Larger projection dimension k improves utility while privacy remains unchanged; smaller k is suggested for more privacy or lower offline latency, and larger k for higher utility.For d = 2048, the authors give k = 25, 100, and 1600 as examples for smaller, moderate, and larger choices, respectively.Source: §4.4 · pp. 6–7, Figure 3 · p. 7
  • Sampling variances larger or smaller than 1/k degrade utility: at k = 1600 and variance 0.22, perplexity is 70.86 with 2.32% leakage; at k = 25 and variance 0.0252, it is 71.38 with 3.1% leakage.The authors conclude that setting the sampling variance to 1/k is optimal.Source: §4.4 · p. 7

4.5 Efficiency and Scalability

  • DPSparseVoteRAG inference takes about 30× longer than RP and vanilla RAG, whereas RP matches RAG’s latency per token; RP also maintains strong privacy as datastore size grows.The multi-round evaluation includes datastores up to about 100K HotpotQA documents, or roughly 200 million tokens; in the ECHR/Qwen3-4B scalability study, vanilla RAG leakage decreases as the datastore grows.Source: §4.5 · pp. 7–8, Table 8 · p. 8, Figure 4 · p. 8

4.6 Why Peformance Degrades More by Differential Privacy Methods

  • For DP-RP-G, larger clipping bounds reduce perplexity and sufficiently large bounds recover performance close to RP; utility loss is attributed to formal-DP constraints rather than RP itself.DP aggregation methods also face a perturbation-magnitude trade-off: looser privacy budgets can improve utility while leaving non-negligible leakage.Source: §4.6 · p. 8, Figure 5 · p. 8

5 Conclusion

The paper proposes random projection to defend RAG datastore privacy against data extraction while retaining semantic utility. It reports protection at approximately the parametric model’s level, with a 7% overall utility decrease versus vanilla RAG.

5 Conclusion

  • Random projection empirically protects against data reconstruction at approximately the parametric model without a RAG datastore’s level while maintaining semantic utility.The reported overall utility decrease is 7% compared with vanilla RAG.Source: §5 · p. 8
  • The proposed method is a simple random-projection defense against data-extraction attacks in RAG applications.Source: §5 · p. 8
  • The method maintains semantic utility with a 7% overall decrease compared with vanilla RAG.Source: §5 · p. 8

Limitations

Large-scale evaluation of privacy-preserving methods in real-world LLM-based RAG systems remains future work. Without system-level optimization, the paper estimates one million queries would take 92 days.

Limitations

  • Without system-level optimization, evaluating one million queries would take 92 days, motivating future large-scale evaluation work.The estimate is based on about 8 seconds per query.Source: Limitations · p. 9
  • Large-scale evaluation of privacy-preserving methods in real-world LLM-based RAG systems remains future work.Source: Limitations · p. 9
  • The stated evaluation constraint concerns throughput at real-world scale, where systems may need to process billions of queries daily.Source: Limitations · p. 9

Ethical Considerations

The experiments use publicly available or synthetic datasets and exclude proprietary data and private user information. The accompanying analysis states that sufficiently large random projections preserve cosine similarity in expectation and preserve the L2 distance.

Ethical Considerations

  • The experiments use publicly available or synthetically generated datasets and do not use proprietary data or private user information.The datasets are licensed for non-commercial research use.Source: Ethical Considerations · p. 9

B Preservation of the L2-Distance and Cosine Similarity of Random Projection

B.1 Proof: Preservation of the Cosine Similarity

The proof treats cosine similarity as the expected projected cosine between normalized query and datastore embeddings. Gaussian projection makes the projected coordinates a sample of correlated bivariate normals, yielding expected distortion that decreases as O(1/k).

B.1 Proof: Preservation of the Cosine Similarity

  • The proof normalizes the query and datastore embeddings and expresses their original cosine similarity as their inner product.Source: §B.1 · p. 13
  • Gaussian projection and rotational invariance let the proof represent projected coordinates as samples from bivariate normal variables with correlation ρ.The proof identifies the projected coordinates’ correlation with the original cosine similarity.Source: §B.1 · p. 13
  • The proof connects projected cosine similarity to the sample Pearson correlation of k IID bivariate normal samples.The cited lemma is presented as established in prior literature.Source: §B.1 · p. 13
  • The proof concludes that expected cosine-similarity distortion decreases as O(1/k), so the designed random projection preserves cosine similarity.Source: §B.1 · p. 14

B.2 Synthetic Data

The sanity check tests whether random projection preserves query-to-datastore embedding distances in synthetic and random-vector settings. Its results show that increasing the projection dimension reduces cosine-similarity error and keeps L2 distance ratios within theoretical bounds.

B.2 Synthetic Data

  • The synthetic setting uses LLAMA3-8B-INSTRUCT to generate 100 datastore entries and 100 queries per entry; the random setting samples both embeddings from N(0, 1).Source: §B.2 · p. 14
  • The check averages L2 distance ratios and cosine-similarity differences first over 100 datastores and then across 100 runs.Source: §B.2 · p. 14
  • As k increases, cosine-similarity error approaches zero, while L2 distance ratios remain within the JL bounds and become more stable.The authors also report that larger k improves perplexity.Source: Figure 6 · p. 14, §B.2 · p. 14

C Proof: Probability of Top-1 Selection Changing after Random Projection

The proof analyzes when random projection can change the top-1 retrieval choice, while the accompanying algorithms apply the same projection to datastore and query embeddings. The KNN-LM algorithm combines projected-neighbor predictions with language-model predictions; its DP variant clips projected embeddings and adds calibrated Gaussian noise.

C Proof: Probability of Top-1 Selection Changing after Random Projection

  • After random projection, embeddings sufficiently close to the original top-1 match can overtake it with non-negligible probability, potentially replacing sensitive tokens with less sensitive alternatives.The authors frame this as privacy protection while preserving general semantic content.Source: C Proof: Probability of Top-1… · p. 15

D Complete Algorithms

  • KNN-LM projects datastore embeddings offline and each query online, then combines the language-model and projected-neighbor token distributions using λ = 0.25.The selected token is appended to the context and generation continues.Source: D Complete Algorithms · p. 16, C Proof: Probability of Top-1… · p. 16
  • DP-RP-G clips projected datastore vectors and adds Gaussian noise calibrated to the privacy budget; its implementation adapts prior work because LLM embeddings do not meet the assumed vector bounds.The DP step is applied to datastore embeddings, not queries.Source: D Complete Algorithms · pp. 16–17

E Implementation Details

The implementation details span leakage and utility evaluations on Enron Email, ECHR, C4, and HotPotQA, with model and generation settings specified. Additional experiments report little utility variation across DPSparseRAG m values and comparable vanilla-RAG performance from Rademacher projections.

E Implementation Details

  • The evaluations use Enron Email for structured-identifier leakage, ECHR for legal-document utility and identity leakage, C4 for untargeted leakage, and HotPotQA for retrieval and QA.HotPotQA uses over 90K datastore documents, with 100 target documents selected for targeted attacks.Source: E Implementation Details · p. 17

F.1 General Settings

  • Generation uses repetition penalty 0.75, no-repeat-ngram-size 0, model-specific last-hidden-state embeddings, and a maximum output length of 2048 tokens.Source: §F.1 · pp. 17–18

F.2 DPSparseRAG

  • Across m values from 10 to 60 at ϵ = 5, DPSparseRAG shows no significant utility difference on Enron Email and ECHR.Source: §F.2 · p. 18

G Random Projection with Other Distributions

  • Rademacher-based projections preserve privacy effectively and achieve performance comparable to vanilla RAG on Enron Email.The authors state that effectiveness depends on preserving similarity structure, not on a particular projection-matrix distribution.Source: G Random Projection with Other… · p. 18, Table 11 · p. 18

H.1 False Positive Classification of Non-Sensitive Information

The discussion distinguishes strong protection for unique identifiers from residual leakage and occasional annotation errors involving common terminology. An example comparison reports relevant RAG and proposed-method generations, unlike often incoherent or unrelated DP-based outputs.

H.1 False Positive Classification of Non-Sensitive Information

  • A small number of items may still be leaked across Enron Email, ECHR, and C4, while the reported failures lacked sensitive information.The authors report 100% leakage prevention for numbers, IDs, and email addresses, but common terms may be misclassified as personal identity information.Source: §H.1 · pp. 18–19

H.2 An Example of Generation

  • In the example responses, RAG and the proposed method produce clear, relevant answers, whereas DP-based methods often produce incoherent or unrelated content.Source: Table 12 · p. 18, §H.2 · p. 19

Related papers

Loading 2610.22297v1…