Shadow Queries for Private Retrieval in Vector Databases

• Embedding inversion attacks can reconstruct text from stored embeddings, exploiting their strong semantic coupling to the underlying documents [41], [24], [17].Access to cloud-hosted embeddings can expose private documents through breaches or insider threats.Source: §I · p. 1
• Noise defenses are constrained by retrieval-utility loss, while secret scaling can be nullified by vector normalization [24], [60].These limitations expose a tension between privacy protection and retrieval effectiveness.Source: §I · pp. 1–2
• SHAQ uses shadow-query embeddings to decompose document semantics into diverse facets and decouple stored vectors from source text.A generative language model creates plausible user queries covering document aspects; the resulting embeddings replace document embeddings, preserving retrieval through semantic matches while avoiding storage of the full document semantics.Source: §I · p. 2
• SHAQ generates shadow queries with a crafted prompt and K-Means clustering, then encodes and randomly indexes them for retrieval-time mapping to documents.This two-stage generation and indexing pipeline is presented as compatible with variable-length documents and different embedding models, with offline processing before retrieval.Source: §I · p. 2
• Recovery averaged 0.2710 (minimum 0.2104 ROUGE-1), adaptive recovery averaged 0.3376 (minimum 0.2083 ROUGE-1), and utility reached NDCG 0.8524.SHAQ reduced recovered tokens by 12.43% on average and up to 19.50% versus prior defenses, while maintaining retrieval performance on par with an undefended system.Source: §I · p. 2

II. BACKGROUND

• II. BACKGROUND introduces the knowledge needed to understand SHAQ.The section then covers information retrieval and embedding inversion attacks and defenses.Source: §II · p. 2

A. Information Retrieval: From Sparse to Dense

• Sparse retrieval uses lexical overlap, such as TF-IDF and BM25, while transformer embedding models [5], [19] encode text as high-dimensional vectors.Sparse methods are efficient but can miss semantic matches when queries and documents use different wording.Source: A. Information Retrieval: From Sparse… · p. 3
• Dense retrievers such as DPR [14] and GTR [26] compare query and document embeddings in a shared vector space to rank semantically similar documents.Retrieval selects the top-k documents according to an embedding similarity function, such as cosine similarity or dot product.Source: A. Information Retrieval: From Sparse… · p. 3
• Vector databases store pre-computed document embeddings so retrieval avoids re-encoding, and cloud-based commercial services are typically used.The passage identifies and as examples of commercial solutions.Source: A. Information Retrieval: From Sparse… · p. 3
• RAG uses a user prompt to retrieve relevant documents, which are concatenated with the prompt as context for LLM generation.Source: A. Information Retrieval: From Sparse… · p. 3

B. Embedding Inversion Attacks and Defenses

• Embedding inversion attacks [41], [24], [17] reconstruct text from embeddings, assuming access to the embeddings and their encoding model.Attacks include gradient-based optimization and generative models such as vec2text [24] and GEIA [17]; the paper targets the risk posed by vec2text on document embeddings in vector databases.Source: B. Embedding Inversion Attacks and… · p. 3
• Noise defenses [24] are constrained by retrieval utility, while secret scaling [60] can be bypassed by vector normalization; both directly modify embeddings.The passage identifies semantic coupling between embeddings and source text as a limitation of existing defenses.Source: B. Embedding Inversion Attacks and… · p. 3

III. APPROACH

• III. APPROACH outlines SHAQ’s threat model, defense overview, and two core components.Source: §III · p. 3

A. Threat Model

• The assumed adversary is honest-but-curious, can access stored embeddings and obtain arbitrary-text embeddings from the model, and aims to reconstruct the associated documents.Access may arise through insider threats or data breaches at the third-party host.Source: A. Threat Model · p. 3

B. Overview

• SHAQ generates plausible retrieval queries spanning document facets, then encodes them to replace document embeddings and decouple stored vectors from source text.The overview describes generation and embedding indexing as the method’s two core components.Source: B. Overview · pp. 3–4
• Generated shadow-query embeddings are indexed in randomized order alongside a mapping from each shadow query to its corresponding document.Source: B. Overview · p. 4

C. Shadow Query Generation

• By decomposing a document into multiple shadow queries, generation avoids directly exposing its content; their embeddings replace the document embeddings during indexing.Source: C. Shadow Query Generation · p. 4
• For each document, SHAQ constructs a prompt using the desired candidate count, then uses a generative language model to produce candidate shadow queries.The prompt guides the model toward diverse, concise queries that capture semantic facets of the document.Source: C. Shadow Query Generation · p. 4
• SHAQ clusters the generated candidates into nk groups with K-Means and randomly selects one query from each cluster for the final set.This procedure encourages coverage of diverse semantic aspects.Source: C. Shadow Query Generation · p. 4

D. Embedding Indexing

• SHAQ replaces document embeddings with embeddings of each document’s selected shadow queries, storing vectors in V and embedding-to-document ID mappings in S.Each shadow query is encoded with ϕ, its document ID is recorded, and the embedding is added to the vector database.Source: D. Embedding Indexing · pp. 4–5
• Because each stored embedding encodes a semantic facet rather than the original document embedding, EIA reconstruction no longer directly yields the original text while retrieval can remain effective.The indexing step is designed to replace document embeddings with shadow-query embeddings that still encode document semantics.Source: D. Embedding Indexing · p. 4
• SHAQ randomly shuffles shadow-query embeddings rather than pooling them, avoiding utility degradation from semantic interference and mitigating adaptive attacks that could group a document’s embeddings.The scattering strategy is intended to make it harder for an attacker to identify all embeddings corresponding to the same document.Source: D. Embedding Indexing · p. 5

IV. EXPERIMENTAL EVALUATION

• The evaluation tests SHAQ’s semantic decomposition and embedding decoupling, retrieval utility, defense efficacy, ablations, and robustness to a challenging adaptive attack.The paper assigns these evaluations to Sections IV-B through IV-F, respectively.Source: §IV · p. 5

A. Experimental Settings

• The experimental settings cover attack and defense baselines, dense retrieval, evaluation metrics, and SHAQ’s settings, with implementation discussion in Appendix C.Source: A. Experimental Settings · p. 5

1) Baselines of Attack and Defenses:

• The attack baseline is vec2text [24], which uses T5-based generative models [35] trained on embedding-text pairs from Natural Questions or MS MARCO to reconstruct documents.The training documents are truncated to 32 tokens.Source: 1) Baselines of Attack and Defenses · p. 5
• Embedding noise [24] and secret scaling [60] are defense baselines, but secret scaling is treated as no defense because vector normalization nullifies its effect.Source: 1) Baselines of Attack and Defenses · p. 5

2) Settings of Dense Retriever:

• The task ranks corpus documents against user queries by cosine similarity or dot product, using FAISS [7]; downstream question answering and generation are out of scope.GTR-T5 produces normalized embeddings and is used throughout for compatibility with the attack model, with cosine similarity as the objective.Source: 2) Settings of Dense Retriever · p. 5
• SHAQ is evaluated on diverse BEIR datasets [46] using development or test splits to avoid potential information leakage from the attack model’s training.The evaluated datasets’ statistical details are presented in Table I.Source: 2) Settings of Dense Retriever · p. 5, Table I · p. 5
• The evaluation samples 100 queries per dataset, excludes nonbinary-label or excessively high-relevance-count datasets, and augments relevant documents with BM25’s top-100 ranked documents.All relevant documents are included for each sampled query; sampled document counts vary across datasets because documents can serve multiple queries.Source: 2) Settings of Dense Retriever · p. 6
• All sampled documents are truncated to 32 tokens before shadow-query generation to match vec2text’s fixed-length setup and avoid an advantage from longer, harder-to-recover texts.Source: 2) Settings of Dense Retriever · p. 6

3) Evaluation Metrics:

• Utility is evaluated with top-10 NDCG, MAP, Recall, Precision, and Accuracy; defense efficacy uses BLEU, ROUGE, and METEOR.Utility metrics range from 0 to 1, with higher values better; defense metrics range from 0 to 1, with lower similarity indicating better defense.Source: 3) Evaluation Metrics · p. 6

4) Settings of

• SHAQ generates shadow queries with QwQ-32B in 8-bit quantization to reduce memory requirements.The paper selects the model for its generation quality and capacity to produce diverse shadow queries.Source: 4) Settings of · p. 6
• The Scifact search selected nk = 10 shadow queries per document and embedding-noise σ = 0.01 for comparable utility and a fair defense-efficacy comparison.The paper reports that nk = 10 yields the best utility–privacy balance and that σ = 0.01 yields utility comparable to SHAQ.Source: 4) Settings of · p. 6

B. Demonstrating Semantic Decomposition and Embedding Decoupling

• A proof-of-concept experiment examines whether SHAQ achieves semantic decomposition and embedding decoupling.The passage introduces the experiment but does not report its results.Source: B. Demonstrating Semantic… · p. 6

2) Embedding Decoupling:

• Shadow query embeddings reach up to 0.8310 cosine similarity with document embeddings, while lexical overlap falls as low as 0.0023 BLEU and 0.1015 METEOR.Other reported lexical minima are 0.1949 R-1, 0.0489 R-2, and 0.1561 R-L, supporting semantic relevance without exposing full document content.Source: 2) Embedding Decoupling · p. 7

C. Retrieval Utility Preservation

• SHAQ’s changes from undefended performance range from −0.0747 to +0.0382 for NDCG and −0.0778 to +0.0390 for MAP, with competitive retrieval effectiveness overall.Retrieval ranks shadow queries by cosine similarity, then maps them to original documents using embedding-to-document-ID links.Source: C. Retrieval Utility Preservation · p. 7

D. Defense Efficacy

• SHAQ achieves the lowest reconstruction scores across all metrics and datasets, outperforming embedding noise by up to −0.1949 in R-1 and −0.1689 in METEOR.The evaluated attack reconstructs a document from one embedding, with SHAQ targets sampled as shadow queries from distinct documents.Source: D. Defense Efficacy · p. 8
• The privacy evaluation samples 100 indexed embeddings; SHAQ uses one shadow query embedding per distinct document, while an adaptive multi-shadow-query attack is discussed separately.The setup assumes the attacker is unaware of SHAQ’s defense mechanism.Source: D. Defense Efficacy · p. 8
• Embedding noise also reduces reconstruction accuracy, but substantial document content remains recoverable in most cases, leaving residual privacy concerns.Its reconstruction-score reductions from undefended performance range from −0.2653 to −0.8581 for BLEU and −0.3127 to −0.5842 for R-1.Source: D. Defense Efficacy · p. 8

E. Ablation Studies

• SHAQ maintains robust retrieval utility when documents are untruncated and when the embedding model uses an alternative similarity measure.The ablations indicate generality to variable-length documents and model-agnostic behavior; full details are in Appendix G.Source: E. Ablation Studies · p. 8

F. Adaptive Attack

• 0.2083 R-1 was the lowest attack efficacy observed against SHAQ under an adaptive attack with knowledge of its mechanism, parameters, and generation LLM.The attacker was assumed to know the observed vectors represented shadow queries and to recover those queries before reconstructing the document.Source: F. Adaptive Attack · p. 8

V. RELATED WORK

• Related work spans embedding inversion attacks, EIA defenses, and document/query generation techniques.These prior approaches provide context for SHAQ’s focus on privacy in vector databases.Source: §V · p. 8, A. Existing EIAs · p. 8, B. Existing Defenses · pp. 8–9

A. Existing EIAs

• Gradient-based EIAs optimize dummy token embeddings, while generation-based GEIA and vec2text decode text from embeddings, with vec2text targeting document-level vectors.Song et al. [41] describe the gradient-based approach; Li et al. [17] introduce sentence-level GEIA, and Morris et al. [24] propose document-level vec2text.Source: A. Existing EIAs · p. 8

B. Existing Defenses

• Noise defenses offer limited utility-privacy balance, and secret scaling can be defeated by vector normalization; SHAQ instead replaces document embeddings with shadow-query embeddings.The scaling method is proposed by Zhuang et al. [60], while Morris et al. [24] inject noise.Source: B. Existing Defenses · pp. 8–9

C. Document/Query Generation

• Unlike doc2query, docTTTTTquery, and query2doc, which pursue retrieval or training utility, SHAQ’s query-generation pipeline targets document privacy.Nogueira et al. [28] propose doc2query and later docTTTTTquery; Wang et al. [53] introduce query2doc.Source: C. Document/Query Generation · p. 9

VI. CONCLUSION

• SHAQ breaks the embedding-text connection through embedding decoupling while preserving document semantic facets through semantic decomposition.Its central design replaces database document embeddings with shadow-query embeddings, shifting from direct embedding modification to decomposition and decoupling.Source: §VI · p. 9

A. Shadow Query Generation Prompt Template

• The prompt asks a language model to generate concise, natural, semantically diverse queries that retrieve a target document without repeating query meanings.Source: Shadow query generation prompt template · p. 10

B. Generated Shadow Queries

• Generated shadow queries cover most target-document content in compact-fact or question formats that resemble plausible retrieval-time queries.Source: B. Generated Shadow Queries · p. 10

C. Empirical Implementation

• Experiments used Python 3.12.3, PyTorch 2.3.0, Transformers 4.44.2, and NVIDIA H100 GPUs, with a fixed random seed across runs.The experiments ran on a Rocky Linux high-performance computing cluster.Source: C. Empirical Implementation · p. 11

D. Comprehensive Details of Evaluated Datasets

• The evaluation spans SciFact [52] scientific fact verification, Natural Questions open-domain QA, MS MARCO web-passage retrieval, and NFCorpus biomedical retrieval.Their queries are respectively scientific claims, real search questions, Bing queries, and non-technical NutritionFacts.org questions.Source: D. Comprehensive Details of Evaluated… · p. 11
• Other benchmarks include HotpotQA [58] Wikipedia-based QA, FiQA finance retrieval, ArguAna counter-argument retrieval, and Quora Question Pairs semantic equivalence.Quora uses one question to retrieve another and assesses whether they are semantically equivalent.Source: D. Comprehensive Details of Evaluated… · p. 11
• Fact-verification benchmarks also include FEVER [47], using mutated Wikipedia sentences and Wikipedia pages, and climate-focused CLIMATE-FEVER [6].Source: D. Comprehensive Details of Evaluated… · p. 11

E. Excluded Datasets from Evaluation

• Excluded datasets’ split and statistics are summarized in Table V, with exclusion grounds including many relevant documents per query, non-binary relevance scores, and non-standard IR status.The supplied passages do not identify which specific dataset matches each ground.Source: Table V · p. 11, E. Excluded Datasets from Evaluation · p. 11

F. Hyperparameter Search

• SHAQ generates 10 shadow queries per sampled document, selects one per K-Means cluster, and uses nk = 10 as the best utility-privacy trade-off.The authors attribute query diversity to prompt design and the generative model; with more computational resources, generating more initial queries is anticipated to improve retrieval performance.Source: F. Hyperparameter Search · p. 11
• σ = 0.01 yields utility comparable to SHAQ at nk = 10, while larger noise levels degrade retrieval utility, consistent with prior suggestions [24] [60].The search tested σ ∈ {0.001, 0.01, 0.2}.Source: F. Hyperparameter Search · pp. 11–12

G. Ablation Studies

• The ablations examine full-length documents and DPR embeddings, but the fixed attack model supports only truncated documents and a fixed embedding model, precluding attack-efficacy evaluation.The main experiments truncate documents to 32 tokens to match vec2text’s training constraint; full-length documents are used to assess utility in a production-like setting.Source: G. Ablation Studies · p. 12, H. Retrieval Utility on Untruncated… · p. 12, §I · p. 12

H. Retrieval Utility on Untruncated Documents

• +0.3684 NDCG, +0.3556 MAP, +0.3934 Recall, +0.0430 Precision, and +0.4000 Accuracy are SHAQ’s gains with full-length versus truncated documents.The full-length setting also outperforms the undefended and embedding-noise settings; the attack model cannot handle variable-length inputs, so defense efficacy is not assessed.Source: H. Retrieval Utility on Untruncated… · p. 12
• Using full-length documents substantially improves retrieval utility across defense scenarios; SHAQ has the largest gains, attributed to shadow queries generated from more comprehensive content.The full-length document setting is evaluated on SciFact and improves utility relative to the truncated counterpart.Source: H. Retrieval Utility on Untruncated… · p. 12

I. Retrieval Utility on Different Embedding Model

• With DPR embeddings, all three scenarios have lower retrieval performance than with GTR-T5 but remain on par with one another, supporting SHAQ’s model-agnostic integration into dense retrievers.GTR-T5 uses a cosine-similarity objective, whereas DPR uses a dot-product objective; the passage attributes the overall decline mainly to DPR’s poorer semantic embedding ability.Source: §I · p. 12

J. Adaptive attack

• The attacker knows SHAQ’s mechanism, the number of shadow queries per document, and has access to the LLM used to generate them.The attacker also knows that the observed database embeddings correspond to shadow queries for target documents.Source: J. Adaptive attack · p. 12
• The attacker groups likely shadow-query embeddings using cosine similarity, reconstructs them individually, then prompts the same LLM to piece together the original document.SHAQ scatters each document’s shadow-query embeddings in the database, so the attacker locates the other queries by selecting the nk −1 embeddings most similar to an observed one.Source: J. Adaptive attack · pp. 12–13
• Under adaptive attack, BLEU ranges 0.0151–0.1446, R-1 0.2083–0.4517, R-2 0.0511–0.2192, R-L 0.1665–0.3530, and METEOR 0.1822–0.3807.The results are attributed to shadow queries covering distinct semantic facets; SHAQ withstands this informed attack compared with an undefended system or one using embedding noise.Source: Table IX · p. 13, J. Adaptive attack · p. 13