PaperlayerView exact revision on arXiv

Source-linked AI summary

Denoising-Aware Inversion: Revealing Privacy Risks in Noise-Protected Text Embeddings

Yubo Wang, Shujie Cui, James Bailey, Hongzhi Yin, Wenyu Liang, Min Tang, Shiyue Qin, Weiqing Wang

arXiv:2608.18610v1cs.LGcs.AIcs.CR

TL;DR

Gaussian-noise protection may not prevent adaptive attackers from recovering sensitive text from embeddings that preserve semantic and lexical information. DAEI combines unsupervised denoising with generative inversion and achieves approximately 154% higher BLEU than noisy Vec2Text, with 32–60% gains in F1 and ROUGE-L.

  • Problem

    It remains unclear whether Gaussian-noise protection prevents adaptive attackers from recovering sensitive text from released embeddings.

  • Method

    DAEI combines an unsupervised denoising autoencoder with generative inversion to recover text from noisy embeddings without clean targets.

  • Results

    Approximately 154% relative BLEU improvement and 32–60% gains in F1 and ROUGE-L over noisy Vec2Text were observed across benchmarks and embedding backbones.

  • Takeaways & Limitations

    Gaussian noise alone is insufficient to prevent textual information leakage, motivating evaluation against adaptive denoising-aware attacks.

Abstract

from arXiv · show

Dense text embeddings are widely used in data mining, retrieval, and downstream machine learning systems due to their compact and semantically rich representations, but recent embedding inversion attacks have shown that they can expose substantial information about the original text, leading to serious privacy leakage risks. A common defense is to release perturbed embeddings by adding Gaussian noise, which is simple yet effective against standard inversion attacks and does not significantly degrade embedding utility for downstream tasks. However, it remains unclear whether such noise-protected embeddings are sufficiently safe against adaptive attackers that explicitly account for the perturbation process. In this paper, we study text embedding inversion in a noise-protected setting, where the attacker can observe only noisy embeddings and has no access to clean embedding targets. We first analyze why existing generative inversion methods fail under this setting and identify a "Double Noise Trap", which fundamentally prevents standard generative inversion models from achieving high-quality reconstruction. To address this challenge, we propose DAEI, a denoising-aware embedding inversion pipeline that combines a residual denoising autoencoder with generative text inversion where the denoiser is trained in an unsupervised manner using Stein's unbiased risk estimate to enable denoising from noisy observations alone. Extensive experiments show that DAEI achieves approximately 154\% relative improvement in BLEU over the existing generative inversion baseline, while also improving token-level F1 and ROUGE-L by 32--60\%. The promising inversion performance of DAEI challenges the prevailing assumption that simple Gaussian perturbation is sufficient to prevent sensitive information leakage from embedding representations.

I. INTRODUCTION · II. PROBLEM FORMULATION AND MOTIVATION · A. Problem Formulation

The paper shows that Gaussian noise does not eliminate privacy leakage from text embeddings: adaptive inversion remains possible when attackers observe only noisy vectors and lack clean targets. It formulates this no-clean-embedding threat model, identifies the Double Noise Trap, and introduces DAEI, which substantially improves reconstruction over noisy inversion baselines.

  • I. INTRODUCTION: Dense embeddings preserve semantic, lexical, and potentially sensitive information, making their release vulnerable to extraction attacks that recover underlying text.Generative attacks can reconstruct full sentences rather than only unordered keywords.
  • I. INTRODUCTION: Adding perturbation can preserve downstream utility while reducing disclosure, but conventional generative inversion methods remain largely ineffective on noise-protected embeddings.Search-based inversion offers some robustness, yet recovered text is often less readable and weakly token-aligned.
  • II. PROBLEM FORMULATION AND MOTIVATION: No-clean-target inversion is fundamentally different because attackers observe only noisy vectors, invalidating supervised denoising against clean references and requiring separation of signal from noise.The paper frames this as an underexplored inversion paradigm rather than merely a harder standard task.
  • I. INTRODUCTION: DAEI couples a residual denoising autoencoder with generative inversion and supports denoising-aware pre-training, joint fine-tuning, and correction without clean embedding targets.The pipeline builds on Vec2Text and uses unsupervised Stein’s unbiased risk estimation for denoising from noisy observations alone.
  • II. PROBLEM FORMULATION AND MOTIVATION: The paper identifies a “Double Noise Trap”: Gaussian perturbation moves embeddings off the clean text manifold, causing vanilla inversion to pursue off-manifold directions.DAEI instead projects noisy representations toward the clean manifold before inversion.
  • I. INTRODUCTION: 154% relative improvement in BLEU is achieved by DAEI over noisy Vec2Text, alongside 32–60% gains in F1 and ROUGE-L and substantial improvement over ZSInvert.These results span multiple benchmarks and embedding backbones, challenging the assumption that noise guarantees embedding privacy.
  • A. Problem Formulation: The threat model gives attackers black-box query access, knowledge that perturbation is additive Gaussian noise, and either the noise scale σ or the ability to estimate it.The clean embedding z remains strictly inaccessible, and attackers may train on embedding–text pairs (y, x).
  • A. Problem Formulation: Given a noisy embedding y and black-box embedder f, the stated objective is to reconstruct the original text x, while directly training on noisy pairs is theoretically analyzed as insufficient.The formulation distinguishes the protected hidden text from attacker-chosen calibration inputs.

B. Vec2Text Fails due to “Double Noise Trap”

Under noisy embeddings, Vec2Text’s correction compares independently perturbed target and hypothesis embeddings, creating a doubled-variance residual that can overwhelm semantic alignment. This “Double Noise Trap” motivates denoising before generation and correction.

  • Mechanism: Vec2Text models pθ(x | z) and refines text hypotheses by comparing the target embedding with each hypothesis embedding.The inverter generates hypotheses, while the corrector uses embedding comparisons for refinement.
  • Double Noise Trap: Under noisy access, the target and hypothesis embeddings contain independent perturbations, ϵ1 and ϵ2, around clean representations.The observed quantities are ytar = z + ϵ1 and yhyp = fclean(ˆx) + ϵ2, with ϵ1, ϵ2 ∼N(0, σ2Id).
  • Double Noise Trap: The correction residual includes ϵ1 −ϵ2 ∼ N(0, 2σ2Id), doubling the residual noise variance.The correction signal combines the desired semantic residual with noise formed by subtracting two independent perturbations.
  • Double Noise Trap: 0.39 residual noise magnitude arises for a 768-dimensional embedding with σ = 0.01, exceeding the single-noise displacement σ.This larger residual illustrates why correction becomes unreliable even when each individual perturbation is small.
  • Double Noise Trap: As semantic residuals shrink, stochastic cross terms and the doubled noise baseline can dominate, rewarding noise-aligned rather than semantically aligned hypotheses.The noisy target is off the text-induced clean-embedding manifold, making direct correction geometrically misguided.
  • Mitigation: The proposed decoupled design first estimates a denoised clean-manifold representation, then performs generation and correction using denoised embeddings.This design directly addresses the failure caused by comparing noisy target and hypothesis embeddings.

C. Infeasible Supervised Denoising

Without access to clean embeddings, standard supervised denoising cannot be trained because its clean target is unavailable. This constraint motivates an inversion pipeline that reconstructs text using only noisy observations.

  • C. Infeasible Supervised Denoising: The no-clean-embedding constraint fundamentally changes the denoising process.The setting excludes clean embedding targets from the attacker’s available information.
  • C. Infeasible Supervised Denoising: Supervised denoising is infeasible because the clean target z is unavailable.The standard objective depends on a denoising network hϕ trained against clean embeddings.
  • C. Infeasible Supervised Denoising: The limitation necessitates a pipeline that reconstructs original text strictly from noisy observations.The proposed direction addresses denoising without relying on clean embedding targets.

III. METHOD · A. SURE-based Unsupervised DAE

DAEI is a four-stage noisy embedding inversion pipeline that combines a noisy embedder, residual denoising autoencoder, generative inverter, and iterative corrector. Its first stage trains the denoiser without clean embeddings by replacing inaccessible oracle-risk terms with SURE and estimating divergence using Hutchinson probes.

  • III. METHOD: DAEI comprises four stages: SURE-based unsupervised DAE pre-training, DAE-shielded inverter training, joint SURE-CE fin, and iterative correction.The pipeline uses a black-box noisy embedder, denoising autoencoder hϕ, generative inverter θ, and iterative corrector.
  • A. SURE-based Unsupervised DAE: Stage 1 learns a denoising module hϕ that constructs a denoised embedding ˜z from the noisy observation y.The DAE is applied after obtaining a noisy embedding and before the inverter generates an initial hypothetical text.
  • A. SURE-based Unsupervised DAE: The oracle denoising risk cannot be optimized directly because the clean embedding z is unobserved.The inaccessible cross term depends on the hidden clean embedding.
  • A. SURE-based Unsupervised DAE: Under y = z + ϵ with ϵ ∼N(0, σ2Id), Stein’s lemma replaces the inaccessible cross term with quantities depending on y, σ2, and denoiser divergence.This yields SURE, whose minimization is equivalent in expectation to minimizing the oracle denoising risk.
  • A. SURE-based Unsupervised DAE: The denoiser uses residual correction, with rϕ(y, σ) estimating the displacement that suppresses noise and moves observations toward the clean embedding manifold.Substituting the residual parameterization into the divergence term produces the training objective after removing constants independent of ϕ.
  • A. SURE-based Unsupervised DAE: DAEI approximates the residual-network divergence with Monte-Carlo Hutchinson estimation using Rademacher probes.The estimator is unbiased, and spectral normalization in the residual MLP stabilizes the trace estimate.
  • A. SURE-based Unsupervised DAE: The Hutchinson estimator’s variance decays at rate O(1/K).Rademacher probes satisfy E[vkv⊤k ] = I, providing the stated unbiasedness guarantee.

B. DAE-Shielded Inverter

The DAE-Shielded Inverter trains the generative inverter on denoised representations paired with original text, reducing the burden of noise filtering and avoiding overfitting to non-semantic perturbations. Its objective is standard token-level cross-entropy over autoregressive target sequences.

  • DAE-Shielded Inverter: The inverter θ is trained on denoised representations ˜z paired with text x instead of raw noisy embeddings.This prevents the generative model from overburdening its capacity with high-dimensional noise filtering.
  • DAE-Shielded Inverter: Training on (˜z, x) avoids overfitting to non-semantic perturbations in noisy embeddings.The denoising stage separates the inverter’s training input from the raw perturbed representation.
  • Training Objective: The inverter uses the standard token-level cross-entropy loss.The loss is defined over target tokens in an autoregressive sequence.
  • Training Objective: At step t, x_t is the target token and x_<t provides the preceding autoregressive context.T denotes the sequence length.

C. Joint SURE-CE Fine-tuning · D. DAE-Shielded Corrector

The method jointly fine-tunes the denoiser and inverter with SURE and cross-entropy supervision, while PCGrad resolves conflicting DAE updates. A DAE Shield then denoises both correction inputs, keeping iterative refinement on the learned clean manifold.

  • C. Joint SURE-CE Fine-tuning: Stage 3 jointly fine-tunes the DAE and inverter to align unsupervised denoising with downstream generative inversion.This follows their independent training.
  • C. Joint SURE-CE Fine-tuning: The joint objective combines LSURE(y; ϕ) with λCE(t)LCE(θ; ϕ), while λCE(t) is warmed up during early training.The objective is written as LSURE-CE(ϕ, θ) = LSURE(y; ϕ) + λCE(t) LCE (θ; ϕ).
  • C. Joint SURE-CE Fine-tuning: Cross-entropy gradients pass through hϕ(y, σ), giving the DAE semantic feedback while SURE preserves alignment with the denoising manifold.The CE signal also preserves latent directions needed for decoding.
  • C. Joint SURE-CE Fine-tuning: Distinct learning rates are used for the DAE and inverter, with PCGrad applied exclusively to DAE parameters ϕ because the objectives can conflict.PCGrad addresses optimization-direction conflicts between SURE and cross-entropy.
  • C. Joint SURE-CE Fine-tuning: When ⟨gCE, gSURE⟩< 0, the CE gradient is projected onto the normal plane of the SURE gradient before updates are merged.This projection lets text feedback shape the representation without unlearning denoising capabilities.
  • D. DAE-Shielded Corrector: Directly comparing a generated hypothesis with a noisy target during correction triggers the Double Noise Trap.Conventional correctors iteratively refine initial text hypotheses through target-embedding comparison.
  • D. DAE-Shielded Corrector: The DAE Shield passes both the noisy target y and the current hypothesis embedding through the fully fine-tuned denoiser hϕ⋆.This replaces raw-representation comparison with denoised correction inputs.
  • D. DAE-Shielded Corrector: The corrector conditions only on the denoised target ˜z, denoised hypothesis ˜e(k), and their residual difference, constraining refinement to the learned clean manifold.The shield directs alignment toward semantic text directions rather than irreducible noise residuals.

IV. EXPERIMENTAL SETUP … B. Target Embedding Models

The experiments test DAEI against inversion baselines, across embedding backbones, and under in-domain and out-of-domain evaluation. Training uses a capped mixture of three question-answering and retrieval corpora across four pipeline stages, while targets span distinct T5-based and BERT-based architectures.

  • IV. EXPERIMENTAL SETUP: The study evaluates DAEI against vanilla Vec2Text and noise-robust baselines, across embedding backbones and deployment conditions.These goals correspond to research questions on recovery performance, backbone transfer, and robustness.
  • 1) Training set:: Training combines Natural Questions, MS MARCO, and Yahoo Answers, capped at 7M raw-text examples for inversion-dataset construction.The corpora cover real search queries, web passages, human-generated answers, and broad everyday question-answering topics.
  • 1) Training set:: Stage 1 pre-trains the residual DAE using noisy embeddings only, then Stage 2 trains the base inverter on denoised-embedding/text pairs.Stage 2 denoised embeddings are produced by the Stage 1 DAE.
  • 1) Training set:: Stage 3 jointly fine-tunes on noisy-embedding/text pairs with on-the-fly DAE denoising during the forward pass.The DAE is actively tuned during end-to-end optimization.
  • 1) Training set:: Stage 4 trains the iterative corrector on denoised embeddings, hypothetical text, and denoised hypothetical embeddings labeled with the original text.The inputs are generated using the fully tuned Stage 3 DAE so correction operates within the optimal denoised distribution.
  • 2) Evaluation sets:: Evaluation includes held-out in-domain data from the training mixture and out-of-domain data from AG News, Anthropic Toxic Prompts, Python Code Alpaca, Climate-FEVER, and MedMCQA.The unseen corpora assess generalization beyond the training distribution.
  • B. Target Embedding Models: GTR-base uses a T5-initialized bi-encoder optimized for robust zero-shot retrieval, producing structured semantic representations.It is one of the two black-box target embedding models.
  • B. Target Embedding Models: GTE-base is an encoder-only model trained on large-scale contrastive data, complementing GTR as a distinct target backbone.Together, GTR and GTE represent T5-based versus BERT-based architectures and different training paradigms.

C. Baselines and Our Models … 3) Adversarial-Decoding-Based Baseline:

The study compares clean and noisy embedding-inversion settings using vanilla Vec2Text baselines, denoising-aware pipelines, and ZSINVERT as the primary external noisy-inversion baseline. DAEI jointly fine-tunes denoising and inversion, while ZSINVERT instead performs adversarial token optimization.

  • C. Baselines and Our Models: ZSINVERT is the primary external baseline because it is the only known prior work demonstrated effective for noisy text embedding inversion.It uses adversarial decoding rather than the mainstream encoder-decoder architecture.
  • 1) Clean Upper Bound:: CLEAN-TO-CLEAN evaluates vanilla Vec2Text on clean embeddings as the noiseless state-of-the-art recovery reference and theoretical ceiling.This upper bound represents ideal conditions that noisy inversion pipelines aim to approach.
  • 3) Adversarial-Decoding-Based Baseline:: Table I summarizes the systems evaluated in the main experiments, including the training-free ZSINVERT base configuration.For ZSINVERT base, the listed training embedding type applies only to corrector training.
  • 2) Generative Inversion under Noise:: CLEAN-TO-NOISY evaluates vanilla Vec2Text trained only on clean embeddings but tested on noisy embeddings, exposing severe distribution shift.The baseline measures performance when the inversion model does not adapt to perturbed inputs.
  • 2) Generative Inversion under Noise:: NOISY INVERTER trains vanilla Vec2Text directly on noisy embeddings, adapting to the corrupted distribution without a denoising mechanism.It provides a stronger noise-adapted baseline than CLEAN-TO-NOISY.
  • 2) Generative Inversion under Noise:: DAE-ONLY first denoises embeddings with a DAE and trains an inverter on the resulting cache, but omits joint fine-tuning.This isolates the effect of sequential denoising followed by inversion.
  • 2) Generative Inversion under Noise:: DAEI jointly fine-tunes the DAE and inverter as the complete proposed pipeline.Joint optimization distinguishes DAEI from the DAE-ONLY configuration.
  • 3) Adversarial-Decoding-Based Baseline:: ZSINVERT frames inversion as adversarial decoding, iteratively optimizing discrete tokens to minimize embedding distance.This provides a non-generative-paradigm comparison for the main experiments.

D. Metrics. … A. Overall Performance (RQ1 & RQ2)

The paper evaluates inversion quality with text-overlap and embedding-similarity metrics, using specified denoising and inversion configurations. DAEI achieves the strongest noisy-inversion results across embedding backbones, while noisy training alone provides limited gains and ZSINVERT poorly recovers text despite high embedding similarity.

  • D. Metrics.: Evaluation reports sentence-level BLEU, token-level F1, ROUGE-L, and cosine similarity between reconstructed and reference-text embeddings.BLEU measures n-gram overlap, token-level F1 measures unigram keyword recovery, and ROUGE-L evaluates structural similarity.
  • D. Metrics.: DAE evaluation additionally reports MSE and embedding cosine similarity against clean embeddings used only for offline evaluation, never training or optimization.
  • E. Implementation Setting: The default setup uses prior-known Gaussian noise with σ = 0.01, while the DAE uses learning rate 1×10−3, hidden dimension 1024, depth 3, spectral normalization, and five Monte-Carlo probes.
  • E. Implementation Setting: All inverters use Vec2Text with a T5 backbone and learning rate 1 × 10−3; training truncates texts to 128 tokens and evaluation to 32 tokens.
  • E. Implementation Setting: Joint fine-tuning reduces the DAE learning rate to 2 × 10−4 and linearly warms the cross-entropy loss weight over 10,000 steps.
  • E. Implementation Setting: Correctors use learning rate 5×10−4 and stop refinement when consecutive-step cosine-similarity gains fall below 1×10−3, limiting residual-noise overfitting.
  • A. Overall Performance (RQ1 & RQ2): 154.4% relative BLEU improvement: on GTR-base, DAEI raises BLEU from 0.1916 to 0.4875 over the NOISY INVERTER baseline and also improves token-level F1 from 0.5814 to 0.7717.
  • A. Overall Performance (RQ1 & RQ2): 5–8 points: noisy training improves reconstruction only modestly over CLEAN-TO-NOISY on GTR-base, while DAEI recovers 78.0%/89.2% of clean-upper-bound BLEU/F1 on GTR-base and 71.7%/83.2% on GTE-base.ZSINVERT achieves around 0.01 BLEU and 0.08–0.15 F1 and ROUGE-L despite embedding cosine similarity of 0.85–0.96; DAEI shows similar gains on GTE-base, though its overall scores and clean upper bound are lower.

B. Out-of-Domain Generalization (RQ3) · C. Components Effectiveness (RQ4 & RQ5) · 1) DAE Effectiveness:

DAEI generalizes across unseen domains and improves inversion by denoising noisy embeddings without clean training targets. Its stages serve distinct purposes: Stage 1 prioritizes denoising, while Stage 3 also steers representations toward semantic directions for reconstruction.

  • B. Out-of-Domain Generalization (RQ3): DAEI consistently outperforms NOISY INVERTER and ZSINVERT on BLEU, F1, and ROUGE-L across all out-of-domain datasets.This evaluation is reported in Table IV, whose baseline is NOISY INVERTER and whose upper bound is CLEAN-TO-CLEAN.
  • B. Out-of-Domain Generalization (RQ3): DAEI performs especially well on Anthropic Toxic Prompts, Python Code, and MedMCQA, whereas AG News scores lower for both the clean upper bound and DAEI.AG News examples average about 61 tokens versus 22–30 tokens elsewhere, and truncation to 32 output tokens may limit recovery.
  • 1) DAE Effectiveness:: 29.2% and 23.8% MSE reductions are achieved by the Stage 1 DAE on GTR-base and GTE-base, respectively, without clean embeddings during training.Cosine similarity rises from 0.9179 to 0.9322 on GTR-base and from 0.9662 to 0.9727 on GTE-base.
  • 1) DAE Effectiveness:: MSE reduction and cosine improvement show that DAE shifts noisy embeddings closer to clean counterparts without using clean embeddings as training targets.The in-domain evaluations are summarized in Tables II and III for GTR-base and GTE-base.
  • 1) DAE Effectiveness:: Stage 3 has slightly weaker MSE and cosine denoising metrics than Stage 1 because it additionally optimizes cross-entropy toward semantic directions for text reconstruction.Stage 1 focuses purely on denoising, while Stage 3 produces a representation better suited to reconstruction.
  • 1) DAE Effectiveness:: 47.65 to 51.15 mean LID increase from Gaussian noise is reversed after DAE projection, with Stage 1 reaching 47.19 relative to clean embeddings.LID is estimated using MLE with 100 nearest neighbors against a held-out clean reference set; Fig. 3 also reports median LID.
  • 1) DAE Effectiveness:: 14–19 percentage-point gains in text reconstruction metrics and 6–7 percentage-point cosine-similarity gains are obtained by DAE-ONLY over NOISY INVERTER across two embedders.Removing perturbation noise makes representations more semantically stable and easier for the generative decoder to interpret.

2) Role of Joint Fine-tuning: … VII. CONCLUSION

The paper shows that joint fine-tuning strengthens DAEI and that the method remains effective across moderate noise-scale mismatches. Its broader review and conclusion frame denoising-aware inversion as evidence that Gaussian noise alone may not prevent textual information leakage.

  • 2) Role of Joint Fine-tuning:: DAEI improves over DAE-ONLY by approximately 5–13 percentage points on reconstruction metrics and 2–4 points in embedding cosine similarity through joint fine-tuning.These gains indicate that jointly tuning the denoiser and inverter can outweigh a small sacrifice in denoising accuracy.
  • D. Generalization Across Varying Noise Levels (RQ6): BLEU stays above 0.4377, F1 remains above 0.7421, and cosine similarity remains around 0.924–0.929 when test noise varies from σ = 0.005 to 0.018.DAEI is trained with σ = 0.01 and evaluated under other Gaussian noise scales on GTR-base.
  • D. Generalization Across Varying Noise Levels (RQ6): At σ = 0.02, DAEI achieves 0.3966 BLEU and 0.7200 F1, tolerating moderate mismatch between training and test-time perturbations.The result remains relevant when the defender samples σ from a small range and the attacker estimates only an average σ.
  • A. Text Embedding Inversion Attacks: Prior embedding inversion work progressed from revealing sensitive attributes and keywords to reconstructing complete sentences and high-fidelity short texts.Later methods also reduce attacker requirements through few-shot, transfer-based, or zero-shot settings.
  • C. Denoising Techniques: Denoising autoencoders reconstruct clean inputs from corrupted observations, while Noise2Noise and SURE enable training without direct clean targets.SURE estimates Gaussian denoising risk from noisy observations alone, and Monte Carlo SURE extends this approach.
  • C. Denoising Techniques: The field lacks a systematic understanding of whether high-quality text can be reconstructed from noise-protected embeddings, motivating denoising-aware inversion in the noisy-only setting.This gap concerns scenarios where attackers observe only noisy embeddings rather than clean reconstruction targets.
  • VII. CONCLUSION: DAEI combines an unsupervised denoising autoencoder with generative inversion and outperforms existing baselines across backbones, domains, and ablation settings.The conclusion states that these results demonstrate Gaussian noise alone is insufficient to prevent textual information leakage.
Loading 2608.18610v1…