PaperlayerView exact revision on arXiv

Source-linked AI summary

Towards Label-Only Membership Inference Attack against Pre-trained Large Language Models

Yu He, Boheng Li, Liu Liu, Zhongjie Ba, Wei Dong, Yiming Li, Zhan Qin, Kui Ren, Chun Chen

arXiv:2502.18943v1cs.CRcs.CL

TL;DR

Because many commercial LLM APIs provide generated text rather than complete logits, this paper studies membership inference in the label-only setting. It introduces PETAL, which estimates token probabilities from semantic similarity and outperforms existing label-only attacks while achieving results comparable to logits-based attacks across benchmarks.

  • Problem

    Because prior LLM membership inference attacks generally require logits unavailable through many commercial APIs, this paper asks whether generated tokens alone can reveal training-set membership.

    Source: §1 · p. 1

  • Method

    PETAL uses token-level semantic similarity to approximate target-token probabilities, derive alternative perplexity, and infer membership by thresholding.

    Source: §1 · p. 2

  • Results

    Across WikiMIA and MIMIR on five open-source LLMs, PETAL outperformed existing label-only attacks and was comparable to logits-based attacks.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The paper demonstrates membership inference against pre-trained LLMs in the label-only setting.

    Source: §9 · p. 14

    Differences between surrogate and target models can introduce errors in PETAL's estimated token probabilities.

    Source: §4.3 · p. 7

Abstract

from arXiv · show

Membership Inference Attacks (MIAs) aim to predict whether a data sample belongs to the model's training set or not. Although prior research has extensively explored MIAs in Large Language Models (LLMs), they typically require accessing to complete output logits (\ie, \textit{logits-based attacks}), which are usually not available in practice. In this paper, we study the vulnerability of pre-trained LLMs to MIAs in the \textit{label-only setting}, where the adversary can only access generated tokens (text). We first reveal that existing label-only MIAs have minor effects in attacking pre-trained LLMs, although they are highly effective in inferring fine-tuning datasets used for personalized LLMs. We find that their failure stems from two main reasons, including better generalization and overly coarse perturbation. Specifically, due to the extensive pre-training corpora and exposing each sample only a few times, LLMs exhibit minimal robustness differences between members and non-members. This makes token-level perturbations too coarse to capture such differences. To alleviate these problems, we propose \textbf{PETAL}: a label-only membership inference attack based on \textbf{PE}r-\textbf{T}oken sem\textbf{A}ntic simi\textbf{L}arity. Specifically, PETAL leverages token-level semantic similarity to approximate output probabilities and subsequently calculate the perplexity. It finally exposes membership based on the common assumption that members are `better' memorized and have smaller perplexity. We conduct extensive experiments on the WikiMIA benchmark and the more challenging MIMIR benchmark. Empirically, our PETAL performs better than the extensions of existing label-only attacks against personalized LLMs and even on par with other advanced logit-based attacks across all metrics on five prevalent open-source LLMs.

1 Introduction

This introduction frames label-only membership inference as a practical challenge for pre-trained LLMs and presents PETAL, which estimates perplexity from token-level semantic similarity. It reports that PETAL outperforms existing label-only attacks and is comparable to logit-based attacks on two benchmarks.

1 Introduction

  • Label-only MIAs address APIs that expose generated text rather than complete logits, which prevents prior logits-based attacks from calculating membership scores directly.MIAs can also audit machine unlearning and whether an LLM used data without authorization for pre-training.Source: §1 · p. 1
  • Prior robustness-gap label-only attacks are largely ineffective on pre-trained LLMs because better generalization makes token-level perturbations too coarse to capture members’ smaller decision-boundary distance gap.The introduction identifies this as a fundamental limitation of existing label-only attacks in the pre-training phase.Source: §1 · p. 2
  • PETAL uses a surrogate LLM and univariate linear regression to map token-level semantic similarity to estimated output probabilities, approximate sample perplexity, and infer membership by thresholding.This relies on the assumption that members are better memorized and have smaller losses.Source: §1 · p. 2
  • Across WikiMIA and MIMIR on five open-source LLMs, PETAL significantly outperforms existing label-only attacks and is comparable to advanced logit-based attacks.The paper also reports studies of PETAL’s parameters, fine-tuned and closed-source LLM case studies, and defenses.Source: §1 · p. 2
  • The evaluation also varies query budget, probability-estimation granularity, surrogate model, input length, decoding, and weighting, and examines fine-tuned LLMs, defenses, Gemini-1.5-Flash, and GPT-3.5-Turbo-Instruct.The two closed-source model case studies address a gap the authors identify in prior MIA research.Source: §1 · p. 2

2 Background & Related Work

The section establishes autoregressive pretraining as token-next likelihood modeling and surveys membership inference attacks, emphasizing that practical label-only access motivates attacks targeting pretraining.

2.1 Pre-trained Language Models

  • Autoregressive LLMs model each next token from its preceding context and are trained by minimizing data-point negative log-probability.The paper treats this objective as qualitatively memorizing tokens following training prefixes; extensive pretraining exposes each sample fewer than three times and yields nearly consistent training and test loss [9, 67].Source: §2.1 · pp. 2–3

2.2 Membership Inference Attacks

  • MIAs range from black-box attacks using output logits to white-box attacks using parameters, while NLP studies mostly use black-box access [56] [60] [77] [78].Choquette et al. explored stricter label-only access and showed logits are not imperative.Source: §2.2 · p. 3
  • Pretraining-phase attacks use loss or perplexity thresholds [12], reference-model sample hardness [44], or low-probability outlier tokens [59]; MIMIR reduces temporal distribution shift [15].For fine-tuning, Mireshghallah et al. and Fu et al. [17] use reference models, while Mattern et al. [42] use neighborhood samples as an alternative.Source: §2.2 · p. 3
  • Because services often expose generated tokens but not full logits, prior robustness-based label-only attack [63] fails on pretrained LLMs, motivating the first effective pretraining-phase label-only MIA.The paper attributes that prior attack’s failure to better generalization and overly coarse perturbations.Source: §2.2 · p. 3

3 Revisiting Label-only MIAs

This section defines the label-only membership-inference setting and evaluates robustness-based attacks for pretrained LLMs. Those attacks are nearly random, which the authors attribute to stronger generalization and perturbations too coarse to capture membership differences.

3.1 Preliminaries

  • An MIA predicts whether a target sample was in pretraining data; label-only attackers see generated tokens, lack training-data knowledge, and are evaluated by balanced accuracy, AUC, and low-FPR TPR.The attacker has no access to the model structure or enough non-member samples to train shadow or reference models; training comparable models at pretraining scale is unaffordable for common attackers.Source: §3.1 · pp. 3–4

3.3 Evaluation and Analysis

  • Robustness-based label-only attacks are nearly random and significantly weaker than logits-based attacks across all metrics on Pythia-6.9B and WikiMIA, as Table 1 summarizes.Extensive pretraining and exposing samples only once or a few times improve generalization and shrink the member–non-member decision-boundary gap; token-level perturbations are too coarse to capture it. Finer embedding-level perturbations would require access to model parameters, outside the threat model.Source: Table 1 · p. 4, §3.3 · p. 5

3.2 Robustness-based Label-only MIA

  • Tang et al. [63] adapt max-margin robustness attacks to language models by perturbing input prefixes and comparing generated outputs with remaining ground-truth tokens using text similarity.The evaluated perturbations are random swapping, back translation, and word substitution; swapping and substitution alter 15% of words, and each sample receives three augmented inputs.Source: §3.2 · p. 4

4 Methodology

PETAL focuses on the first generated token to preserve membership signals, uses token-level semantic similarity to approximate output probabilities, and infers membership from approximated perplexity. Its method uses surrogate-model regression to calibrate the similarity-to-probability mapping.

4.1 Focusing on the First Generated Token

  • Only the first generated token preserves effective membership signals because errors from iterative sampling accumulate and obscure member–non-member differences.The WikiMIA analysis found no notable score gap when generation was iterated, supporting the focus on the first token.Source: §4.1 · p. 5, Figure 2 · p. 6

4.2 Approximating Output Probabilities via Token-level Semantic Similarity

  • A WikiMIA Pearson correlation of 0.765 (p < 0.001) supports estimating each token’s output probability from its semantic similarity to the first generated token.The observed trend motivates univariate linear regression to approximate probabilities from token-level semantic similarity.Source: §4.2 · p. 6, Figure 3 · p. 6
  • PETAL obtains regression parameters on an open-source surrogate model, relying on the similarity-to-probability mapping being close across models.The paper reports this mapping is close for GPT-2 xl, Pythia-2.8B [7], phi-2 [38], and gemma-2-2B [65].Source: §4.2 · p. 6, Table 2 · p. 6, §4.3 · p. 7

4.3 Membership Inference based on Approximated Perplexity

  • Thresholding approximated perplexity identifies members because the model is expected to assign member sequences lower perplexity.Surrogate–target differences can introduce probability-estimation errors, but the paper says similarity scores can align with output probabilities despite their narrower range, which may inflate scores for low-probability tokens.Source: §4.3 · pp. 6–7

5 Experiments

Experiments show that PETAL makes effective label-only membership inferences across pre-trained LLM benchmarks, remains robust to parameter choices, and also works on fine-tuned models.

5.1 Attack Setup

  • WikiMIA uses Wikipedia events labeled by model release dates, while MIMIR draws members and non-members from Pile training and test splits; evaluations use post-2021 models and Pythia models, respectively.Baselines include five logits-based attacks and three data-augmentation label-only variants; PETAL uses GPT-2 xl as surrogate and greedy decoding in the main experiments.Source: §5.1 · pp. 7, 9

5.2 Main Results

  • All three robustness-based attacks are near random across models, except on GitHub, where code-data difficulty increases the membership signal.Token-level perturbations are too coarse to detect the small member/non-member boundary-distance gap; uncertainty on unseen code increases that gap.Source: Table 3 · p. 8, Table 4 · p. 8, §5.2 · p. 9
  • PETAL outperforms other label-only attacks across datasets and models and matches logits-based attacks, including at 1% FPR, except that logits attacks lead on arXiv.Its AUC differences from the PPL attack are within ±0.03; the authors suggest arXiv’s LaTeX code and special characters challenge GPT-2 xl’s probability approximation.Source: Table 3 · p. 8, Table 4 · p. 8, §5.2 · pp. 9–10
  • Attack performance improves slightly with model size, while PETAL remains on par with logits-based attacks across varying memorization capabilities.The authors attribute the trend to larger pre-trained LLMs memorizing faster and better.Source: Table 4 · p. 8, §5.2 · p. 10

5.3 Parameter Analysis

  • Reducing queries generally lowers performance, but using at least half the tokens retains parity with logits-based attacks; estimating probabilities from more generated tokens weakens PETAL.The exception to the query-reduction decline is TPR at 1% FPR, which the authors relate to outlier tokens rather than average token probabilities.Source: §5.3 · pp. 10–11, Figure 4 · p. 11
  • PETAL remains stable across surrogate-model combinations, with AUC changes within 0.06; no weighting strategy improves AUC across all four tested models, and sentence-transformer choice does not undermine its effectiveness.About 5% of sampled weighting parameter sets yield slight gains for individual models, but none improve on unweighted perplexity across all four.Source: §5.3 · pp. 11–12, Figure 5 · p. 11, Table 9 · p. 12
  • Longer texts improve all attacks, while PETAL stays comparable to logits-based attacks; it is stable across decoding strategies, with a marginal AUC increase under greedy search.The text-length tests use 64, 128, and 256 words; the authors link longer-text gains to more memorized prefix-token pairs.Source: §5.3 · p. 11, Figure 6 · p. 11, Table 8 · p. 12

5.4 Results on Fine-tuned LLMs

  • On fine-tuned LLaMA-Doctor and OPT-History, PETAL remains on par with logits-based attacks and significantly outperforms label-only attacks; MIAs are more effective in fine-tuning, with LLaMA-Doctor AUC reaching 0.87.The evaluated models were fine-tuned by Hugging Face developers on Chatdoctor and History datasets, respectively.Source: §5.4 · p. 12, Table 11 · p. 12

6 PETAL against Defenses

PETAL remains effective when inputs are paraphrased or training data are deduplicated, and MemGuard does not stop it. Differential privacy is described as costly to apply during LLM pre-training, with evaluation on differentially private pre-trained models left for future work.

Text Paraphrasing

  • On paraphrased WikiMIA, PETAL shows little to no performance decline, especially in AUC, because paraphrasing preserves the token semantics it uses.The evaluation uses the paraphrased WikiMIA benchmark from Zhang et al. [79] across four LLMs.Source: Table 12 · p. 13, §6 · p. 13

Training Data Deduplication

  • On Pythia-dedup models, data deduplication reduces MIA performance only slightly, to an almost negligible extent.The authors argue coarse-grained deduplication is insufficient; finer-grained deduplication might remove useful knowledge, and deduplication requires retraining, which may be unaffordable for typical companies.Source: Table 13 · p. 13, §6 · p. 13

MemGuard

  • MemGuard [28] is ineffective against PETAL because PETAL approximates probabilities from semantic similarity rather than using the model’s actual output probabilities.MemGuard's noise preserves predicted labels to limit utility loss, so PETAL can still compute semantic similarity and approximate the probability vectors.Source: §6 · p. 13

Differential Privacy

  • Differential privacy bounds MIA success in theory, but applying methods such as DPSGD [1] to LLM pre-training is prohibitively costly, and evaluation on differentially private pre-trained LLMs remains future work.The authors report no publicly available instances of LLMs pre-trained with differential privacy, to their knowledge.Source: §6 · pp. 13–14

7 Case Studies on Closed-source LLMs

The paper addresses the lack of closed-source LLM membership-inference studies with case studies of Gemini-1.5-Flash and GPT-3.5-Turbo-Instruct. PETAL is evaluated using updated WikiMIA, with a March 2024 membership cutoff, despite limited knowledge of the models’ training data.

Attack Setup

  • The case studies target Gemini-1.5-Flash and GPT-3.5-Turbo-Instruct using updated WikiMIA, which sets March 2024 as the member/non-member cutoff.The setup may be imperfect because detailed training-data information for the two models is unavailable, but the authors consider it sufficient to validate PETAL on real-world LLMs.Source: §7 · p. 14

Attack Results

  • PETAL achieves an attack AUC of 0.67 on GPT-3.5-Turbo-Instruct without access to model parameters or output probability distributions.The authors present this as evidence of membership-inference privacy threats in real-world scenarios.Source: §7 · p. 14
  • Gemini-1.5-Flash appears to have weaker memorization than GPT-3.5-Turbo-Instruct, possibly because safety alignment mitigates training-data leakage [46].The authors describe safety alignment as one possible explanation.Source: §7 · p. 14

8 Discussion

The discussion argues that PETAL is practical despite query costs, sketches why semantic similarity can provide membership signals, and acknowledges resource and application-scope limitations.

Efficiency and Practicality

  • PETAL is practical, costing $0.003 to attack a 128-token sample with GPT-3.5-Turbo-Instruct and performing on par with logits-based attacks without output probabilities.Its query budget can be reduced from approximately (n −1)(n + 2)/2 to (n−1)(n+2)/8 input tokens.Source: §8 · p. 14

Limitations

  • PETAL requires a surrogate model, adding GPU-resource demands, while surrogate-free attacks and applications to unlearning audits and copyright detection remain future work.The discussion frames the paper’s focus as privacy leakage from MIAs, consistent with existing work [31] [59] [79].Source: §8 · p. 14

Theoretical Underpinnings

  • Distributed representations imply that training on a sentence raises probabilities for semantically similar neighbors, motivating PETAL’s use of token-level semantic similarity as an MIA signal.Because generated tokens were sampled at high probability, higher token-level semantic similarity is expected to indicate higher output probability; the theory remains for future work.Source: §8 · p. 14

9 Conclusion

The paper studies label-only membership inference against pre-trained LLMs, finding existing attacks ineffective and proposing PETAL. Experiments indicate privacy risk persists even under this stringent access setting.

9 Conclusion

  • PETAL uses per-token semantic similarity to approximate output probabilities, calculate perplexity, and infer membership.The method is based on the assumption that members are better memorized and have smaller perplexity.Source: §9 · p. 14
  • Existing label-only attacks are ineffective against pre-trained LLMs, motivating PETAL as a method targeting this gap.The paper focuses on pre-training because it is more prevalent and presents greater technical challenges.Source: §9 · p. 14
  • Experiments across various metrics demonstrate PETAL’s effectiveness, and the analysis finds equivalent privacy risk even in the most stringent label-only setting.The authors hope the analysis motivates development of more effective defenses.Source: §9 · p. 14

Ethics Considerations

The authors frame the work as a warning about LLM membership-inference risks, not an endorsement of attacks, and call for stronger defenses and caution with privacy-sensitive pre-training data.

Ethics Considerations

  • The authors do not endorse practical attacks and urge deeper study of LLM memorization, stronger pre-training defenses, and avoiding privacy-sensitive training data without robust safeguards.They note that memorization may be inherent and membership inference difficult to fully resolve.Source: Ethics Considerations · p. 15
  • The authors shared findings with open-source model authors and disclosed commercial-LLM vulnerabilities to OpenAI and Google, allowing 60 days to address them before publication.Source: Ethics Considerations · p. 15
  • The study uses publicly available WikiMIA and MIMIR data for research, which the authors say raises no privacy concerns and generally does not violate licenses.Source: Ethics Considerations · p. 15

Open Science · A Rationale behind Our Case Studies Setup

The paper commits to public research artifacts and reproducibility, and argues that its commercial-model case studies are likely to reflect real-world scenarios despite uncertainty about training data.

Open Science

  • The team intends to publish datasets, scripts, and source code on GitHub to support open access, collaboration, and technological progress.The paper frames freely accessible research materials as a commitment to open science and wider adoption of open practices.Source: Open Science · p. 15
  • Pythia, LLaMA, and OPT are freely accessible, while Gemini-1.5-Flash and GPT-3.5-Turbo-Instruct are accessible through their providers’ official APIs and documentation.The paper identifies Hugging Face as an example source for the open-source models.Source: Open Science · p. 15
  • WikiMIA [59] and MIMIR [15] are openly accessible on Hugging Face and released under the MIT license.The paper presents their accessibility and licensing as supporting transparency.Source: Open Science · p. 15
  • The paper will document and publicly release artifacts, environment configurations, source code, hyperparameters, and experimental details for replication and validation.The stated goal is to provide a verifiable foundation for further scientific exploration.Source: Open Science · p. 15

A Rationale behind Our Case Studies Setup

  • The case studies are likely to align with real-world scenarios despite limited knowledge of Gemini-1.5 and GPT-3.5 training data.The rationale cites Wikipedia as a common high-quality pre-training source [59], reported Wikipedia pre-training for the GPT-3 series and PaLM [14], and knowledge cutoffs before March 2024.Source: A Rationale behind Our Case Studies… · p. 18
  • Wikipedia’s use as a high-quality pre-training text source and the reported Wikipedia pre-training of GPT-3 and PaLM motivate the experimental setup.The paper also notes that Gemini-1.5 and GPT-3.5 knowledge cutoffs precede March 2024.Source: A Rationale behind Our Case Studies… · p. 18

B Balanced Accuracy Results on MIMIR … E Balanced Accuracy Results on Paraphrased WikiMIA

PETAL substantially outperforms prior label-only attacks and achieves performance comparable to logits-based attacks on MIMIR; its ROC performance, granularity behavior, and paraphrase robustness are also reported.

B Balanced Accuracy Results on MIMIR

  • PETAL substantially outperforms prior label-only attacks and achieves balanced accuracy comparable to logits-based attacks on MIMIR.The passage notes that balanced accuracy and AUC typically show similar trends because both reflect average privacy leakage.Source: B Balanced Accuracy Results on MIMIR · p. 18
  • Balanced accuracy and AUC typically exhibit similar trends because both reflect the average level of privacy leakage.Source: B Balanced Accuracy Results on MIMIR · p. 18

C ROC Curves of Main Experiments

D Additional Experimental Results on Different Estimation Granularity

  • Unlike WikiMIA and the other six MIMIR subsets, arXiv attack results anomalously increase as estimation granularity rises from 4 to 16.The authors suggest this may relate to the arXiv dataset’s distribution characteristics and leave it for future work.Source: D Additional Experimental Results on… · p. 19, Figure 8 · p. 19
  • Figure 8 reports PETAL’s additional AUC results on Pythia-6.9B across estimation granularities of 1, 2, 4, 8, and 16.Source: Figure 8 · p. 19

E Balanced Accuracy Results on Paraphrased WikiMIA

F Additional Experimental Results on Training Data Deduplication

Training-data deduplication only slightly reduces balanced accuracy and leaves TPR at 1% FPR unchanged, indicating that coarse-grained deduplication does little to mitigate MIA privacy threats.

F Additional Experimental Results on Training Data Deduplication

Related papers

Loading 2502.18943v1…