PaperlayerView exact revision on arXiv

Source-linked AI summary

On the Evidentiary Limits of Membership Inference for Copyright Auditing

Murat Bilgehan Ertan, Emirhan Böge, Min Chen, Kaleel Mahmood, Marten van Dijk

arXiv:2601.12937v1cs.CRcs.AI

TL;DR

Copyright auditing needs reliable evidence of whether a suspect text or semantic equivalent appeared in an LLM’s training data, but MIAs may be vulnerable to adversarial paraphrasing. The paper formalizes an adversarial auditing protocol and evaluates SAGE-guided semantic paraphrases, finding that MIA signals degrade sharply despite preserved meaning and stable semantic representations.

  • Problem

    Copyright auditing lacks reliable black-box procedures for determining whether a suspect text or semantically equivalent variant was used during LLM training.

  • Method

    The paper formalizes a judge–prosecutor–accused protocol and evaluates SAGE, a Sparse Autoencoder-guided paraphrasing framework that preserves semantic content while altering lexical structure.

  • Results

    MIAs show signals on unmodified text but degrade sharply under semantics-preserving paraphrasing, even when the model’s underlying semantic representations remain stable.

  • Takeaways & Limitations

    Current MIAs are brittle under adversarial, meaning-preserving rewrites and are insufficient alone as robust technical evidence for copyright auditing.

  • Takeaways & Limitations

    Paraphrasing does not fully eliminate memorization, with MIAs retaining AUC 0.7–0.8 under full fine-tuning, and the findings are specific to text.

Abstract

from arXiv · show

As large language models (LLMs) are trained on increasingly opaque corpora, membership inference attacks (MIAs) have been proposed to audit whether copyrighted texts were used during training, despite growing concerns about their reliability under realistic conditions. We ask whether MIAs can serve as admissible evidence in adversarial copyright disputes where an accused model developer may obfuscate training data while preserving semantic content, and formalize this setting through a judge-prosecutor-accused communication protocol. To test robustness under this protocol, we introduce SAGE (Structure-Aware SAE-Guided Extraction), a paraphrasing framework guided by Sparse Autoencoders (SAEs) that rewrites training data to alter lexical structure while preserving semantic content and downstream utility. Our experiments show that state-of-the-art MIAs degrade when models are fine-tuned on SAGE-generated paraphrases, indicating that their signals are not robust to semantics-preserving transformations. While some leakage remains in certain fine-tuning regimes, these results suggest that MIAs are brittle in adversarial settings and insufficient, on their own, as a standalone mechanism for copyright auditing of LLMs.

1. Introduction

Opaque LLM training corpora may contain copyrighted material, while memorization and extraction vulnerabilities motivate membership inference attacks as potential audit evidence. This section argues that MIAs may fail as admissible evidence because accused developers can use semantics-preserving paraphrases that disrupt their signals.

  • Motivation: Opaque LLM training and fine-tuning corpora raise concerns that copyrighted material may be included without permission.LLMs can also memorize and reproduce training data and remain vulnerable to extraction and inference attacks.
  • Existing Auditing Methods: MIAs estimate training-set membership using loss reduction, likelihood ratios, or calibration signals, but their reliability varies across fine-tuning regimes.Prior work reports LLM vulnerability to MIAs under certain fine-tuning conditions.
  • Evidentiary Challenge: Adversarial copyright evidence must withstand manipulation, remain interpretable, and remain stable under distribution shifts, requirements existing MIAs do not guarantee.An accused party can paraphrase or otherwise obfuscate text before training, while judges cannot observe the original corpus.
  • Contributions: The paper formalizes a judge–prosecutor–accused communication protocol requiring MIA-based claims to remain valid when suspected texts are paraphrased or obfuscated.The protocol makes evidentiary assumptions, adversarial incentives, and admissibility requirements explicit.
  • Contributions: SAGE uses Sparse Autoencoders and a Semantic Persistence Score to preserve meaning and document structure while suppressing surface-level token overlap.The framework enables ablations separating structural and semantic effects.
  • Main Result: Existing MIA signals are dominated by lexical and distributional artifacts, so semantics-preserving obfuscation limits their suitability as technical evidence in adversarial copyright auditing.This structural limitation persists even when the underlying meaning is preserved.

2. Copyright Auditing

This section frames copyright auditing as a black-box, adversarial test of whether a suspect text or semantically equivalent variant was used in LLM training. It formalizes robustness to meaning-preserving transformations as a necessary reliability condition for membership-inference evidence.

  • Evidentiary reliability: Reliability requires robustness because an accused party can invalidate a membership-inference claim through semantics-preserving transformations.Such invalidation prevents the claim from constituting reliable evidence, regardless of its stability or explanatory framing.
  • Problem formulation: Copyright auditing asks whether a suspect text or semantically equivalent variant was used to train a black-box-accessible LLM.Copyright concerns protected expression rather than exact duplication, so auditing must account for meaning-preserving transformations.
  • Adversarial protocol: The protocol models a prosecutor presenting membership-inference evidence, an accused model owner who may obfuscate training text, and a judge evaluating the claim.The setting includes opaque training data and adversarial incentives, while allowing legally or strategically meaning-preserving transformations.
  • Threshold limitations: Membership thresholds are sensitive to input difficulty and model generalization, so thresholds calibrated on one distribution may fail under distribution shift.The appropriate τmia depends on the model, dataset, and input distribution, creating false-positive-control concerns.
  • Robustness criterion: An auditing method is robust when semantic-preserving transformations do not change sufficiently decisive membership decisions.The formal framework uses an auditing statistic A(x; fθ), threshold τmia, and a transformation family T; paraphrasing is one important subclass.
  • Scope: The remainder of the work evaluates whether existing membership-inference attacks satisfy this robustness requirement rather than proposing a legal standard.The formalization is presented as a technical threat model for information asymmetries and adversarial incentives in litigation.

3. Structure Aware SAE-Guided Extraction

SAGE is a structure-aware, metric-guided paraphrasing pipeline that obfuscates training data while preserving semantic content. It preserves structural text, rewrites narrative text, and selects candidates to balance semantic persistence against lexical divergence.

  • SAGE generates obfuscated training data while preserving semantic content through structure-aware, metric-guided paraphrasing.
  • The pipeline decomposes each document into ordered sections and classifies them as structural or narrative.Structural examples include headers, citations, code blocks, and identifiers.
  • Structural sections are preserved verbatim, while N = 3 narrative paraphrase candidates are generated under semantic and surface-form constraints.The final paraphrase is selected from these candidates using a trade-off between semantic preservation and lexical divergence.
  • Semantic persistence is measured with SAE feature activations, using average cosine similarity across corresponding narrative spans.The semantic observer is a Sparse Autoencoder attached to an intermediate layer of a separate fixed language model, with structural spans excluded to avoid formatting artifacts.
  • Surface-form divergence is measured with WordSim, where lower values indicate less word-level overlap between original and paraphrased text.WordSim combines word-token Jaccard similarity, word-trigram overlap, and character 5-gram overlap.

4. Results

Across datasets and fine-tuning regimes, MIAs detect membership in unmodified training data but degrade sharply after semantics-preserving paraphrasing. SAGE and SAGE-R suppress leakage while preserving semantic content and downstream utility, with full fine-tuning producing stronger but harder-to-remove leakage.

  • Overall MIA robustness: MIAs show detectable signals on unmodified text but degrade sharply under semantics-preserving paraphrasing, despite stable underlying semantic representations.This pattern is consistent across all evaluated settings.
  • LoRA fine-tuning: Under LoRA fine-tuning, original-data models show strong attack performance, while SAGE reduces AUC and low-FPR metrics and SAGE-R approaches the pretrained baseline.The degradation occurs consistently across attacks and datasets.
  • Full fine-tuning: Under full fine-tuning, attack success is uniformly higher than under LoRA, although SAGE variants still outperform SOFT and reduce leakage relative to standard fine-tuning.Stronger memorization from full fine-tuning is more difficult to eliminate entirely through paraphrasing.
  • Stability across regimes: Across LoRA and full fine-tuning, defense effectiveness follows the stable ordering FT > SOFT > SAGE > SAGE-R.SOFT reduces leakage but remains weaker than SAGE and SAGE-R, while requiring model-internal access that SAGE does not require.
  • Utility and semantic persistence: SAGE retains conceptual meaning while substantially reducing surface overlap, supporting MIA suppression without degrading downstream fine-tuning utility.Semantic consistency is measured with SPS, surface overlap with WordSim, and downstream utility with an LLM-as-a-Judge framework using averaged jury scores.

5. Discussion

Under the communication protocol, the results reveal a robustness limitation: semantics-preserving transformations can substantially change membership scores, weakening the evidentiary value of current MIAs despite residual leakage. The discussion therefore motivates semantic, protocol-aware, and adaptive auditing strategies for adversarial settings.

  • Robustness and evidentiary limits: Current MIAs can change substantially under semantics-preserving transformations, revealing a robustness limitation for copyright auditing.The authors do not claim that MIAs are universally ineffective, but report sensitivity even under favorable conditions.
  • Robustness and evidentiary limits: This sensitivity suggests that current MIAs rely strongly on surface-level lexical regularities rather than meaning-invariant features.The resulting brittleness undermines their evidentiary value in adversarial auditing settings.
  • Limitations: AUC 0.7–0.8 remains under full finetuning, showing that paraphrasing does not fully eliminate memorization.Residual statistical signal alone does not establish evidentiary reliability, which requires a decision threshold τmia.
  • Future work: Future work should target semantic or functional behavior and develop protocol-aware, game-theoretic, and adaptive auditing strategies.One proposed strategy aggregates membership signals across multiple semantically equivalent paraphrases to test evidence stability under adversarial obfuscation.

6. Related Works

Related work spans model-centric privacy defenses and dataset copyright-auditing methods. Privacy techniques formally limit individual-sample influence, while auditing approaches use watermarking, dataset inference, and document-level membership tests.

  • Model-Centric Privacy Defenses: Differential-privacy defenses such as DP-SGD and DP-LoRA bound individual training-sample influence by injecting optimization noise.These methods provide formal privacy guarantees but typically incur utility degradation and require careful hyperparameter tuning.
  • Dataset Copyright Auditing: Copyright-auditing approaches include intrusive watermarking, dataset inference, and document-level membership inference attacks.DE-COP probes whether a model distinguishes verbatim text from paraphrased variations, whereas dataset inference targets aggregate dataset inclusion rather than record-level membership.

7. Conclusion … C.3. Bug 3: Model Loading Mismatch

The paper concludes that membership inference attacks are brittle evidence for copyright auditing under adversarial, semantics-preserving transformations and should not stand alone. Its appendices document dataset-comparison limitations, SOFT reproduction bugs, SAGE implementation details, metric analyses, paraphrase examples, factual-anchor ablations, and extensive MIA evaluations.

  • 7. Conclusion: MIAs degrade under semantics-preserving paraphrasing, indicating dependence on surface artifacts rather than semantic use and limiting their evidentiary value for copyright auditing.The paper recommends protocol-aware approaches aligned with legal standards of evidence rather than relying on membership inference alone.
  • A. Appendix Organization: The appendices cover dataset-level comparison limitations, SOFT reproduction issues, SAGE implementation and utility-evaluation procedures, metric ablations, paraphrase examples, and factual-anchor ablations.These materials address theoretical, practical, algorithmic, experimental, and qualitative aspects of the auditing framework.
  • A. Appendix Organization: Appendix H and Appendix I report extensive MIA results for Llama-3.2-3B and EleutherAI/pythia-6.9b across fine-tuning regimes, defenses, and evaluator models.The appendix organization also identifies these appendices as the main repositories of model-specific MIA performance results.
  • B. On Dataset-level Comparisons: Dataset-level comparisons are evidentially unreliable because accused parties can omit items, disclose partial or unverifiable transformed corpora, and potentially dilute comparisons with large volumes of unrelated data.The passage frames dataset disclosure and similarity matching as an inadequate way to bypass the limitations of MIAs.
  • C. Reproduction Issues in SOFT Framework: Reproducing SOFT revealed critical implementation issues in its official codebase that prevent faithful reproduction of the reported results.The authors document these issues for transparency and to justify a corrected reimplementation.
  • C.1. Bug 1: Inconsistent Training Regime: The SOFT code applies LoRA and then enables gradients for all parameters, creating an undefined regime that is neither standard LoRA nor full fine-tuning.The paper notes that its reported results instead use full fine-tuning without LoRA, leaving the implementation discrepancy unexplained.
  • C.2. Bug 2: Incorrect Model Checkpointing: Because the model is a PeftModel, saving stores only LoRA adapter weights, discarding the simultaneously trained base-model weights.The resulting checkpoint cannot faithfully represent the training performed under the inconsistent regime.
  • C.3. Bug 3: Model Loading Mismatch: Standard model loading expects complete weights, so the adapter-only checkpoint causes the system to load the untuned base model because no PEFT-specific loader is implemented.The passage further states that adding PEFT-aware loading would not resolve the underlying issue.

C.4. Impact on Reproducibility … D.4. LLM-as-a-Judge Utility Evaluation

The paper corrects reproducibility bugs that invalidate original-model MIA evaluations, then details SAGE and related obfuscation procedures while evaluating whether semantic utility persists after paraphrasing.

  • C.4. Impact on Reproducibility: All MIA attacks accidentally evaluated the original pretrained model, producing SOFT AUC ≈0.50 because that model lacked membership-specific leakage.This bug explains the initial near-random-guessing reproduction results.
  • C.4. Impact on Reproducibility: The flawed pipeline reproduced SOFT’s near-random-guessing results but not the reported baseline of approximately 80% AUC, whose origin remains unclear.Evaluation always ran against the pretrained model, regardless of whether SOFT obfuscation was applied.
  • D. Implementation Details: The corrected implementation removed the LoRA wrapper, saved complete model weights, and verified that loading restored the fine-tuned weights.These changes addressed the training and model-loading failures in the reproduction pipeline.
  • C.4. Impact on Reproducibility: The authors retained SOFT’s data obfuscation, loss-based high-risk sample selection, and dynamic swapping while retraining models with the corrected pipeline.This preserved the original defense methodology while fixing the evaluation pipeline.
  • D.2. Additional Details on SAGE Generation: SAGE generates iterative paraphrases that preserve structural sections verbatim, rewrite narrative sections, and use semantic and surface-form constraints.The procedure is formalized as a sequence of attempts for each input document.
  • D.2. Additional Details on SAGE Generation: SAGE returns candidates early when SPS(x, ˜x) ≥0.60 and WordSim(x, ˜x) ≤0.35; otherwise, it updates prompts and selects the candidate maximizing SPS(x, ˜x) −WordSim(x, ˜x).WordSim is aggregated over narrative sections and computed with a regex-based tokenizer.
  • D.3. SAGE-R: Factual Anchor Identification and Removal: SAGE-R and FT-F remove factual anchors such as entities, numbers, and dates, replacing consistently detected values with canonical placeholders through deterministic regex substitution.Anchors are identified by fixed LLM prompting and then ordered by first occurrence.
  • D.4. LLM-as-a-Judge Utility Evaluation: The study uses LLM-as-a-Judge evaluation to test whether fine-tuned models retain semantic and conceptual utility despite degraded membership-inference signals.Questions are generated with GPT-4o-mini from randomly sampled MIMIR member documents, using conceptual, self-contained prompts and the first 4,000 characters.

D.5. Question Set Composition · E. Ablation and Analysis on SPS & WordSim · E.1. Paraphrase Quality

The evaluation uses 325 fixed questions spanning complementary forms of training-data familiarity, while SPS, WordSim, and judge-based utility assess semantic preservation under paraphrasing. SAGE preserves semantics more strongly than SAGE-R and rewrites surface form substantially, though metric values depend on datasets and paraphrasers.

  • D.5. Question Set Composition: Each dataset uses the same fixed set of 325 evaluation questions across models, enabling controlled comparisons beyond brittle surface memorization.The set probes multiple forms of training-data familiarity.
  • D.5. Question Set Composition: The question set combines specific-content, closed-book topic, tiered-difficulty, contrastive, cloze, dataset-specific topic, and passage-completion tasks.These tasks target document-specific knowledge, semantic recognition, contextual understanding, in-domain discrimination, objective completion, domain or style recognition, and source-consistent continuation.
  • D.5. Question Set Composition: Responses use stochastic decoding and are graded against source-derived references by an averaged panel of GPT-4o-mini, DeepSeek-V3.2, and Grok-4.1-fast judges.The subject model generates up to 512 new tokens at temperature 0.7.
  • E. Ablation and Analysis on SPS & WordSim: SPS and LLM-as-a-Judge jointly operationalize semantic equivalence by linking representational similarity with preserved downstream functional utility.The evaluation is aligned with the judge–prosecutor–accused protocol, where meaning and utility can remain preserved despite eliminated membership signals.
  • E. Ablation and Analysis on SPS & WordSim: SAGE, and sometimes SAGE-R, achieves higher judge scores than standard fine-tuning, which the paper attributes to semantic-preserving paraphrasing that regularizes against surface-level overfitting.The effect is strongest in structured technical domains such as ArXiv and PubMed, and smaller or absent in noisier HackerNews data.
  • E. Ablation and Analysis on SPS & WordSim: Utility comparisons are interpreted within each dataset, with the core conclusion being no systematic utility degradation under semantic-preserving paraphrasing rather than absolute performance gains.This qualification preserves the paper’s interpretation of the utility results.
  • E.1. Paraphrase Quality: SAGE consistently achieves higher SPS than SAGE-R, while all configurations maintain low WordSim values across datasets and paraphrasing models.SAGE’s advantage indicates stronger semantic persistence when structural and factual anchors are preserved, alongside substantial surface-level rewriting.
  • E.1. Paraphrase Quality: SPS varies substantially by dataset and paraphraser, with technical corpora generally scoring higher and no paraphraser uniformly dominating across domains.The absolute scale is dataset- and model-dependent, motivating analysis across semantic observers.

E.2. SAE Implementation Details & Ablation · F. Qualitative Examples of SAGE variants · G. Ablation on Factual Anchors

The ablations show that SPS depends on the semantic observer but consistently ranks SAGE above SAGE-R, supporting relative rather than absolute interpretation. Factual-anchor removal alone only partially suppresses membership attacks, whereas broader surface and structural obfuscation achieves stronger suppression.

  • E.2. SAE Implementation Details & Ablation: SPS varies substantially across probe models and SAE layers, with deeper layers generally producing higher scores.Results are averaged across paraphrasing models to isolate the semantic-representation effect.
  • E.2. SAE Implementation Details & Ablation: SPS has no universal threshold because its absolute scale depends on the probe model and layer.The authors recommend calibrating SPS to a specific observer or aggregating multiple observers.
  • E.2. SAE Implementation Details & Ablation: SAGE achieves higher SPS than SAGE-R for every dataset, probe model, and layer tested.The ordering persists in both shallow and deep representations despite fluctuations in absolute scores.
  • E.2. SAE Implementation Details & Ablation: SPS captures relative semantic degradation induced by stronger obfuscation rather than functioning merely as a similarity heuristic.The invariant SAGE-versus-SAGE-R ordering across observers and layers supports SPS as a semantic-persistence signal.
  • F. Qualitative Examples of SAGE variants: Tables 8–11 provide qualitative examples showing how different SAGE variants behave on representative samples.These examples complement the quantitative ablations with variant-level illustrations.
  • G. Ablation on Factual Anchors: Factual-anchor removal alone partially reduces attack effectiveness but leaves substantial residual leakage and remains weaker than SAGE and SAGE-R.FT-F consistently lowers AUC and TPR@FPR relative to standard fine-tuning across datasets and attacks.
  • G. Ablation on Factual Anchors: The gap between FT-F and SAGE shows that removing facts alone is insufficient because attacks exploit surviving lexical and structural regularities.SAGE reduces surface-form overlap while preserving semantics, and SAGE-R additionally removes structural and factual cues for the strongest suppression.

H. Detailed Results on meta-llama/Llama-3.2-3B

On meta-llama/Llama-3.2-3B, pretrained-model MIAs perform near random guessing, whereas full fine-tuning produces near-perfect leakage. SAGE substantially reduces this leakage, and SAGE-R reduces it further across datasets and attacks.

  • Pretrained baseline: Pretrained-model MIAs perform at or near random guessing, with AUC values around 0.5 and low TPR@0.01%FPR.This provides a lower-bound baseline and validates the experimental setup.
  • Full fine-tuning: Without defenses, full fine-tuning yields near-perfect AUCs across all five datasets and TPR@1%FPR values near 1.0.The effect is consistent across datasets, attacks, and evaluator backends.
  • Defenses: SAGE substantially reduces membership leakage, while SAGE-R reduces it further across datasets.Both defenses produce large drops in AUC and TPR relative to full fine-tuning without defenses.

I. Detailed Results on EleutherAI/pythia-6.9b

On EleutherAI/pythia-6.9b, the ablation reproduces the main ordering: pretrained attacks are near chance, LoRA produces moderate leakage, and full fine-tuning produces the strongest leakage. SAGE reduces leakage under adaptation, with SAGE-R generally suppressing it most strongly.

  • Pretrained model: On the pretrained model, membership-inference attacks remain near chance.This result is summarized in Table 29 for EleutherAI/pythia-6.9b.
  • Overall finding: Overall, the EleutherAI/pythia-6.9b ablation confirms the ordering observed in the main results.The comparison spans pretrained, LoRA, and full-fine-tuning settings.
Loading 2601.12937v1…