PaperlayerView exact revision on arXiv

Source-linked AI summary

Do Defenses Against LLM Extraction Work Across Attacks? A Lifecycle Benchmark of Black-Box Model Extraction

Shuze Liu, Kaixiang Zhao, Runyang Xu, Jingzhi Chen, Nathan Wu, Yu Wang, Yushun Dong

arXiv:2610.00839v1cs.CR

TL;DR

Text-only LLM APIs expose responses that can train surrogates, yet fragmented evaluations make extraction attacks and defenses difficult to compare. This benchmark evaluates six attacks, ten defenses, and two adaptive rewriting attacks under matched conditions while preserving method-specific procedures. It finds that rankings and defense effectiveness vary across metrics and attacks, and rewriting can reduce provenance scores while also reducing surrogate utility.

  • Problem

    Text-only APIs expose model behavior that adversaries can use to train surrogates, while varied evaluation conditions and security objectives make prior extraction and defense results difficult to compare.

    Source: §1 · pp. 1–2

  • Method

    A lifecycle benchmark compares six attacks, ten defenses, and two adaptive rewriting attacks under matched conditions while preserving method-specific acquisition and training procedures.

    Source: §1 · p. 2, §5 · p. 10

  • Results

    Attack rankings vary across capability, fidelity, and repetition, defense effectiveness varies across attacks, and paraphrasing or back-translation lowers some provenance scores while reducing surrogate capability or fidelity.

    Source: §5 · p. 10

  • Takeaways & Limitations

    The controlled comparisons provide a basis for comparing extraction methods and their interactions with defenses under text-only access.

    Source: §1 · p. 2, §5 · p. 10

    Lineage verification with DuFFin does not reliably rank extracted students above unrelated controls across extraction procedures, and its evidence depends on behaviors covered by the probe set.

    Source: §4.2 · p. 8

Abstract

from arXiv · show

Large language models (LLMs) deployed through text-only APIs face model extraction risks, as adversaries can collect their responses to train surrogates that reproduce their capabilities. While prior work has developed diverse attacks and defenses, evaluations remain fragmented across access assumptions, model configurations, query budgets, and security objectives, limiting comparability across methods. To address this gap, we introduce a unified benchmark covering six extraction attacks, ten defenses, and two adaptive attacks that paraphrase or back-translate protected responses before surrogate training. The benchmark controls model configurations, query data, budgets, and held-out evaluation conditions within each comparison while preserving attack-specific querying and training procedures. We measure surrogate capability, fidelity to the victim, output quality using Rep-4, and query-budget sensitivity; defenses use their own security metrics paired with surrogate performance. For the adaptive attacks, we jointly measure provenance-detector scores and the capability and fidelity of surrogates trained on rewritten responses. The benchmark thus provides a reproducible basis for comparing extraction methods and their interactions with defenses under text-only access. Code and artifacts are available at https://github.com/sliu11-byte/MEA-Bench.

1 INTRODUCTION

Black-box extraction lets attackers use responses from text-only APIs to train surrogates, but fragmented evaluations make attack and defense results difficult to compare. The benchmark matches conditions while preserving method-specific procedures and evaluates defenses against their own objectives, including after adaptive rewriting.

1 INTRODUCTION

  • Existing studies vary access assumptions, models, query data, budgets, and objectives, making extraction and defense results difficult to reconcile.The benchmark addresses three evaluation gaps: confounded attack comparisons, incommensurable defense outcomes, and adaptive evaluations that stop before surrogate training.Source: §1 · pp. 1–2
  • Attack comparisons must match models, data, and budgets; defense evaluations must pair objective-specific security evidence with surrogate utility; adaptive tests must assess surrogates trained on rewritten responses.Rewritten text alone cannot show whether useful supervision or a detectable defense signal survives surrogate training.Source: §1 · p. 2
  • The lifecycle benchmark compares six attacks under shared configurations and data, evaluates ten defenses by intervention point and objective, and tests DIPPER paraphrasing and back-translation before surrogate training.It pairs defense metrics with surrogate capability and fidelity, and measures adaptive provenance evidence alongside resulting surrogate performance.Source: §1 · p. 2
  • Across six attacks, ten defenses, and two adaptive attacks, the benchmark reports metric-dependent attack rankings, limited defense transferability, and utility costs when provenance evidence is weakened.The protocol matches models, query data, budgets, and evaluation conditions while retaining method-specific procedures.Source: §1 · p. 2

2 BLACK-BOX EXTRACTION LIFECYCLE AND METHODS

The paper frames extraction as a lifecycle in which attackers acquire queries, obtain victim responses, train a surrogate, and evaluate it. Attacks and defenses differ in the stages they change or inspect, while adaptive attacks rewrite protected responses before training.

2.1 BLACK-BOX MODEL EXTRACTION LIFECYCLE

  • Victim responses, protected responses, or adaptively rewritten responses become training supervision paired with queries to train the surrogate.The lifecycle distinguishes query acquisition, victim interaction, surrogate training, and evaluation.Source: §2.1 · p. 3

2.2 EXTRACTION AND ADAPTIVE ATTACK MODELS

  • The six attacks share text-only victim interaction but differ in query acquisition and training; adaptive attacks additionally rewrite protected responses before training.Acquisition ranges from shared-pool selection to method-specific query construction, while training ranges from response-supervised fine-tuning to preference-based or discriminator-based objectives.Source: §2.2 · pp. 3–4

2.3 DEFENSE MODELS

  • Defenses inspect query streams, modify released responses, embed provenance signals, or probe trained models, so their success requires different measurements.Query detectors classify traffic; anti-distillation targets surrogate performance; provenance targets detectability; lineage verification compares a suspect model with the victim.Source: §2.3 · p. 4

3 BENCHMARK EVALUATION DESIGN

The benchmark compares attacks, defenses, adaptive rewriting, and query budgets while preserving method-specific procedures and holding surrounding conditions fixed. It reports capability, fidelity, output quality, and objective-specific defense evidence rather than collapsing them into one score.

3 BENCHMARK EVALUATION DESIGN

  • The design studies attack effectiveness, defense goals and utility costs, adaptive robustness, and query-budget effects through comparisons that vary one target while holding surrounding conditions fixed.Capability and fidelity are reported with the relevant defense result, such as query classification or provenance or lineage scores.Source: §3 · p. 5

3.1 ATTACK EVALUATION ACROSS LIFECYCLE STAGES

  • Six attacks are compared under common query budgets and shared pools where applicable, while retaining their acquisition and training procedures and evaluating fresh surrogates on held-out conditions.Adaptive tests compare direct training on protected responses with paraphrased or back-translated responses using the same SeqKD configuration; budget sensitivity varies queries for four attacks.Source: §3.1 · p. 5

3.2 DEFENSE EVALUATION ACROSS LIFECYCLE STAGES

  • Each defense family is measured against its own goal: query classification, surrogate degradation, provenance detection, or lineage separation, with no combined defense score.Anti-distillation compares matched clean and protected training; provenance reports detector scores separately from capability and fidelity; DuFFin separates extracted surrogates from unrelated controls.Source: §3.2 · pp. 5–6

3.3 SHARED CONTROLS AND BENCHMARK CONFIGURATION

  • Comparisons hold relevant model, query, surrogate, and evaluation conditions fixed; the extraction pool combines open-ended and task inputs, and capability, fidelity, and quality use separate measures.The default query budget is 1,000, with budget tests at 100, 1,000, and 10,000; attack and adaptive experiments use Llama models, while defense experiments use Qwen models.Source: §3.3 · p. 6

4 EXPERIMENTS

The experiments compare extraction attacks, defenses, adaptive rewriting, and query budgets under controlled evaluations. Results show that rankings and defense effects vary across objectives and attack procedures.

4.1 EXTRACTION ATTACK EFFECTIVENESS

  • No attack ranks best across capability, fidelity, and output repetition: task-dependent accuracy, QEDKS’s highest fidelity, and Model Leeching’s low repetition capture distinct outcomes.GAD performs well on selected tasks despite lower average accuracy, while its high Rep-4 indicates degraded output quality.Source: §4.1 · pp. 6–7

4.2 DEFENSE EVALUATION

  • ADS suppresses QEDKS but has little effect on SeqKD or SODA, while DOGe is mixed and Trace Rewriting causes more consistent but modest degradation.The differences are consistent with how each attack uses protected responses; a modification may be attenuated or repurposed by another training objective.Source: §4.2 · pp. 7–8
  • Provenance signals do not consistently survive surrogate training: GINSEW separates most clearly under QEDKS, ADFP often returns no signal, and Radioactivity can move in the wrong direction.The evaluation finds little change in capability or fidelity alongside detector-score variation across attacks.Source: §4.2 · p. 8
  • Query detectors recognize QEDKS traffic more readily than fixed SeqKD queries, while DuFFin does not consistently rank extracted students above unrelated controls.Query detection reflects acquisition-pattern shifts, and lineage scores depend on whether the probe set covers behaviors transferred by an extraction procedure.Source: §4.2 · p. 8

4.3 ADAPTIVE ROBUSTNESS

  • Rewriting yields defense-dependent, partial detector-score reductions while reducing fidelity across all three defenses; back-translation causes the larger capability loss.ADFP has no inherited signal, and GINSEW lacks positive clean–protected separation; lower detector scores therefore do not uniformly establish evasion.Source: §4.3 · pp. 8–9

4.4 QUERY-BUDGET SENSITIVITY

  • Budget effects depend on acquisition and training design: capability does not improve monotonically, and fidelity or output quality can decline even when capability is retained.At B = 10000, fidelity and output quality fall sharply for SeqKD and SODA; QEDKS preserves both despite lower capability.Source: §4.4 · p. 9

5 CONCLUSION AND FUTURE WORK

The paper concludes that extraction outcomes and defense effectiveness vary across metrics and attacks, and that adaptive rewriting can reduce detector scores while also reducing surrogate capability or fidelity. It positions the benchmark as a controlled lifecycle comparison and outlines its scope and reproducibility practices.

5 CONCLUSION AND FUTURE WORK

  • The benchmark compares six attacks, ten defenses across four security objectives, and two adaptive attacks; outcomes vary by metric and defense effectiveness by attack.Paraphrasing and back-translation lower some provenance-detector scores but also reduce surrogate capability or fidelity.Source: §5 · p. 10

AI USE STATEMENT

  • The authors used AI to retrieve and discover literature and polish author-written text, then reviewed sources and checked edits for accuracy and consistency.They did not use AI to generate synthetic datasets, prove mathematical claims, or conduct experiments.Source: AI USE STATEMENT · p. 10

ETHICS STATEMENT

  • Experiments used public datasets and open-weight models in a controlled research environment, without targeting proprietary production services or collecting private user data.The authors state that use should comply with model and data licenses, service terms, and applicable law.Source: ETHICS STATEMENT · p. 10

REPRODUCIBILITY STATEMENT

  • The paper points to its sections and appendices for evaluation flows, implementations, settings, data processing, metrics, calibration, and detailed numerical results.Code, configuration files, and processed evaluation artifacts are available in the MEA-Bench repository.Source: REPRODUCIBILITY STATEMENT · p. 10

A RELATED WORK

  • Prior benchmarks cover narrower settings, while MEA-Bench connects diverse attacks, defenses, response transformations, surrogate training, and post-training evaluation in one empirical lifecycle.The related-work discussion also describes extraction attacks by query selection, supervision construction, and surrogate optimization.Source: A RELATED WORK · p. 15, Table 3 · p. 16

B.1 SHARED GENERATION AND TRAINING SETTINGS

  • Shared generation settings reuse victim transcripts across attacks unless online interaction is required, and use greedy decoding, fixed seeds, and common sequence-processing rules.Trainable adapters use the same LoRA configuration rather than method- or budget-specific tuning.Source: §B.1 · p. 16

B.2 ATTACK IMPLEMENTATIONS

The appendices document how attacks acquire queries and train surrogates, how defenses and adaptive rewriting are implemented, and the shared experimental data and model conditions. They also specify software and preprocessing choices for reproducibility.

B.2 ATTACK IMPLEMENTATIONS

  • Attacks differ in query acquisition, supervision construction, and optimization: some use direct response supervision, while LoRD, SODA, and GAD add distinct student-based objectives.QEDKS expands seed queries online; Model Leeching templates prompts and parses answers, while SeqKD, LoRD, SODA, and GAD use a fixed pool.Source: §B.2 · pp. 16–17

B.3 DEFENSE IMPLEMENTATIONS AND CALIBRATION

  • Query detectors classify observed query streams; anti-distillation defenses modify released responses; provenance methods test inherited signals, while DuFFin verifies lineage after training.Defense comparisons retain attack-specific procedures and change only victim responses for matched clean–protected pairs.Source: §B.3 · pp. 17–18

B.4 ADAPTIVE-ATTACK IMPLEMENTATIONS

  • Adaptive attacks paraphrase or back-translate protected responses before training a fresh SeqKD surrogate, using protected text alone and keeping queries and the B = 1000 budget unchanged.The detector is rerun separately on clean, defense-only, and rewritten branches.Source: §B.4 · p. 18

B.5 SOFTWARE, HARDWARE, AND CHECKPOINT SELECTION

  • The experiments record software versions, accelerator details, configurations, inputs, and output checksums in run manifests, and evaluate the final completed adapter checkpoint.SODA also validates its matched SeqKD initialization, and GAD stores generator and discriminator adapters separately.Source: §B.5 · pp. 18–19

C.1 VICTIM AND SURROGATE MODELS

  • Attack experiments use a Llama-3.3-70B-Instruct victim and Llama-3.1-8B-Instruct surrogate; defense experiments use a Qwen2.5-72B-Instruct victim and base Qwen2.5-7B surrogate.Attackers observe submitted queries and returned text, not victim parameters, probabilities, hidden states, or gradients.Source: §C.1 · p. 19

C.2 EXTRACTION QUERY POOL

  • The 100,000-query pool combines 80% general instructions with 20% task-focused questions from eleven public datasets; QEDKS instead constructs queries online.Every query received by the victim, including follow-ups, counts toward the budget.Source: §C.2 · p. 19

C.3 PROMPT FORMATTING, PARSING, AND DEDUPLICATION

  • Queries are normalized and deduplicated, held-out evaluation inputs are removed, and budget sets are nested prefixes of one seeded ordering.The prompt pool is not replicated to fill its target size.Source: §C.3 · p. 19

C.4 HELD-OUT EVALUATION TASKS

The evaluation uses held-out task examples for capability and a separate prompt set for behavioral fidelity and output quality. It defines aggregate accuracy, response similarity, and within-response repetition measures.

C.4 HELD-OUT EVALUATION TASKS

  • Capability evaluation uses zero-shot runs on six tasks, with overall ACC defined as the unweighted macro average of task-level accuracies.Behavioral fidelity and output quality use 3,000 separate open-ended prompts, disjoint from extraction sets at budgets up to 10,000.Source: §C.4 · pp. 19–20

D.1 CAPABILITY, FIDELITY, AND OUTPUT QUALITY

  • ACC gives each task equal weight, BERTScore F1 compares paired surrogate and victim responses, and Rep-4 measures within-response four-gram repetition rather than correctness or victim similarity.Lower Rep-4 indicates less repetition; responses shorter than four tokens are excluded.Source: §D.1 · p. 21

D.2 DEFENSE-SPECIFIC METRICS

The benchmark preserves defense-specific metrics and pairs them with surrogate performance, while reporting query detection, lineage verification, and adaptive robustness separately. Comparisons use calibrated detectors and paired conditions under a fixed evaluation protocol.

D.2 DEFENSE-SPECIFIC METRICS

  • Provenance scores retain each defense’s native statistic and scale, while adaptive comparisons separately track provenance, ACC, and BERTScore F1.Radioactivity is displayed as −log10(p) in Figures 3 and 4 for visual consistency.Source: §D.2 · pp. 21–22
  • Query detection treats each 50-query group as one decision unit for attack and held-out benign batches.Each detector’s decision rule is calibrated on benign batches before attack and held-out benign batches are evaluated.Source: §D.2 · p. 21, §D.3 · p. 22
  • DuFFin lineage verification computes victim–model answer agreement on valid probes and evaluates model-level ROC-AUC against four negative controls.The comparison uses 508 common valid probes; tied positive–negative pairs contribute 0.5.Source: §D.2 · p. 22
  • Adaptive rewrites are compared with matched defense-only conditions using separate provenance, ACC, and BERTScore F1 measurements.The measurements are reported separately so signal removal is not conflated with degradation of rewritten training data.Source: §D.2 · p. 22

D.3 DETECTION CALIBRATION AND STATISTICAL AGGREGATION

  • Evaluation comparisons preserve shared task examples and held-out prompts, and are reported as paired descriptive comparisons under a fixed protocol.Capability is aggregated within tasks and then across tasks; BERTScore F1 and Rep-4 are macro-averaged over held-out responses.Source: §D.3 · p. 22

E DETAILED EXPERIMENTAL RESULTS

  • Detailed-results tables preserve raw or native-scale measurements from the same runs as paired main-text figures.Figures emphasize changes or trajectories, while tables support reproducibility.Source: E DETAILED EXPERIMENTAL RESULTS · p. 22

E.1 ADDITIONAL QUERY-BUDGET RESULTS

The query-budget analysis reports exact results for four attacks at budgets of 100 and 10,000, with the 1,000-query results in Table 1.

E.1 ADDITIONAL QUERY-BUDGET RESULTS

  • Table 8 reports exact results for four attacks at query budgets B = 100 and B = 10000.The table caption identifies aggregate attack results at both budgets.Source: §E.1 · p. 22
  • The four attacks’ B = 1000 results are reported in Table 1.Source: §E.1 · p. 22

E.2 ANTI-DISTILLATION RESULTS

The appendices report exact clean and defended anti-distillation results, alongside surrogate performance and detector results for provenance and query-detection defenses.

E.2 ANTI-DISTILLATION RESULTS

  • Table 9 reports clean and defended values underlying Figure 2: ACC uses the conditional-log-likelihood evaluation protocol, while BERT F1 is baseline-rescaled BERTScore F1 against the same undefended teacher.Source: §E.2 · p. 23

E.3 PROVENANCE RESULTS

  • Table 10 reports capability and fidelity for students trained on clean and protected responses under provenance defenses.Source: §E.3 · p. 23
  • Table 11 preserves the provenance defenses’ native detector scales and directions.Source: §E.3 · p. 23

E.4 QUERY-DETECTION RESULTS

  • Table 12 reports batch-level query-detection results.Source: §E.4 · p. 23

E.5 LINEAGE-VERIFICATION RESULTS

DuFFin’s model-level ROC-AUC is 0.167 across three extracted students and four negative models evaluated on shared valid probes.

E.5 LINEAGE-VERIFICATION RESULTS

  • 0.167 is the DuFFin model-level ROC-AUC across three extracted students and four negative models.The seven models were evaluated on 508 shared valid probes.Source: §E.5 · p. 23

E.6 ADAPTIVE-ATTACK RESULTS

Adaptive-attack results report provenance signals before and after response rewriting alongside detector-specific tables for provenance, query detection, and lineage.

E.6 ADAPTIVE-ATTACK RESULTS

  • ADFP is zero in every condition, so Table 14 retains its scores while Figure 4 omits them.The two rewriting branches use the same clean and defense-only detector baselines.Source: §E.6 · p. 23, Table 14 · p. 24
  • Table 11 reports native provenance-detector scores for clean and protected students at B = 1000.The measures are ADFP −log10(p), GINSEW Psnr, and Radioactivity native p-values; arrows indicate stronger evidence.Source: Table 11 · p. 24
  • Table 12 reports query-detection results at B = 1000, with bold marking best values within each query group, including ties.Rates use calibrated thresholds.Source: Table 12 · p. 24
  • Table 13 reports DuFFin lineage-verification scores, where higher values indicate stronger predicted lineage.Source: Table 13 · p. 24
  • Table 14 reports provenance signals before and after adaptive response rewriting for SeqKD at B = 1000.It gives ADFP −log10(p), GINSEW Psnr, and Radioactivity native p-values; arrows indicate stronger evidence.Source: Table 14 · p. 24
Loading 2610.00839v1…