PaperlayerView exact revision on arXiv

Source-linked AI summary

Model Extraction and Adversarial Transferability, Your BERT is Vulnerable!

Xuanli He, Lingjuan Lyu, Qiongkai Xu, Lichao Sun

arXiv:2103.10013v1cs.CL

TL;DR

Commercial BERT-based APIs may be vulnerable to model extraction, while the transferability of adversarial examples for BERT APIs—especially across different architectures—has been an open question. The paper extracts local models from API queries and uses them to test adversarial transfer, finding attacks effective across challenging settings and defenses inadequate unless victim performance is sacrificed.

  • Problem

    Model extraction can imitate NLP APIs, and the extent of adversarial transferability for BERT APIs, especially across different architectures, remains underexplored.

    Source: §1 · p. 1, §2.2 · p. 2

  • Method

    The authors query BERT-based APIs to train extracted models, then generate adversarial examples on those models and test transfer to the victims.

    Source: §1 · p. 1, §4.3 · p. 4

  • Results

    Extracted models achieved 0.85-0.99× victim models’ accuracies across the reported query settings, and the pseudo white-box attack was more than twice effective in the best case versus black-box counterparts.

    Source: §4.2 · p. 3, §4.3 · p. 4

  • Takeaways & Limitations

    Model extraction and adversarial transferability can compromise BERT-based APIs despite defenses unless victim-model performance is sacrificed.

    Source: §1 · p. 1

    The experiments use representative NLP datasets and tasks; extending the work to more complex NLP tasks is left for future work.

    Source: §6 · p. 5

Abstract

from arXiv · show

Natural language processing (NLP) tasks, ranging from text classification to text generation, have been revolutionised by the pre-trained language models, such as BERT. This allows corporations to easily build powerful APIs by encapsulating fine-tuned BERT models for downstream tasks. However, when a fine-tuned BERT model is deployed as a service, it may suffer from different attacks launched by malicious users. In this work, we first present how an adversary can steal a BERT-based API service (the victim/target model) on multiple benchmark datasets with limited prior knowledge and queries. We further show that the extracted model can lead to highly transferable adversarial attacks against the victim model. Our studies indicate that the potential vulnerabilities of BERT-based API services still hold, even when there is an architectural mismatch between the victim model and the attack model. Finally, we investigate two defence strategies to protect the victim model and find that unless the performance of the victim model is sacrificed, both model ex-traction and adversarial transferability can effectively compromise the target models

1 Introduction

Fine-tuned BERT models support commercial pay-per-query NLP APIs, while prior work and this paper raise concerns about their exposure to model extraction and adversarial attacks. The paper investigates extracting local copies and using them to transfer adversarial examples against the victim APIs.

  • Commercial task-oriented NLP models built on BERT are often deployed as pay-per-query APIs to protect data privacy, system integrity, and intellectual property.Source: §1 · p. 1
  • Carefully designed queries and API outputs can enable local imitation of NLP APIs, raising concerns about model extraction and competitors’ ability to reproduce services.The passage describes how sidestepping annotation and algorithm-design investment could let competitors offer an identical service at a lower price.Source: §1 · p. 1
  • The paper studies a two-stage attack that extracts a local model and uses it to craft adversarial examples transferable to the victim, alongside two defense strategies.The authors report that both attacks remain resilient to defenses without corrupted victim predictions, and that attacks can cost as little as $7.1.1.Source: §1 · p. 1

2 Related Work

Prior work frames model extraction as a way to imitate black-box models and facilitate follow-up attacks. In NLP, adversarial transferability has received some study, but its success for BERT APIs—especially across different architectures—remains an open question in this account.

2.1 Model Extraction Attack (MEA)

  • Model extraction, also called model stealing, has been applied to computer-vision and NLP tasks.Source: §2.1 · p. 2
  • An extracted model can closely imitate a black-box victim, raising intellectual-property concerns and providing a basis for follow-up adversarial attacks.Source: §2.1 · p. 2

2.2 Adversarial Transferability in NLP

  • Whether adversarial examples transfer across NLP models remains underexplored for BERT APIs, particularly when substitute and victim architectures differ.The passage notes prior exploration in NLP systems [Sun et al., 2020a; Wallace et al., 2020], while describing the BERT API question as largely unexplored.Source: §2.2 · p. 2

3 Attack on BERT-based API

The attack has two phases: query a black-box victim to build an extracted model, then use that model to generate adversarial examples for transfer to the victim. The extraction phase trains on query-prediction pairs, while the transfer phase uses the extracted model as a white-box surrogate.

3 Attack on BERT-based API

  • The attack on a BERT-based API consists of Model Extraction Attack (MEA) followed by Adversarial Example Transfer (AET).Source: §3 · p. 2

3.1 Model Extraction Attack (MEA)

  • Attackers query the victim API and use the resulting query-prediction pairs to train a local extracted model intended to match the victim’s accuracy.The described setup fine-tunes a public release of BERT on the collected dataset.Source: §3.1 · p. 2
  • Table 1 presents statistics for datasets used in sentiment analysis and topic classification.Source: Table 1 · p. 2

3.2 Adversarial Example Transfer (AET)

  • The AET phase generates adversarial examples on the extracted model and tests their transfer to the victim, using the extracted model as a white-box surrogate.Source: §3.2 · p. 2

4 Experiments and Analysis

Across sentiment and topic-classification tasks, the experiments test model extraction under varied query distributions and budgets, then assess adversarial transfer and architectural mismatch. The results show strong extraction and transfer attacks, with transferability reduced by architectural mismatch.

4.1 NLP Tasks and Datasets

  • The experiments cover TP-US and Yelp sentiment analysis, plus AG news and Blog topic classification.Queries are drawn from the same data as the victim or from a different distribution in the same domain; the authors note they lack access to the victim’s original training data.Source: §4.1 · p. 2, §4.2 · p. 3

4.2 MEA Setup and Results

  • With 1x or 5x query budgets, extraction reaches 0.85–0.99× of victim accuracy across differing query distributions, while domain closeness correlates with extraction success.Using the same data can even outperform the victim model; 0.1x and 0.5x query budgets are often sufficient.Source: §4.2 · p. 3, Table 2 · p. 3
  • The authors estimate extraction costs across classification datasets and describe the cost as highly economical and worthwhile.The estimates use Google and IBM API pricing.Source: §4.2 · pp. 3–4, Table 3 · p. 4

4.3 AET Setup and Results

  • Adversarial examples are generated on the extracted model using gradient-selected tokens and typo corruptions, then evaluated on the victim API by misclassification rate.The workflow first labels queries through the victim API and trains an extracted model on those labels, then transfers its adversarial typo examples to the victim.Source: Figure 1 · p. 3, §4.3 · p. 4
  • The extracted-model pseudo-white-box attack transfers adversarial examples to victim APIs at more than twice the rate of black-box counterparts in the best case.The comparison is reported across the attacks evaluated in Table 4.Source: Table 4 · p. 4, §4.3 · p. 4
  • Increasing the query budget from 1x to 5x improves transfer attack performance, which the authors attribute to higher extraction fidelity.The results compare 5x versus 1x queries.Source: §4.3 · p. 4

4.4 Architecture Mismatch

  • On AG news, architectural mismatch alleviates AET efficacy, while matched BERT-large architectures magnify victim vulnerability across attacks.The authors also report that mismatch can reduce AET efficacy.Source: §4.4 · p. 4, Table 5 · p. 4

5 Defence

The section evaluates softmax temperature and prediction perturbation as defenses against model extraction, but reports that protection generally comes with trade-offs. Both model extraction and adversarial transfer remain threats under the evaluated defenses.

5 Defence

  • SOFT applies temperature τ to the softmax posterior: higher values smooth the probabilities, while τ=0 yields hard labels.The section describes softening as manipulating the posterior probability distribution.Source: §5 · pp. 4–5
  • Temperature changes fail to defend against MEA except at τ=0, while perturbation offers significant defense at the cost of victim accuracy.Table 6 covers NO DEF., SOFT., and PERT. across datasets; lower attack scores indicate better defenses, and parenthetical numbers give victim accuracy.Source: Table 6 · p. 5, §5 · p. 5
  • PERT adds normal noise with variance σ to predicted probabilities, with larger variance described as a stronger defense.Source: §5 · p. 5
  • At σ=0.50, MEA surpasses the victim model; the authors conjecture that extraction still captures informative knowledge from perturbed probabilities.They identify this result for future in-depth study.Source: §5 · p. 5
  • Both MEA and AET remain threats to BERT-based APIs even when attackers have access only to limited or erroneous predictions.Source: §5 · p. 5

6 Conclusions

The paper concludes that extracted models can enhance adversarial transferability in difficult settings, including limited query budgets, distribution differences, and architectural mismatch. It identifies more complex NLP tasks and more effective defenses as future work.

6 Conclusions

  • Extracted models can substantially enhance adversarial-example transferability with limited queries, different query distributions, or architectural mismatch.The conclusion reports extensive experiments on representative NLP datasets and tasks under varied settings as evidence of attack effectiveness.Source: §6 · p. 5
  • The authors plan to extend the work to more complex NLP tasks and develop more effective defenses.Source: §6 · p. 5

A Dataset Description

The study uses four datasets spanning sentiment analysis and topic classification, with dataset-specific inputs and label constructions. The Trustpilot dataset is restricted to its US subset.

A Dataset Description

B Training Details

The experiments use Huggingface as their codebase and train each model for four epochs on an NVIDIA V100 GPU. Training uses batches of 64 and AdamW with a learning rate of 5e-5.

B Training Details

C Performance of Different Query Size

With 0.1x and 0.5x of the victim models’ training data, extracted models generally achieved accuracy comparable to the victims, with performance varying by dataset and domain.

C Performance of Different Query Size

D Architectural Mismatch

Architectural mismatch between the victim and attack models can reduce attack performance, but the proposed attacks remain effective overall.

D Architectural Mismatch

Loading 2103.10013v1…