PaperlayerView exact revision on arXiv

Source-linked AI summary

Black-Box Attacks on Sequential Recommenders via Data-Free Model Extraction

Zhenrui Yue, Zhankui He, Huimin Zeng, Julian McAuley

arXiv:2109.01165v1cs.CRcs.AIcs.IR

TL;DR

The paper asks whether sequential recommenders can be extracted and attacked without access to their training data. It studies extraction using a white-box model that behaves similarly to the black-box victim, then reports 0.747 top-10 agreement on ML-1M. Its conclusion is that sequential recommenders are threatened by model extraction.

  • Problem

    The paper examines whether sequential recommenders are vulnerable to model extraction, a risk not previously studied in recommender systems according to the authors.

    Source: §6 · p. 14

  • Method

    The paper learns a white-box model that behaves similarly to a black-box recommender without access to its training data.

    Source: §6 · p. 14

  • Results

    0.747 top-10 agreement on the ML-1M dataset shows similarity between the extracted white-box model and the black-box recommender.

    Source: §6 · p. 14

  • Takeaways & Limitations

    The authors conclude that sequential recommender systems are threatened by model extraction attacks.

    Source: §6 · p. 14

    The attack setting limits the total number of victim API requests to avoid large numbers of queries.

    Source: §3.1 · p. 4

Abstract

from arXiv · show

We investigate whether model extraction can be used to "steal" the weights of sequential recommender systems, and the potential threats posed to victims of such attacks. This type of risk has attracted attention in image and text classification, but to our knowledge not in recommender systems. We argue that sequential recommender systems are subject to unique vulnerabilities due to the specific autoregressive regimes used to train them. Unlike many existing recommender attackers, which assume the dataset used to train the victim model is exposed to attackers, we consider a data-free setting, where training data are not accessible. Under this setting, we propose an API-based model extraction method via limited-budget synthetic data generation and knowledge distillation. We investigate state-of-the-art models for sequential recommendation and show their vulnerability under model extraction and downstream attacks. We perform attacks in two stages. (1) Model extraction: given different types of synthetic data and their labels retrieved from a black-box recommender, we extract the black-box model to a white-box model via distillation. (2) Downstream attacks: we attack the black-box model with adversarial samples generated by the white-box recommender. Experiments show the effectiveness of our data-free model extraction and downstream attacks on sequential recommenders in both profile pollution and data poisoning settings.

1 INTRODUCTION

The paper examines data-free model extraction against sequential recommenders, where existing attack methods are scarce or rely on exposed training data. It uses autoregressive sequence generation to build a surrogate and applies attacks from that model to the black-box recommender.

1 INTRODUCTION

  • The experiments show that model extraction poses a threat to sequential recommender systems.The approach builds a white-box surrogate and transfers attacks from it to the black-box victim.Source: §1 · pp. 1–2
  • Prior recommender attacks have limited applicability to sequential models or rely on heuristics, while many assume attackers can access full victim training data.The paper notes that such data can be difficult to obtain in implicit-feedback settings such as clicks and views.Source: §1 · pp. 1–2
  • Extraction is challenging because attackers lack training data and semantic surrogate datasets, while APIs may return only rankings under limited query budgets.The paper considers a data-free setting and constructs a surrogate using limited API queries.Source: §1 · p. 2
  • Autoregressive training lets the recommender generate sequences resembling real training data, supporting synthetic data for extraction.The paper reports that generated data can resemble normal user behavior, which is difficult to detect.Source: §1 · p. 2
  • The downstream attacks include profile pollution that appends items to user logs and data poisoning that uses fake users to bias retrained models.The attacks are evaluated when the black-box sequential model returns top-k ranked lists.Source: §1 · p. 2

2 RELATED WORK

Prior model-extraction approaches rely on data or methods that do not transfer directly to discrete sequential recommendation, while recommender attacks face heuristic and architecture-specific limits. This work presents data-free weight extraction and attacks as a response to those gaps.

2.1 Model Extraction in Image and Text Tasks

  • Existing image-extraction methods often assume partial training data or semantically similar surrogates; discrete inputs block their direct transfer to sequential data.Data-free methods for images include synthetic samples and inputs optimized to maximize disagreement with the target.Source: §2.1 · pp. 2–3
  • NLP extraction can use random word sequences or surrogate datasets, but semantic-similarity surrogates are hard to use in recommendation and random item sequences perform poorly.The authors therefore generate data following sequential recommenders' autoregressive property.Source: §2.1 · p. 3
  • Sequential extraction needs autoregressive synthetic data and pair-wise ranking loss, while downstream attacks after extraction remain underexplored.The authors identify recommendation's ranking outputs as harder to distill than NLP classification.Source: §2.1 · p. 3

2.2 Attacks on Recommender Systems

  • Recommender attacks include profile pollution at test time and poisoning at training time, with fake-user ratings used to poison training data.Profile pollution targets a user's recommendation results by changing their profile, such as their view history.Source: §2.2 · p. 3
  • Profile pollution can inject fake views into user logs, but the work assumes interactions can be appended via implanted malware and excludes malware development and activity injection from scope.The cited prior strategy for choosing injected items is heuristic and lacks victim-recommender knowledge.Source: §2.2 · p. 3
  • Prior recommender poisoning can be heuristic or require complete interaction data; LOKI is infeasible for deep networks because of Hessian-computation cost, whereas this work extracts weights without real training data.The paper also proposes approximated attacks against the extracted recommenders.Source: §2.2 · p. 3

3 FRAMEWORK

The framework extracts a white-box surrogate from synthetic sequences and black-box ranked-list responses, then uses it to generate attacks against the victim. It is scoped to settings with unavailable training data and limited API queries.

3 FRAMEWORK

  • The framework first distills a white-box recommender from informative synthetic data, then uses it to generate adversarial sequences for profile pollution or data poisoning against the victim.The attack generation is gradient-based and operates in the discrete item space.Source: §3 · p. 4

3.1 Setting

  • The setting assumes unknown victim weights, unavailable training data and item statistics, limited API queries, and a known or partially known architecture.Each input sequence counts as one query-budget unit.Source: §3.1 · p. 4

3.2 Threat Model

  • The threat model uses timestamp-ordered item sequences drawn from a discrete item space to query a victim recommender.The victim's weights are unknown, and its output is a truncated ranked list of next-item recommendations.Source: §3.2.1 · p. 4

3.2.1 Black-box Victim Recommender.

  • The black-box victim returns a truncated top-k ranked list of recommended next items for an input sequence.The listed items are drawn from the discrete item space.Source: §3.2.1 · p. 4

3.2.2 White-box Surrogate Recommender.

  • The white-box surrogate exposes output scores over the item space, even though the black-box victim provides only a ranked list.The surrogate is composed of an embedding layer and a sequential model.Source: §3.2.2 · p. 4

3.3 Attack Goal

  • The attack pipeline first extracts a white-box model from the trained black-box victim without accessing its weights.The surrogate then stands in for the victim when constructing downstream attacks.Source: §3.3.1 · p. 4, §3.3.2 · p. 5

3.3.1 Model Extraction.

  • The extraction objective distills the victim's ranked-list responses on generated sequences by minimizing the distance between black-box and white-box outputs.The query sequences and their labels are synthetic rather than real training data; the output distance can use a ranking loss.Source: §3.3.1 · pp. 4–5

3.3.2 Downstream Attacks.

  • Downstream attacks seek to raise a target item's exposure through profile pollution or data poisoning, with retraining required for poisoning but not profile pollution.Profile pollution appends attack items to an accessible user sequence; poisoning injects fake profiles into normal training data.Source: §3.3.2 · p. 5

4 METHODOLOGY

The method extracts a black-box recommender from synthetic sequences and returned rankings, then uses the surrogate for profile-pollution and data-poisoning attacks. Its generation and distillation steps adapt to the data-free setting and the API’s top-k outputs.

4.1 Data-Free Model Extraction

  • The framework first generates synthetic inputs and black-box ranking labels, then distills a white-box recommender and uses it to attack the black box.The extraction process has data-generation and model-distillation steps; downstream attacks use the extracted surrogate.Source: §4.1 · p. 5, §4.2 · p. 7

4.1.1 Data Generation.

  • Autoregressive generation extends sequences with sampled recommendations, aiming to create more diverse, user-like data that better represents real behavior than random sequences.The method samples from the recommender’s top-k list; the authors note that it does not exploit ranking outputs or item properties such as similarity.Source: §4.1.1 · p. 6

4.1.2 Model Distillation.

  • Because the black box returns top-k rankings rather than probability distributions, distillation uses a ranking loss on surrogate scores for ranked items and sampled negatives.The loss compares neighboring top-k items and penalizes negatives that score above top-k items; generated sequences are also split into prefixes for training.Source: §4.1.2 · pp. 6–7

4.2 Downstream Attacks

  • The extracted white-box recommender supplies adversarial samples for two downstream attacks: profile pollution and data poisoning.The attacks test whether adversarial examples transfer from the trained surrogate to the black-box recommender.Source: §4.2 · p. 7

4.2.1 Profile Pollution Attack.

  • Profile pollution appends adversarial items to a user’s history, adapting gradient-based attacks to discrete item IDs under the assumption that useful items lie near the target in embedding space.The method searches embedding-level perturbations and tests candidate items with the white-box model, retaining the candidate that gives the target the highest score.Source: §4.2.1 · p. 7

11 end

Algorithm 2 takes a target, a desired profile length, a white-box model, and attack parameters as inputs.

11 end

  • Algorithm 2’s inputs are the target item, expected profile length, white-box model, and parameters ε and n.Source: §11 · p. 8

12 end

The poisoning method constructs fake sequences intended to promote a target item, including by placing it among seemingly irrelevant items. It also notes that the alternating pattern may be made less rigid by varying target-item proportions or adding noise or user-like data.

12 end

  • Repeated injection items can improve attack performance, while avoiding consecutive target items limits their disproportionate presence in the injection sequence.Source: §12 · p. 8

4.2.2 Data Poisoning.

  • Poisoning sequences alternate the target with items selected to be irrelevant or unlikely, so the target is next even after seemingly irrelevant histories.Candidate fillers are chosen using gradients and T-FGSM, selecting items with the lowest similarity to the perturbed sequence representation.Source: §4.2.2 · p. 8
  • The method repeats the sequence-and-filler construction to generate poisoning data, drawing on co-visitation and adversarial-example approaches [8].Source: §4.2.2 · p. 8
  • The authors note that the alternating pattern may be detectable, but varying target proportions or adding noise or user-like data can make it less noticeable.Source: §4.2.2 · p. 8

5 EXPERIMENTS

The experiments evaluate extraction on three recommendation datasets and three sequential recommender architectures, using held-out interactions and fixed implementation and evaluation protocols. The supplied passages describe the setup but do not report the experimental results.

5.1 Setup

  • The evaluation uses Movielens-1M, Steam, and Amazon Beauty, with rating data processed as implicit feedback.Source: §5.1.1 · p. 9

5.1.1 Dataset.

  • For each sequence, the last two items are held out for validation and testing, with the remaining items used for training.Source: §5.1.1 · p. 9

5.1.2 Model.

  • The extraction study implements NARM, BERT4Rec, and SASRec, representing different model architectures and training schemata.Source: §5.1.2 · p. 9

5.1.3 Implementation Details.

  • Models use Adam with weight decay 0.01, learning rate 0.001, batch size 128, and 100 warmup steps; generated sequence lengths match dataset limits.The allowed lengths for ML-1M, Steam, and Beauty are 200, 50, and 50, respectively; poisoning profiles are 1% of a real profile’s size.Source: §5.1.3 · p. 9

5.1.4 Evaluation Protocol.

  • Evaluation measures ranking with HR@K and NDCG@K and output similarity with Agreement@K between the black-box and extracted white-box models.The reported test ranking averages performance across each positive item and 100 uniformly sampled negatives.Source: §5.1.4 · pp. 9–10

5.2 RQ1: Can We Extract Model Weights without Real Data?

The experiments show that sequential recommenders can be extracted without real training data, with extraction quality varying by synthetic-data strategy, dataset sparsity, and model architecture. Cross-architecture tests identify NARM as the strongest white-box extractor overall.

5.2.1 Standard Model Extraction.

  • Random and autoregressive extraction retain about 94.54% of the black-box models’ R@10 on average, despite lacking real training data.For black-box NARM on ML-1M, R@10 drops 1.34% with random data and 0.98% with autoregressive data.Source: §5.2.1 · pp. 10–11
  • Autoregressive synthetic data yields higher output agreement than random data, averaging Agr@10 0.574 versus 0.452 across datasets.The authors also report that autoregressive data better resembles the true training-data distribution.Source: §5.2.1 · p. 11
  • Denser datasets generally support stronger extraction, while sparse data can increase divergence and reduce recommendation agreement.The standard extraction setup assumes the white-box and black-box use the same architecture; the cross-architecture experiment evaluates differing architectures.Source: §5.2.1 · p. 11
  • NARM has the best overall same-architecture extraction capability, while SASRec and BERT4Rec perform satisfactorily on ML-1M and Steam.SASRec shows slight improvements over BERT4Rec in most cases.Source: §5.2.1 · p. 11

5.2.2 Cross Model Extraction.

  • Across the reported comparison, NARM performs best overall as a white-box architecture, reproducing most target recommender systems with average top-10 agreement of 0.597, versus 0.548 for SASRec and 0.471 for BERT4Rec.Source: §5.2.2 · p. 11

5.3 RQ2: How do Dataset Sparsity and Budget Influence Model Extraction?

The experiments find that extraction and profile-pollution outcomes depend on data density, query budget, and victim architecture. Attacks transferred from extracted models are generally effective, but their performance varies with dataset sparsity and target popularity.

5.3 RQ2: How do Dataset Sparsity and Budget Influence Model Extraction?

  • As Beauty data become denser, both black-box and extracted models perform better, and extracted-model Agr@10 increases.The density experiment uses k-core Beauty datasets with increasing k and a fixed 5000-sequence extraction budget.Source: §5.3 · p. 12
  • Increasing the query budget brings rapid initial gains in white-box similarity, but gains become marginal beyond a certain point.The authors frame the aim as finding a budget sufficient for a white-box model to generate adversarial examples.Source: §5.3 · p. 12

5.4 RQ3: Can We Perform Profile Pollution Attacks using the Extracted Model?

  • For NARM, average metrics of extracted models on ML-1M reach 94.8% of black-box attack performance, compared with 66.2% on Steam and 55.6% on Beauty.The method achieves the best targeted-item promotion results on all datasets; on Steam, targeted-item N@10 scores are significantly improved.Source: §5.4 · p. 13
  • On Steam and Beauty, SASRec is the most vulnerable tested victim: its average N@10 rises from 0.071 to 0.571, versus 0.0680 to 0.286 for NARM.The authors report that victim-model robustness varies across recommenders.Source: §5.4 · p. 13
  • Profile pollution works across popularity groups, but target promotion declines as item popularity falls, making unpopular items harder to attack.Target groups are defined by item frequency: head is the top 20%, tail the bottom 20%, and middle the remainder.Source: §5.4 · p. 13

5.5 RQ4: Can We Perform Data Poisoning Attacks using the Extracted Model?

Extracted-model data-poisoning methods outperform the compared baselines, although their overall promotion is less effective than profile pollution. The relative improvement is especially large for less popular target items.

5.5 RQ4: Can We Perform Data Poisoning Attacks using the Extracted Model?

  • Data-poisoning methods outperform the baselines, but overall promotion is less effective than profile pollution when multiple targets are poisoned simultaneously.The authors contrast this with profile pollution, which targets specific users and can use stronger attack examples.Source: §5.5 · p. 14
  • On Beauty, average N@10 for targeted items rises from 0.066 before attack to 0.240 with the proposed method across three models.The authors note that DQN performs worse than in profile pollution and can sometimes reduce performance.Source: §5.5 · p. 14
  • Comparable poisoning results from black-box and white-box attacks suggest that the generation algorithm, rather than white-box similarity, may be the bottleneck.Source: §5.5 · p. 14
  • For N@10, relative improvement is 30.8% for head items, over 300% for middle items, and over 1000% for tail items.The results indicate greater exposure gains for less popular items, while promoting popular items through profile injection may be harder in this setting.Source: §5.5 · p. 14

6 CONCLUSION AND FUTURE WORK

The paper concludes that sequential recommender models can be extracted without training data and that attacks generated with extracted models can bias or corrupt black-box victims. Future work points to broader settings, defenses, and more query-efficient extraction.

6 CONCLUSION AND FUTURE WORK

  • A white-box recommender can closely reproduce black-box behavior without training data, reaching 0.747 top-10 agreement on ML-1M.The conclusion presents this as evidence that black-box sequential recommenders are vulnerable to model extraction.Source: §6 · p. 14
  • Experiments show that transferred attacks can drastically bias and corrupt well-trained black-box recommenders under both profile pollution and data poisoning.The authors conducted further experiments on both attack settings after observing the potential for transfer from white-box to black-box models.Source: §6 · p. 14
  • Future work includes extending the framework to more universal settings, developing defenses or robust training pipelines, and using active learning to reduce extraction queries.Source: §6 · pp. 14–15
Loading 2109.01165v1…