Source-linked AI summary
Model Extraction Warning in MLaaS Paradigm
Manish Kesarwani, Bhaskar Mukhoty, Vijay Arya, Sameep Mehta
TL;DR
Hosted MLaaS models can be extracted through prediction APIs, creating risks to query revenue and training-data privacy. The paper develops a cloud-based monitor with entropy-based information gain and input-space coverage metrics for individual and colluding users, and evaluates them on BigML decision trees. The results show that the metrics provide approximate information about knowledge extracted by adversaries.
Problem
Hosted ML models accessed through pay-per-query APIs are susceptible to extraction that can avoid future charges and expose private training information.
Method
A cloud-based monitor observes query streams and uses entropy-based information gain and query-summary coverage of source-model partitions to estimate extraction by individual or colluding users.
Results
The metrics provided approximate information about adversaries’ extracted knowledge for decision-tree models evaluated with known attack techniques on BigML datasets.
Takeaways & Limitations
Extraction-status warnings can inform model owners about adversarial learning and support decisions about query pricing and model packaging.
Takeaways & Limitations
The evaluation focuses on decision-tree models and depends on the validation set being accurately classified and informative.
Abstract
from arXiv · showhide
Cloud vendors are increasingly offering machine learning services as part of their platform and services portfolios. These services enable the deployment of machine learning models on the cloud that are offered on a pay-per-query basis to application developers and end users. However recent work has shown that the hosted models are susceptible to extraction attacks. Adversaries may launch queries to steal the model and compromise future query payments or privacy of the training data. In this work, we present a cloud-based extraction monitor that can quantify the extraction status of models by observing the query and response streams of both individual and colluding adversarial users. We present a novel technique that uses information gain to measure the model learning rate by users with increasing number of queries. Additionally, we present an alternate technique that maintains intelligent query summaries to measure the learning rate relative to the coverage of the input feature space in the presence of collusion. Both these approaches have low computational overhead and can easily be offered as services to model owners to warn them of possible extraction attacks from adversaries. We present performance results for these approaches for decision tree models deployed on BigML MLaaS platform, using open source datasets and different adversarial attack strategies.
1. INTRODUCTION
MLaaS models exposed through pay-per-query APIs can be extracted through intelligent queries, threatening future revenue and training-data privacy. The paper proposes a cloud-based monitor using information gain and input-space coverage to warn about extraction by individual or colluding users.
- Pay-per-query MLaaS APIs expose hosted models to extraction attacks that can replicate models, avoid future charges, and leak training-data information.
- The proposed cloud-based monitor estimates extraction status from query and response streams, with warning quality improving when it has greater visibility of the deployed model.
- An entropy-based metric measures information learned by decision trees against a model-owner validation set and supports identifying colluding users with maximum combined information.
- A second metric uses compact query summaries and input-space coverage relative to source-model partitions to identify users with maximum model information.
- The approaches were evaluated on decision trees deployed through BigML using open datasets and random-query and path-finding attack strategies.
- The work addresses a gap in prior extraction research, which analyzed attacks but did not estimate adversaries’ extraction status or warn model owners.
2. PROBLEM FRAMEWORK
The framework models MLaaS prediction APIs as interfaces through which individual or colluding adversaries query a source model and learn a comparable model from query responses.
- An ML model is represented as a function mapping a d-dimensional input feature vector x_i to a categorical prediction y_i.
- A model extraction attack abuses the prediction API to learn a model f̂ with performance similar to the deployed source model f.
- Adversaries may act alone or collude, know the model type or training-data kind, and issue intelligent queries to learn model parameters.
1 −Rtest
The test-set metric compares an adversary’s learned model with the deployed model using the source model’s test data.
- The source model’s dataset is split into training and test sets, and the test set T evaluates the source and learned models.
- 1 −Rtest represents the accuracy of an adversary’s model with respect to the deployed model.
- The metric uses an indicator function when computing the test-set comparison.
1 −Runif
The uniform-input metric evaluates an adversary’s learned model over data generated uniformly from the input feature space.
- The learned model is tested on a uniformly generated dataset U in addition to test data following a specific distribution.
- Uniform-input evaluation supplements distribution-specific testing with coverage across the input feature space.
- 1 −Runif represents the learned model’s performance compared with the deployed model under the uniform-input evaluation.
3. MODEL EXTRACTION WARNING
The monitor estimates how much of a hosted model users can recover from query-response pairs. It uses information gain on a validation set to track learning and issue warnings when extraction status exceeds a threshold.
- The monitor detects whether clients can jointly reconstruct a deployed model beyond a specified accuracy threshold.It observes query-response streams and evaluates locally learned models against the source model’s behavior.
- It estimates user learning by incrementally training a local decision tree from query-response pairs and measuring its information gain on a validation set.The validation set provides the basis for comparing the learned tree with the source model.
- Information gain generalizes entropy reduction to quantify information acquired by a decision tree from a validation set partitioned by its leaves.The monitor computes this metric for user-trained trees and for the owner’s source tree.
- Information gain accurately captures the user model’s learning rate as training data increases.Figure 2 compares decision-tree accuracy with information gain over increasing amounts of training data.
- Users whose extraction status exceeds the owner-defined threshold receive an extraction warning sent to the relevant model owner.The warning is generated after the monitor computes each user’s extraction status.
Model Extraction Status with Collusion.
For colluding users, the monitor searches for groups whose combined queries reveal the most information about the deployed model. A greedy selection method reduces the exponential cost of exhaustive group search, while warning quality depends on the validation set’s information content.
- Model Extraction Status with Collusion.: The collusion extension identifies whether any k of n users jointly accumulate enough information to trigger an extraction warning.The monitor combines users’ query data to estimate their shared knowledge of the deployed model.
- Model Extraction Status with Collusion.: Exhaustive colluding-user selection evaluates every k-user combination, making its running time exponential in the number of users.For each combination, the monitor trains a decision tree on the combined query points and computes information gain.
- Model Extraction Status with Collusion.: The greedy algorithm selects users iteratively, choosing the next query set that yields the highest information gain with already selected users.It returns a set of k users and their extraction status without enumerating every combination.
- Model Extraction Status with Collusion.: Warning accuracy depends on the validation set being accurately classified by the source model and containing information across classes.If classification is inaccurate, the set may contain less information because samples may belong to only a few classes; an information-content score can provide confidence.
Initialization
The alternate monitor represents each user’s learned coverage of the source decision tree as feature intervals associated with leaf nodes. It aggregates covered areas by class and weights class-specific extraction by source-model class probabilities.
- Initialization: The alternate strategy maintains compact summaries of user queries instead of storing all historical queries and repeatedly training combined decision trees.This approach is intended to reduce computational expense when users and queries are numerous.
- Initialization: The monitor updates a user’s summary using the source tree’s leaf identifier and returned class label for each query.The summary stores rectangular regions in feature space corresponding to the user’s observed queries.
- Initialization: Each user summary is a leaf-by-feature matrix whose entries record learned minimum and maximum feature values for each source-model leaf.The matrix has one row per leaf and one column per continuous input feature.
- Initialization: It computes each leaf’s area or hypervolume from the stored feature intervals, then aggregates leaves sharing a class to estimate class-specific extraction.For at least three features, the geometric measure is a hypervolume rather than an area.
- Initialization: Overall extraction status is a probability-weighted sum of per-class extraction percentages, and users above threshold t generate warnings.The weights are the source model’s class probabilities.
Model Extraction Status with Collusion.
The collusion strategy greedily selects users whose combined summaries maximize covered input-space hypervolume. Its use is constrained because hypervolume coverage does not apply to categorical features.
- Model Extraction Status with Collusion: The hypervolume strategy does not support categorical features because area or hyper-volume is undefined for them.The limitation applies to the strategy based on input-space coverage.
- Model Extraction Status with Collusion: The monitor selects colluding users greedily by maximizing the hypervolume of their combined model summaries.The combined summary is illustrated for users A and B, and selection is performed in polynomial time.
- Model Extraction Status with Collusion: The collusion-selection algorithm differs from the information-gain approach only in choosing users by maximum combined-summary hypervolume.The approach identifies a set of k users with maximum model knowledge.
4. EXPERIMENTS
Experiments evaluate two extraction-warning strategies on BigML decision-tree models under random and Tramer attacks, including colluding users. The monitor’s estimates generally track adversarial model performance, with strategy-specific sensitivity to query distributions and feature types.
- Experimental setup: The evaluation uses BigML decision-tree APIs, open-source datasets, random-query attacks, and Tramer path-finding attacks.The Tramer attack recursively searches for decision-tree leaf rules using returned leaf IDs and class labels.
- Individual adversaries: 1 −Rtest steadily increases as adversaries issue Tramer queries and recover the rule sets of successive leaf nodes.Random-query decision-tree performance varies across datasets because query and training-data distributions may differ.
- Individual adversaries: At higher query counts, strategy 1 more closely matches the adversary’s 1 −Rtest under the Tramer attack.The monitor’s decision tree can nevertheless have lower 1 −Rtest because its training and query distributions differ.
- Individual adversaries: Strategy 2 more closely matches 1 −Rtest for both random-query and Tramer attacks by summarizing input-space coverage.Its results are less affected by the distribution of queries, but the strategy is more suited to continuous features.
- Colluding adversaries: At 1000 queries per user, extraction status rises from 28% for k = 1 to 51% for k = 3 and 69% for k = 5 colluding users.These results use strategy 2 for simultaneous Tramer attacks by n = 10 users.
- Colluding adversaries: Extraction status consistently increases as the number of colluding users grows under simultaneous random attacks.For n = 10 users, the random-attack setting uses strategy 1 for k ∈ {1, 3, 5}.
5. CONCLUSION AND FUTURE WORK
The work presents extraction monitoring metrics for decision-tree models and evaluates them against attacks on BigML datasets. The authors report approximate measurement of adversaries’ extracted knowledge while identifying broader model and dataset coverage as future work.
- Cloud-based machine learning services face attacks that can steal deployed models and compromise future payments or training-data privacy.
- The monitor uses entropy-based information gain and compact model summaries to estimate extraction by individual and colluding adversaries.The metrics can also support greedy selection of users with maximum model knowledge under collusion.
- Experiments on BigML datasets and two known attack techniques show that the metrics provide approximate information about adversaries’ extracted knowledge for decision-tree models.
- Future work will extend evaluation to more datasets and non-interpretable models such as neural networks.The authors also plan proxy models for broader source-model coverage and information-theoretic measures of training-data privacy loss.