Model extraction from counterfactual explanations

I. INTRODUCTION

• The work is motivated by the use of machine learning in high-stakes decisions and calls for transparency, including explanations of consequential automated decisions under the GDPR.Transparency approaches include intrinsically interpretable models and post-hoc explanations of black-box models.Source: §I · p. 1
• Counterfactual explanations are perturbed input instances that receive a different prediction, and are one form of post-hoc explanation for black-box models.Post-hoc methods explain how black-box models produce outcomes; other families include global, local, feature-relevance, and visualization-based explanations.Source: §I · p. 1
• Post-hoc explanations face privacy inference attacks that can reveal training-set membership or a model’s structure and parameters.The introduction also identifies explanation manipulation as a separate threat, including fairwashing that makes an unfair model appear fair.Source: §I · pp. 1–2
• The paper provides the first study of model extraction attacks exploiting counterfactual explanations and evaluates attack performance across different adversary models.The adversary models differ in knowledge of the target’s training-data distribution and architecture, and in whether the explanation algorithm uses training data.Source: §I · p. 2
• The attack achieves high-fidelity and high-accuracy model extraction under a limited query budget.The paper also studies attack performance across adversary models that vary in distribution knowledge, architecture knowledge, and whether the explanation algorithm uses training data.Source: §I · p. 2

II. PRELIMINARIES

• The preliminaries establish background concepts in machine learning, counterfactual explanations, and model extraction attacks.The supplied machine-learning subsection describes a general supervised-learning setup.Source: §II · p. 2

A. Machine learning

• A supervised-learning model is a parameterized function from feature inputs to outputs, trained from labeled examples to fit the data.The training set consists of i.i.d. input-label pairs.Source: A. Machine learning · p. 2
• The generic training framework uses a regularizer to prevent overfitting, with λ controlling its strength; deep neural networks are compositions of parametric layer functions.The paper states that this framework applies to random forests, decision lists, and deep neural networks.Source: A. Machine learning · p. 2

B. Counterfactual explanations

• Counterfactual explanations are nearby instances whose predictions differ from the original, such as a loan-denied profile paired with loan-approved alternatives.The Adult Income examples show an original instance alongside counterfactuals, with dashed marks indicating unchanged features.Source: B. Counterfactual explanations · p. 2, Figure 1 · p. 3, Table I · p. 3
• The counterfactual objective seeks a different prediction while keeping the generated instance close to the original.The loss term enforces a different outcome; the distance term encourages proximity.Source: B. Counterfactual explanations · p. 2
• Diverse counterfactual frameworks return several varied options, helping users choose a way to reach the desired outcome while exposing more information to attackers.The paper uses DiCE but states that its attack is generic to other counterfactual frameworks.Source: B. Counterfactual explanations · p. 2
• DiCE balances counterfactual proximity and diversity, and the study examines attacks using both single and diverse explanations.Its objective combines outcome validity, distance from the query instance, and a diversity metric, balanced by hyperparameters λ1 and λ2.Source: B. Counterfactual explanations · pp. 2–3

C. Model extraction

• Accuracy-based extraction seeks a surrogate whose accuracy approaches the target’s, while fidelity-based extraction seeks to maximize agreement with the target.The former is also called theft-motivated extraction; the latter is reconnaissance-motivated, with fidelity measured over a reference set.Source: C. Model extraction · p. 3
• Model extraction can be an early step toward model inversion or adversarial-example discovery.Source: C. Model extraction · p. 3
• Functionally equivalent extraction matches the target across the whole input space and requires model-specific techniques, unlike more generic learning-based attacks.In learning-based extraction, the target can serve as a labeling oracle.Source: C. Model extraction · pp. 3–4
• Adversary capabilities cover task-related domain knowledge, deployment knowledge of model characteristics, and model access, including query limits and output types.Source: C. Model extraction · p. 4

III. MODEL EXTRACTION FROM COUNTERFACTUAL EXPLANATIONS

• The section introduces the general extraction problem, focuses on counterfactual explanations, and proceeds to adversary models and their attacks.Source: §III · p. 4

A. Problem formulation

• Explanation-based extraction uses both a target’s predictions and explanations to build a surrogate model.Source: A. Problem formulation · p. 4
• In the counterfactual setting, the explanation API returns a counterfactual and its outcome for each input, or a set of counterfactuals when explanations are diverse.Source: A. Problem formulation · p. 4

B. Attack description

• The adversary is assumed to know the task and has bounded black-box access, while deployment knowledge varies across data-distribution, architecture, and training-data-use scenarios.Source: B. Attack description · p. 4
• The attack set is labeled through prediction and explanation APIs, and the surrogate is trained on the original attack data plus both API outputs.The transfer set is defined as TA = {DA, B(DA)} ∪ E(DA).Source: B. Attack description · p. 4
• Counterfactual explanations balance the attack set by construction, which the paper says can help surrogates approximate the target’s decision boundary with few queries.The passage contrasts this with traditional extraction, whose transfer sets may be class-imbalanced.Source: B. Attack description · p. 4

IV. EXPERIMENTAL EVALUATION

• The evaluation section reports on the performance of counterfactual-explanation-based extraction attacks using real datasets.Source: §IV · p. 4

A. Experimental setting

• Experiments use Adult Income (48,842 people, 11 attributes), COMPAS (7,214 people, 8 attributes), and Default Credit (29,986 Taiwanese card users, 23 attributes).The tasks predict income above $50,000, re-offending within two years, and credit default, respectively.Source: A. Experimental setting · pp. 4–5
• Surrogate success is evaluated primarily by fidelity to the target model, with surrogate accuracy also reported.Testing sets assess both black-box and surrogate accuracy, as well as surrogate fidelity.Source: A. Experimental setting · p. 5
• Each dataset is split 67%/16.5%/16.5% for training, testing, and attack pools; target classifiers are MLPs with dataset-specific hidden-layer configurations.Adult Income and COMPAS use two hidden layers with 75 and 50 neurons, respectively; Default Credit uses one hidden layer with 50 neurons.Source: A. Experimental setting · p. 5
• Five scenarios vary whether the adversary has known, partially known, or unknown training-data distributions, receives multiple counterfactuals, or changes proximity and diversity settings.All scenarios are evaluated across assumptions about target architecture knowledge and whether the explanation API uses training data; unknown architecture is simulated by trying five architectures.Source: A. Experimental setting · p. 6
• In S1–S3, the API returns one counterfactual per query, while the adversary uses known, partially known, or unknown training-data distributions to form the attack set.S1 tests 100, 250, 500, or 1,000 queries over 10 random samples and compares against a surrogate trained on the complete attack pool; S3 uniformly samples the input space, which can yield unrealistic points.Source: A. Experimental setting · p. 6
• S4 tests 3, 5, or 7 counterfactuals per query; S5 varies proximity and diversity weights with 1,000 queries and five counterfactuals per query.S4 uses λ1 = 0.5 and λ2 = 1.0; S5 uses λ1 from 0.5 to 3.0 and λ2 from 1.0 to 3.0.Source: A. Experimental setting · p. 6

Experimental setup

• For Adult Income and COMPAS, Arch 5 is the target architecture; for Default Credit, Arch 1 is the target, with remaining architectures as candidates when the target is unknown.The experiments also find that knowing the target architecture does not significantly improve attack success.Source: B. Experimental results · p. 6, Table III · p. 7

S1: Single counterfactual with known training data distribution

• 250 queries yield 90% fidelity across Adult Income, COMPAS, and Default Credit, exceeding traditional extraction baselines that use 8059, 1192, and 4948 queries, respectively.At 1000 queries, fidelity reaches 94%, 93%, and 98% on those datasets, with test accuracy matching the target model; Table IV reports fidelity and accuracy averaged over 10 attacks.Source: B. Experimental results · p. 6, Table IV · p. 7

S3: Single counterfactual with unknown training data distribution

• With unknown data distribution on Adult Income, surrogate fidelity is 7.79% below the partial-knowledge setting and 9.14% below the full-knowledge setting.Despite this decrease, the surrogate outperforms traditional extraction using eight times more labels and full distribution knowledge; Table VI reports scenario performance.Source: B. Experimental results · p. 7, Table VI · p. 8

S4: Multiple counterfactuals with known training data distribution

• Surrogate fidelity improves as the number of counterfactuals per query increases, while using the target architecture provides no significant advantage.Using training data in the explanation API seems to help slightly at query budgets of 500 or fewer, but not above 500.Source: B. Experimental results · p. 7, Figure 4 · p. 8

S5: Impact of proximity and diversity

• Higher proximity and diversity constraints are associated with higher-fidelity surrogates, and target-architecture knowledge does not provide a significant advantage.The reported comparison concerns the Adult Income dataset.Source: B. Experimental results · pp. 7–8

S2: Single counterfactual with partially known training data distribution

• With 1000 queries on Adult Income, partial knowledge of feature marginals yields 93% fidelity and accuracy close to the target model’s test accuracy.Table V reports the scenario’s fidelity and accuracy averaged over 10 attacks.Source: B. Experimental results · p. 7, Table V · p. 8

Summary of the results

• 500 queries suffice for high-fidelity, high-accuracy surrogates when the adversary has partial or complete data-distribution knowledge; without that knowledge, extraction still outperforms traditional attacks using full distribution knowledge.The paper reports lower surrogate performance when the distribution is unknown, but still better performance than traditional extraction with eight times more labels.Source: B. Experimental results · p. 8

Learning-based attacks

• Learning-based extraction trains a surrogate on labels collected from the target model, using non-adaptive queries, active learning, or semi-supervised learning.This work’s attacks are non-adaptive, so queries can be sent together in a batch while still producing high-fidelity, high-accuracy surrogates with low query budgets.Source: §V · pp. 8–9

Functionally equivalent and hybrid attacks

• Functionally equivalent attacks solve equations for unknown model parameters; hybrid methods combine extracted parameters with subsequent surrogate training.Other cited approaches also use power side channels, gradient explanations, or infer model architecture and training hyperparameters.Source: §V · p. 9

Extraction using explanations

• The cited prior explanation-based extraction work uses gradients rather than counterfactuals and targets two-layer neural networks, whereas this work focuses on tabular data and is architecture-agnostic.The passage notes that gradient explanations are challenging to interpret in tabular prediction problems.Source: §V · p. 9

Countermeasures

• Watermarks can help detect stolen surrogates made public for queries, but are inefficient against adversaries who use stolen models internally.The discussion distinguishes deterrence of theft-motivated extraction from prevention of high-fidelity extraction.Source: §VI · pp. 9–10
• Query monitoring and auditing may struggle to detect non-adaptive, low-query attacks, and Sybil accounts can distribute queries among apparently regular users.These defenses analyze query patterns to distinguish adversaries from normal users.Source: §VI · p. 10

Tension between privacy and explainability

• More realistic post-hoc explanations can leak more model information, creating a trade-off with privacy; differential privacy’s effects on explanation robustness and trust remain open.For high-stakes decisions, the discussion suggests inherently transparent models designed with privacy-preserving techniques as a safer solution.Source: §VI · p. 10

• Across five adversarial scenarios, counterfactual explanations support high-fidelity, high-accuracy model extraction even under low query budgets.Source: §VII · p. 10
• Requiring counterfactual explanations to be diverse directly improves the performance of the surrogate models learned by the attacks.Source: §VII · p. 10
• The more faithful machine-learning models become, the more powerful model-extraction attacks an adversary can perform.The authors identify privacy-preserving transparent-box design as future work addressing the tension between privacy and explainability.Source: §VII · p. 10