Black-Box Dissector: Towards Erasing-based Hard-Label Model Stealing Attack

1 Introduction

• Cloud-model predictions can enable model functionality theft, which can also support further adversarial attacks.The paper frames deployed models as valuable intellectual property that AI companies seek to keep private.Source: §1 · p. 1
• Existing methods assume complete probability predictions, while practical APIs may return top-k values or only top-1 labels; hard-label performance can fall by 30.50%.The introduction also notes lower accuracy with hard labels and declining performance as fewer class predictions are available.Source: §1 · p. 2, Figure 1 · p. 2
• CAM-driven erasing removes regions highlighted by the substitute model and queries the victim again, aiming to recover class information obscured by the top-1 prediction.The pipeline selects erased images with the largest substitute-output difference from their originals.Source: §1 · p. 2, Figure 2 · p. 3
• Random-erasing self-KD averages substitute-model outputs across erased versions into pseudo-labels, which are used with victim hard labels to mitigate overfitting.The introduction attributes overfitting to hard, noisy labels on synthetic or substitute datasets and reports improved generalization with self-KD.Source: §1 · pp. 2–3
• Across four datasets, black-box dissector achieves 43.04–90.57% test accuracy and 47.60–91.37% agreement, up to 8.27% above state of the art.The method is also reported effective against defenses and real-world APIs, and improves transfer adversarial attacks by 4.91%–16.20%.Source: §1 · p. 3

2 Background and Notions

• Model-stealing attacks typically construct a transfer dataset and train a substitute; transfer data may be synthesized or selected, with query budgets limiting its size.Selection approaches include reinforcement learning, active learning, uncertainty-based, k-Center, and DFAL strategies.Source: §2 · p. 4
• In the hard-label setting, ϕ truncates the victim prediction to a one-hot vector, retaining only the top-1 class.The substitute is optimized on transfer data using a loss such as cross-entropy, against either the full soft prediction or truncated hard output.Source: §2 · p. 4
• Knowledge distillation transfers knowledge from teacher to student; model stealing is its black-box form, but hard labels discard probability-based class-similarity information.Unlike regular KD, self-KD uses the model itself as teacher rather than a larger, better teacher.Source: §2 · p. 4
• The paper adapts erasing by using a prior to choose pixels for occlusion, with Grad-CAM attention serving as the prior for re-mining information hidden by hard labels.Random erasing is described as a data-augmentation method that can reduce overfitting and improve robustness.Source: §2 · pp. 4–5

3 Method

• Black-box dissector combines conventional transfer-set construction and substitute training with CAM-driven erasing and random-erasing-based self-knowledge distillation.The CAM-driven strategy digs out missing inter-class information and aligns attention, while self-distillation mitigates overfitting and enhances generalization.Source: §3 · p. 5, §3.2 · pp. 8–9

3.1 A CAM-driven erasing strategy

• CAM-driven erasing uses the substitute model’s Grad-CAM map as a pixelwise prior for randomly erasing image regions, targeting the hard-label class’s response area.Because the substitute’s attention map may be inaccurate, the method repeats erasing and selects the variation with the largest label difference; erased samples can provide useful information even when the selected area is inaccurate.Source: §3.1 · pp. 5–7

3.2 A random-erasing-based self-KD module

• Random-erasing-based self-distillation averages the substitute’s outputs on repeatedly erased versions of an unlabeled image to form its soft pseudo-label.This consistency regularization encourages matching predictions across perturbed images and is intended to enhance generalization.Source: §3.2 · pp. 7–8
• The new substitute is trained using hard labels from the victim on transfer and erased samples together with soft pseudo-labels from the previous substitute.The objective combines losses on the transfer set and erased sample set with a loss on the pseudo-label set.Source: §3.2 · p. 8

4.1 Experiment settings

• Experiments used ResNet-34 victims on CIFAR10, SVHN, Caltech256, and CUBS200, with 1.2M unlabeled ILSVRC-2012 images for attacks and query budgets up to 30k.The study compared test accuracy and agreement against KnockoffNets and ActiveThief.Source: §4.1 · p. 9

4.2 Experiment results

• Under 30k queries, our method exceeded the previous best method in test accuracy by 4.85%, 1.72%, 3.88%, and 8.27% across four datasets; 8.27% was largest, and agreement showed similar results.Source: §4.2 · p. 10, Table 1 · p. 10
• Black-box dissector evaded adaptive misinformation and prediction poisoning on CIFAR10, while removing self-KD greatly reduced performance.The authors attribute the defenses’ limited effectiveness to self-KD alleviating overfitting from noisy labels.Source: §4.2 · p. 11, Table 2 · p. 11
• Removing CAM-driven erasing greatly reduced performance; replacing it with CutOut caused 3.36% degradation, and self-KD outperformed the tested regularizers.The substitute’s attention map tended to fit the victim’s across training stages; results on CUBS200 were similar to CIFAR10.Source: §4.2 · pp. 11–13, Figure 5 · p. 12, Table 3 · p. 12
• For an AWS Marketplace waste-classifier API usable only black-box, using ILSVRC-2012 for attack and a 2,527-image public test set, our method improved test accuracy by 12.63% and 7.32% over two previous methods.Source: §4.2 · p. 13
• Adversarial samples generated from its substitute models transferred more successfully across all tested substitute architectures, improving attack success rates by 4.91%–16.20%.The evaluation used PGD with a maximum L∞ perturbation of 8/255 on the CIFAR10 test set.Source: §4.2 · p. 14, Table 4 · p. 14

• The method’s superiority is reported on four widely-used datasets, with effectiveness also verified against defense methods, real-world APIs, and downstream adversarial attacks.Source: §5 · p. 14
• Black-box dissector combines a CAM-driven erasing strategy with a RE-based self-KD module for hard-label model stealing.Source: §5 · p. 14
• The authors state the approach may extend beyond image data when CAM and similar erasing methods work, citing synonym saliency-word replacement for NLP as an example.Source: §5 · p. 14
• The authors propose aligning attention maps with masked image patches in transformer classifiers as a direction for future validation.Source: §5 · p. 14