PaperlayerView exact revision on arXiv

Source-linked AI summary

Copycat CNN: Stealing Knowledge by Persuading Confession with Random Non-Labeled Data

Jacson Rodrigues Correia-Silva, Rodrigo F. Berriel, Claudine Badue, Alberto F. de Souza, Thiago Oliveira-Santos

arXiv:1806.05476v1cs.CVstat.ML

TL;DR

The paper asks whether a black-box CNN exposed through an API can be copied without access to its confidential training data or problem-domain examples. It queries the target with random unlabeled images, trains a copycat on the resulting stolen-label dataset, and evaluates this across three classification problems and Microsoft Azure's Emotion API. Copycats reached at least 93.7% of original-model performance with non-problem-domain data, at least 98.6% with additional problem-domain data, and at least 97.3% of the Azure API's performance.

  • Problem

    The paper investigates whether companies' black-box CNN models can be copied when exposed through APIs, including without problem-domain training examples.

  • Method

    The method queries a target CNN with random unlabeled images, uses its predictions as stolen labels for a fake dataset, and trains a copycat network on that dataset.

  • Results

    Copycat networks achieved at least 93.7% of original-model performance with non-problem-domain data, at least 98.6% with additional problem-domain data, and at least 97.3% of Microsoft Azure Emotion API performance.

  • Takeaways & Limitations

    The study concludes that creating a copycat CNN is feasible by querying a target black box with random non-labeled data.

  • Takeaways & Limitations

    The investigated problems have few output classes, and the technique was evaluated on only a limited number of problems.

Abstract

from arXiv · show

In the past few years, Convolutional Neural Networks (CNNs) have been achieving state-of-the-art performance on a variety of problems. Many companies employ resources and money to generate these models and provide them as an API, therefore it is in their best interest to protect them, i.e., to avoid that someone else copies them. Recent studies revealed that state-of-the-art CNNs are vulnerable to adversarial examples attacks, and this weakness indicates that CNNs do not need to operate in the problem domain (PD). Therefore, we hypothesize that they also do not need to be trained with examples of the PD in order to operate in it. Given these facts, in this paper, we investigate if a target black-box CNN can be copied by persuading it to confess its knowledge through random non-labeled data. The copy is two-fold: i) the target network is queried with random data and its predictions are used to create a fake dataset with the knowledge of the network; and ii) a copycat network is trained with the fake dataset and should be able to achieve similar performance as the target network. This hypothesis was evaluated locally in three problems (facial expression, object, and crosswalk classification) and against a cloud-based API. In the copy attacks, images from both non-problem domain and PD were used. All copycat networks achieved at least 93.7% of the performance of the original models with non-problem domain data, and at least 98.6% using additional data from the PD. Additionally, the copycat CNN successfully copied at least 97.3% of the performance of the Microsoft Azure Emotion API. Our results show that it is possible to create a copycat CNN by simply querying a target network as black-box with random non-labeled data.

I. INTRODUCTION

The paper investigates whether black-box CNNs exposed through APIs can be copied using random unlabeled data, motivated by their commercial value and adversarial-example vulnerability. It proposes querying a target network to create a fake labeled dataset, training a copycat network, and evaluating the approach across local problems and a cloud API.

  • CNNs deliver state-of-the-art performance across visual classification, recognition, and object detection, motivating companies to provide models through APIs.
  • Adversarial-example vulnerability motivates the hypothesis that CNNs may operate in a problem domain without being trained on problem-domain examples.
  • The method queries a black-box target with random unlabeled data, uses its labels to build a fake dataset, and trains a copycat network on that dataset.
  • 93.7% of original-model performance was achieved by all copycat networks using non-problem-domain data, rising to 98.6% with additional problem-domain images.
  • 97% of Microsoft Azure Emotion API performance was copied, and the study reports that black-box CNN copying with random unlabeled data is possible.

II. RELATED WORKS

Earlier model-extraction studies copied conventional classifiers, text classifiers, or deep models for adversarial-example transfer. This paper extends black-box copying to deep CNNs using real-world, high-dimensional problems, random public images, and a public API.

  • Prior work copied decision trees, logistic regression, SVMs, multilayer perceptrons, Naive Bayes, and text-classification models through black-box queries.
  • A previous substitute-DNN approach achieved over 80% accuracy on augmented MNIST while approximating decision boundaries for adversarial-example generation.
  • This study differs by copying one deep model into another deep model across three real-world problems and high-dimensional inputs.
  • The method additionally uses random natural images from public datasets or the Internet, alongside problem-domain data, and attacks a public API.

III. COPYCAT CONVOLUTIONAL NEURAL NETWORK

The copycat procedure generates a fake dataset by labeling problem-domain and/or random non-problem-domain images with a black-box target CNN, then trains a separate network on those stolen labels. The copycat architecture and output layer are adapted to the target task, with pre-trained weights used where available.

  • A. Fake Dataset Generation: Fake datasets consist of random natural images labeled automatically by the target network, with labels called stolen labels.
  • A. Fake Dataset Generation: The attacker treats the target CNN as a black box that receives images and returns class labels, while discarding any available original labels.
  • A. Fake Dataset Generation: The approach can use either problem-domain images or publicly available non-problem-domain images, with the latter representing an attacker lacking same-domain data.
  • B. Copycat Network Training: The attacker may choose a copycat architecture different from the target architecture, provided its output layer matches the target network's number of classes.
  • B. Copycat Network Training: Copycat training fine-tunes a pre-trained model on the fake dataset, initializing the changed output layer with random weights.

IV. EXPERIMENTAL METHODOLOGY

The methodology compares an original target network and a smaller problem-domain baseline with three copycat configurations, evaluating all networks on a held-out test dataset.

  • Three copycats are evaluated: one trained on non-problem-domain data, one on problem-domain data, and one fine-tuned across both.The methodology distinguishes NPD-SL, PD-SL, and NPD+PD-SL configurations.
  • The experiment compares the original target network and a smaller problem-domain network trained with original labels as two baselines.The original network uses the original dataset, while the alternative uses less problem-domain data.
  • All baselines and copycats are evaluated on a non-overlapping problem-domain test dataset that none of the models access during training.The test dataset is reserved for performance evaluation.

1) Test Dataset (TD):

The test dataset measures every baseline and copycat on unseen problem-domain images, while the baseline comparison represents a cheaper model trained with limited labeled domain data.

  • Network Trained with Problem Domain Data and Original Labels (PD-OL): The PD-OL baseline uses a smaller, separately collected problem-domain dataset with original labels to represent a cheaper model.The dataset can be augmented to support a fair comparison because deep models require substantial training data.
  • Network Trained with Problem Domain Data and Original Labels (PD-OL): The experiments compare whether copycats can replace cheaper problem-domain models when those models do not fully replicate the original network.The comparison uses data different from the original training images but from the same domain.

1) Network Trained with Non-Problem Domain and Stolen Labels (NPD-SL):

The study tests copycats built from target-generated labels on non-problem-domain or problem-domain images, including a combined configuration fine-tuned with domain data, across several real-world tasks.

  • Network Trained with Non-Problem Domain and Stolen Labels (NPD-SL): NPD-SL tests whether random natural images unrelated to the problem domain can support copying through target-generated labels.The setup assumes the attacker lacks same-domain data and uses publicly available images instead.
  • Network Trained with Problem Domain and Stolen Labels (PD-SL): PD-SL uses a small set of unlabeled problem-domain images, assigning them labels predicted by the target network.The problem-domain dataset is shared with PD-OL, but its original labels are replaced by stolen labels.
  • Combined copycat: NPD+PD-SL fine-tunes the non-problem-domain copycat with the problem-domain data used for PD-SL.This configuration assumes access to both public unrelated images and a small unlabeled domain-specific set.
  • Evaluation scope: The copy attacks are evaluated on facial expression, object, and satellite crosswalk classification, plus a publicly available Microsoft API.The design compares non-problem-domain and problem-domain random data.

2) General Object Classification (GOC):

The experiments define task datasets and evaluate copycat performance with macro-averaged accuracy and ratios relative to the target and smaller problem-domain baseline; the Microsoft API evaluation also involves paid label generation.

  • Crosswalk classification: Crosswalk classification is binary satellite-image classification across Europe, America, and Asia, with European images split between testing and original-domain training.The cited procedure assigns 15% of European images to TD and 85% to ODD.
  • Microsoft Azure Emotion API: The Microsoft Azure Emotion API returns face locations and recognized emotions, and its pricing tiers affect the feasibility of generating stolen labels at experimental scale.The free tier is described as potentially prohibitive for the required number of queries.
  • Evaluation metrics: Performance is measured with macro-averaged accuracy and ratios against both the target network and the smaller problem-domain baseline.Macro averaging computes a simple average over classes, while a perfect copycat would match target accuracy and reach 100% of target performance.

E. General Setup

The experiments compare copycat networks trained with stolen labels against original-label baselines across three classification problems. Training used SGD, while the FER results show copycats surpassing the problem-domain baseline.

  • Training setup: SGD with a Step Down learning-rate policy trained the models, with copycat training capped at five epochs and target training continuing until loss plateaued.Target training used 30 epochs for FER, 20 for GOC, and 5 for SCC.
  • Experimental comparison: All experiments compared NPD-SL, PD-SL, and NPD+PD-SL copycats with OD and PD-OL baselines across three problems.The comparisons assessed whether random natural images and stolen labels could produce equivalent or near-equivalent performance.
  • FER results: 88.7% target accuracy and 82.5% PD-OL accuracy yielded 93.0% PD-OL performance over the FER target.All other copycat networks improved over the PD-OL baseline.
  • FER results: 93.7% target-relative performance was achieved by NPD-SL in FER, exceeding PD-OL performance by 100.7%.NPD-SL also performed better than the network trained with problem-domain data.

B. GOC

In GOC, copycats trained with stolen labels substantially outperformed the original-label problem-domain baseline and approached the target network. Adding problem-domain data did not improve the NPD-SL result in SCC.

  • GOC: 95.3% target accuracy and 82.1% PD-OL accuracy corresponded to 86.2% PD-OL performance over the GOC target.These values define the GOC baseline comparison.
  • GOC: 98.6% target-relative performance was achieved by NPD-SL in GOC, exceeding PD-OL performance by 14.4%.PD-SL was 9.5% better than PD-OL over the target, indicating the importance of stolen labels in the copy process.
  • GOC: NPD-SL and NPD+PD-SL achieved the greatest GOC accuracy, while PD-OL achieved the lowest.The PD-SL dataset did not improve NPD-SL in this problem domain.
  • SCC: 95.8% target accuracy and 92.8% PD-OL accuracy corresponded to 96.9% PD-OL performance over the SCC target.This establishes the SCC original-label baseline.
  • SCC: All stolen-label networks achieved SCC performance within less than 1% of the target and exceeded PD-OL.NPD-SL reached 99.2% of target performance, while NPD+PD-SL improved performance by only 0.1% over the other configurations.

D. Microsoft Azure Emotion API

The Microsoft Azure Emotion API experiment evaluated whether copycats could reproduce a cloud model’s performance. Random natural images achieved 97.3% of the API’s performance, while problem-domain data produced a comparison above the API.

  • API comparison: The Microsoft Azure Emotion API achieved 35.1% on the test dataset, substantially below the FER experiment results.Copycat networks nevertheless achieved performance very close to the API.
  • API comparison: 97.3% of target performance was achieved without problem-domain data, showing that random natural images supported the API copy attack.Using only problem-domain data reached 100.8% of API performance, while combining both data types was worse.
  • Overall comparison: Copycat networks trained with stolen labels outperformed the PD-OL baseline and achieved high performance over the target networks.Figure 3 compares PD/NPD inputs and original/stolen labels relative to the target network.
  • Interpretation: Copying target predictions, including mistakes, may be more effective than using a smaller correctly labeled dataset.The discussion therefore considers stolen labels appropriate for this task.
  • Experimental caveat: Preliminary experiments found that matching class distributions by discarding more than 40% of images was not beneficial to copying.The authors suggest the reduced image count may have limited the copycat’s ability to mimic decision frontiers and identify this for further exploration.

VI. CONCLUSION

The paper concludes that black-box CNNs can be copied using random unlabeled natural images and stolen labels, with performance close to original networks and a cloud API. It also identifies low output-class counts and limited problem coverage as scope boundaries.

  • Conclusion: The method creates a copycat of a deep CNN using only random unlabeled natural images.The paper frames this as an attack relevant to protecting cloud-based deep-learning models.
  • Conclusion: At least 93% of original-network performance was achieved without problem-domain data, and NPD+PD-SL reached 98.6%, 98.6%, and 99.3% over target for FER, GOC, and SCC.Copycats were always better than PD-OL networks and achieved at least 97.3% of Microsoft Azure Emotion API performance.
  • Limitations: The investigated problems had low numbers of output classes, and the technique was evaluated on only three problems.The paper proposes testing it with hundreds or thousands of classes and across more problems.
Loading 1806.05476v1…