PaperlayerView exact revision on arXiv

Source-linked AI summary

SCALE-UP: An Efficient Black-box Input-level Backdoor Detection via Analyzing Scaled Prediction Consistency

Junfeng Guo, Yiming Li, Xun Chen, Hanqing Guo, Lichao Sun, Cong Liu

arXiv:2302.03251v2cs.CR

TL;DR

MLaaS users need backdoor detection without access to deployed model internals, while many existing defenses require inspecting or modifying models. SCALE-UP uses prediction consistency during pixel-wise amplification to detect malicious inputs, and experiments report effectiveness, efficiency, and resistance to low-poisoning-rate adaptive attacks. Its scope is filtering poisoned testing samples rather than removing model vulnerability or recovering triggers.

  • Problem

    MLaaS users need to identify malicious inputs despite black-box deployed models, while many existing defenses require model access or modification.

  • Method

    SCALE-UP analyzes prediction consistency while amplifying all pixel values and is designed for black-box input-level detection under data-free and data-limited settings.

  • Results

    SCALE-UP shows promising detection across evaluated attacks, requires approximately 5% more inference time than standard inference, and achieves AUROC > 0.95 for successfully attacking samples at a 0.4% poisoning rate.

  • Takeaways & Limitations

    The method provides a black-box input-level defense for MLaaS and is reported to remain effective against potential adaptive attacks with low poisoning rates.

  • Takeaways & Limitations

    SCALE-UP filters poisoned testing samples but does not reduce deployed-model backdoor vulnerability or recover trigger patterns.

Abstract

from arXiv · show

Deep neural networks (DNNs) are vulnerable to backdoor attacks, where adversaries embed a hidden backdoor trigger during the training process for malicious prediction manipulation. These attacks pose great threats to the applications of DNNs under the real-world machine learning as a service (MLaaS) setting, where the deployed model is fully black-box while the users can only query and obtain its predictions. Currently, there are many existing defenses to reduce backdoor threats. However, almost all of them cannot be adopted in MLaaS scenarios since they require getting access to or even modifying the suspicious models. In this paper, we propose a simple yet effective black-box input-level backdoor detection, called SCALE-UP, which requires only the predicted labels to alleviate this problem. Specifically, we identify and filter malicious testing samples by analyzing their prediction consistency during the pixel-wise amplification process. Our defense is motivated by an intriguing observation (dubbed scaled prediction consistency) that the predictions of poisoned samples are significantly more consistent compared to those of benign ones when amplifying all pixel values. Besides, we also provide theoretical foundations to explain this phenomenon. Extensive experiments are conducted on benchmark datasets, verifying the effectiveness and efficiency of our defense and its resistance to potential adaptive attacks. Our codes are available at https://github.com/JunfengGo/SCALE-UP.

1 INTRODUCTION

The paper addresses black-box input-level backdoor detection for MLaaS, where users can query deployed models but cannot inspect or modify them. SCALE-UP exploits scaled prediction consistency: poisoned samples remain more prediction-consistent than benign samples when all pixel values are amplified.

  • Third-party DNNs enable practical deployment but expose MLaaS users to hidden backdoors embedded during training.Attacked models behave normally on benign samples, making malicious prediction manipulation difficult to identify.
  • Most existing backdoor defenses require accessing or modifying model weights, preventing their use in typical MLaaS settings.Some black-box defenses exist, but they rely on implicit assumptions about attacks.
  • SCALE-UP detects whether a suspicious input is malicious using only predictions from the deployed model.The paper frames input-level detection as a practical firewall for blocking and tracing malicious samples.
  • Poisoned samples show significantly more consistent predictions than benign samples when all pixel values are amplified.The paper names this phenomenon scaled prediction consistency and provides theoretical insights to explain it.
  • Experiments evaluate SCALE-UP under data-free and data-limited settings and report effectiveness against potential adaptive attacks.The contribution statement also characterizes the method as simple and effective for black-box input-level detection.

2 RELATED WORK

The related work reviews backdoor attacks by trigger type and defenses by defender access. It distinguishes white-box methods from black-box model-level and input-level defenses relevant to MLaaS.

  • 2.1 BACKDOOR ATTACK: Backdoor attacks inject adversary-specified triggers into selected training samples, causing attacked models to target an attacker-chosen label.Attacked models can behave normally on benign samples, making the attacks stealthy.
  • 2.1 BACKDOOR ATTACK: Patch-based attacks stamp localized trigger patches onto benign training samples and change their labels to a predefined target.BadNets is presented as the first backdoor attack in this category.
  • 2.1 BACKDOOR ATTACK: Non-patch-based attacks use full-image perturbations, image warping, or sample-specific triggers rather than small patches.These methods can break assumptions that triggers are sample-agnostic.
  • 2.2 BACKDOOR DEFENSE: White-box defenses require access to suspicious model source files and include model repair approaches such as pruning and distillation.Their goal is to remove hidden backdoors from attacked networks.
  • 2.2 BACKDOOR DEFENSE: Black-box defenses query deployed models and divide into model-level detection of attacked models and input-level detection of malicious inputs.The paper focuses on the input-level category.

3 THE PHENOMENON OF SCALED PREDICTION CONSISTENCY

The paper identifies scaled prediction consistency by amplifying every image pixel and comparing prediction behavior under benign and attacked models. It supports the phenomenon with experiments and an NTK-based theorem.

  • Phenomenon setup: The analysis studies prediction behavior of benign and poisoned samples as the basis for black-box input-level defense.The section first reviews the general backdoor-training process before presenting its observations.
  • Backdoor process: Backdoor training combines benign and poisoned samples, where poisoned inputs add a generated trigger and carry an attacker-specified target label.The attacked network is trained on both sample types using a loss objective.
  • Experimental setup: The experiment scales all pixel values because increasing trigger values does not hinder or may improve attack success, while trigger locations are unknown to defenders.BadNets and ISSBA are used on CIFAR-10 with ResNet, with attack success rates of at least 99%.
  • Empirical observation: Under attacked models, benign-sample confidence decreases during amplification while poisoned-sample confidence remains relatively stable.Under benign models, confidence for both sample types decreases; the contrast is named scaled prediction consistency.
  • Theoretical explanation: Theorem 1 states that when poisoned and benign sample counts approach equality, scaled attacked samples are constantly predicted as the target label under its kernel-regression assumptions.The paper interprets this as consistent target-label prediction when the attacked model over-fits poisoned samples.
  • Data-limited analysis: Benign samples have different SPC values across classes, so high-SPC benign samples may be mistakenly treated as malicious.This class variation can reduce the method’s precision in the data-free analysis.

4 SCALED PREDICTION CONSISTENCY ANALYSIS (SCALE-UP)

SCALE-UP detects malicious inputs in black-box MLaaS settings by analyzing prediction consistency across pixel-wise amplified images. It supports data-free and data-limited operation, with local benign samples used to normalize SPC values in the latter setting.

  • Method: SCALE-UP analyzes amplified versions of suspicious inputs using only deployed-model predictions to detect input-level backdoors.The method is designed for black-box MLaaS settings where defenders can query the model but lack model information or attack knowledge.
  • Data-free analysis: SPC measures the proportion of amplified-image labels that remain consistent with the original input label.Scaled images are generated using a defender-specified scaling set while constraining amplified pixel values to [0, 1].
  • Detection rule: Inputs with SPC(x) > T are classified as backdoor samples using a defender-specified threshold T.The threshold provides the decision rule for converting prediction consistency into a malicious-input judgment.
  • Data-free analysis: Data-free SCALE-UP can misclassify benign samples because SPC varies across classes under attacked models.Class-specific differences mean some benign samples have high SPC values and may reduce detection precision.
  • Data-limited analysis: Data-limited SCALE-UP uses a few benign samples from each class to estimate SPC statistics and normalize the suspicious input’s SPC value.The normalized value is computed relative to local benign samples and used in the main detection pipeline.

5 EXPERIMENTS

Experiments evaluate SCALE-UP across benchmark datasets, black-box baselines, efficiency settings, and adaptive attacks. The method performs strongly across attacks, remains effective under several adaptive settings, and adds little inference overhead.

  • Main Settings: Experiments use CIFAR-10 and Tiny ImageNet with ResNet models, evaluating six representative patch-based and non-patch-based attacks.
  • Main Results: SCALE-UP outperforms or matches baseline defenses across evaluated attacks, including a 0.5 AUROC advantage over all baselines for WaNet on Tiny ImageNet.Its effectiveness is also on par with or better than baselines for classical patch-based attacks and exceeds STRIP, which requires predicted probabilities.
  • Efficiency: SCALE-UP requires fewer inference times than almost all baselines and is approximately 5% slower than standard inference without defense.ShrinkPad is the only faster exception, but its effectiveness is significantly lower.
  • Resistance to Adaptive Attacks: SCALE-UP maintains AUROC above 0.93 across different trigger sizes under both data-free and data-limited settings.These experiments use BadNets and PhysicalBA on Tiny ImageNet.
  • Resistance to Adaptive Attacks: At a 0.4% poisoning rate, SCALE-UP achieves AUROC above 0.95 on successfully attacking poisoned samples and above 0.8 on all poisoned samples.The attack success rate is below 70% at the 0.4% poisoning rate.
  • Resistance to Adaptive Attacks: A fully adaptive attack bypasses SCALE-UP with AUROC 0.467, but small random noise below 0.3 substantially reduces its attack success rate.The noise has minor adverse effects on the vanilla attack, suggesting a practical defense against this adaptive strategy.
  • Additional Discussion: The paper also examines whether alternative augmentation methods could preserve SCALE-UP effectiveness, motivated by scaling being a form of data augmentation.

6 CONCLUSION

The paper proposes SCALE-UP, a black-box input-level backdoor detector for MLaaS based on scaled prediction consistency. Experiments support its effectiveness, efficiency, and resistance to potential adaptive attacks.

  • SCALE-UP detects poisoned inputs from model predictions by exploiting their greater prediction consistency during pixel-wise amplification.

ETHICS STATEMENT

The ethics statement frames the work as defensive while identifying boundaries on what SCALE-UP can address. The method filters poisoned testing samples but does not remove model vulnerability or recover triggers.

  • SCALE-UP is presented as purely defensive, but it only filters poisoned testing samples and does not reduce deployed models’ intrinsic backdoor vulnerability.It also cannot recover trigger patterns.

A THE OMITTED PROOF OF THEOREM 1

The proof analyzes an RBF-kernel model's prediction for an attacked sample under pixel scaling and shows convergence toward the target label as poisoned and benign sample counts become comparable.

  • Theorem 1: Theorem 1 states that C(n·x′) converges to the target label y_t as N_p approaches N_b for n ≥ 1.The theorem assumes uniform sampling, K classes, an RBF kernel, and cross-entropy optimization.
  • Proof setup: The derivation starts from the NTK regression solution with an RBF kernel K(x,x_i)=e^-2γ||x−x_i||2 and simplifies it for a backdoored sample.The proof defines φ_t as the predictive probability for target class t and uses the poisoned-sample formulation x′=(1−m)⊙x+m⊙t.
  • Case n = 1: For n = 1, the positive internal term implies that the attacked sample is predicted as the target label y_t.The proof notes that 1 − e^-2γ||m⊙(t−x_i)||2 is always positive.
  • Case n > 1: For n > 1, the proof compares kernel sums at the scaled attacked input to characterize prediction behavior under amplification.The comparison relies on the internal distance term involving the trigger and benign samples.
  • Case n > 1: Assuming pixel-level trigger residues have approximately zero mean, the proof derives φ_t(n·x′) ≥ 0.5 and obtains the limiting result as N_p approaches N_b.The zero-mean assumption is applied to rewrite the distance expression before combining the relevant equations.

C THE DETAILS FOR TRAINING ATTACKED MODELS

This section describes attacked-model training, evaluation datasets, baseline implementations, inference-time measurement, and experiments across varied trigger and label settings.

  • Training attacked models: Backdoor-infected models are trained for 200 epochs with a 5–10% poisoning rate, while infected labels are randomly selected to preserve benign-input stealthiness.Except for PhysicalBA, training uses no additional data augmentation to expose attack properties.
  • Evaluation: The evaluation reports model accuracy and attack success rate for each task, with detailed results provided in Table 4.These metrics characterize benign performance and attack effectiveness before detection comparisons.
  • Inference-time measurement: Inference time measures identifying poisoned inputs and then predicting benign inputs, using batch queries where multiple augmented predictions are required.SCALE-UP, STRIP, ShrinkPad, and DeepSweep reuse target-model predictions, whereas Frequency uses a secondary network.
  • Multiple triggers: Under multiple triggers targeting one label, SCALE-UP remains resilient as the number of injected triggers increases.The experiment uses varied poisoned samples and reports the result in Figure 9.
  • Multiple labels and coefficients: Multiple infected labels slightly reduce SCALE-UP performance, while increasing the scaling coefficient n improves performance and usually reaches optimal performance at n ≥11.The coefficient study compares six attacks in both data-limited and data-free settings.
  • Local samples: With at least 100 local samples per label, SCALE-UP achieves optimal performance in the data-limited setting.The sensitivity study evaluates six attacks on Tiny ImageNet with ResNet-34.
  • Partial backdoors: For source-label-specific backdoors, SCALE-UP outperforms all baseline defenses, whereas STRIP is not resilient in this scenario.The comparison concerns partial backdoor attacks in which effectiveness is restricted to images from a specific source class.

L THE ROBUSTNESS OF SCALE-UP

The robustness evaluation tests SCALE-UP under noisy inputs and compares its performance with documented baselines and random-noise variants.

  • Robustness evaluation: SCALE-UP is evaluated separately on benign and poisoned samples under varying random-noise magnitudes while preserving benign accuracy and attack success rate.The evaluation uses inference-phase robustness tests and reports results in Figure 13.
  • Noise setting: The robustness experiment uses Gaussian noise on Tiny ImageNet and tests BadNets and PhysicalBA after filtering failed poisoned samples.These attacks were selected because they remain robust against random noise in the referenced analysis.
  • Reported metrics: Table 6 reports AUROC for Tiny ImageNet under VGG-19, while Table 7 reports AUROC for random-noise SCALE-UP variants on CIFAR-10 and Tiny ImageNet.Failed cases are defined as AUROC below 0.55 in both table descriptions.
  • Result: SCALE-UP is robust against noisy poisoned samples under the tested conditions.The reported result concerns the selected attacks and the Tiny ImageNet evaluation.

M ADDITIONAL RESULTS UNDER VGG ARCHITECTURE

Additional VGG-19 experiments on Tiny ImageNet test whether scaled prediction consistency and SCALE-UP effectiveness extend beyond the ResNet architecture.

  • Scaled prediction consistency: Under VGG-19, scaled prediction consistency still holds across all tested cases.The additional experiments are conducted on Tiny ImageNet to examine architectural validity.
  • Confidence behavior: Benign-sample confidence decreases significantly faster than poisoned-sample confidence as the multiplication time increases.This behavior is reported in Figure 14.
  • Scaling-process ablation: Replacing scaling with varied random noise is far less effective, especially for WaNet and ISSBA.The authors speculate that invisible full-image triggers make trigger-related features less robust, while noting that they do not yet provide a theoretical analysis.

P POTENTIAL LIMITATIONS AND FUTURE WORK

SCALE-UP has limitations tied to its overfitting assumption and may degrade on simple datasets, while the authors plan broader validation and application.

  • Evaluation Context: Figure 14 measures average confidence under pixel-wise multiplications, while Figures 15 and 16 show defense ROC curves on CIFAR-10 and Tiny ImageNet.
  • Limitations: SCALE-UP requires attacked DNNs to overfit poisoned samples; low attack success rates may sharply degrade detection performance.
  • Limitations: The defense may fail on simple tasks such as MNIST and GTSRB, where models can also overfit benign samples.
  • Future Work: The authors plan to generalize the methods to continual, non-transferable, federated, audio, and visual tracking settings, and evaluate ViT and GCN architectures.
Loading 2302.03251v2…