PaperlayerView exact revision on arXiv

Source-linked AI summary

Enhancing Robustness of Machine Learning Systems via Data Transformations

Arjun Nitin Bhagoji, Daniel Cullina, Chawin Sitawarin, Prateek Mittal

arXiv:1704.02654v4cs.CRcs.LG

TL;DR

Machine-learning classifiers remain vulnerable to evasion attacks, while existing defenses have limited applicability. The paper uses linear data transformations, including PCA and anti-whitening, and evaluates them across attacks, classifiers, and domains. The defense reduces adversarial success rates while imposing modest benign-accuracy costs, but does not eliminate evasion attacks in all cases.

  • Problem

    Existing defenses against evasion attacks are few and limited to particular attacks and classifier types.

  • Method

    The paper applies linear data transformations, including dimensionality reduction and anti-whitening, during classification and training to improve robustness.

  • Results

    Across attack strategies, classifiers, and applications, the defense reduces adversarial success rates by about 2×, with a 0.5-2% benign-classification utility reduction.

  • Takeaways & Limitations

    The tunable, classifier- and dataset-agnostic defense supports utility-security tradeoffs and can be combined with other defenses.

  • Takeaways & Limitations

    The defense does not make adversarial success rates negligible in all cases.

Abstract

from arXiv · show

We propose the use of data transformations as a defense against evasion attacks on ML classifiers. We present and investigate strategies for incorporating a variety of data transformations including dimensionality reduction via Principal Component Analysis and data `anti-whitening' to enhance the resilience of machine learning, targeting both the classification and the training phase. We empirically evaluate and demonstrate the feasibility of linear transformations of data as a defense mechanism against evasion attacks using multiple real-world datasets. Our key findings are that the defense is (i) effective against the best known evasion attacks from the literature, resulting in a two-fold increase in the resources required by a white-box adversary with knowledge of the defense for a successful attack, (ii) applicable across a range of ML classifiers, including Support Vector Machines and Deep Neural Networks, and (iii) generalizable to multiple application domains, including image classification and human activity classification.

I. INTRODUCTION

Machine-learning systems are increasingly deployed in adversarial settings, but existing evasion defenses are few and narrowly applicable. This paper proposes linear data transformations and evaluates them across attacks, classifiers, and application domains.

  • Evasion attacks perturb test inputs to cause misclassification, threatening machine-learning applications in adversarial settings.
  • Existing defenses are few and limited to particular known attacks and machine-learning classifiers.
  • The paper proposes linear data transformations, including dimensionality reduction, to improve resilience during classification and training.
  • The evaluation covers SVMs and DNNs, multiple evasion attacks including white-box attacks, and MNIST and UCI HAR datasets.
  • Up to 5× more modification is required for successful attacks, while adversarial success rates decrease by around 2−50× at fixed perturbation levels.
  • The defense incurs a modest 0.5-2% loss in classification success on benign samples in most cases.

A. Attacks against machine learning systems

The paper models evasion attacks as perturbations of correctly classified inputs and distinguishes adversaries by their knowledge of the classifier and defense. It reviews attacks for linear SVMs and neural networks.

  • An adversary applies an attack algorithm to a correctly classified input to produce a modified input intended to cause misclassification.
  • Targeted attacks seek classification as a specific incorrect class, while adversarial examples should remain close to the benign input under a chosen p-norm.
  • In the white-box setting, the adversary knows the classification function, its parameters, and any defense.
  • Adversarial knowledge: In classifier mismatch, the adversary knows the original training setup but is unaware of the defense and attacks a surrogate classifier.
  • Adversarial knowledge: In architecture mismatch, the adversary is unaware of the classifier architecture and trains another model on part of the training data.
  • Linear SVM attacks: For multiclass linear SVMs, an untargeted attack selects the smallest perturbation among targeted class-specific attacks.

2) Gradient based attacks on neural networks:

The paper describes gradient-based and optimization-based attacks against neural networks, including an ℓ2-constrained modification of FGS for comparison across classifiers.

  • FGS generates neural-network adversarial examples under the ℓ∞ metric using noise proportional to the sign of the loss gradient.
  • Backpropagation efficiently computes the gradient, while η controls the magnitude of the adversarial perturbation.
  • The paper introduces Fast Gradient, an ℓ2-constrained modification of FGS to compare classifier robustness and defense effectiveness.
  • Optimization-based attacks: Direct adversarial-sample optimization minimizes input distance subject to a misclassification constraint, with Carlini-Wagner providing a studied relaxation.
  • Optimization-based attacks: The neural-network experiments use a Carlini-Wagner loss with the ℓ2 distance metric and evaluate the defense against this state-of-the-art attack.

III. DATA TRANSFORMATIONS AS A DEFENSE

The defense selects a data-dependent linear transformation, transforms training and classification inputs consistently, and retrains the classifier. For SVMs, this changes regularization and can exclude or penalize weight directions, altering robustness-performance tradeoffs.

  • Defense overview: A selected linear transformation is applied to training data before retraining, and every classification input is transformed identically.The resulting classifier composes the trained model with the transformation.
  • Defense overview: Standard training may learn available linear mappings without optimizing adversarial robustness, so explicitly specifying a transformation can change the learned classifier.The paper argues that carefully selected transformations can produce beneficial changes.
  • Support Vector Machines: For invertible transformations, the defense is equivalent to modifying the quadratic regularization form in the SVM loss.The transformed classifier can be rewritten using w = B^T u.
  • Support Vector Machines: More general linear transformations provide flexibility beyond ordinary scalar SVM regularization and expose additional robustness-performance tradeoffs.Ordinary regularization corresponds to scaling by B = 1/√λ I.
  • Support Vector Machines: Non-invertible transformations both modify weight costs and rule out weight vectors outside the image of B^T.The regularization can be viewed as assigning infinite cost to vectors in ker B.
  • Support Vector Machines: The transformation choice can influence the final classifier substantially, allowing essentially any hard-decision classifier positively correlated with fixed true labels.This expressivity motivates careful transformation selection.

C. Defense using PCA

PCA identifies maximum-variance directions and projects data onto selected principal axes, reducing dimensionality while retaining a chosen amount of variance. The paper formalizes this through the covariance eigendecomposition and projected data matrix.

  • PCA in brief: PCA identifies principal axes as directions of maximum data variance and projects the data along selected axes.Choosing k axes reduces dimensionality, with k determined by the desired retained variance.
  • PCA in brief: The centered-data covariance matrix decomposes as C = XX^T = UΛU^T, with eigenvalues ordered λ1 ≥ ... ≥ λd ≥ 0.The columns of U are covariance eigenvectors, and λi is the variance along the ith component.
  • PCA in brief: Each column of U^T X represents a sample in the principal-component basis, while X_k projects samples onto the subspace of the k largest components.The projection uses the k-dimensional subspace associated with the largest eigenvalues.
  • PCA in brief: The retained variance equals the sum of the k largest eigenvalues.The passage expresses retained variance as Σ from i = 1 to k of λi.

2) Implementing the defense:

The implementation retains top principal components or applies anti-whitening, then uses the transformed representation to reshape which classifier directions are learned. PCA removes low-variance directions, while anti-whitening penalizes them more softly.

  • Implementing the defense: A compact projection B = I_k^T U and a full-dimensional transformation B = U I_k I_k^T U preserve the k largest principal components.The compact projection is more efficient for training, while the full-dimensional form facilitates comparison with original data.
  • Intuition behind the PCA defense: For linear SVMs, low-variance principal components tend to receive large coefficients because equal influence requires |(U^T w)_i| proportional to 1/√λ_i.The classifier must use a large coefficient to exploit information in a component with small variation.
  • Intuition behind the PCA defense: Figure 2 reports that low-variance components often have large SVM coefficients, while coefficients generally decrease as √λ_i increases.The first principal component is an exception because it is strongly aligned with the difference of class means.
  • Intuition behind the PCA defense: The PCA defense denies attackers access to low-variance directions by forcing the classifier to assign them no weight.The classifier loses information, but exploiting that information required large coefficients and therefore exposed useful attack directions.
  • Implementing the defense: Eliminating low-variance components mostly removes the relationship between component variance and the corresponding coefficient of U^T w.This changes the weight structure learned by classifiers trained on the projected data.
  • Anti-whitening: Anti-whitening exaggerates variance disparities and introduces C^-c regularization, increasing the price of low-variance components without eliminating them.It is a softer alternative to the hard cutoff imposed by PCA.

2) Neural networks:

For neural networks, explicitly inserting a transformation differs from learning a first linear layer because ordinary training does not optimize that layer for robustness. White-box attacks therefore operate on the composed defended classifier.

  • Neural networks: A learned first linear layer in a neural network does not necessarily provide the desired PCA-based robustness because training does not optimize it for robustness.The defense instead separately specifies the linear-layer weights using PCA.
  • Neural networks: For neural networks, usual weight-matrix regularization does not follow the same robustness intuition as the proposed transformation.The paper distinguishes this behavior from linear SVMs.
  • White-box attacks: The proposed attacks remain applicable because defended classifiers belong to the same family as undefended classifiers, allowing simple attack modifications.This applies to both linear classifiers and neural networks.
  • White-box attacks: In the white-box setting, the adversary attacks the overall classifier f(x) = f_aux(Bx), accounting for the known transformation.For SVMs, the input-operating weight term is replaced by B^T w_i.
  • White-box attacks: Gradient-based attacks compute loss with respect to Bx but perturb the original input x to maximize the resulting loss increase.The paper describes this as the first-order attack objective for the defended network.
  • Neural networks: In experiments, the precomputed transformation is inserted as a linear layer after the neural network input layer.The optimization-based attack keeps the same objective while changing the classifier used to compute loss.

2) Classifier mismatch attacks:

The classifier mismatch setting models adversaries using a classifier or architecture that differs from the defended system. The defense is presented as mitigating adversarial-sample transferability while preserving broad experimental applicability.

  • Classifier mismatch attacks: The adversary may mimic the original classifier without knowing the defense, or use a different training procedure and classifier family.Examples include a defended three-layer neural network paired with an adversary’s five-layer neural network.
  • Classifier mismatch attacks: Linear transformations can mitigate threats arising from adversarial-sample transferability in classifier and architecture mismatch settings.The paper frames these settings as relevant because transferability remains an open research question.
  • Experimental scope: The evaluation spans MNIST and human activity recognition using Linear SVMs and varied neural networks.The experiments use 60,000 training and 10,000 test images for MNIST, while the activity dataset contains 7,352 training and 2,947 test samples.
  • Experimental scope: The reported classification accuracies are 91.5% for MNIST and 96.7% for the human activity recognition dataset.These values describe the evaluated datasets before considering robustness improvements.

2) Neural networks:

The neural-network evaluation examines transformed-data defenses across architectures and experimental attack settings. Results are summarized across classifier families, datasets, attacks, and linear transformations, with adversarial success measured against correctly classified benign inputs.

  • Neural networks: The evaluated neural networks include FC100-100-10 and FC200-200-200-10 fully connected architectures.The first architecture has two hidden layers with 100 neurons each; the second is a larger variant.
  • Linear transformation techniques: The defense applies PCA and other linear transformations to training data and classification pipelines to improve resilience against evasion attacks.PCA projects data onto selected components, while the experiments also investigate anti-whitening.
  • Metrics: Adversarial success counts attacks that change a correct classification into an incorrect one and divides by correctly classified benign samples.The evaluation relates this rate to the perturbation distance ϵ = ∥x −˜x∥.
  • Evaluation design: The evaluation varies datasets, classifiers, attacks, and linear transformations while fixing the remaining pipeline components.The results are summarized in Table II and address mismatch, white-box, classifier-family, dataset, and transformation questions.
  • Classifier mismatch attacks: Figure 4 plots adversarial success against perturbation magnitude for different reduced dimensions in the MNIST classifier-mismatch setting.The attack targets the original classifier while the defense effect is shown for each reduced dimension k.
  • White-box attacks: Figure 5 evaluates the defense against optimal white-box attacks on MNIST Linear SVMs.The figure concerns an adversary aware of the dimensionality-reduction defense.

1) Defense in the classifier mismatch setting:

Linear data transformations substantially reduce evasion success in classifier-mismatch and white-box settings, while allowing robustness to be tuned through the reduced dimension. The defense applies across SVMs and neural networks, but neural-network and SVM robustness comparisons use attacks of different optimality.

  • Classifier mismatch setting: At ϵ = 1.0, PCA with k = 80 reduces adversarial success from 100% to 3.4%, a 96.6% or around 29.4× decrease.At ϵ = 0.5, success falls from 99% to 2%, a 49.5× decrease.
  • Classifier mismatch setting: At ϵ = 1.0, reducing k from 331 to 10 lowers adversarial success from 56.7% to 1.42%.The intermediate rates are 5.9% at k = 100 and 4.4% at k = 30.
  • White-box setting: A 5.83× larger budget is required for 86.6% misclassification with k = 40, increasing from 0.3 without defense to 1.75.For 98% misclassification, the budget rises from 0.5 to 2.5, a 5× increase.
  • Utility-security tradeoff: At k = 80, benign classification falls from 91.5% to 90.64%, while the perturbation needed for 50% misclassification rises from 0.16 to 0.95.The reported utility loss is about 1.2%, alongside a 5.9× security gain.
  • Cross-classifier applicability: The defense is effective for Linear SVMs and neural networks across multiple attack types, supporting applicability across classifier architectures.The neural-network evaluation includes FG, FGS, and Carlini’s attack, while the comparison notes that FG is only a first-order approximation to an optimal attack.

2) Defense against Fast Gradient Sign attack in the white box setting:

The defense reduces attack success against FGS and Carlini attacks in white-box neural-network experiments, and also transfers across datasets and settings. Anti-whitening provides a tunable alternative to PCA, with modest benign-accuracy costs reported across evaluated configurations.

  • Fast Gradient Sign attack: At η = 0.05, PCA with k = 40 reduces FGS adversarial success from 41.64% to 10.14%, a 31.5% or 4.1× reduction.At η = 0.11, success falls from 91.59% to 48.92%, a 42.67% or 1.87× reduction.
  • Fast Gradient Sign attack: A 2.1× larger perturbation is required for 90% FGS misclassification with k = 40, increasing from 0.11 to 0.23.The FGS parameter η is restricted to less than 0.25 because MNIST pixels are normalized to [0, 1].
  • Carlini attack: Against Carlini’s white-box ℓ2 attack, k = 40 lowers success at norm 1.0 to 29.5%, a drop of 35.5% or 2.2×.The budget for 90% misclassification increases from 1.5 to 2.16.
  • Architecture mismatch: In the architecture-mismatch setting, k = 40 reduces adversarial success at norm 1.5 from 51.2% to 13.2%, a 38.0% or 3.9× drop.The results are described as mitigating transferability of adversarial samples to a large extent.
  • HAR dataset: On HAR, PCA with k = 70 reduces white-box success at ϵ = 0.5 from 77.3% to 48.3%, while the budget for 90% misclassification increases from 0.65 to 0.93.The passage reports a 2× adversarial-budget increase and a 2.3% utility drop.
  • Anti-whitening defense: Anti-whitening with c = 2 reduces success at ϵ = 1.0 from 41.42% to 17.06%, a 24.36% or 2.4× fall.Its benign accuracy is 97.32%, compared with 97.47% without defense; the transformation boosts rather than drops high-variance components.

VI. DISCUSSIONS, LIMITATIONS AND FUTURE WORK

The defense improves robustness across attacks, classifiers, and application settings, but PCA has important CNN-specific limitations and may need to complement other defenses.

  • Limitations: The defense reduces adversarial success rates across varied settings, but non-trivial success remains in some cases.The authors suggest combining it with adversarial training or ensemble-based detection when residual attack success is substantial.
  • Limitations: PCA provides only marginal security improvement for the Papernot-CNN, likely because dimensionality reduction removes local information used by convolutional layers.The authors identify alternative transformations, including autoencoders and kernel PCA, as future directions.
  • Related work: Existing defenses often target specific classifier families or domains and may remain vulnerable to strategic adversaries who know the defense.Prior approaches include regularization for SVMs, tree-specific optimal-attack defenses, neural-network structural changes, image transformations, compression, and ensembles.
  • Future work: The defense is classifier- and dataset-agnostic and can be combined with adversarial training for greater robustness.The proposed combination could train neural networks on reduced-dimension samples while incorporating adversarial loss.
  • Conclusion: Across attack strategies, classifiers, and applications, the proposed defenses produced a 2x reduction in adversarial success rates with a 0.5-2% utility reduction.The authors also report computational efficiency and describe the approach as a foundation for countering evasion attacks.

IX. APPENDIX

The appendix describes PCA’s computational overhead, adversarial-training evaluation on MNIST Linear SVMs, input projection costs, and the CNN architecture used in experiments.

  • Computational overhead: PCA incurs a one-time O(d2n + d3) overhead to compute the covariance matrix and eigenvector decomposition.Training a classifier on reduced-dimension data adds one-time overhead but should require less time than training on the original dimensionality.
  • Adversarial training: Figure 14 evaluates adversarial training on MNIST against optimal white-box attacks targeting Linear SVMs.The Linear SVM is trained by gradient descent with periodically augmented training sets containing adversarial samples at specified perturbation values.
  • Computational overhead: Each subsequent input incurs O(dk) overhead for matrix multiplication that projects it onto the principal components.This is the recurring inference-time cost after the principal components have been computed.
  • CNN architecture: The evaluated CNN uses two convolutional layers with 32 filters, two with 64 filters, max pooling after each pair, and fully connected layers with 200 neurons.Its softmax output has 10 neurons corresponding to MNIST’s 10 classes.
Loading 1704.02654v4…