PaperlayerView exact revision on arXiv

Source-linked AI summary

Generative Poisoning Attack Method Against Neural Networks

Chaofei Yang, Qing Wu, Hai Li, Yiran Chen

arXiv:1703.01340v1cs.CRcs.LGstat.ML

TL;DR

Neural-network poisoning remains comparatively underexplored despite the security risk of retraining on publicly collected data. The paper examines a direct gradient attack, proposes a reward-updated generative method and loss-based detection, and reports substantially faster poisoned-data generation with slightly lower accuracy degradation.

  • Problem

    Poisoning attacks against neural networks, especially DNNs, have been studied far less than attacks against SVMs despite the risk of poisoned retraining data.

  • Method

    The paper examines a direct gradient method, proposes an autoencoder-based generator updated by loss rewards, and designs loss-based attack detection.

  • Results

    The generative method substantially improves poisoned-data generation rate, reaching up to 239.38x speedup over the direct gradient method with slightly lower accuracy degradation.

  • Takeaways & Limitations

    Experiments indicate that the generative method has potential for attacking larger neural-network models and bigger datasets, while loss can support low-overhead detection.

Abstract

from arXiv · show

Poisoning attack is identified as a severe security threat to machine learning algorithms. In many applications, for example, deep neural network (DNN) models collect public data as the inputs to perform re-training, where the input data can be poisoned. Although poisoning attack against support vector machines (SVM) has been extensively studied before, there is still very limited knowledge about how such attack can be implemented on neural networks (NN), especially DNNs. In this work, we first examine the possibility of applying traditional gradient-based method (named as the direct gradient method) to generate poisoned data against NNs by leveraging the gradient of the target model w.r.t. the normal data. We then propose a generative method to accelerate the generation rate of the poisoned data: an auto-encoder (generator) used to generate poisoned data is updated by a reward function of the loss, and the target NN model (discriminator) receives the poisoned data to calculate the loss w.r.t. the normal data. Our experiment results show that the generative method can speed up the poisoned data generation rate by up to 239.38x compared with the direct gradient method, with slightly lower model accuracy degradation. A countermeasure is also designed to detect such poisoning attack methods by checking the loss of the target model.

I. INTRODUCTION

The paper frames neural-network poisoning as an underexplored causative attack and proposes faster generation plus loss-based detection. It examines direct gradients, introduces a generative alternative, and evaluates both schemes experimentally.

  • Threat and research gap: Poisoning attacks manipulate training data with artificial labels to alter neural-network parameters, but comparatively few methods target NNs.Prior poisoning work has focused extensively on SVMs, whose mathematical basis is clearer.
  • Proposed approach: The generative method uses an autoencoder to produce poisoned data, updates it through a loss-based reward, and sends outputs to the target NN.The target model acts as a discriminator by calculating loss relative to normal data and returning gradients.
  • Detection: A loss-based countermeasure detects poisoning with very low computational overhead by checking loss differences between normal and poisoned inputs.Normal inputs are expected to produce smaller loss because they remain close to the original decision region.
  • Proposed approach: Traditional gradient-based poisoning is examined first, with poisoned-data generation identified as its implementation bottleneck.The paper explicitly investigates the direct gradient method for NNs.
  • Evaluation: The schemes are extensively evaluated on MNIST and CIFAR-10 under different configurations.The evaluation covers both the attack methods and the proposed detection approach.

III. POISONING ATTACK AGAINST NEURAL NETWORKS

The paper establishes notation for normal and poisoned data, model versions, layer parameters, loss timestamps, and activation functions used in its neural-network attack formulation.

  • Notation: x_i denotes normal data, while x_p denotes poisoned data; subscripts identify the corresponding variable type.The notation distinguishes ordinary inputs from attacker-generated inputs.
  • Notation: Superscripts (o) and (p) distinguish the original model from the poisoned model.For example, w^(o) denotes the original model’s full parameter set.
  • Notation: L_i^(p)(t) denotes the poisoned model’s loss on normal data at time t.The timestamp tracks the loss during the attack process.
  • Network notation: w_k^(o) denotes the parameters of layer k, and ϕ denotes an activation function such as Tanh or ReLU.The notation covers both layer-specific parameters and the nonlinear activation used by the network.

B. The Attacking Process

The attacking process repeatedly generates and injects poisoned data, observes the poisoned model on normal inputs, and updates the generation method using the resulting information.

  • Attack assumptions: The attacker is assumed able to insert arbitrary poisoned data x_p with artificial label t_p into the training dataset.This is the paper’s operational assumption for poisoning attacks.
  • Attack assumptions: The attacker is assumed to know the original NN’s structure and exact weights, while lacking direct ability to modify the model.The attack instead seeks to degrade model accuracy through training-data poisoning.
  • Attack loop: The process begins by generating poisoned data and injecting it into the original model to produce the poisoned model.These are the first two steps of the repeated attack loop.
  • Attack loop: Normal data are sent through the poisoned model to obtain information such as loss and gradients at different timestamps.This feedback supplies the measurements used to adjust data-generation behavior.
  • Attack loop: The generation method is updated from that information, and the procedure repeats from poisoned-data generation.The loop alternates between producing poison, measuring its effect, and revising the generator.

C. The Data Gradient and Direct Gradient Method

The direct gradient method formulates poisoning as maximizing the target model’s loss on normal data, then updates poisoned inputs along their loss gradients. Its element-wise gradient computation can effectively degrade accuracy but creates a scalability bottleneck proportional to input dimension and model complexity.

  • Poisoning minimizes target-model accuracy by maximizing loss on normal data xi after injecting poisoned data xp.
  • The gradient calculation requires second partial derivatives because the target model parameters and poisoned input are complex functions of each other.The second-derivative term is difficult to calculate explicitly or through a backpropagation-like method.
  • The procedure initializes and trains a normal model, selects mislabeled poisoned data, repeatedly injects it, evaluates loss, and updates the poisoned input until convergence.The algorithm outputs poisoned data xp(t) with attacking label tp.
  • The direct gradient method computes gradients with respect to each element of xp and updates xp using the gradient direction and coefficient α.The coefficient may be fixed, such as α = 0.1, or changed adaptively.
  • The direct method can degrade target-model accuracy, but its element-wise generation is time-consuming and scales with input dimension and target-model complexity.This may create severe scalability issues as neural networks and training datasets grow.

D. A Generative Method to Accelerate Poisoned Data Generation

The generative method uses an autoencoder generator and target-model feedback to produce poisoned data while bypassing costly direct gradient calculation. A loss-difference reward and target-model gradients guide generator updates, reducing generation time.

  • Generative method: The method trains an autoencoder generator to produce poisoned data and uses the target NN as a discriminator.The generator receives loss and gradient information from the discriminator to guide its updates.
  • Generative method: The generative method collects target-model gradients with respect to normal data and updates the generator rather than calculating poisoned-data gradients element-wise.Only one target-model update is needed per iteration, reducing generation time especially for complex models.
  • Generative method: The algorithm initializes a normal network, selects and relabels starting poisoned data, and iteratively injects generated data while updating the generator.Iterations stop when the loss condition involving threshold Lth is met, producing poisoned data with attacking label tp.
  • Generative method: A reward based on the difference between consecutive attack losses evaluates generator updates.The reward is combined with target-model gradients to form a weighted generator gradient; poor updates are punished.

E. The Loss-based Countermeasure against Poisoning Attack

The countermeasure monitors target-model loss after each input is injected and uses thresholds to trigger an accuracy check for possible poisoning attacks.

  • Detection procedure: The detector records target-model loss, raises warnings when loss exceeds Lth, and triggers an accuracy check after warnings exceed Wth.It relies on poisoned input-label pairs usually producing larger loss than normal pairs.
  • Detection rationale: The method distinguishes inputs by monitoring loss because poisoning attacks seek to maximize loss and alter the target model’s decision boundary.Normal data generally remains inside the decision region and induces relatively smaller loss.

IV. EXPERIMENT RESULTS

Experiments evaluate direct-gradient and generative poisoning attacks on MNIST and CIFAR-10, comparing accuracy degradation, generation time, and loss-based detection feasibility.

  • Experimental setup: Experiments use MNIST and CIFAR-10 to examine poisoned-data generation under different configurations.The evaluation compares target-model accuracy degradation and time consumption for direct-gradient and generative attacks.
  • Experimental setup: The study also evaluates the feasibility of a loss-based countermeasure against poisoning attacks.All experiments are performed on a GeForce GTX 1080 GPU platform.
  • Experimental setup: Figure 3 compares poisoned-data generation processes under different configurations.The figure presents selected sample-image generation processes for the evaluated attack methods.

A. The Effectiveness of Poisoning Attacks Against NN

Experiments on MNIST and CIFAR-10 show that poisoned data can compromise neural networks, with attack effectiveness depending on initialization and group size. The generative method produces similar poisoning behavior while the experiments also identify practical limits in larger-scale retraining.

  • Experimental setup: MNIST and CIFAR-10 experiments evaluate poisoning attacks under different configurations, using single poisoned-data injections during retraining.The MNIST target is a 784-64-10 network with 96.82% original accuracy; CIFAR-10 uses LeNet with 71.20% original accuracy.
  • Initialization and group size: Normal initial data increases loss and degrades accuracy faster than random initial data, making it the better choice for the direct gradient attack.Figure 4 compares loss and accuracy trends across group sizes for MNIST.
  • Initialization and group size: At group size 100, initial poisoning reduces accuracy to approximately 70% by the second iteration, although the initial-data attack remains insufficient overall.The first plotted point is the original model and the second is the model after poisoning by the initial data.
  • Initialization and group size: Larger group sizes produce more stable curves and lower converged accuracy from random starts, improving attack effectiveness while increasing computation overhead.Group size affects the convergence behavior and the balance between attack effectiveness and computational cost.
  • Scope boundary: The small MNIST target model converges to approximately 10% accuracy, but substantially larger neural networks with continuous retraining may show less severe degradation.The authors present this larger-scale retraining scenario as a practical qualification on the observed degradation.

B. Direct Gradient vs. Generative Methods

The paper compares direct gradient and generative poisoning methods using generation time, target-model accuracy, and normal-data loss. The generative method is substantially faster, with accuracy degradation that is slightly weaker on MNIST but similar on CIFAR-10.

  • Evaluation design: The comparison evaluates average generation time, least target-model accuracy, and largest average normal-data loss across MNIST and CIFAR-10.Experiments use group sizes 1000, 100, and 10 with normal initial data.
  • Results: The generative method is faster than the direct gradient method, especially as group size increases for the direct gradient method.The direct method’s time cost depends strongly on group size, whereas this dependency is not observed for the generative method.
  • Results: For MNIST, the generative method reaches a higher worst-case accuracy of 16.59% versus 8.84%, indicating slightly weaker accuracy degradation.The difference is attributed to the small network and dataset, where accuracy degradation converges too quickly for the generative method.
  • Results: For CIFAR-10, the two methods achieve similar accuracy levels, suggesting better scalability of the generative method for larger neural networks.The paper summarizes the generative method as more effective for larger models and datasets.

C. Evaluating The Proposed Poisoning Attack Detection

The proposed detector monitors abnormal loss during retraining to identify poisoning attacks. Because target-model loss is already calculated during retraining, the method adds only marginal computation and hardware overhead.

  • Detection mechanism: The detector measures the difference between normal and poisoned losses using randomly selected inputs with artificial labels.Figure 5 plots input samples on the horizontal axis and normalized retraining loss on the vertical axis.
  • Detection mechanism: Periodic monitoring of loss anomalies is presented as an effective way to detect poisoning attacks during target-model retraining.The proposed detection method is based on poisoned data deviating the decision boundary and aggravating model loss.
  • Overhead and extension: The detection technique requires marginal extra computation and hardware overhead because target-model loss is usually calculated during retraining.The authors state that the mechanism can be extended to other model parameters and criteria for faster alarms.

V. CONCLUSION

The paper studies poisoned-data generation for neural networks, proposes two attack methods and a loss-based detector, and finds the generative method substantially faster than the direct gradient method.

  • The study examines the general process of generating poisoned data for neural networks.
  • It proposes two poisoning methods against neural networks: a direct gradient method and a generative method.
  • It develops a preliminary loss-based countermeasure for detecting poisoning attacks.
  • Experiments on MNIST and CIFAR-10 show that the generative method substantially improves poisoned-data generation rate over the direct gradient method.
  • The generative method demonstrates potential for attacking large neural network models and big datasets.
  • Future work targets a better generator and gradient policy to improve both poisoned-data generation rate and model attack effectiveness.
Loading 1703.01340v1…