PaperlayerView exact revision on arXiv

Source-linked AI summary

Manipulating Machine Learning: Poisoning Attacks and Countermeasures for Regression Learning

Matthew Jagielski, Alina Oprea, Battista Biggio, Chang Liu, Cristina Nita-Rotaru, Bo Li

arXiv:1804.00308v3cs.CRcs.GTcs.LG

TL;DR

Poisoning attacks can manipulate training data in linear regression, but their impact and effective countermeasures were not systematically understood. The paper develops regression-specific optimization and statistical attacks plus the TRIM defense, finding large attack effects and substantially greater robustness for TRIM.

  • Problem

    The impact of training-data manipulation and the design of resilient learning algorithms for linear regression were not yet well understood.

  • Method

    The paper develops a regression-specific optimization attack, a fast statistical attack requiring limited process knowledge, and the principled TRIM defense, evaluated across models and application datasets.

  • Results

    TRIM’s median MSE increase was only 6.1% across attacks, models, and datasets, while the optimization attack changed 75% of patients’ Warfarin dosages by an average of 93.49%.

  • Takeaways & Limitations

    The study provides a systematic basis for assessing linear-regression poisoning and shows that TRIM can provide robust protection against a large class of attacks.

Abstract

from arXiv · show

As machine learning becomes widely used for automated decisions, attackers have strong incentives to manipulate the results and models generated by machine learning algorithms. In this paper, we perform the first systematic study of poisoning attacks and their countermeasures for linear regression models. In poisoning attacks, attackers deliberately influence the training data to manipulate the results of a predictive model. We propose a theoretically-grounded optimization framework specifically designed for linear regression and demonstrate its effectiveness on a range of datasets and models. We also introduce a fast statistical attack that requires limited knowledge of the training process. Finally, we design a new principled defense method that is highly resilient against all poisoning attacks. We provide formal guarantees about its convergence and an upper bound on the effect of poisoning attacks when the defense is deployed. We evaluate extensively our attacks and defenses on three realistic datasets from health care, loan assessment, and real estate domains.

I. INTRODUCTION

The paper studies poisoning attacks against linear regression, where attackers manipulate training data to alter predictions, and introduces attacks and the TRIM defense. Experiments across regression models and real-world domains show substantial attack impact and strong defense robustness.

  • Motivation: Poisoning attacks manipulate training data to alter predictive models, posing risks as machine learning supports increasingly consequential automated decisions.The paper distinguishes poisoning from evasion and privacy attacks and notes practical poisoning demonstrations across several applications.
  • Contributions: The paper presents the first systematic study of poisoning attacks and countermeasures for linear regression under different adversarial models.Its contributions include regression-specific attacks, a statistical attack, a robust defense, and extensive evaluation.
  • Attacks: Optimization-based attacks tune initialization, objectives, and poisoning variables to maximize attack impact, while statistical attacks trade effectiveness for lower computational and information requirements.The optimization framework is tailored to regression, where attackers can also choose continuous response values.
  • Defense: TRIM iteratively estimates regression parameters with a trimmed loss that removes large-residual points, isolating poisoning samples and providing convergence and MSE guarantees.The defense is designed to remain resilient even when poisoned points resemble the training distribution.
  • Evaluation: 75% of patients had Warfarin dosages changed by the optimization attack, with an average change of 93.49%.One tenth of these patients had dosage changes of 358.89%.
  • Evaluation: TRIM’s median MSE increase was only 6.1% across attacks, models, and datasets, while improving over Huber, RANSAC, and RONI on one dataset.Only 20% of attacks caused more than 27.2% MSE increase; the reported improvement factors were 131.8, 17.5, and 20.28, respectively.

A. Adversarial model

The adversarial model formalizes attackers by their goals, knowledge, and ability to inject bounded poisoning points. The study focuses on availability attacks and defenses, while leaving detailed integrity-attack analysis for future work.

  • Adversary’s Goal: Availability attacks seek indiscriminate prediction changes, whereas integrity attacks target specific test-time mispredictions while preserving other predictions.Both attack types corrupt the model during training by injecting poisoning samples.
  • Adversary’s Knowledge: White-box attackers know the training data, features, learning algorithm, and trained parameters; black-box attackers use substitute data and estimate the model parameters.The black-box setting evaluates attack transferability across training sets.
  • Adversary’s Capability: Attackers inject at most p poisoning points before training, choosing their features and responses within a specified range, typically [0, 1].The poisoning budget constrains the number of injected samples, while their values are attacker-controlled within the allowed domain.
  • Attack Strategy: Poisoning scenarios are represented as bilevel optimization: the outer problem selects poisoning points to maximize a loss, while the inner problem retrains the regression model.The loss can target selected points for integrity attacks or an untainted set for availability attacks.
  • Scope: The paper focuses experimentally on poisoning availability attacks and their defenses, leaving a detailed investigation of integrity attacks to future work.The authors state that the same technical derivation can implement integrity attacks, but do not provide a detailed effectiveness study.

III. ATTACK METHODOLOGY

The attack methodology adapts gradient-based poisoning to regression by jointly optimizing features and continuous responses, then uses theoretical insights to motivate faster attack strategies. The framework varies objectives and initialization schemes to maximize model disruption.

  • Optimization-based Poisoning Attacks: The regression attack extends classification poisoning by jointly optimizing each poisoning point’s features and response variable.This differs from classification methods that initialize categorical labels and keep them fixed during optimization.
  • Theoretical Insights: The theoretical update interpretation drives a faster statistical attack that requires minimal black-box access and less information about the training process.The response update moves farther from the original regression line, while feature updates combine directional effects relative to that line.
  • Baseline Gradient Descent: The baseline gradient-descent attack updates both poisoning features and responses along the gradient while enforcing feasible-domain constraints through projection.The attack iterates until the outer objective stops changing sensibly.
  • Optimization-based Poisoning Attacks: Gradient-based attacks update poisoning samples through the outer objective while accounting for the poisoned model parameters’ dependence on those samples.For convex inner learning problems, this dependence can be computed using differentiated KKT equilibrium conditions.
  • Objective Functions: The outer objective may use MSE on an untainted validation set or regularized training data, with both alternatives evaluated in the experiments.The training-data objective serves as a proxy for generalization error without requiring an additional data set.
  • Initialization Strategies: Regression poisoning initializes response values with Inverse Flipping, yc = 1−y, or Boundary Flipping, yc = round(1−y), after randomly selecting training points.Feature initialization methods did not significantly improve over simple uniform random selection.

B. Statistical-based Poisoning Attack (StatP)

StatP generates poisoning points from estimated training-data statistics, moves features toward distribution corners, and queries the model to select responses. It therefore needs only black-box access and limited information about the training process.

  • StatP Construction: StatP samples poisoning features from a multivariate normal distribution fitted to the training-data mean and covariance.It then rounds feature values toward corners, reflecting the observed effectiveness of corner-near poisoning points.
  • Information Requirements: StatP queries the model to determine poisoning responses and requires only black-box access, unlike optimization attacks that need more training-process information.The attack is agnostic to the exact regression algorithm, its parameters, and the training set.

IV. DEFENSE ALGORITHMS

Existing defenses address noise and outliers, but inlier poisoning can still mislead regression models. The paper introduces TRIM as a defense designed for adversarially corrupted training data.

  • TRIM: TRIM iteratively estimates regression parameters while training on the subset of points with the lowest residuals.Its procedure removes high-residual points from the optimization objective and refines the retained subset across iterations.
  • Existing defenses: Existing defenses include noise-resilient regression methods and adversarially resilient defenses.The former identify or downweight outliers, while the latter often rely on restrictive distributional assumptions.
  • Limitations of existing defenses: Inlier poisoning can resemble legitimate data while still misleading the estimated regression model.StatP selects poisoned points with the same mean and covariance as the training distribution, exposing a weakness in existing regression defenses.
  • Limitations of existing defenses: Prior adversarially resilient methods rely on assumptions such as identity covariance, sub-Gaussian data, or low-rank feature matrices.The paper notes that these assumptions are not usually satisfied in practice.

B. TRIM algorithm

TRIM jointly selects a low-residual training subset and estimates regression parameters through alternating updates. It is guaranteed to terminate, and its analysis bounds model error under worst-case poisoning when fewer than half the training points are poisoned.

  • Design: TRIM alternates between estimating regression parameters and retaining the n points with the smallest residuals.The retained inliers are used to estimate the next parameter vector, while large-residual points are excluded because they increase MSE.
  • Design: TRIM trains on a subset of n points selected from the poisoned dataset of size N, assuming α < 1 so pristine data remains the majority.The attacker injects p = α · n poisoned samples, giving N = (1 + α)n.
  • Convergence: Algorithm 2 terminates in a finite number of iterations.The paper provides a formal convergence guarantee, while empirical evaluation finds convergence within at most a few dozen iterations.
  • Guarantees: TRIM’s global optimum fits at least (1 − α) · n pristine samples without assumptions on how poisoned data is generated.This gives a worst-case-adversary guarantee relating the poisoned-data solution to the pristine-data estimator.
  • Guarantees: When α ≤20%, the theoretical OLS worst-case-to-best-case MSE factor is at most 1.25×.Because the implemented algorithm may reach only a local minimum, experiments measure this ratio empirically; it is below 1.46× at α = 12% and reaches 2.39× for one dataset at α = 20%.
  • Limitations: For LASSO, ridge, and elastic net, regularization may allow a larger worst-case MSE blowup relative to the best-case MSE.The authors report no known technique that triggers this worst-case scenario.

V. EXPERIMENTAL EVALUATION

The evaluation uses three public regression datasets spanning health care, loan assessment, and house pricing. Experiments compare attack and defense methods across multiple models and preprocessing settings.

  • Evaluation design: The experimental workflow analyzes new attacks, compares them with a baseline, and evaluates TRIM against robust-statistics defenses.The study uses cross-validation with separate training, testing, and validation splits.
  • Datasets: The experiments use three public regression datasets from health care, loans, and house pricing.The health care dataset contains 5700 patients; the loan and house-pricing datasets provide additional financial and real-estate settings.
  • Datasets: The health care dataset predicts Warfarin dosage from demographic, clinical, medication, and genetic features.It contains 67 original features and 167 features after one-hot encoding and numerical normalization.
  • Attack evaluation: The evaluation includes a table of the best-performing optimization attack, OptP, for Ridge and LASSO regression.The supplied table material identifies the compared models but does not provide its numerical entries.
  • Datasets: The loan dataset predicts interest rates using loan and borrower attributes, while the house-pricing dataset predicts sale prices from property and location variables.The loan dataset contains 887,383 loans, and the house-pricing dataset contains 1460 houses.

B. New poisoning attacks

The paper evaluates optimization and statistical poisoning attacks across three datasets and four regression models, finding that optimization choices strongly affect attack success. OptP generally outperforms baseline and statistical attacks, but StatP offers a faster, lower-information alternative.

  • OptP improves upon the baseline BGD attack by a factor of 6.83 in the best case and reaches MSEs 155.7 times higher than unpoisoned models.
  • The framework varies initialization, optimization variables, and objective, with each dimension influencing attack success and optimal choices depending on data distribution.
  • Optimizing both x and y is more effective than optimizing x alone, achieving a 6.83-fold MSE improvement over BGD for LASSO on the house dataset.
  • Wval improves over Wtr by 7.09-fold on house LASSO, 17.5% on loan LASSO, and 30.4% on loan ridge under matched initialization and optimization variables.
  • Optimization attacks generally outperform StatP, although StatP exceeds OptP by 11% for loan LASSO and performs similarly in two ridge cases.
  • OptP takes 337 seconds for ridge and 408 seconds for LASSO on house data, whereas StatP generates the same number of points in roughly tenths or hundredths of a second.

3) Question 3: What is the potential damage of poisoning in real applications?:

The experiments show that poisoning can substantially alter application predictions, while transferability extends attack effectiveness across training sets and TRIM is the only defense consistently effective against OptP.

  • For 10% of patients, LASSO dosage increases reach a maximum of 359%, while at 8% poisoning half of patients experience a 75.06% dosage change.
  • Both StatP and OptP retain similar effectiveness when transferred across training sets, with StatP transferred MSEs within 11.4% of original attacks.
  • Chen’s performance is highly variable because its sub-Gaussian feature and identity-covariance assumptions are not met by the real-world datasets.
  • TRIM is the only evaluated defense that consistently performs well against OptP on ridge and LASSO, whereas Huber, RANSAC, and RONI can underperform the undefended model.
  • Known defenses are not consistently effective: Huber increases MSE over undefended ridge models by a factor of 2781 in one reported case.

2) Question 2: What is the robustness of the new defense TRIM compared to known methods?:

TRIM is substantially more robust than the evaluated existing defenses against the paper’s poisoning attacks, while also remaining computationally efficient. Across attacks, models, and datasets, its degradation is usually small and it can outperform unpoisoned models in some cases.

  • 6.1% is the median MSE increase for TRIM across all datasets and attacks, while only 20% of attacks cause more than 27.2% MSE increase.
  • TRIM improves MSE over Huber by a factor of 131.8, RANSAC by 17.5, and RONI by 20.28.
  • TRIM achieves MSE 3.47% lower than unpoisoned models for LASSO on the health care dataset at α = 8%.
  • TRIM is consistently faster than the other evaluated defenses, including an average runtime of 0.02 seconds on the house dataset.
  • Previous defenses RANSAC, Huber, Chen, and RONI can sometimes increase MSE beyond unpoisoned models under the proposed attacks.

APPENDIX A THEORETICAL ANALYSIS OF LINEAR REGRESSION

The appendix establishes equivalence properties for poisoned OLS constructions and extends the argument to ridge regression. Under the stated invertibility conditions, the learned parameters and objective gradients remain unchanged between the compared poisoned datasets.

  • The appendix proves equivalence between the models learned from the two compared poisoned datasets.
  • For OLS, equality of X^T Y and X^T Y′ yields identical learned models after adding any poisoning set Dp.
  • The same equivalence holds for ridge regression because its Hessian includes an invertible λI term.
  • Both the learned parameters and objective gradients are identical regardless of the poisoned data added.
  • The appendix derives the gradient form used to characterize the adversary’s optimization steps.

APPENDIX B ANALYSIS OF TRIM ALGORITHM

The TRIM analysis provides a finite-termination guarantee and an estimation analysis under a bounded poisoning fraction. Its convergence proof relies on a decreasing finite-valued objective-related quantity.

  • The appendix presents a convergence and estimation analysis for the TRIM algorithm.
  • Algorithm 2 terminates in a finite number of iterations.
  • The termination proof uses a strictly decreasing R(i) sequence and the finite number of possible subsets of the required size.
  • The estimation analysis assumes α < 1 and identifies a subset containing (1 − α)·n pristine training samples.

APPENDIX C BASELINE ATTACK

The baseline-attack appendix describes parameter selection and compares OptP, StatP, and BGD on a PDF malware dataset. StatP is especially effective for ridge regression, where the authors attribute its advantage to boundary-seeking feature perturbations.

  • The experiments use a PDF malware dataset containing 5,000 benign and 5,000 malicious files with 137 features.
  • The attack search uses η, β, and ϵ to control gradient steps, learning-rate decay, and stopping.
  • StatP outperforms BGD and OptP by 40% for ridge regression on the evaluated dataset.
  • The authors associate StatP’s ridge advantage with pushing feature values to boundaries, where the convex-loss maximum is achieved at corners.
Loading 1804.00308v3…