PaperlayerView exact revision on arXiv

Source-linked AI summary

Poisoning and Backdooring Contrastive Learning

Nicholas Carlini, Andreas Terzis

arXiv:2106.09667v2cs.LG

TL;DR

Multimodal contrastive models benefit from cheap, robust training on noisy Internet data, but the paper asks whether that openness creates a serious security risk. It adapts existing poisoning and backdoor techniques to this setting and finds that extremely small data modifications can control model behavior, motivating defenses and broader study.

  • Problem

    Noisy, uncurated Internet training offers cost and robustness benefits, but the security and reliability of multimodal contrastive learning in adversarial settings is not well understood.

  • Method

    The paper empirically evaluates adapted targeted-poisoning and patch-based backdoor attacks against multimodal contrastive models.

  • Results

    0.01% poisoning suffices for many backdoor attacks, while targeted poisoning requires 0.0001% of the training data.

  • Takeaways & Limitations

    Training on noisier data with less human review may increase poisoning risk, making defenses important for trustworthy contrastive classifiers.

  • Takeaways & Limitations

    In contrastive classification, attackers control an image’s embedding but must rely on downstream classification to produce the desired misclassification.

Abstract

from arXiv · show

Multimodal contrastive learning methods like CLIP train on noisy and uncurated training datasets. This is cheaper than labeling datasets manually, and even improves out-of-distribution robustness. We show that this practice makes backdoor and poisoning attacks a significant threat. By poisoning just 0.01% of a dataset (e.g., just 300 images of the 3 million-example Conceptual Captions dataset), we can cause the model to misclassify test images by overlaying a small patch. Targeted poisoning attacks, whereby the model misclassifies a particular test input with an adversarially-desired label, are even easier requiring control of 0.0001% of the dataset (e.g., just three out of the 3 million images). Our attacks call into question whether training on noisy and uncurated Internet scrapes is desirable.

1 INTRODUCTION

Multimodal contrastive learning gains cost and robustness advantages from noisy, uncurated Internet data, but that same openness enables practical poisoning and backdoor attacks with very few injected examples.

  • Benefits of uncurated data: Uncurated training avoids expensive labeling and lets contrastive models learn from arbitrary images without predefined supervised categories.This expands usable data beyond datasets restricted to fixed object labels.
  • Benefits of noisy data: Training on noisy Internet data improves robustness, including CLIP’s reported effective robustness on ImageNet-V2.Supervised models trained only on ImageNet can overfit that dataset and generalize poorly to other test sets.
  • Security risk: Internet-scraped datasets receive no human review before training, making adversarial contamination a plausible concern.The paper links this risk to the datasets’ explicitly noisy and uncurated nature.
  • Attacks studied: The paper evaluates targeted poisoning, which misclassifies one chosen input, and patch-based backdoors, which misclassify inputs carrying a trigger pattern.Both attacks target multimodal contrastive models.
  • Attack practicality: 0.01% of training data suffices for many backdoor attacks, while poisoning attacks require only 0.0001%.These rates are orders of magnitude below the approximately 1% average required by prior backdooring attacks cited by the authors.

2 BACKGROUND, NOTATION, AND RELATED WORK

The paper frames poisoning and backdooring as attacks on multimodal embedding-based classifiers, defines their objectives and threat model, and describes how patches can control predictions.

  • Attack concepts: A poisoning attack injects examples into a benign dataset, producing a model that behaves normally in standard settings but is attacker-controlled in other settings.The poisoned dataset is formed as X′ = X ∪ P before training.
  • Attack concepts: Targeted poisoning makes a selected input x′ receive an adversary-chosen label y′, and the paper argues this works on uncurated datasets.Prior attacks commonly targeted curated datasets, limiting practical deployment.
  • Attack concepts: A backdoor makes any image containing a chosen patch receive a target label, unlike targeted poisoning’s focus on one particular image.The paper studies checkerboard patches placed either consistently in the upper-left corner or randomly.
  • Multimodal contrastive learning: Multimodal contrastive learning maps images and captions into a shared embedding space and increases similarity for paired examples while reducing similarity to other examples.The image and text encoders are represented as f: A → E and g: B → E.
  • Model uses and objective: Contrastive models can support downstream feature extraction with a trained classifier or zero-shot classification using text-derived class embeddings.The paper’s adversary objective covers incorrect behavior in either use case, while experiments focus on the image encoder.
  • Adversary capabilities: The threat model allows injection of a small number of training examples and assumes no control over later fine-tuning data or algorithms.The authors consider poisoning 100–10,000× fewer images than prior supervised attack rates would require.

3 POISONING AND BACKDOORING ATTACK ALGORITHM

The attacks adapt standard poisoning and backdoor procedures to multimodal contrastive models by controlling image embeddings through poisoned image-text pairs. Diverse captions and patched images steer either a targeted input or any triggered input toward a desired label.

  • 3.1 OUR MULTI-SAMPLE POISONING ATTACK: Targeted poisoning injects image-text pairs so the learned image embedding causes a chosen test input to receive an adversarially desired label.The attack constructs captions related to the target label and forms the poisoned dataset by union with the benign training data.
  • 3.1 OUR MULTI-SAMPLE POISONING ATTACK: Automated duplicate cleaning is not a security defense because tiny image noise or minor caption changes can evade exact-match detectors without degrading attack quality.The authors also connect robust duplicate detection under adversarial manipulation to an unsolved adversarial-examples problem.
  • 3.1 OUR MULTI-SAMPLE POISONING ATTACK: The caption set can be built by searching training captions for the target label, while prompt-engineered captions provide an alternative requiring additional adversary knowledge.The naive method tolerates misleading matches because most retrieved captions are correct; the second method uses CLIP’s 80 classification prompts.
  • 3.2 HOW CONTRASTIVE ATTACKS DIFFER: Contrastive poisoning is harder than supervised poisoning because the adversary controls an embedding, while optimization may instead modify the text encoder or distribute changes across both encoders.A diverse caption set encourages the model to modify the poisoned image embedding rather than every caption embedding.
  • 3.3 EXTENDING THE ATTACK TO BACKDOOR MODELS: Backdooring replaces the single target image with different patched images, training the model so inputs containing the trigger pattern map to the desired label.The attack uses x_i ⊕bd for poison samples and captions found in the training dataset, avoiding assumptions about zero-shot prompts.

4 EVALUATION

The evaluation tests adapted attacks against CLIP trained on Internet-scraped captioned-image datasets under zero-shot and linear-probe settings. Targeted poisoning succeeds with only a few samples, while backdooring reaches substantial success at a 0.01% poison ratio; the two downstream evaluations can diverge for backdoors.

  • 4 EVALUATION: The experiments use Conceptual Captions-3M and a 15-million-image YFCC subset, both consisting of captioned images scraped from the Internet.CLIP uses its default ResNet-50 vision model and Transformer language model with the stated training protocol.
  • 4.2 POISONING EVALUATION: Targeted poisoning three of 3 million Conceptual Captions examples fools zero-shot CLIP into the target class with 40% probability.The attack targets one of 1,000 ImageNet class labels; poisoning 15 of 15 million YFCC images succeeds 20% of the time.
  • 4.3 BACKDOORING EVALUATION: A 0.01% poison ratio reaches a 50% attack success rate against zero-shot classifiers in the backdoor evaluation.The evaluation varies backdoor poison ratios and requires separate models per configuration because each model can contain only one backdoor.
  • 4.3 BACKDOORING EVALUATION: Backdoor vulnerability can occur in the zero-shot model even when the corresponding linear-probe model is not vulnerable.The authors identify this contrast with poisoning evaluation as an open direction for further understanding.

5 ABLATION STUDY

The ablations introduce backdoor z-score as a lower-variance proxy for attack success and examine how poisoning strategy, dataset size, model size, and patch size affect backdoor efficacy. Random patch placement becomes especially effective with more poisoned examples, while larger models and patches make attacks more reliable.

  • 5.1 A STABLE METRIC: BACKDOOR Z-SCORE: Backdoor z-score measures how atypically similar two patched images are relative to random non-backdoored images.The metric compares the similarity distributions of backdoored and ordinary image pairs.
  • 5.1 A STABLE METRIC: BACKDOOR Z-SCORE: Similarity between image pairs has an exponential relationship with the probability that the model assigns them the same class, enabling lower-variance attack evaluation.Each datapoint uses 8 trained CLIP models to estimate variance while reducing computation relative to direct downstream evaluation.
  • 5.1.2 BACKDOOR ATTACK SUCCESS RATE AS A FUNCTION OF MODEL AND DATA SCALE: 300 poisoned examples maintain nearly constant attack success across datasets from 50,000 to 3,000,000 samples, whereas 75 examples degrade beyond one million samples.The 75-example condition corresponds to a poison ratio below 0.01% at one million samples.
  • 5.1.1 BACKDOOR ATTACK SUCCESS RATE AS A FUNCTION OF POISONED FRACTION: 75 poisoned examples, or 0.0025% of the dataset, produce a z-score of 2.5 when patches are consistently placed and consistently tested.Randomly placed poisoned patches yield a z-score near zero under this condition because the model does not learn a reliable patch-similarity association.
  • 5.1.1 BACKDOOR ATTACK SUCCESS RATE AS A FUNCTION OF POISONED FRACTION: With more poisoned examples, random patch placement becomes orders of magnitude more effective than consistent placement and remains effective when patches are randomly placed at test time.Below 0.01% poisoning, consistent placement is better; above that threshold, random placement is better.
  • 5.1.2 BACKDOOR ATTACK SUCCESS RATE AS A FUNCTION OF MODEL AND DATA SCALE: Larger models are easier and more consistent to backdoor: a 30 million-parameter model has z-scores always between 5.1 and 5.9, compared with 0.3 to 5.4 for a 5 million-parameter model.The 1 million-parameter model is never successfully backdoored.
  • 5.1.3 BACKDOOR ATTACK SUCCESS RATE AS A FUNCTION OF PATCH SIZE: A 4 × 4 patch is already sufficient for attack success at a poisoning rate of 0.0025%, while larger patches increase success through 16 × 16 but not significantly beyond it.Patches larger than 16 × 16 may even decrease attack success at 32 × 32.

6 CONCLUSION

The paper argues that noisy, uncurated training data brings cost and robustness benefits but also intensifies poisoning risks. It motivates defenses and broader study of adversarial threats to self-supervised learning.

  • 6 CONCLUSION: Noisy and uncurated datasets offer cost benefits and robustness improvements, but their lack of human review increases exposure to poisoning attacks.Internet-scraped data can be modified by adversaries before training.
  • 6 CONCLUSION: The attacks require far less training-data modification than fully supervised attacks, and scaling dataset size does not prevent them from succeeding.
  • 6 CONCLUSION: Developing defenses that filter malicious samples is challenging but important for making contrastive classifiers trained on uncurated data trustworthy.Manual review is impractical because it would remove the value of uncurated training data.
  • 6 CONCLUSION: The paper presents these attacks as an early warning for broader security and reliability problems in self-supervised learning.

ETHICS STATEMENT

The authors argue that publishing practical attacks on multimodal contrastive classifiers has limited immediate harm and can reduce future risk. Early disclosure can warn users and give researchers more time to develop defenses.

  • ETHICS STATEMENT: The attack can be carried out by anyone able to post Internet images, with little to no technical skill.
  • ETHICS STATEMENT: The authors state that these classifiers are not yet used in security-critical situations, so publication does not currently cause direct harm.
  • ETHICS STATEMENT: Publicizing the classifiers’ limitations early may prevent organizations from deploying them in safety-critical settings while assuming they are robust.
  • ETHICS STATEMENT: Early disclosure maximizes the time available for defense researchers to develop stronger defenses before the attacks are needed.

REPRODUCIBILITY STATEMENT

The paper’s attacks are relatively easy to reproduce with existing open-source CLIP training tools and a modified dataset, but reproducing training requires substantial computational resources.

  • REPRODUCIBILITY STATEMENT: The attacks can be reproduced by running existing open-source CLIP training tools on a slightly modified dataset containing poisoned samples.
  • REPRODUCIBILITY STATEMENT: Reproduction is nevertheless computationally demanding because training a single CLIP model is slow.
Loading 2106.09667v2…