PaperlayerView exact revision on arXiv

Source-linked AI summary

Leveraging Imperfect Restoration for Data Availability Attack

Yi Huang, Jeremy Styborski, Mingzhi Lyu, Fan Wang, Adams Kong

arXiv:2609.04627v1cs.AI

TL;DR

Existing data availability attacks often work well in either supervised or self-supervised learning, and CUDA remains limited by weak SSL poisoning and an image-quality trade-off. The paper analyzes CUDA’s gradients and class-wise bias, then proposes Imperfect Restoration Poisoning (IRP) to preserve image quality while strengthening poisoning. Across supervised and self-supervised learning, defenses, architectures, and datasets, IRP consistently outperforms prior attacks.

  • Problem

    Existing data availability attacks often specialize in supervised or self-supervised learning, while CUDA has underwhelming SSL effectiveness and a severe image-quality–poisoning trade-off.

  • Method

    The paper analyzes CUDA with a deep learning model and proposes IRP based on sub-optimal gradients, class-wise bias, and imperfect restoration.

  • Results

    IRP outperforms previous attacks across supervised and self-supervised learning, five defense methods, multiple architectures, and multiple datasets while maintaining high image quality.

  • Takeaways & Limitations

    IRP provides a data availability attack that combines strong effectiveness across learning settings and defenses with high image quality.

  • Takeaways & Limitations

    Prior CUDA analysis relies on Gaussian independent data, two classes, and a non-deep-learning setting, limiting its design insights for stronger poisoning methods.

Abstract

from arXiv · show

The abundance of online data is at risk of unauthorized usage in training deep learning models. To counter this, various Data Availability Attacks (DAAs) have been devised to make data unlearnable for such models by subtly perturbing the training data. However, existing attacks often excel against either Supervised Learning (SL) or Self-Supervised Learning (SSL) scenarios. Among these, a model-free approach that generates a Convolution-based Unlearnable Dataset (CUDA) stands out as the most robust DAA across both SSL and SL. Nonetheless, CUDA's effectiveness against SSL is underwhelming and it faces a severe trade-off between image quality and its poisoning effect. In this paper, we conduct a theoretical analysis of CUDA, uncovering the sub-optimal gradients it introduces and elucidating the strategy it employs to induce class-wise bias for data poisoning. Building on this, we propose a novel poisoning method named Imperfect Restoration Poisoning (IRP), aiming to preserve high image quality while achieving strong poisoning effects. Through extensive comparisons of IRP with eight baselines across SL and SSL, coupled with evaluations alongside five representative defense methods, we showcase the superiority of IRP. Code: https://github.com/lyumingzhi/IRP

1 Introduction

Existing DAAs often specialize in either supervised or self-supervised learning, while CUDA is more robust across both but remains limited by weak SSL poisoning and an image-quality trade-off. This paper analyzes CUDA and introduces IRP to improve poisoning strength while preserving image quality.

  • Motivation: Most DAAs designed for supervised learning fail to generalize effectively to self-supervised learning because augmentation-invariant training can ignore their poison features.The paper identifies CUDA and CP as notable exceptions, while CP loses effectiveness in supervised learning.
  • Motivation: CUDA is the strongest existing DAA across supervised and self-supervised learning, but SSL models trained on CUDA-poisoned data still retain about 70% clean test accuracy.Larger CUDA kernels strengthen poisoning but visibly blur the generated images.
  • Motivation: The authors identify limitations in prior CUDA analysis, which assumes Gaussian independent data, two classes, and a non-deep-learning setting.These assumptions make it difficult to derive design insights for a more effective poison method.
  • Contributions: IRP analyzes CUDA with a deep learning model, revealing sub-optimal clean-data gradients and class-wise bias produced through random filters.The analysis motivates a poisoning method based on imperfect restoration.
  • Contributions: IRP is designed to maintain high image quality while producing stronger poisoning effects than CUDA.The paper presents IRP as a novel data availability attack derived from the CUDA analysis.
  • Evaluation: Across supervised and self-supervised learning, five defense techniques, CIFAR-10, and a subset of ImageNet, IRP outperforms baseline poisons by a large margin.Additional evaluations cover five architectures and two supplementary datasets, with high effectiveness reported throughout.

2 Related Works

DAA research spans model-reliant and model-free approaches, but many methods are designed for supervised learning and do not transfer effectively to self-supervised learning. CUDA offers moderate SSL robustness, yet its effectiveness and image-quality trade-off remain limitations, motivating evaluation against mitigation strategies.

  • DAA Methods: DAAs include model-reliant methods based on bilevel optimization and model-free methods that exploit shortcut learning in deep neural networks.Model-free methods leverage easily learnable shortcuts rather than semantic features to induce class distinctions.
  • DAA Methods: Adversarial Poisons and related methods are primarily tailored for supervised learning, limiting their suitability for self-supervised scenarios.The related work identifies supervised-learning specialization as a limitation of earlier approaches.
  • DAA Methods: CUDA uses randomly generated class-wise convolutional filters to create shortcut features for data protection.These filters are intended to exploit shortcut learning in target models.
  • DAA Methods: CUDA shows moderate robustness to SSL, but its SSL effectiveness remains insufficient and it faces a severe trade-off between image quality and poison efficacy.This trade-off is presented as a central limitation of CUDA.
  • DAA Mitigation Strategies: Adversarial Training mitigates DAA potency but has significant computational demands, while augmentation methods generally have weaker effects.ISS is reported to outperform earlier augmentation countermeasures and achieve results comparable to adversarial training.

3 Analysis

The analysis explains CUDA’s poisoning through non-optimal gradients and class-wise bias, then develops IRP by imperfectly restoring CUDA-filtered images to improve quality while retaining poisoning.

  • Notation and setup: CUDA applies class-specific filters to published images, while the attacker assumes no knowledge of downstream training processes.The poisoned images are intended to inject class-based poison features across supervised, self-supervised, and adversarial training settings.
  • The Non-Optimal Gradient: CUDA’s poisoned-data gradient deviates from the optimal clean-data descent direction, increasing loss and disrupting recognition of clean data.For sufficiently small learning rates, any deviation from the clean-data optimal direction increases loss.
  • Poisoning through Class-Wise Bias: CUDA creates class-wise bias because self-correlation preserves the poisoned class’s information, whereas cross-correlation shifts other-class information to incorrect positions and adds blur.For κ = 9, the stated shift probability is 0.988; non-peak values also contribute to degrading other-class information.
  • Poisoning with Imperfect Restoration: IRP filters CUDA-poisoned images, learns a class-wise restoration filter through mean-square regression, and then uses the imperfectly restored filter Pc = Rc ⋆ Ac for poisoning.Theorem 2 states that perfect restoration is impossible under the stated spanning condition.
  • Poisoning with Imperfect Restoration: IRP retains CUDA-like peak shifts but adds complex turbulent patterns around the peaks, producing higher-frequency attack patterns dependent on training data.The corresponding IRP self-correlation peaks centrally, while cross-correlation peaks at a different position.
  • Poisoning with Imperfect Restoration: IRP-generated poisoned examples are visually clearer than CUDA examples, with sharper faces, backgrounds, and fine image details.The paper specifically notes sharper facial features and more distinguishable salamander and stingray spots under IRP protection.

4 Experiments

Experiments compare IRP with DAAs across supervised and self-supervised learning, defenses, partial poisoning, architectures, and image quality. IRP generally produces stronger poisoning while preserving higher image quality than CUDA.

  • Experimental setup: IRP is evaluated against eight DAAs across SL and SSL, representative defenses, partial-poisoning settings, architectures, and datasets.The experiments use CIFAR-10, CIFAR-100, STL-10, and ImageNet-100, with ResNet-18 as the default model.
  • Baseline comparisons: IRP achieves the lowest clean test accuracies in both SL and SSL on CIFAR-10 and ImageNet-100.This establishes its strongest poisoning performance across both learning paradigms and both reported datasets.
  • Baseline comparisons: On CIFAR-10 SSL, IRP reduces accuracy to 43.24%, compared with 66.58% for CUDA and 86.86% for AR.Under CIFAR-10 SL, AR and IRP both reduce accuracy to around 10%.
  • Baseline comparisons: On ImageNet-100, IRP reaches 1.98% accuracy in SL and 9.30% in SSL, outperforming the other reported DAAs.CUDA, EM, AP, and LSP reduce SL accuracy below 10%, while CUDA and CP reach 21.89% and 18.76% in SSL.
  • Defense evaluation: Only CUDA and IRP remain effective across all CIFAR-10 countermeasures, while IRP permits a lower maximum accuracy of 43.24% than CUDA’s 48.58%.Across all testing cases, CUDA permits 66.58%, which is 23.24% higher than IRP’s maximum accuracy.
  • Additional SSL algorithms: IRP lowers ImageNet-100 accuracy to around 10% across SimSiam, MoCoV3, and BYOL, with 11.6% in its worst case.This evaluation extends the SSL comparison beyond SimCLR.
  • Partial poisoning: Effectiveness drops quickly when less than 100% of CIFAR-10 training data is poisoned, and models trained on the remaining clean subset perform similarly.The same partial-poisoning trend applies to the other DAAs tested.
  • Transferability and quality: IRP outperforms CUDA across examined target architectures and generates poisoned images with better quality than CUDA.Image quality is assessed with LPIPS, SSIM, MS-SSIM, CLIP-IQA, and BRISQUE; filter ablations remain less effective than IRP.

5 Conclusion

The paper analyzes CUDA theoretically and uses those insights to introduce IRP. IRP is reported as effective across SL, SSL, and defense scenarios while maintaining high image quality.

  • Conclusion: Theoretical analysis identifies CUDA’s sub-optimal gradients and its class-wise bias strategy for data poisoning.These findings motivate the design of the proposed method.
  • Conclusion: IRP demonstrates high effectiveness in SL, SSL, and state-of-the-art defense scenarios while maintaining high image quality.The paper presents this combination as suitable for real-world data protection applications.

A.1 Proof of Theorem 1

The proof derives properties of CUDA filter convolutions from the distribution of squared random-filter entries. It shows distinct peak behavior for identical and different filters.

  • Preliminaries: For a uniform random variable, the proof derives the density and expectation of its square, which support later peak calculations.The squared variable has support [0, p_u^2] and expectation p_u^2/3.
  • Theorem statement: Under the stated condition, the proof establishes all four convolution properties for CUDA filters.These properties concern peak location, central-peak probability, and the relative expected peak heights.
  • Filter properties: Identical CUDA filters have their convolution peak at the center, where the expected peak exceeds values at other positions.The proof compares complete central overlap with cases involving fewer or displaced overlaps.
  • Filter properties: For different filters, the convolution peak occurs where their one-valued entries overlap, with probability 1/κ^2 of occurring at the center.The expected peak for identical filters is higher than that for different filters.

A.2 Proof of Theorem 2

The proof models convolution patches as linear transformations and analyzes expected filter-overlap peaks. It establishes when perfect reconstruction is impossible and extends peak properties to expected values.

  • Linear reconstruction: A convolution patch is represented as η_cij = ℘x_cij, where ℘ maps a larger input patch to the filtered patch.The matrix has κ^2 rows and (2κ−1)^2 columns.
  • Linear reconstruction: If the relevant columns of ℘ span R^κ^2, no reconstruction vector can recover the target pixel exactly from the filtered patch.The proof shows that the required orthogonality condition cannot hold under this span assumption.
  • Application to CUDA: For CIFAR-10’s p_u = 0.3 and κ = 3, the stated condition holds, so CUDA filters satisfy the theorem’s analogous properties.The conclusion applies the theorem to the parameter setting used for CIFAR-10.
  • Expected peak properties: The expected convolution peak for identical filters is centered, while for different filters it occurs where their one-valued entries overlap.The center probability for different filters is 1/κ^2, and identical-filter peaks are higher in expectation.
  • Expected peak properties: The proof compares four overlap cases and identifies the same-location overlap case as the largest expected contribution.This case yields the centered expected peak for identical filters.

B.1 Training details

Training settings for supervised and self-supervised learning are summarized in Table 7, with augmentation terminology defined for pretraining and linear probing.

  • For VGG-19 and ViT, SL uses a 0.01 learning rate; ViT training lasts 200 epochs.
  • SL uses standard SSL linear-probing augmentation because it improves clean test accuracy and can defend against certain DAAs.
  • Table 7 lists training settings for SL and SSL, with paired values representing CIFAR-10 and ImageNet-100 configurations.
  • ‘Pretrain’ denotes the standard augmentation used during SSL pretraining, while ‘LinProbe’ denotes the augmentation used during SSL linear probing.

C Additional Results

Additional image examples compare Double Blur, Random Sharpness, and IRP, with IRP producing the clearest outputs through customized filter optimization.

  • IRP produces the clearest images among the three methods because its blurriness filter is customized through optimization.
  • Double Blur generates the blurriest outputs, while Random Sharpness produces varying blur because its sharpness filters are randomly generated class-wise.
  • Figure 7 presents ImageNet-100 examples from Double Blur, Random Sharpness, and IRP in separate rows.

C.2 Additional Defenses

Additional evaluations examine IRP against recent purification defenses and compare its robustness with CUDA under multiple defense settings.

  • AVATAR and COIN defend against both CUDA and IRP poisons, while UEraser is less effective than standard defenses such as SSL and AT.
  • 71.90% clean test accuracy is achieved for CUDA-poisoned CIFAR-10 under COIN purification.
  • 54.78% is the maximum clean test accuracy for IRP-poisoned CIFAR-10 after AVATAR purification, remaining severely below clean-data training.
  • Across all defenses, IRP is more potent than CUDA.
  • Table 8 reports top-1 clean test accuracies for ResNet-18 trained on CIFAR-10 poisoned by various DAAs, with Max denoting each poison’s worst-case performance.
Loading 2609.04627v1…