Source-linked AI summary
Disrupting Deepfakes: Adversarial Attacks Against Conditional Image Translation Networks and Facial Manipulation Systems
Nataniel Ruiz, Sarah Adel Bargal, Stan Sclaroff
TL;DR
Unauthorized face manipulation motivates attacks that disrupt the outputs of conditional image translation systems. The paper adapts adversarial attacks, introduces class-transferable disruptions, adversarial training, and spread-spectrum evasion, and reports successful disruption and defense-evasion results. Its scope includes a limitation observed for attention-driven GANimation systems.
Problem
The paper addresses how to prevent unauthorized deepfake generation when conditional image translation systems modify facial expressions, poses, or attributes.
Method
The paper adapts adversarial attacks to image translation, develops class-transferable and spread-spectrum disruptions, and trains GANs adversarially to alleviate disruptions.
Results
The proposed disruptions successfully transfer across conditioning attributes, while adversarial training alleviates disruptions and spread-spectrum disruption evades diverse blur defenses.
Takeaways & Limitations
Disrupting conditional image translation provides a defense against unauthorized facial manipulation, including settings where the conditioning attribute or blur defense is not fully known.
Takeaways & Limitations
Attention-driven GANimation systems present an issue with class transfer because they generate color and attention masks that designate replacement regions.
Abstract
from arXiv · showhide
Face modification systems using deep learning have become increasingly powerful and accessible. Given images of a person's face, such systems can generate new images of that same person under different expressions and poses. Some systems can also modify targeted attributes such as hair color or age. This type of manipulated images and video have been coined Deepfakes. In order to prevent a malicious user from generating modified images of a person without their consent we tackle the new problem of generating adversarial attacks against such image translation systems, which disrupt the resulting output image. We call this problem disrupting deepfakes. Most image translation architectures are generative models conditioned on an attribute (e.g. put a smile on this person's face). We are first to propose and successfully apply (1) class transferable adversarial attacks that generalize to different classes, which means that the attacker does not need to have knowledge about the conditioning class, and (2) adversarial training for generative adversarial networks (GANs) as a first step towards robust image translation networks. Finally, in gray-box scenarios, blurring can mount a successful defense against disruption. We present a spread-spectrum adversarial attack, which evades blur defenses. Our open-source code can be found at https://github.com/natanielruiz/disrupting-deepfakes.
1 Introduction
Face manipulation systems can generate realistic changes to expressions, poses, and attributes, creating a need to prevent unauthorized modifications. This section introduces deepfake disruption through imperceptible adversarial perturbations, class-transferable attacks, adversarial training, and spread-spectrum evasion of blur defenses.
- Face manipulation systems generate realistic changes to expressions, poses, age, hair color, gender, and other attributes.
- Unauthorized deepfake creation motivates defenses because malicious actors can modify people’s images without consent.
- Deepfake disruption adds imperceptible perturbations that deteriorate translated outputs until they must be discarded or the manipulation becomes perceptually evident.
- Class-transferable disruptions target conditional image translation without requiring knowledge of the selected conditioning attribute.
- The work adapts adversarial attacks to conditional image translation and presents baseline disruption methods for deepfake systems.
- Adversarial training for GANs is proposed to alleviate disruptions, while spread-spectrum disruption is designed to evade blur defenses in gray-box settings.
2 Related Work
Related work covers image translation, adversarial attacks, attacks on generative models, and defenses. The paper distinguishes its focus on disrupting conditional image translation and transferring attacks across conditioning classes from prior work.
- Prior image-translation systems generate faces with modified expressions or attributes using deep neural networks.
- Classification attacks include FGSM, iterative FGSM, PGD, and feature-space attacks, whereas this paper targets image translation systems.
- Attacks on VAEs and VAE-GANs change reconstructed images to different semantic classes, unlike this paper’s objective of disrupting deepfake generation.
- Prior work on CycleGAN-related self-adversarial attacks seeks higher translation quality or more honest translations, whereas this paper seeks to disrupt translation.
- Earlier image-translation attacks did not address conditional image translation, transfer across conditioning classes, or defenses for these attacks.
- The paper proposes a faster heuristic iterative spread-spectrum disruption for evading blur defenses, extending ideas related to watermarking and transformation-robust attacks.
3 Method
The paper develops disruptions for conditional image translation by adapting adversarial attacks, extending them across conditioning classes, training GANs against disruptions, and evading blur defenses.
- Image Translation Disruption: The authors adapt adversarial attacks to disrupt image translation outputs by adding human-imperceptible perturbations to input images.A disruption is successful when the translated output becomes perceptibly corrupted or modified.
- Image Translation Disruption: The disruption objective maximizes output distortion relative to a reference, while targeted disruption instead pushes the translated output toward that reference.The ideal disruption uses the original generator output as the reference and is a special case of targeted disruption.
- Image Translation Disruption: FGSM, I-FGSM, and PGD are adapted to generate targeted disruptions through one-step or iterative gradient-based updates.Adapted PGD initializes the disrupted image within the perturbation bound and uses the I-FGSM update function.
- Conditional Image Translation Disruption: For class-conditional systems, the proposed class-transferable disruption increases transferability across different target classes by cycling through all classes during optimization.The method increments the class index through K classes and resets it after reaching the final class.
- GAN Adversarial Training: The paper proposes adversarial training for GANs by training the generator, or both generator and discriminator, on adversarial examples.The G+D approach also trains the discriminator using perturbed real and generated images.
- Spread-Spectrum Evasion of Blur Defenses: The spread-spectrum disruption evades blur defenses by iteratively optimizing across different blur types and magnitudes.The method applies a modified I-FGSM update while cycling through K blurring operations.
4 Experiments
The experiments evaluate image-level disruptions across multiple translation architectures, class-transfer settings, adversarial defenses, and blur-evasion scenarios. The results show successful disruption across architectures, transfer across conditioning classes, improved resistance from G+D training, and spread-spectrum evasion of unknown blur defenses.
- 4.1 Image Translation Disruption: FGSM, I-FGSM, and PGD disruptions affect GANimation, StarGAN, pix2pixHD, and CycleGAN, with stronger disruptions successful across tested architectures.GANimation is more robust at ϵ = 0.05 than StarGAN but can be disrupted with ϵ = 0.1.
- 4.1 Image Translation Disruption: L2 ≥0.05 corresponds to very noticeable output distortions and defines the reported percentage of successfully disrupted images.The experiments use this threshold for the % dis. metric.
- 4.1 Image Translation Disruption: Away From Output is most effective with I-FGSM and PGD, while Away From Input achieves similar effectiveness without computing G(x), saving one generator forward pass.The comparison is performed on StarGAN using CelebA.
- 4.2 Class Transferable Adversarial Disruption: Iterative and joint class-transferable disruptions successfully transfer across action units, whereas targeting an incorrect action unit is ineffective.The qualitative examples and Table 4 evaluate transfer using distance between undisrupted and disrupted outputs.
- 4.3 GAN Adversarial Training and Other Defenses: G+D adversarial training alleviates disruptions more effectively than generator adversarial training, while combining G+D training with blur provides the greatest resistance in the white-box setting.Strong PGD disruptions remain relatively successful against the combined defense.
- 4.4 Spread-Spectrum Evasion of Blur Defenses: Spread-spectrum disruption and EoT defeat diverse blur types and magnitudes; spread-spectrum performs better for high-magnitude Gaussian blur and is roughly K times faster than EoT.EoT outperforms spread-spectrum on box blur cases on average.
5 Conclusion
The paper defends against unauthorized deepfake generation by disrupting conditional facial-manipulation networks with adapted adversarial attacks. It also addresses transfer across conditioning attributes, GAN robustness, and blur defenses.
- The proposed defense disrupts conditional image-translation facial-manipulation networks using adapted adversarial attacks.
- The authors report class-transferable disruptions that remain effective across target attributes, rather than requiring knowledge of the selected conditioning class.
- Adversarial training for GANs is presented as a first step toward image-translation networks resistant to disruption.
- Blurring can defend against naive disruptions in gray-box settings, while the proposed spread-spectrum disruption evades a wide range of blur defenses.