PaperlayerView exact revision on arXiv

Source-linked AI summary

Adversarial Perturbations Against Real-Time Video Classification Systems

Shasha Li, Ajaya Neupane, Sujoy Paul, Chengyu Song, Srikanth V. Krishnamurthy, Amit K. Roy Chowdhury, Ananthram Swami

arXiv:1807.00458v1cs.LGcs.CRcs.CVstat.ML

TL;DR

Research on adversarial perturbations has largely focused on images rather than temporally varying video, despite real-time video classifiers being used in safety-relevant settings. This paper develops GAN-based universal perturbations for video, including circular perturbations that handle clip-boundary shifts. It reports over 80% misclassification on targeted activities while largely preserving classification of other classes.

  • Problem

    Adversarial-perturbation studies have mostly focused on images, leaving open whether stealthy attacks can target real-time video classifiers used in safety-relevant applications.

    Source: §1 · p. 1

  • Method

    The paper extends GAN-based universal perturbations for video and introduces circular dual-purpose perturbations that remain valid under cyclic frame shifts.

    Source: §1 · p. 2, §10 · p. 12

  • Results

    Over 80% misclassification rates were achieved on targeted activities across UCF101 and Jester while other classes remained correctly classified with relatively high accuracy.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Stealthy attacks on deployed video-classification systems could cause unnoticed pecuniary loss and compromise personal safety.

    Source: §1 · p. 1

    The approach does not account for contextual consistency between misclassified actions and their surrounding video context.

    Source: §8 · p. 12

Abstract

from arXiv · show

Recent research has demonstrated the brittleness of machine learning systems to adversarial perturbations. However, the studies have been mostly limited to perturbations on images and more generally, classification that does not deal with temporally varying inputs. In this paper we ask "Are adversarial perturbations possible in real-time video classification systems and if so, what properties must they satisfy?" Such systems find application in surveillance applications, smart vehicles, and smart elderly care and thus, misclassification could be particularly harmful (e.g., a mishap at an elderly care facility may be missed). We show that accounting for temporal structure is key to generating adversarial examples in such systems. We exploit recent advances in generative adversarial network (GAN) architectures to account for temporal correlations and generate adversarial samples that can cause misclassification rates of over 80% for targeted activities. More importantly, the samples also leave other activities largely unaffected making them extremely stealthy. Finally, we also surprisingly find that in many scenarios, the same perturbation can be applied to every frame in a video clip that makes the adversary's ability to achieve misclassification relatively easy.

1 INTRODUCTION

The paper investigates stealthy adversarial attacks on real-time video classifiers, where temporal inputs make online targeting and clip alignment challenging. It develops universal perturbations and GAN-based methods, including circular and frame-wise variants, and reports strong targeted misclassification while largely preserving other classes.

1 INTRODUCTION

  • Real-time video attacks must be generated efficiently, target only selected actions, and cope with unknown clip boundaries as activities change over time.The attacker cannot know in advance which frames correspond to the target action or where the classifier's clips begin and end.Source: §1 · pp. 1–2
  • Universal perturbations are generated offline with a GAN, avoiding intensive online computation for each incoming video clip.The approach is designed to work on unseen inputs.Source: §1 · p. 2
  • Dual-purpose universal perturbations target misclassification of one class while minimizing effects on other classes to make attacks stealthier.Blindly perturbing all frames can also misclassify non-target activities and expose an attack.Source: §1 · p. 2
  • C-DUP uses a 3D perturbation clip that remains valid under cyclic frame shifts, addressing sensitivity to misaligned clip boundaries.The method adds a circular-perturbation unit between the generator and discriminator.Source: §1 · p. 2
  • 2D-DUP applies a simple 2D perturbation across every video frame, but works on only a subset of videos.The paper uses this variant to investigate the effect of temporal structure.Source: §1 · p. 2
  • Over 80% misclassification rates on targeted activities were achieved across UCF101 and Jester while other classes remained correctly classified with relatively high accuracy.UCF101 covers coarse-grained human actions, while Jester covers fine-grained hand gestures.Source: §1 · p. 2

2 BACKGROUND

The background introduces real-time video classification, the C3D classifier used in the study, and GAN architectures that the attack adapts. It frames online recognition as classification from evolving score curves and describes the temporal modeling and efficiency of C3D.

2.1 Real-time video-based classification systems

  • Sliding-window classifiers produce evolving score curves for each action class, which are smoothed to predict ongoing actions online.Low scores for the true class in each window can prevent recognition of the true action.Source: §2.1 · p. 2, Figure 1 · p. 3

2.2 The C3D classifier

  • C3D is a real-time video-classification target that uses 3D convolution and pooling to model temporal information.Its input is a 16-frame clip with 112-by-112 RGB frames, followed by a softmax class-score output.Source: §2.2 · p. 3

2.3 Generative Adversarial Networks and their relevance

  • Unlike a video-generation GAN, the attack adapts a GAN-like architecture so a fixed video classifier evaluates perturbed clips and the generator learns perturbations that fool it.The paper builds on prior GAN-like work for image perturbations [29] and extends it for video classification.Source: §2.3 · p. 3

3 THREAT MODEL AND DATASETS

The study evaluates attacks under a white-box threat model in which the adversary can inject perturbations into a video stream. It uses UCF-101 and Jester to represent coarse- and fine-grained action classification.

3.1 Threat model

  • The white-box adversary knows the training data and classifier and can inject perturbations into the real-time stream.The assumed access could come from intercepting the stream or previously installed malware.Source: §3.1 · p. 3
  • Attacks aim to misclassify every clip from a target class while correctly classifying clips from non-target classes.The threat model defines stealth in terms of leaving other actions unaffected.Source: §3.1 · p. 3

3.2 Our datasets

  • UCF-101 represents coarse-grained actions and Jester represents fine-grained gestures; UCF-101 split 1 supplies 9,537 training and 3,783 testing clips, while Jester uses 148,092 training and 14,787 validation clips for testing.Jester's validation set is used as the test set because test labels are unavailable.Source: §3.2 · pp. 3–4
  • The illustrated C3D architecture comprises eight convolutional, five max-pooling, and two fully connected layers, followed by softmax.The architecture figure specifies 3 × 3 × 3 convolution kernels and a 1 × 2 × 2 pool1 kernel.Source: Figure 2 · p. 4
  • The attack architecture differs from a traditional GAN by using a video classifier as discriminator and generating perturbations rather than direct classifier inputs.Its objective is to make the classifier misclassify perturbed inputs.Source: Figure 3 · p. 4

4 GENERATING PERTURBATIONS FOR REAL-TIME VIDEO STREAM

The attack computes perturbations offline for unseen incoming video, using a GAN-like generator trained against a fixed video classifier. The generator produces universal perturbation clips and is optimized to lower the score of the true class.

4 GENERATING PERTURBATIONS FOR REAL-TIME VIDEO STREAM

  • Because incoming video is unavailable in advance, the attack computes universal perturbations offline and applies them online to unseen inputs.Universal perturbations were previously studied for image classification [29].Source: §4 · p. 4
  • A 3D generator produces universal perturbation clips, a post-processor provides a place for later modifications, and a fixed pre-trained video classifier evaluates perturbed clips.Only the generator is trained, unlike traditional GANs that train generator and discriminator together.Source: §4 · p. 4
  • The generator maps latent noise to a perturbation clip, adds it to a training clip, and seeks to lower the classifier's score for that clip's true class.The classifier returns a score vector for the perturbed input.Source: §4 · pp. 4–5
  • Training minimizes cross-entropy across training data so the learned perturbation can misclassify incoming samples from the same distribution.Generator feedback from the discriminator drives updates across training samples.Source: §4 · p. 5
  • Universal perturbations alone can misclassify unintended clips, potentially compromising the attack's stealth requirement.The paper addresses this issue in its subsequent stealth-focused approach.Source: §4 · p. 5

5 MAKING PERTURBATIONS STEALTHY

Because attackers cannot know in advance which frames contain a target action, perturbations must be applied across frames; blindly applying universal perturbations can also misclassify non-target actions. The section introduces dual-purpose universal perturbations to target one class while minimizing effects on others.

5 MAKING PERTURBATIONS STEALTHY

  • Universal perturbations applied blindly can misclassify non-target actions, while attackers cannot know in advance which frames capture the target action.For example, benign customer actions could be mapped to shoplifting and trigger alarms.Source: §5 · p. 5
  • Dual-Purpose Universal Perturbations (DUPs) are universal perturbations intended to misclassify target-class inputs while leaving non-target classifications correct.Because target frames cannot be selected in advance, perturbations are added to each frame.Source: §5 · p. 5
  • The DUP objective minimizes target-class cross-entropy while maximizing cross-entropy for other classes, with λ weighting the target-class term.A smaller λ places less emphasis on target misclassification when stealth is more important.Source: §5 · p. 5

6 THE TIME MACHINE: HANDLING THE TEMPORAL DIMENSION

Video clips are captured with sliding windows, but the attacker cannot know the classifier's starting position in advance, so perturbations can depend on clip alignment. The section examines this boundary effect and introduces cyclically shifted perturbations and a same-frame special case.

6.1 The boundary effect

  • 0.39 average normalized correlation for the same frame at positions {5, 9} shows that perturbations can differ across staggered clips.Aligned positions {7, 7} have correlation 1.00.Source: §6.1 · p. 6, Figure 5 · p. 6
  • Average perturbation magnitude is larger on the first and last few clip frames than in the middle.The authors conjecture that nearby frame locations have similar perturbations, while larger offsets involve more diverse clips.Source: §6.1 · p. 6, Figure 6 · p. 6
  • As the tested clip's start offset grows, attack success initially falls and then rises as the clip becomes similar to the subsequent clip.A stride smaller than the clip size creates overlapping input clips and the same offset-related boundary effect.Source: §6.1 · pp. 6–7, Figure 7 · p. 7

6.2 Circular Dual-Purpose Universal Perturbation

  • C-DUP makes every cyclic shift of a universal perturbation clip a valid perturbation, addressing unknown classifier clip boundaries.The generator output is rolled through offsets before being added to training inputs.Source: §6.2 · pp. 7–8

6.3 2D Dual-Purpose Universal Perturbation

  • 2D-DUP constrains all frames to share one perturbation, a simpler special case of C-DUP that may reduce attack success.Its generator outputs one frame, which is tiled to form the perturbation clip.Source: Figure 9 · p. 7, §6.3 · p. 8

7 EVALUATIONS

Evaluations compare universal perturbations on UCF-101 and Jester, testing circular C-DUP against a single-frame 2D-DUP. C-DUP is more robust to temporal misalignment, while 2D-DUP performs comparably on UCF-101 but less well on Jester when temporal changes are stronger.

7.1 Experimental Setup

  • Experiments use C3D classifiers, with UCF-101 tested on applying-lipstick and Jester on sliding-hand-right and shaking-hands targets; benign accuracies reported are 96.1% and 90.03%.Training balances target and non-target inputs in each batch; non-target examples are undersampled for both datasets.Source: §7.1 · pp. 8–9
  • The evaluation measures quasi-imperceptibility and attack success, defined by target-class misclassification and correct classification of non-target classes.The experiments compare DUP with C-DUP and report attack rates across perturbation/input-clip alignments.Source: §7.1 · p. 9, §7.2 · p. 9, §7.2.1 · p. 10

7.2 C-DUP Perturbations

  • Across UCF-101 alignments, C-DUP averages 84% target-class success versus DUP’s 68.26%; at zero misalignment, DUP reaches 84.49% target and 88.03% non-target success.UP achieves 84.01% target misclassification but only 45.2% correct classification for non-target classes; it is excluded from further evaluations.Source: §7.2.1 · p. 10, Figure 14 · p. 10
  • C-DUP perturbations show more pronounced evolution on Jester than UCF-101; the authors conjecture UCF-101 emphasizes spatial appearance, while Jester’s fine-grained actions make temporal information more important.The added perturbation shown across original and perturbed frames is described as quasi-imperceptible to human eyes.Source: §7.2.2 · p. 10, §7.3.1 · p. 11
  • For sliding-hand-right on Jester, C-DUP averages 85.14% target and 81.03% non-target success across alignments, versus DUP’s 52.42% and 82.36%.C-DUP is slightly lower at zero offset, where DUP is tailored, but outperforms it under misalignment.Source: Figure 14 · p. 10, §7.2.2 · p. 11
  • On Jester’s shaking-hands target, C-DUP’s target-class attack success is 79.03%, versus DUP’s 57.78%; the authors attribute its advantage to robustness against temporal misalignment.The target class has no temporally similar class, making misclassification harder; the authors say stealth is affected to some extent but remains reasonably high.Source: Figure 14 · p. 10, §7.2.2 · p. 11

7.3 2D-Dual Purpose Universal Perturbations

  • On UCF-101, 2D-DUP reaches 87.58% target and 83.37% non-target attack success, versus C-DUP’s 87.52% and 84.00%, respectively.The authors describe the results as on par and report that the same perturbation is added to every frame.Source: §7.3.1 · p. 11
  • On Jester, 2D-DUP reaches 84.64% target and 80.04% non-target success for sliding-hand-right; for shaking-hands, rates fall to 70.92% and 54.83%.The authors attribute the latter degradation to stronger temporal changes, which a single 2D perturbation manipulates less effectively than evolving C-DUP.Source: §7.3.2 · pp. 11–12

8 DISCUSSION

The discussion identifies white-box knowledge and unmodeled video context as limitations. It also outlines consistency-based defenses, while warning that frame-level consistency may be less useful for strongly temporal gestures.

8 DISCUSSION

  • The attacks assume the adversary knows the deployed DNN; black-box attacks are proposed as future work, not evaluated here.The authors cite transferability studies as motivation for exploring black-box attacks.Source: §8 · p. 12
  • The approach does not account for context, so implausible action combinations or sequences may alert a human observer.The authors give running, kicking a ball, and applying makeup as an example of actions unlikely to occur together.Source: §8 · p. 12
  • Frame- and clip-consistency checks are suggested defenses, but may be less effective on Jester, where one image may not reveal hand-movement direction.The authors say such checks may work well when temporal flow is not pronounced, as in UCF-101.Source: §8 · p. 12
  • The authors also propose checking action-object consistency with an additional image classifier, such as whether a kicking action matches a soccer ball.This defense is described as future work.Source: §8 · p. 12

9 RELATED WORK

Prior adversarial-perturbation work largely addresses images or other modalities rather than real-time video’s distinctive input characteristics. This paper extends image-focused generative perturbation work to video classification.

9 RELATED WORK

  • Earlier adversarial-input studies showed that small perturbations can change model predictions, but generally did not address real-time temporally varying video inputs.The paper positions its focus as adversarial perturbations for real-time video action-recognition systems.Source: §9 · p. 12
  • Image research established hardly perceptible and image-agnostic adversarial perturbations, including universal perturbations that misclassify target-class images with high probability [39] [28].The paper contrasts those image-focused methods with its study of video perturbations.Source: §9 · p. 12
  • Mopuri et al. [29] used a GAN generator to model universal image perturbations; this paper extends that generative framework to attacks on videos.GANs had also been used to generate natural images [34].Source: §9 · p. 12
  • Studies of face recognition, voice recognition, and malware classification did not account for real-time video activity recognition’s distinctive input characteristics [25] [36] [12].Source: §9 · p. 12

10 CONCLUSIONS

The paper addresses real-time generation, stealth, and temporal structure in video attacks by extending GAN architectures. Its approaches achieve around 80% attack success rates across coarse- and fine-grained activity datasets.

10 CONCLUSIONS

  • The approach extends GAN architectures to address real-time generation, stealth, and temporal structure in video classification attacks.Experiments cover coarse-grained actions and fine-grained hand gestures.Source: §10 · p. 12
  • Around 80% attack success rates were achieved in both cases.The experiments use one coarse-grained action dataset and one fine-grained hand-gesture dataset.Source: §10 · pp. 12–13
  • The authors propose investigating possible defenses in future work.Source: §10 · p. 13

A APPENDIX: VISUALIZING PERTURBATIONS ON UCF101

The UCF101 appendix visualizes C-DUP and 2D-DUP perturbations for the target class apply lipstick, then shows their effects on a clean clip. Both perturbations are described as quasi-imperceptible to human vision.

A APPENDIX: VISUALIZING PERTURBATIONS ON UCF101

B APPENDIX: VISUALIZING PERTURBATIONS ON JESTER

The Jester appendix visualizes C-DUP and 2D-DUP perturbations for the target class shaking hand, then shows their effects on a clean clip. The perturbation is described as quasi-imperceptible to human vision.

B APPENDIX: VISUALIZING PERTURBATIONS ON JESTER

Loading 1807.00458v1…