PaperlayerView exact revision on arXiv

Source-linked AI summary

Stop My Dancing! Understanding, Detecting and Attributing Motion-Aware Deepfake Videos

Fazhong Liu, Yan Meng, Tian Dong, Guoxing Chen, Haojin Zhu

arXiv:2610.11496v1cs.CVcs.CR

TL;DR

Pose-guided diffusion models can produce coherent full-body motion, creating a detection and attribution challenge that existing MAD-specific benchmarks and methods do not adequately address. The paper constructs a MAD benchmark and proposes MoDA, combining spatial-semantic and frequency cues. MoDA outperforms prior detectors across reported settings, reaching 94.8% in-distribution accuracy.

  • Problem

    MAD’s coherent full-body synthesis challenges existing detection and attribution methods, while prior work lacks a standardized multi-model benchmark designed for this setting.

    Source: §1 · p. 1, §2.3 · p. 3

  • Method

    The authors construct a MAD benchmark and propose MoDA, a framework that combines spatial-semantic and frequency features for detection and attribution.

    Source: §1 · p. 3, §3.2.1 · p. 4

  • Results

    MoDA outperforms prior detectors across in-distribution, cross-dataset, and adaptive-attack settings, reaching 94.8% in-distribution accuracy.

    Source: §1 · p. 2, §8 · p. 14

  • Takeaways & Limitations

    The benchmark and MoDA provide a basis for evaluating MAD detection and attribution across real-world and adaptive-attack settings.

    Source: §1 · p. 3, §8 · p. 14

    Because benign uploads can greatly outnumber MAD clips, MoDA should screen or rank videos rather than trigger automatic takedowns, with thresholds calibrated to the expected ratio.

    Source: §5.3.4 · p. 11

Abstract

from arXiv · show

Pose-guided diffusion models can now synthesize entire human figures in motion, spawning a new class of deepfakes: Motion Aware Deepfake (MAD) that have already reached hundreds of millions of viewers. To better understand this emerging threat, we construct the first MAD-specific benchmark and measurement framework, containing over 1.5 million frames that mix 1,363 real and 30,122 synthetic videos from six controllable generators, with realistic perturbations and open-world evaluation splits. Then, we dissect MAD and discover that, despite their global coherence, these videos betray faint yet reliable cues: because the model relies on limited input frames for motion synthesis, it must predict and simulate coherent movement at motion boundaries, thereby producing high-frequency artifacts along with model-specific spectral fingerprints. Based on the observations obtained from analysis on dataset, we propose MoDA, the first defense framework tailored to detect and attribute MAD videos. MoDA couples spatial semantics with steganalysis-rich frequency features via cross-domain alignment and multi-scale aggregation, achieving 94.8% in-distribution and 89.1% cross-dataset detection accuracy gains of 10% to 25% over prior work and 91.5% model attribution accuracy. MoDA achieves 81.94% accuracy on 200 clips produced by two unseen commercial MAD platforms, indicating promising zero-shot transfer, and 78.13% detection accuracy on 1,200 unseen MAD video clips (55k frames in total) collected from the open Internet. Under white-box, gray-box, and black-box adaptive attacks, MoDA maintains relatively stable detection and attribution performance while the accuracies of the baselines drop rapidly.

1 Introduction

The introduction frames Motion-Aware Deepfakes as pose-guided videos that synthesize coherent full-body motion, challenging existing detectors and attribution methods. It presents a dedicated benchmark and MoDA, which combines spatial-semantic and frequency cues to detect and attribute MAD.

1 Introduction

  • MAD synthesizes coherent full-body motion, creating detection and attribution challenges that existing methods and benchmarks do not adequately address.The introduction identifies motion-identity invariance, suppressed high-frequency and physiological cues, and a lack of unified cross-generator and open-world measurement.Source: §1 · pp. 1–2
  • The paper introduces a multi-model MAD benchmark and reports that architectural differences leave quantifiable spectral fingerprints in generated videos.Its contributions also include evaluating detection across cross-dataset, no-prior-knowledge, motion-level, and adaptive-attack settings.Source: §1 · pp. 2–3
  • 94.8% in-distribution and 89.1% transferability accuracy, versus baseline averages of 81.6% and 66.2%, respectively, with 91.5% average model-attribution accuracy.Joint spatial-semantic and frequency cues improve accuracy by 3.62% over a pure spatial detector and 6.50% over a pure frequency baseline.Source: §1 · p. 2
  • 78.13% accuracy on 1,200 open-Internet clips and 81.94% on 200 clips from two commercial services, exceeding existing methods by over 25% in both cases.The authors caution that evolving generators may evade detection until labeled examples are collected, while MoDA can be adapted with a small fine-tuning budget.Source: §1 · p. 2
  • Across 200 held-out clips, MoDA outperforms the strongest competitor by 9.9%, 16.4%, and 20.8% under white-box, gray-box, and black-box adaptive attacks, respectively.The attack suite includes white-box PGD and frequency-specific SSA, gray-box MI-FGSM, and black-box platform degradations and perturbations.Source: §1 · pp. 2–3
  • MoDA combines spatial-semantic and frequency features to detect MAD and attribute videos to their source generators.It uses high-pass residuals as fingerprints; the analysis associates distinct ringing periods with U-Net skip-connection density and additional high-frequency differences with architectural choices.Source: §1 · pp. 2–3

2 Background and related work

The background reviews diffusion-based video generation and controllable synthesis before outlining AIGC detection and attribution approaches. It identifies the lack of MAD-specific methods and a standardized benchmark as unresolved gaps.

2.1 Diffusion Models for Video Generation

  • Diffusion models generate data through forward noising and learned backward denoising, with the denoiser trained to estimate injected noise.Latent diffusion reduces computational complexity by denoising in latent space.Source: §2.1 · p. 3

2.2 Controllable Video Generation

  • Controllable video generation research improves quality and control through inter-frame attention and motion modules, while portrait-conditioned approaches combine reference images with pose guidance.The cited examples include AnimateDiff, DisCo, and Animate Anybody.Source: §2.2 · p. 3

2.3 AIGC Detection and Attribution

  • Existing detection and attribution methods span spatial, temporal, frequency, reconstruction, vision-language, and video-transformer approaches, but are not designed for MAD’s full-body motion setting.The paper identifies missing MAD-specific interpretability, attribution, robustness, and standardized multi-model benchmarks; it also notes benchmark risks such as identity leakage, source bias, and class imbalance.Source: §2.3 · p. 3

3 Dissecting Motion-aware Deepfake

MAD differs from conventional face-focused deepfakes by using diffusion-based methods to synthesize a victim’s full body and motion. The paper characterizes this as producing a more potent deceptive effect.

3 Dissecting Motion-aware Deepfake

  • MAD uses advanced diffusion-based methods to generate fake videos encompassing a victim’s full body and motion.This contrasts with conventional deepfakes that primarily manipulate facial attributes [35].Source: §3 · p. 4
  • The paper describes MAD as more deceptive than conventional deepfakes that primarily manipulate facial attributes [35].Source: §3 · p. 4
  • MAD’s defining contrast in this passage is whole-body motion generation rather than facial-attribute manipulation [35].Source: §3 · p. 4

3.1 Threat Model

The threat model considers attackers using accessible motion-controllable generation services to create and disseminate full-body motion forgeries. It assumes limited technical expertise and hardware resources.

3.1 Threat Model

  • The attack pipeline pairs a target’s publicly available portrait with an arbitrarily acquired motion sequence, then synthesizes a full-body motion forgery.The motion sequence may come from an unrelated third party or maliciously constructed content.Source: §3.1 · p. 4
  • The assumed attacker has limited technical expertise and hardware resources and needs only accessible motion-controllable generation services.Source: §3.1 · p. 4
  • Attackers may add facial reenactment or perturbations and disseminate the resulting videos through social networks.The passage characterizes the pipeline as highly automated, with low barriers to entry and potentially severe consequences.Source: §3.1 · p. 4

3.2 MAD Dataset

The paper builds the MAD dataset as a benchmark for pose-guided controllable video generation and evaluates it with complementary open-world splits. Its videos use diverse dance-like motion, and its evaluation also includes perturbations and composite edits.

3.2.1 Real & Synthetic Videos.

  • The MAD dataset benchmarks diffusion-based pose-guided video generation using six representative models, with clips generated from reference portraits and driving motion.The dataset is designed to address the lack of a dedicated benchmark for this task.Source: §3.2.1 · p. 4
  • The benchmark contains 1,363 real videos and 30,122 synthetic clips, with dance-heavy motion and identity- and motion-disjoint train/test splits.Real videos come from screened TikTok and social-media sources; the benchmark also includes low-motion speech-style evaluation to bound generalization beyond dancing.Source: §3.2.1 · p. 4, Table 1 · p. 5
  • Facial reenactment and eight common image perturbations are applied to test robustness to composite attacks and platform-like degradation.The reenactment is a secondary editing step applied to MAD videos.Source: §3.2.1 · p. 4

3.2.2 Open-Internet & Commercial.

  • Two complementary open-world splits test generalization to unseen commercial generators and manually screened MAD clips collected from the Internet.The commercial split has 200 clips from I2VControl and Wan2.2-Animate; the open-Internet split contains 1,200 short clips corroborated by two annotators.Source: §3.2.2 · p. 5

3.3 Understanding MAD and Insights of MoDA

The analysis finds that MAD’s holistic synthesis and temporal smoothness weaken conventional spatial and temporal detection cues, while motion-related high-frequency artifacts and model-specific spectral signatures remain measurable.

3.3.1 Limitation of Image-level Inconsistency Detection.

  • MAD synthesizes whole frames, erasing local spatial discrepancies that image-level detectors such as FaceX-Ray seek.Figure 2 presents multi-view evidence, and the analysis concludes that local pixel consistency alone is no longer a reliable cue.Source: §3.3.1 · p. 5, Figure 2 · p. 5

3.3.2 Limitation of Temporal Inconsistency Detection.

  • MAD’s strict temporal smoothness suppresses frame-to-frame jitter and physiological anomalies, weakening detectors that rely on those signals.Gaze-angle traces are smoother for MAD than for real videos, while conventional deepfakes show pronounced jitter.Source: §3.3.2 · pp. 5–6

3.3.3 Persistence of High-Frequency Artifacts.

  • Although diffusion attenuates generic frequency fingerprints, MAD leaves high-frequency anomalies around portrait boundaries and large-motion regions.These artifacts arise as the model hallucinates coherent motion and content exposed after the portrait moves.Source: §3.3.3 · p. 6

3.3.4 Model-Specific Fingerprint.

  • The reported spectral metrics distinguish the three models: Animate Anyone has small T, low H, and low Elow/Ehigh; MimicMotion has large T, high H, and high Elow/Ehigh; MusePose has small T, medium-high H, and the highest Elow/Ehigh.The metrics are dominant period T, radial entropy H, and low-to-high frequency energy ratio Elow/Ehigh.Source: §3.3.4 · p. 6
  • High-frequency residuals form distinct, architecture-linked clusters that provide a basis for attributing the generating model.Figure 3 visualizes the separability in a feature space defined by spectral metrics.Source: Figure 3 · p. 6, §3.3.4 · p. 6
  • The metric patterns are interpreted in relation to model design: temporal modeling is associated with larger T or higher Elow/Ehigh, while fusion and attention choices relate to entropy.Animate Anyone’s dense spatial attention is associated with low entropy; MimicMotion’s fusion and MusePose’s complex spatiotemporal attention with higher entropy.Source: §3.3.4 · p. 6

4 System Design of MoDA

MoDA combines learnable high-pass filtering, spatial-semantic and frequency features, attention to motion-sensitive regions, and multi-scale fusion for MAD detection and attribution. Its pipeline also trains a video-level detector and a generator classifier.

4.1 High-Pass Filter Based Processing

  • The learnable high-pass filter adaptively highlights motion-boundary inconsistencies and texture anomalies, supplying a complementary high-frequency view to spatial semantics.Its channel-wise depthwise filters begin with distinct high-pass patterns and are constrained to have zero mean.Source: §4.1 · p. 7

4.2 Spatial-Semantic and Frequency-Domain Feature Extraction

  • Spatial-semantic features preserve structural information, while frequency features expose subtle artifacts that may be imperceptible in pixels.Source: §4.2.1 · p. 7

4.2.1 Cross-Domain Feature Alignment.

  • Cross-domain alignment computes pixel-wise cosine-similarity affinities between spatial-semantic and frequency features to establish correlations while preserving each stream’s characteristics.A learnable scalar controls fusion strength.Source: §4.2.1 · p. 7

4.2.2 Key-Region Focus.

  • The Key-Region Focus Module uses self-attention to direct the model toward motion-related regions, with stronger activation on pronounced motion than subtle movement.The module is applied three times for multi-scale learning of location-enhanced features.Source: §4.2.2 · pp. 7–8

4.2.3 Multi-Scale Feature Aggregation.

  • Multi-scale aggregation aligns and fuses different-resolution features through local windows, retaining both global semantics and local details.Feature maps are resized to a shared resolution before concatenation and window-based fusion.Source: §4.2.3 · p. 8

4.3 MAD Video Detection

  • The detection classifier is trained with binary cross-entropy and labels a video as MAD when its predicted fake probability exceeds 0.5.Source: §4.3 · p. 8

4.4 MAD Model Attribution

  • For attribution, MoDA reuses its detector’s feature backbone with a lightweight task-specific head and assigns each video the modal generator prediction across frames.The learned representation encodes spectral and structural fingerprints rather than relying on handcrafted statistics and threshold tuning.Source: §4.4 · p. 8

5 Evaluation of MoDA

MoDA is evaluated on a benchmark with identity- and motion-separated splits, under perturbations, generator shifts, and error-analysis settings. Across these tests, its detection remains comparatively strong, while ablations examine the contributions of frequency cues, localization, alignment, and multi-scale features.

5.1 Experiment Details

  • Experiments include random clipping, compression, blurring, and noise; evaluation reports AUC, accuracy, FPR, FNR, and F1-score, with frame predictions aggregated into video decisions.The study also compares methods under face reenactment and common perturbations.Source: §5.1 · pp. 8–9, §5.3 · p. 9

5.2 Dataset Preparation for Experiments

  • The benchmark contains 1,363 real videos and 30,122 synthetic clips from six generators, with roughly 1.4 million frames; identity and motion sequences are disjoint across train and test splits.The final protocol uses 241,344 labeled frame samples, evenly divided between training and testing.Source: §5.2 · p. 9

5.3.1 In-distribution Performance.

  • Above 94.8% average accuracy across in-distribution perturbation regimes, MoDA exceeds RECCE by 13.2 percentage points.Under common perturbation, it records 95.2%, 14.9 percentage points above DeFake.Source: §5.3.1 · p. 9, Table 3 · p. 10

5.3.2 Transferability Performance.

  • MoDA’s average cross-dataset recall improves by 12% to 25% over prior methods and never falls below 75% when trained on a single generator.Training on Mimic yields 89.2% recall on Muse and 77.5% on Animate, compared with DFGaze’s 73.1% and 25.6%.Source: §5.3.2 · p. 10, Table 4 · p. 10

5.3.3 Ablation Test.

  • The Frequency-only model achieves 84.53% cross-dataset ACC, while another variant scores 73.06%; the passage attributes stronger generalization to motion-boundary-induced high-frequency artifacts being more stable across generators.Attention raises in-distribution AUC from 0.756 to 0.854 and cross-dataset AUC from 0.724 to 0.739. Combining CDFA with KRFM raises in-distribution AUC from 0.764 to 0.871 and reduces the corresponding error rate by 57% relative to CDFA.Source: §5.3.3 · p. 11

5.3.4 Error Analysis and Misses.

  • MoDA’s cross-dataset average FPR/FNR is 0.041/0.086; observed misses include motion blur in real videos, weak boundary artifacts in low-motion MAD, and occlusion or incomplete motion.Recall remains above 0.96 on high-motion dance, low-motion dance, and TED-Talk speech clips.Source: §5.3.4 · p. 11, §5.3.5 · p. 11
  • Because benign uploads can outnumber MAD clips by orders of magnitude, the authors recommend screening or ranking rather than automatic takedowns, with thresholds calibrated to the expected benign-to-fake ratio.Source: §5.3.4 · p. 11

5.4 Attribution Performance of MoDA

MoDA’s learned spatial-frequency representation supports generator attribution under realistic perturbations, while a truncated variant targets on-device pre-screening. The full model remains the authors’ primary choice for backend moderation and high-confidence auditing.

5.4 Attribution Performance of MoDA

  • 91.5% average attribution accuracy: MoDA’s spatial-frequency signature remains stable across three generators, though MusePose is hardest to distinguish from MimicMotion.The evaluation distorts videos with compression, noise, and clipping to mimic real-world uploads.Source: §5.4 · p. 11

5.5 Mobile-Oriented Lightweight Variant

  • The lightweight variant reaches 0.756 AUC with 0.067 FPR while reducing inference memory from 1.6GB to 164MB and runtime from 7.6ms/frame to 2.0ms/frame.The authors position it as a front-end filter for resource-constrained devices, while the full model remains the backend option.Source: §5.5 · p. 12, Table 7 · p. 12

6 Robust Validation of MoDA

The section tests MoDA beyond generators known during training and against adaptive adversaries. It evaluates commercial and open-Internet videos, unknown-generator attribution, and multiple attack and mitigation settings.

6.1.1 Detection of Unknown MAD Models.

  • 81.94% positive-class recall on clips from two unseen commercial platforms, rising to 95.10% after fine-tuning on three labeled clips (≈80 frames).The pretrained detector outperformed RECCE (52.58%), DFGaze (43.33%), and DeFake (23.09%) without retraining.Source: §6.1.1 · p. 12

6.1.2 Open-Internet Cases.

  • 78.1% accuracy on 1,200 open-Internet clips, exceeding RECCE (52.0%), XCeption (43.0%), and DeFake (39.0%).Errors concentrate in clips with weak subject motion, very small human regions, or aggressive platform compression.Source: §6.1.2 · p. 12

6.1.3 Attribution of Unknown MAD Models.

  • 77.60% accuracy separating known from unknown generators using an open-set rule based on maximum softmax probability.The tested unseen generators were Disco [47], MagicAnimate [50], and MagicPose [6].Source: §6.1.3 · pp. 12–13

6.2 Adaptive Attacks towards MoDA

  • MoDA degrades more gracefully than baselines under adaptive attacks, but stronger perturbations reduce accuracy while visibly degrading clips.The section identifies simultaneous suppression of high-frequency residuals and preservation of motion realism as constraints on attackers.Source: §6.2 · p. 13

6.2.1 White-box attack.

  • 39.4% detection accuracy under PGD, and 57.12% accuracy under the frequency-oriented SSA white-box attack.The PGD evaluation used three iterations with α=1/255 and ε=5/255; under SSA, RECCE achieved 43.66%, XCeption 26.86%, and DeFake 11.62%.Source: §6.2.1 · p. 13

6.2.2 Gray-box attack.

  • 55.41% accuracy under the gray-box MI-FGSM attack, versus 39.0% for RECCE and 6.4% for DeFake.The evaluation used 200 victim-split samples and 10 iterations per sample.Source: §6.2.2 · p. 13

6.2.3 Black-box attack.

  • 96.6% average robustness to black-box post-processing, at least 11.9% above the strongest competitor.MoDA retained 96.7% accuracy after lossy compression, 96.0% after pixel noise, and 94.6% after visual effects; motion-boundary attacks caused less than a 5% accuracy drop over 1,000 frames.Source: §6.2.3 · pp. 13–14

6.3 Mitigation against Adaptive Attacks

  • Adversarial training is stronger than input preprocessing in white-box and gray-box settings, while preprocessing is cheaper but smooths some benign high-frequency evidence.Adversarial training improves robustness by preserving discriminative motion-boundary cues under small worst-case perturbations.Source: §6.3 · p. 14

7 Discussion and Limitations

MoDA faces a detection window for new generators and leaves room to improve on open-Internet clips. The discussion identifies adaptation and training-data additions as next steps.

Scalability window

  • A new generator can create a detection window until defenders collect labeled examples, though small-budget fine-tuning can improve transfer to unseen commercial models.The discussion also notes that larger perturbation budgets visibly hurt video quality, while complex optimization adds runtime overhead.Source: §7 · p. 14

Improving open-Internet performance

  • 78.13% open-Internet accuracy leaves room for improvement, especially for platform-native compression and small-subject cases.The proposed next steps include continual adaptation as commercial generators emerge and better calibration or multi-stage moderation cascades.Source: §7 · p. 14

Adaptive attacks and trade-offs

  • Adversarial training and stronger defenses are identified as likely needs for improving robustness against adaptive attacks.Larger perturbation budgets visibly degrade video quality, and more complex optimization adds runtime overhead.Source: §7 · p. 14

8 Conclusion

The conclusion defines and studies Motion-Aware Deepfakes and presents a dataset and MoDA, a detector and attribution method. Experiments report performance across benchmark, real-world, and adaptive-attack settings.

8 Conclusion

  • The paper defines Motion-Aware Deepfakes as fake videos manipulated by pose-guided controllable video-generation models.It also formally defines the MAD process.Source: §8 · p. 14
  • The authors construct a MAD dataset and design MoDA using frequency-domain and spatial-semantic features to detect and attribute fake videos.Source: §8 · p. 14
  • Experiments on the curated benchmark and real-world clips report that MoDA surpasses prior detectors across in-distribution, cross-dataset, and adaptive-attack settings.Source: §8 · p. 14
Loading 2610.11496v1…