PaperlayerView exact revision on arXiv

Source-linked AI summary

A Generalizable and Explainable Framework for Synthetic Video Detection Using First-Digit Gradient Statistics

Sidharth Shanu, Gautam Kumar, Tej Singh

arXiv:2609.39585v1cs.CV

TL;DR

Photorealistic synthetic videos make detection harder, while end-to-end detectors offer limited insight into what they learn. This paper analyzes Sobel-gradient first digits and uses LDA and an MLP; despite deviations from Benford’s law, the statistics discriminate real and AI-generated videos.

  • Problem

    As synthetic videos become more photorealistic, end-to-end detectors provide limited insight into what they learn, while forensic approaches seek compact, interpretable signals.

    Source: §I · p. 1

  • Method

    The pipeline analyzes first-digit statistics of Sobel gradients, visualizes class separation with LDA, and classifies using an MLP.

    Source: §I · p. 1, §III · p. 2

  • Results

    Sobel-gradient first-digit statistics do not follow Benford’s law, yet an MLP achieves over 90% accuracy and AUC on AI-generated video detection, with LDA showing strong class separation.

    Source: §I · p. 1

  • Takeaways & Limitations

    First-digit distributions of non-scale-invariant properties can carry discriminatory signal, and the distribution remains relatively stable across different scenes when frame rate and filter size are uniform.

    Source: §VI · p. 10

Abstract

from arXiv · show

AI video generators have not only become harder to detect but are used to generate a diverse set of scenarios from landscapes to street views to animal videos. This creates a problem where CNN-based detectors are effective but offer no insight into their inner workings, while forensics-based detectors are often pretrained for a set scenario or become too complex to derive meaningful insights. We present a novel approach to AI video detection using Sobel gradient values analysed with the first-digit law. Using linear discriminant analysis, we visualise the discriminatory signal, while a multi-layer perceptron is used for classification. The detection method has no generator- or scenespecific features, and the model has no knowledge of container formats, codec, bitrate, or compression artefacts. The model is trained and tested on GenBuster-200K, GenBusterBench, GenVA, FaceForensics++ C23, and CelebDF. We also show how zero-shot detection fails even though the feature set carries a discriminatory signal.

I. INTRODUCTION

As synthetic videos become more photorealistic, detection needs interpretable signals that can keep pace with generators. This paper introduces Sobel-gradient first-digit analysis and situates it among forensic and machine-learning approaches to synthetic-media detection.

I. INTRODUCTION

  • Published detection artefacts can become training signals for generators, while detectors gain no comparable advantage from advances in generation.The paper describes this asymmetry as a reason to keep refining detection artefacts.Source: §I · p. 1
  • Forensics-based detectors isolate human-selected signals with low-capacity models, whereas end-to-end CNNs and larger models learn from raw pixels but provide limited insight.The cited discussion also notes that heavy models require substantial training data and computation.Source: §I · p. 1
  • First-digit statistics compact video information, and this paper searches for a discriminatory signal in Sobel gradients rather than the DCT coefficients used in prior work.Earlier first-digit-law approaches mainly used DCT and other frequency transforms, [15],.Source: §I · p. 1

II. RELATED WORK

  • CNN-based detectors can generalise across some generators, but their performance degrades when generator architecture or post-processing differs from training data,.Video benchmarks also report poor transfer between forgery pipelines [3],.Source: §II · p. 2
  • Prior spectral fingerprints rely on high-frequency DCT components that JPEG recompression can discard, while earlier video first-digit work targeted double-compression artefacts,,,, [15],.These limitations motivate forensic features beyond such frequency-domain signals.Source: §II · p. 2
  • The paper adds a 3-D Sobel temporal-derivative bank to the spatial bank, encoding frame-to-frame pixel-change rates that synthetic generators produce inconsistently relative to genuine optical flow.The approach is presented as a quantitative forensic feature intended to generalise across generators and withstand post-processing.Source: §II · p. 2

III. METHODOLOGY

The method resamples videos, computes spatial and temporal-derivative Sobel features, and summarizes gradient responses as first-digit histograms and energy terms. It concatenates the features into a fixed 800-dimensional vector for LDA visualization or MLP classification.

III. METHODOLOGY

  • Ten channels capture luminance, sharpness, color-plane correlations, and chrominance, and each is processed with two kernel sizes and two gradient directions.The channels include luminance, Laplacian-magnitude sharpness, signed channel-correlation differences, and BT.601 chrominance.Source: §III · pp. 2–3
  • Videos are resampled at uniform temporal intervals, with the target rate no higher than the dataset’s lowest native frame rate, and temporal features use consecutive sampled-frame differences.No additional temporal filtering is applied before differencing.Source: §III · pp. 2–3
  • The two feature banks are concatenated into an 800-dimensional vector, which is projected with LDA for visualization or classified directly by an MLP.The vector is the classifier’s sole input.Source: §III · pp. 2, 4, Figure 1 · p. 3
  • The spatial bank measures edge energies within each frame, while the temporal-derivative bank measures edge energies in frame-to-frame changes using the same Sobel machinery.The banks differ in whether Sobel kernels operate on the resampled frame or its temporal-difference image.Source: §III · p. 3
  • For each bank, channel, kernel size, and direction, the method computes a nine-bin first-significant-digit histogram from nonzero responses and an RMS energy over all sampled responses.The energy term measures overall gradient strength and can help distinguish smoother or blurred synthesized regions from regions with stronger edge structure.Source: §III · pp. 3–4

IV. EXPERIMENTAL SETUP

The experiments evaluate the fixed feature pipeline across five video datasets, using dataset-specific splits and feature configurations. An MLP performs classification, while LDA visualizes class separability and Cohen’s d measures per-feature discriminability.

IV. EXPERIMENTAL SETUP

  • Each video is represented by a fixed 800-dimensional vector from spatial and temporal-derivative Sobel responses across ten channels, two kernel sizes, and two directions.The feature extraction stage uses non-overlapping temporal windows and no data augmentation.Source: §IV · p. 4
  • A three-layer MLP with ReLU activations and dropout classifies the feature vectors, using focal loss to address class imbalance.Source: §IV · p. 4
  • LDA visualizes class separability and quantifies between-class versus within-class variance, while Cohen’s d measures per-feature discriminability.Source: §IV · p. 4
  • The evaluation covers GenBuster-200K, GenBusterBench, GenVA, FaceForensics++ C23, and CelebDF, with dataset-specific splits and roles.GenVA is used for zero-shot cross-domain testing with real videos drawn from the GenBuster-200K test split; the other datasets include specified in-domain or benchmark evaluation protocols.Source: §IV · p. 4

A. GenBuster-200K: In-Domain Binary Detection

On GenBuster-200K, the in-domain detector separates real videos from the twelve generators and achieves strong binary metrics. Its signal spans multiple gradient features, while detection rates vary across generators.

A. GenBuster-200K: In-Domain Binary Detection

B. GenBusterBench: Benchmark Generalisation

GenBusterBench performance remains comparable across the standard and extended subsets, then drops on the wild subset of latest-generation models. The analysis links this pattern to differences in artefact directions and a distributed gradient-energy signal.

B. GenBusterBench: Benchmark Generalisation

  • AUC is 0.976 on the standard, 0.984 on the extended, and 0.939 on the wild GenBusterBench subset.The wild subset contains latest-generation models whose synthesis quality has advanced furthest and artefact magnitudes are smallest.Source: B. GenBusterBench: Benchmark… · p. 6
  • Wild-subset performance drops because its artefacts diverge in direction from those represented during training, which a fixed decision boundary cannot fully intercept.The passage identifies this directional divergence as the structural reason for the drop, independent of training-set size.Source: B. GenBusterBench: Benchmark… · p. 6, Figure 7 · p. 6
  • The top GenBusterBench features show fake gradient energy below real, with energy features dominating and discriminative power spread across many features.The cumulative curve reaches 50% at ∼174 features and 80% at ∼399; all top-20 features have |d| values extending above 1.3.Source: B. GenBusterBench: Benchmark… · p. 6, Figure 9 · p. 6

C. GenVA: Zero-Shot Cross-Domain Generalisation

A model trained on GenBuster-200K achieves strong zero-shot results on GenVA without fine-tuning. The analysis reports separated fake clusters and extreme effect sizes, while noting that two generator families overlap with training.

C. GenVA: Zero-Shot Cross-Domain Generalisation

  • Without fine-tuning on GenVA, overall AUC is 0.993 and balanced accuracy is 0.979.The LDA projection shows all three GenVA generators clustered apart from real videos despite the model never having been trained on GenVA.Source: Figure 11 · p. 6, C. GenVA: Zero-Shot Cross-Domain… · p. 7
  • Unseen VideoCrafter-2 achieves 97.1% detection, while Pika v1 and Sora rates of 99.7% and 99.2% are partly expected because both families occur in training.The passage describes the cross-domain transfer as partially in-distribution for Pika v1 and Sora.Source: C. GenVA: Zero-Shot Cross-Domain… · p. 7
  • GenVA effect sizes reach |d| ≈6, with the top features all negative, indicating fake gradient statistics below real.The cumulative curve reaches 50% of discriminative power at ∼132 features and 80% at ∼291.Source: C. GenVA: Zero-Shot Cross-Domain… · p. 7, Figure 12 · p. 7

D. FaceForensics++: Face-Swap Analysis

FaceForensics++ C23 results are substantially weaker than those for text-to-video datasets. The analysis attributes the limited signal to the small manipulated region and to composited patches retaining real-pixel statistics.

D. FaceForensics++: Face-Swap Analysis

  • Experiments use C23 compression and a 1,600-dimensional feature combining the standard video feature with an additional face-crop feature; DeepFakeDetection is eval-only.Its 994 clips were withheld to avoid dominating the Deepfakes direction and inflating headline accuracy.Source: D. FaceForensics++: Face-Swap Analysis · p. 7
  • The six-class classifier reaches balanced accuracy 0.524 and collapsed binary AUC 0.717 on FaceForensics++.DeepFakeDetection, withheld from training, has a 79.4% detection rate at inference.Source: D. FaceForensics++: Face-Swap Analysis · p. 7, Table VI · p. 7
  • Face-swap signals are diluted by unmodified background pixels, and stitching preserves natural-image statistics because it transplants real pixel patches.The additional face-crop feature directly exposes the classifier to manipulated pixels, but results remain limited for stitched content.Source: D. FaceForensics++: Face-Swap Analysis · p. 8
  • Text-to-video generators synthesize every pixel and produce frame-wide gradient deviations that the feature captures reliably.This contrasts with face swaps, where surrounding pixels remain unmodified real footage.Source: D. FaceForensics++: Face-Swap Analysis · p. 8
  • Recall ordering tracks synthesized frame area; FaceShifter and NeuralTextures typically occupy larger blend regions than earlier Deepfakes outputs, while low original-class recall reflects the chosen threshold.The threshold was tuned to maximise fake recall, so the passage calls the 29.5% original-class recall a tuning artefact rather than a fundamental feature limitation.Source: D. FaceForensics++: Face-Swap Analysis · pp. 7–8, Table VII · p. 7

E. CelebDF: Cross-Dataset Face-Swap Check

On CelebDF, detection is above chance but trails results on fully synthesized benchmarks. Across datasets, Sobel-gradient first-digit histograms show structured deviations from Benford’s curve, while class discrimination reflects shifted probability mass rather than Benford adherence.

E. CelebDF: Cross-Dataset Face-Swap Check

  • CelebDF uses an 800-D video feature and an 80/20 stratified split, with the same MLP and focal-loss recipe but a fixed, uncalibrated 0.5 threshold.Its face-swap fakes alter the face region, unlike the text-to-video generators, and are visually subtle.Source: E. CelebDF: Cross-Dataset Face-Swap… · p. 8
  • ROC-AUC 0.624 and balanced accuracy 0.615 on CelebDF trail the 0.94–0.99 AUC range on fully synthesized benchmarks and are near FF++’s AUC 0.717.At the uncalibrated 0.5 threshold, fake recall is 98.8% versus 24.2% for real videos; the authors attribute this to class imbalance and a conservative threshold.Source: E. CelebDF: Cross-Dataset Face-Swap… · pp. 8–9, Table VIII · p. 9

F. Benford’s Law Deviation Analysis

  • The Benford comparison covers 80 spatial- and temporal-derivative modes across five datasets, comparing empirical first-significant-digit histograms with the theoretical curve.The five datasets are GenBuster-200K, GenBusterBench, GenVA, FaceForensics++, and CelebDF.Source: F. Benford’s Law Deviation Analysis · p. 9
  • Across datasets and modes, FSD histograms depart reproducibly from Benford’s curve, with localized digit excesses most pronounced in raw-colour and channel-correlation modes.Real and synthetic videos share this deviation structure; discrimination instead comes from shifts in probability-mass placement away from a shared non-Benford baseline.Source: F. Benford’s Law Deviation Analysis · p. 9
  • Across 80 modes, histogram correlations are 0.93–0.999 and cross-dataset JS distances are ≤0.12, while cross-generator distances are ≤0.06.Cross-generator shape stability exceeds cross-dataset stability; residual variation concentrates in chrominance and channel-correlation modes at k=1, while sharpness and base-colour modes at k=3 are most stable (JSD ≈0.03).Source: Table IX · p. 9, F. Benford’s Law Deviation Analysis · p. 9

VI. CONCLUSION

The paper presents a Sobel first-digit-law detection pipeline and reports stronger AUCs on fully generated videos than on face-swap datasets. It also argues that first-digit signals remain relatively stable across scenes when frame rate and filter size are uniform.

VI. CONCLUSION

  • AUCs were 0.99 on GenBuster-200K and GenVA, versus 0.717 on FF++ C23 and 0.624 on CelebDF; the authors hypothesize deviation scales with generated-pixel count.Source: §VI · p. 9
  • At uniform frame rate and filter size, first-digit distributions remain relatively stable across scenes, suggesting the signal is more universal than previously thought.Source: §VI · p. 10
  • Future work should test other non-scale-invariant image properties for discriminatory strength and stability, and may combine this approach with other forensic tools.Source: §VI · p. 10
  • The analysed datasets are publicly available, while the study’s analysis code is available from the corresponding author on reasonable request.Source: §VI · p. 10

APPENDIX

Figure 18 presents representative empirical first-significant-digit histograms alongside the theoretical Benford curve for five datasets. It shows real/original and fake/manipulated class distributions across selected temporal-derivative modes.

APPENDIX

  • Figure 18 includes representative overlays for GenBuster-200K, GenBusterBench, GenVA, FaceForensics++, and CelebDF.Source: Figure 18 · p. 10
  • Each panel-block shows two representative temporal-derivative modes at k=1, spanning both gradient directions across all ten feature channels.The full grids cover two feature banks, two kernel sizes, two directions, and ten channels.Source: APPENDIX · p. 10
  • The overlays compare the theoretical Benford curve with empirical mean ±1 standard deviation histograms, using blue for real/original and red for fake/manipulated videos.Source: APPENDIX · p. 10

Related papers

Loading 2609.39585v1…