PaperlayerView exact revision on arXiv

Source-linked AI summary

Adaptive Gated Deepfake Detection for Low-Resolution and Resource-Constrained Environments

Vaishnavi Sen, Cody Laurie, Rashida Hasan

arXiv:2609.05320v1cs.CVcs.LG

TL;DR

Deepfake detectors can be costly and brittle on low-quality inputs because they often use fixed inference paths. AdaGate-DF combines RGB and DCT features with quality-aware multi-exit routing. On Celeb-DF, it achieves competitive detection while improving with resolution and balancing uncertainty-aware prediction with computational efficiency.

  • Problem

    Fixed-path deepfake detectors face challenges with low-resolution inputs and resource-constrained deployment.

  • Method

    AdaGate-DF combines RGB and DCT representations with image-quality-aware gating across fast, medium, and full multi-exit paths.

  • Results

    AdaGate-DF achieves an AUC of 0.9370 on Celeb-DF, while its AUC increases from 0.8954 at 128×128 to 0.9708 at 384×384.

  • Takeaways & Limitations

    The results support quality-aware adaptive routing as a practical balance among detection performance, uncertainty-aware prediction, and computational efficiency.

Abstract

from arXiv · show

Deepfake detection models often rely on high-quality inputs, fixed inference paths, and computationally expensive architectures, limiting their use in low-resolution and resource-constrained settings. This paper proposes AdaGate-DF, an adaptive gated deepfake detection framework that uses image-quality cues to route samples through a dual multi-exit system so high-quality images can exit earlier and save compute. We evaluated AdaGate-DF against MaD-CoRN, DefakeHop++, and ShuffleNetV2 on two benchmark datasets (Celeb-DF and FaceForensics++) under multiple configurations to test image resolution dependence and training and inference efficiency. On Celeb-DF, AdaGate-DF achieves an AUC of 0.9370, outperforming MaD-CoRN and DefakeHop++ while maintaining a low inference latency. Resolution-based testing shows consistent improvement as input resolution increases, reaching an AUC of 0.9708 at 384 by 384. The FaceForensics++ results highlight that AdaGate-DF remains effective under class imbalance, following competitive results with evaluated models. Overall, AdaGate-DF demonstrated a practical balance between detection performance, uncertainty-aware prediction, and computational efficiency for variable-quality deepfake detection.

I. INTRODUCTION

AdaGate-DF addresses low-resolution, variable-quality deepfake detection and fixed-cost inference by combining dual representations with quality-aware adaptive routing. It is evaluated across benchmark datasets, uncertainty behavior, resolution robustness, and computational efficiency.

  • Motivation: Low-resolution, compressed, blurred, or inconsistently sized inputs can suppress facial artifacts used for deepfake detection.These conditions motivate methods that remain effective when fine-grained manipulation cues are degraded.
  • Motivation: Fixed inference paths process every sample identically, creating inefficiency for real-time deployment under limited computational resources.This is especially relevant when large volumes of images or video frames must be processed.
  • Proposed approach: AdaGate-DF combines RGB and DCT-based frequency features with fast, medium, and full paths selected using image-quality cues.The design adapts computation to input quality instead of using a fixed-depth pipeline.
  • Evaluation: The evaluation covers Celeb-DF and FF++, comparisons with state-of-the-art models, uncertainty-aware prediction, resolution-based performance, training time, and inference latency.The study examines both detection quality and deployment-oriented efficiency.
  • Reproducibility: The paper provides code, dataset sources, preprocessing, and evaluation scripts for reproducibility.

II. RELATED WORK

Prior work separately addresses low-resolution robustness, frequency-aware representations, lightweight architectures, and adaptive inference. AdaGate-DF combines these directions through quality-aware routing over RGB and DCT representations.

  • Low-resolution detection: Low-resolution methods target degraded inputs, while spatial-only detection can become less reliable when resizing, compression, down-sampling, or blur suppresses facial artifacts.
  • Frequency-aware detection: Frequency-domain methods use DCT, wavelet, and spectral representations to capture manipulation or compression inconsistencies outside the RGB domain.
  • Efficient detection: Lightweight detectors reduce computational cost but generally retain a fixed computational pathway for every input.
  • Adaptive inference: Adaptive inference assigns different network depths to samples, reducing unnecessary computation for easier inputs while reserving capacity for difficult cases.
  • Research gap: AdaGate-DF combines RGB and DCT representations with a quality-aware gate that routes samples through fast, medium, or full inference paths.

III. METHODOLOGY

AdaGate-DF uses dual spatial and frequency branches with multiple exits, while an adaptive gate selects the computation path from image-quality cues.

  • Architecture: The framework combines spatial features with DCT-derived frequency features for deeper analysis.An adaptive gating module selects a dynamic inference path based on image quality.

A. Dataset and Preprocessing

Training uses randomly selected resolutions and complementary RGB and DCT inputs, while lightweight quality features drive adaptive routing. The gate adjusts computation to input quality and reserves deeper processing for challenging samples.

  • Resolution sampling: Training randomly samples resolutions from 128 × 128, 224 × 224, 256 × 256, and 384 × 384 to improve robustness to varying input resolution.
  • Input representations: The preprocessing pipeline normalizes images and generates a grayscale DCT representation alongside the RGB input.
  • Quality features: Quality features include resolution, sharpness, blur, compression artifacts, brightness, and contrast.These features are assembled into a vector denoted q.
  • Adaptive routing: A lightweight gating network maps q to a routing decision that determines the computation level for each input.The design processes high-quality samples efficiently while reserving deeper computation for challenging inputs.

C. Dual-Brach Gated Architecture

AdaGate-DF combines spatial RGB and frequency DCT branches with multiple exits, enabling inference paths that trade computational cost against detection depth. The fast path uses shallow features for high-quality or less complex inputs, while deeper paths support more challenging inputs.

  • C. Dual-Brach Gated Architecture: AdaGate-DF combines RGB spatial and DCT-based frequency branches, each implemented with three exit points.The exits correspond to shallow, intermediate, and deep representations.
  • C. Dual-Brach Gated Architecture: Three inference paths provide different computational budgets: fast, medium, and full.The multi-exit design supports adaptive selection among these paths.
  • C. Dual-Brach Gated Architecture: The fast path uses shallow RGB and frequency features with minimal computational cost for high-quality or less complex inputs.
  • C. Dual-Brach Gated Architecture: The medium path uses intermediate features to balance computational efficiency and detection performance.
  • C. Dual-Brach Gated Architecture: The full path fuses the deepest features from both branches to produce the most accurate prediction for low-quality or challenging inputs.

D. Training and Inference Strategy

AdaGate-DF jointly trains its branch exits and gating network, then uses quality features during inference to select a fast, medium, or full path. The experiments assess uncertainty-aware performance, resolution robustness, and computational efficiency across balanced and imbalanced conditions.

  • D. Training and Inference Strategy: All branches and exit points are optimized jointly with classification losses applied to every exit.
  • D. Training and Inference Strategy: Quality-based routing rules supervise the gating network, encouraging simpler inputs toward shallow paths and complex inputs toward deeper paths.
  • D. Training and Inference Strategy: During inference, quality features produce a routing decision that selects the fast, medium, or full path.The stated goal is to reduce computation for simpler inputs while maintaining strong detection performance for challenging cases.
  • D. Training and Inference Strategy: Experiments measure uncertainty-aware performance, resolution robustness, and computational efficiency under balanced and imbalanced conditions.

A. Dataset split and configuration

The evaluation uses Celeb-DF and FF++ with specified train, validation, and test splits, multiple input resolutions, and consistent preprocessing and hardware conditions. AdaGate-DF additionally uses RGB and frequency branches with lightweight image-quality features.

  • A. Dataset split and configuration: Celeb-DF contains 80,824 training, 10,104 validation, and 10,103 testing images with approximately balanced real and fake classes.
  • A. Dataset split and configuration: FF++ contains 7,000 videos with an approximate 1:6 real-to-fake ratio and produces 55,953 frames after extraction.
  • A. Dataset split and configuration: All models are evaluated at 128 × 128, 224 × 224, 256 × 256, and 384 × 384 resolutions, with 224 × 224 as the base comparison setting.
  • A. Dataset split and configuration: AdaGate-DF uses RGB and DCT frequency branches alongside quality features including resolution, blur, sharpness, compression proxy, brightness, and contrast.
  • A. Dataset split and configuration: AdaGate-DF training uses AdamW with learning rate 2 × 10^-4, weight decay 1 × 10^-4, batch size 32, and 15 epochs.
  • A. Dataset split and configuration: Training and evaluation use NVIDIA L40 GPUs with 48 GB of memory, and the same environment is used for all compared models.

C. Evaluation Metrics

The evaluation reports standard classification metrics together with uncertainty rate, training time, and inference latency. AUC is emphasized for FF++ because its class imbalance can make accuracy misleading, while comparisons include three efficient detectors.

  • C. Evaluation Metrics: Evaluation includes accuracy, AUC, precision, recall, F1-score, uncertainty rate, training time, and inference latency.
  • C. Evaluation Metrics: AUC is particularly important for FF++ because its highly imbalanced split can influence accuracy through the majority class.
  • C. Evaluation Metrics: Uncertainty rate measures low-confidence predictions, using a 0.65 maximum-confidence threshold for AdaGate-DF and [0.4, 0.6] for probability-based models.
  • C. Evaluation Metrics: AdaGate-DF is compared with MaD-CoRN, DefakeHop++, and ShuffleNetV2 as efficient deepfake detection models.MaD-CoRN uses convolutional reservoir networks, DefakeHop++ uses handcrafted features with classifier-based prediction, and ShuffleNetV2 is designed for low-latency inference.

V. RESULTS AND DISCUSSION

The experiments assess uncertainty-aware performance, benchmark detection quality, and gated-versus-fixed inference behavior across Celeb-DF and imbalanced FF++. Results show strong Celeb-DF performance and reduced uncertainty, while FF++ requires cautious interpretation because class imbalance weakens class-separation evidence.

  • 0.9370 AUC and 0.8654 F1-score on Celeb-DF show the proposed gated model outperforming MaD-CoRN and DefakeHop++ while remaining competitive with ShuffleNetV2.
  • The gated configuration improves substantially over fast-only and medium-only variants, while full-only achieves the strongest proposed-variant performance at higher computational cost.The gated model trades slight performance degradation against forced binary decision for faster training and broader image coverage.
  • 22.23% uncertainty for the gated model contrasts with 83.31% for fast-only and 85.01% for MaD-CoRN on Celeb-DF.The results associate adaptive routing with fewer low-confidence predictions, while noting that uncertainty should be interpreted alongside AUC and F1-score.
  • 0.5163 AUC on imbalanced FF++ outperforms DefakeHop++, although ShuffleNetV2 reaches 0.5693 and low AUC values indicate limited class separation.The approximately 1:6 real-to-fake ratio makes AUC the primary comparison metric because threshold-dependent metrics can be inflated.
  • FF++ results remain competitive but require caution because confidence can remain high despite weak discrimination under class imbalance.The authors recommend supplementing future interpretation with balanced accuracy, specificity, false positive rate, false negative rate, and confusion-matrix analysis.

B. Resolution-Based Evaluation

Resolution-based evaluation tests whether models remain reliable across low, standard, and high input resolutions. AdaGate-DF improves consistently on Celeb-DF with increasing resolution and remains more stable than DefakeHop++ on imbalanced FF++.

  • 128×128, 224×224, 256×256, and 384×384 represent low-, standard-, and higher-resolution evaluation conditions.The tests examine manipulation detection when fine-grained artifacts are reduced and when additional spatial and frequency information is available.
  • Celeb-DF AUC rises from 0.8954 at 128×128 to 0.9708 at 384×384 for the proposed gated model.The authors attribute this pattern to the spatial-frequency representation benefiting from additional image detail while retaining strong lower-resolution performance.
  • The proposed method performs more consistently than MaD-CoRN and DefakeHop++ across tested Celeb-DF resolutions.ShuffleNetV2 leads at 224×224 and 256×256 but declines at 384×384, so higher resolution does not uniformly improve every architecture.
  • FF++ AUC ranges from 0.5043 to 0.5253 for the proposed method, compared with 0.2331 to 0.4883 for DefakeHop++.Despite class-imbalance effects, the gated model maintains more consistent class-separation behavior across resolutions.

C. Training and Inference Efficiency

AdaGate-DF offers a favorable training–inference efficiency profile, combining low latency with adaptive routing for resource-constrained deployment. The evaluation also identifies remaining needs for broader class-balanced metrics and more standardized timing analysis.

  • Efficiency results: 6.07 hours and 0.14 ms: AdaGate-DF’s Celeb-DF training and inference profile supports efficient deployment.On FF++, it requires 5.58 hours of training and 0.25 ms inference time.
  • Efficiency results: Training time and inference time do not always follow the same trend across evaluated models.DefakeHop++ has relatively low training time but the highest inference latency on both datasets, while MaD-CoRN has the highest training time.
  • Efficiency results: AdaGate-DF achieves lower inference latency than ShuffleNetV2 on Celeb-DF while also providing adaptive routing and uncertainty-aware prediction.
  • Efficiency results: AdaGate-DF has the lowest inference latency among the compared methods on FF++, supporting its use as a practical resource-constrained inference framework.
  • Future work: Future evaluation should add class-balanced metrics, standardize hardware timing, and quantify gate assignments across resolutions.The proposed extensions include balanced accuracy, specificity, false positive rate, false negative rate, confusion matrices, and gate-allocation analysis.
Loading 2609.05320v1…