Source-linked AI summary
Learning Human-Perceived Fakeness in AI-Generated Videos via Multimodal LLMs
Xingyu Fu, Siyi Liu, Yinuo Xu, Pan Lu, Guangqiuse Hu, Tianbo Yang, Taran Anantasagar, Christopher Shen, Yikai Mao, Yuanzhe Liu, Keyush Shah, Chung Un Lee, Yejin Choi, James Zou, Dan Roth, Chris Callison-Burch
TL;DR
Existing video-generation evaluations largely overlook whether humans can detect and ground fine-grained visual traces of machine generation. DeeptraceReward supplies expert spatial-temporal annotations and trains multimodal reward models, with a 7B model outperforming GPT-5 by 34.7% across identification, grounding, and explanation while revealing a consistent difficulty gradient.
Problem
Existing evaluation methods overlook human perception of fine-grained clues of inauthenticity in increasingly realistic AI-generated videos.
Method
DeeptraceReward annotates 3.3K generated videos with 4.3K natural-language, bounding-box, and timestamp annotations, then trains multimodal language models as reward models.
Results
The trained 7B reward model achieves 70.2% average performance and surpasses GPT-5 by 34.7%, while classification is easier than trace detection and temporal labeling is hardest.
Takeaways & Limitations
DeeptraceReward provides a testbed and training signal for more human-aligned video generation and understanding.
Takeaways & Limitations
The dataset retains high-quality dynamic videos with object or human movement, excluding ambiguous, severely degraded, or still videos.
Abstract
from arXiv · showhide
Can humans identify AI-generated (fake) videos and provide grounded reasons? While video generation models have advanced rapidly, a critical dimension -- whether humans can detect deepfake traces within a generated video, i.e., spatiotemporal grounded visual artifacts that reveal a video as machine generated -- has been largely overlooked. We introduce DeeptraceReward, the first fine-grained, spatially- and temporally- aware benchmark that annotates human-perceived fake traces for video generation reward. The dataset comprises 4.3K detailed annotations across 3.3K high-quality generated videos. Each annotation provides a natural-language explanation, pinpoints a bounding-box region containing the perceived trace, and marks precise onset and offset timestamps. We consolidate these annotations into 9 major categories of deepfake traces that lead humans to identify a video as AI-generated, and train multimodal language models (LMs) as reward models to mimic human judgments and localizations. On DeeptraceReward, our 7B reward model outperforms GPT-5 by 34.7% on average across fake clue identification, grounding, and explanation. Interestingly, we observe a consistent difficulty gradient: binary fake v.s. real classification is substantially easier than fine-grained deepfake trace detection; within the latter, performance degrades from natural language explanations (easiest), to spatial grounding, to temporal labeling (hardest). By foregrounding human-perceived deepfake traces, DeeptraceReward provides a rigorous testbed and training signal for socially aware and trustworthy video generation.
1 INTRODUCTION
DeeptraceReward addresses whether humans can identify AI-generated videos through grounded visual traces and explanations. It introduces a fine-grained benchmark and shows that multimodal models struggle more with trace localization than binary fake-versus-real classification.
- 1 INTRODUCTION: DeeptraceReward is the first benchmark of human-perceived deepfake traces with fine-grained spatial and temporal expert annotations.It contains 3.3K generated videos and 4.3K annotations, each pairing an explanation with bounding boxes and onset–offset timestamps.
- 1 INTRODUCTION: The benchmark categorizes human-perceived traces into nine major types, including object distortion, movement anomalies, and commonsense violations such as object disappearance.These artifacts can reveal machine generation despite strong surface realism.
- 1 INTRODUCTION: Multimodal baselines exceed 70% on binary real-versus-fake classification but remain below 36% on fine-grained deepfake-trace grounding.This gap shows that recognizing a video as fake is easier than identifying the specific evidence for that judgment.
- 1 INTRODUCTION: The paper trains multimodal language models as reward models to capture human judgments and localizations of deepfake traces.The experiments evaluate 13 baseline multimodal language models on DeeptraceReward.
- 1 INTRODUCTION: 70.2% average performance across identification, grounding, and explanation lets the trained 7B reward model surpass GPT-5 by 34.7% and Gemini 2.5 Pro by 40.2%.The model reaches 99.4% on binary classification but about 70% on the other tasks.
2 DEEPTRACEREWARD DATASET
DEEPTRACEREWARD is built to study human-perceived deepfake traces in realistic, dynamic AI-generated videos through fine-grained spatial, temporal, textual, and categorical annotations. The benchmark combines balanced fake and real videos, expert review, nine major trace categories, and reward-model evaluation.
- 2.1 VIDEO COLLECTION: Dynamic scenes are retained because unnatural trajectories, object distortions, and sudden blurring are more likely to emerge during movement than in static videos.The collection excludes ambiguous, severely degraded, implausibly interacting, or incoherently moving samples.
- 2.1 VIDEO COLLECTION: 6.0% of videos remain ambiguous to annotators even after manual filtering, marking a boundary on human-perceived fakeness judgments.This residual ambiguity is reported during annotation rather than attributed to a specific artifact category.
- 2.2 ANNOTATION PIPELINE: Each annotation combines temporally aware bounding boxes, natural-language explanations, and structured category tags identifying localized visual anomalies.Experts inspect videos frame by frame through the LabelBox platform, with additional review and consensus procedures for ambiguous cases.
- 2.1 VIDEO COLLECTION: 4,334 annotations cover 3,318 high-quality fake videos from seven state-of-the-art generators, paired with 3,318 real videos matched by video-length distributions.The real videos are sampled from LLaVA-Video-178K, while the fake videos are collected from diverse generators.
- 2.3 DATASET ANALYSIS: Nine movement-centric categories explain 90% of categorized cases, and one trace may belong to multiple categories.The categories include object disappearance, distortion, blurring, trajectory, redundancy, merging, splitting, and background distortion, among others.
- 2.3 DATASET ANALYSIS: The trained reward model outperforms baselines across categories, with especially strong gains on object splitting and object merging.Figure 5 summarizes performance comparisons between baseline models and the best reward model trained on DEEPTRACEREWARD.
3 EXPERIMENTS
Experiments evaluate multimodal LMs on DEEPTRACEREWARD using classification, explanation, spatial grounding, and temporal localization metrics, then compare baselines with supervised-finetuned 7B models. The results show a substantial advantage for the trained model and a consistent difficulty gradient from classification to temporal localization.
- 3.3 RESULTS AND ANALYSIS: 70.2% overall performance lets the best VideoLLaMA 3-based 7B model surpass GPT 5 by 34.7% and Gemini 2.5 Pro by 40.2%.The comparison is reported on the DEEPTRACEREWARD test set after training on its high-quality data.
- 3.1 EXPERIMENTAL SETUPS: Evaluation uses seven metrics covering fake-versus-real accuracy, explanation quality, bounding-box overlap and distance, and temporal distance.Bounding-box IoU measures predicted-region overlap with the ground truth, while time distance measures the predicted trace start relative to the annotated start.
- 3.3 RESULTS AND ANALYSIS: Baseline models remain below 37% overall, while GPT 5, GPT 4.1, and Gemini 2.5 Pro are the only baselines above 30%.GPT 4.1 exceeds GPT 5 overall by 1%, despite GPT 5 producing stronger explanations.
- 3.3 RESULTS AND ANALYSIS: The trained 7B model reaches 99%+ classification accuracy, 70.6 explanation performance, 32.6 bounding-box IoU, and 21.6 time distance.These results substantially improve trace identification across the reported metrics, although performance remains below human-level behavior.
- 3.3 RESULTS AND ANALYSIS: Binary real-versus-fake classification is easier than fine-grained trace detection, whose difficulty rises from explanations to spatial grounding and then temporal localization.SOTA baselines average 85.6% on classification but only 27.3 explanation, 11.9 bounding-box IoU, and 99.8 time-distance scores.
- 3.3 RESULTS AND ANALYSIS: Ablations show only minor fine-grained metric differences across supervision settings, with all supervision signals producing the best overall performance.The compared settings remove temporal annotations, explanations, or both.
4 RELATED WORK
Existing video evaluations emphasize global or holistic quality, while prior fake-video detection often targets human-face deepfakes. DEEPTRACEREWARD instead captures fine-grained human-perceived fakeness with spatial and temporal localization.
- Existing evaluation methods mainly score global video characteristics or holistic preferences, lacking fine-grained sources of perceived fakeness.
- DEEPTRACEREWARD collects spatially and temporally localized annotations to identify and justify human-perceived fakeness in generated videos.
- Prior AI-generated video detection methods largely focus on human-face deepfakes.
5 CONCLUSION
The paper introduces DEEPTRACEREWARD to address the neglect of human perception in fine-grained video-fakeness evaluation. It finds that existing multimodal LMs struggle with trace detection, while a dedicated reward model improves performance and supports more human-aligned video research.
- DEEPTRACEREWARD is a benchmark with expert-annotated, spatially and temporally localized deepfake traces.
- Existing multimodal LMs fall short on deepfake trace detection, whereas a dedicated reward model demonstrates significant performance gains.
- The benchmark is intended to support future research toward more human-aligned video generation and understanding.
A ADDITIONAL EXAMPLES
The appendix presents category-based examples from DEEPTRACEREWARD and notes that the dataset offers interpretable trace-based feedback while carrying potential misuse risks.
- Figure 6 presents DEEPTRACEREWARD examples organized by deepfake-trace category.
- The dataset makes generated videos more interpretable by tracing deepfake artifacts through human visual-perception feedback.
- If misused, the dataset could support video generators that produce higher-quality fake videos for deceptive purposes.
C LIMITATIONS AND FUTURE DIRECTIONS.
The paper identifies annotation cost and scope as limitations, while leaving the effect of DEEPTRACEREWARD on video generation for future work.
- DEEPTRACEREWARD requires intensive human effort throughout the annotation process.
- The study focuses on existing off-the-shelf video generators and multimodal language models.
- Future work may test whether using DEEPTRACEREWARD as a fine-grained reward model improves video generation through reinforcement learning.
D EXPLANATION EVALUATION PROMPT
The evaluation prompt scores explanations by whether they identify both the correct object and reason, while the inference prompt requests fake/real classification, spatial coordinates, onset time, and an explanation. Inference settings standardize frame sampling where possible but retain model-specific constraints.
- D EXPLANATION EVALUATION PROMPT: Explanations receive 0, 0.5, or 1 according to whether the predicted object and reason are absent, partially correct, or both correct.A score of 0 denotes a wrong or missing explanation; 0.5 denotes the same object with an incorrect reason; 1 denotes correct object and reason identification.
- D EXPLANATION EVALUATION PROMPT: A clock-related prediction scores 0 when it names a tilted clock face instead of floating hands without support.
- D EXPLANATION EVALUATION PROMPT: A prediction scores 1 when it exactly identifies that the bottle cap is warped at the hinge.
- D EXPLANATION EVALUATION PROMPT: A prediction scores 0.5 when it identifies pixelation in the lamp bulb but misses that the bulb flickers.
- E INFERENCE SETTING AND PROMPT: Models with adjustable frame rates use 2 fps, whereas models without that capability retain their default settings.
- E INFERENCE SETTING AND PROMPT: Video-LLaVa processes exactly 8 frames through VLMEvalKit, while Llava-onevision-qwen2-7b-ov-hf supports 1 fps and was configured for 20 maximum frames.
- E INFERENCE SETTING AND PROMPT: All zero-shot models use the same baseline prompt for inference.
- E INFERENCE SETTING AND PROMPT: The baseline prompt requires an AI-generated-versus-real decision, [x0, y0, x1, y1] coordinates, a fake onset time, and an explanation enclosed in tagged fields.
F QUALITATIVE ANALYSIS
The paper directs readers to a qualitative analysis in Figure 7, which compares reference explanations with outputs from GPT and the best 7B reward model.
- F QUALITATIVE ANALYSIS: The text identifies Figure 7 as the location of the paper’s qualitative analysis.
- F QUALITATIVE ANALYSIS: Figure 7 compares ground-truth explanations, GPT-generated explanations, and explanations from the best 7B reward model based on Video-LLaMa3.
G FINETUNING SETUPS
The fine-tuning experiments use eight H100 GPUs, with epoch duration differing between the VideoLLaMA 3 7B and Qwen 2.5 VL 7B base models. Training, validation, and test data are randomly split by unique video in an 8:1:1 ratio.
- G FINETUNING SETUPS: Training, validation, and test data are randomly sampled from DeeptraceReward by unique video using an 8:1:1 ratio, with each table cell reporting fake/real counts.
- G FINETUNING SETUPS: Fine-tuning runs on 8 × NVIDIA H100 80GB SXM GPUs; one epoch takes about 40 minutes for VideoLLaMA 3 7B and 70 minutes for Qwen 2.5 VL 7B.
H VAL SET RESULTS
The section provides the hyper-parameter settings for the best fine-tuned models and reports their validation results on DeeptraceReward.
- H VAL SET RESULTS: Table 4 lists hyper-parameter settings for the best fine-tuned models built on the two base models.
- H VAL SET RESULTS: Table 5 reports validation-set results on DeeptraceReward.