PaperlayerView exact revision on arXiv

Source-linked AI summary

AV-Deepfake1M: A Large-Scale LLM-Driven Audio-Visual Deepfake Dataset

Zhixi Cai, Shreya Ghosh, Aman Pankaj Adatia, Munawar Hayat, Abhinav Dhall, Tom Gedeon, Kalin Stefanov

arXiv:2311.15308v2cs.CV

TL;DR

Highly realistic deepfakes and the localization of short manipulations embedded in real videos remain challenging, while many benchmarks assume whole clips are real or fake. AV-Deepfake1M is a large-scale dataset built with content-driven audio, video, and audiovisual manipulations. Its benchmark shows a significant performance drop compared with previous datasets, and the paper presents it as an asset for developing next-generation localization methods.

  • Problem

    Many benchmarks assume whole audio, visual, or audiovisual clips are real or fake, while only a few works address locating small manipulations embedded in real videos.

    Source: §1 · p. 2

  • Method

    AV-Deepfake1M is a large-scale audiovisual dataset for temporal localization, generated with content-driven manipulations and state-of-the-art text, audio, and video methods.

    Source: §1 · p. 2

  • Results

    Benchmark methods show a significant drop in performance on AV-Deepfake1M compared with previous datasets.

    Source: §5.1 · p. 6, §6 · p. 8

  • Takeaways & Limitations

    The paper presents AV-Deepfake1M as an asset for building next-generation temporal deepfake localization methods.

    Source: §6 · p. 8

    AV-Deepfake1M, like other deepfake datasets, has an imbalance in the numbers of fake and real videos.

    Source: §6 · p. 8

Abstract

from arXiv · show

The detection and localization of highly realistic deepfake audio-visual content are challenging even for the most advanced state-of-the-art methods. While most of the research efforts in this domain are focused on detecting high-quality deepfake images and videos, only a few works address the problem of the localization of small segments of audio-visual manipulations embedded in real videos. In this research, we emulate the process of such content generation and propose the AV-Deepfake1M dataset. The dataset contains content-driven (i) video manipulations, (ii) audio manipulations, and (iii) audio-visual manipulations for more than 2K subjects resulting in a total of more than 1M videos. The paper provides a thorough description of the proposed data generation pipeline accompanied by a rigorous analysis of the quality of the generated data. The comprehensive benchmark of the proposed dataset utilizing state-of-the-art deepfake detection and localization methods indicates a significant drop in performance compared to previous datasets. The proposed dataset will play a vital role in building the next-generation deepfake localization methods. The dataset and associated code are available at https://github.com/ControlNet/AV-Deepfake1M .

1 INTRODUCTION

Rapidly improving content generation makes real-versus-fake discrimination increasingly difficult, while many benchmarks assume entire clips are real or fake. AV-Deepfake1M targets the resulting gap in temporal localization of small manipulations within otherwise real content.

1 INTRODUCTION

  • Highly realistic generated text, video, and audio make distinguishing real from fake content increasingly difficult, even for humans.The paper connects this challenge to risks including misinformation and fraud.Source: §1 · p. 2
  • Most benchmarks treat entire audio, visual, or audiovisual content as either real or fake, leaving small manipulations embedded in otherwise real content underrepresented.The paper notes that this gap may leave detection methods unreliable on partial manipulations.Source: §1 · p. 2
  • AV-Deepfake1M is a large-scale audiovisual dataset designed specifically for temporal deepfake localization.Its generation pipeline incorporates ChatGPT and state-of-the-art audio and video generation methods.Source: §1 · p. 2
  • The dataset’s content-driven generation pipeline combines novel manipulation strategies with state-of-the-art text, video, and audio generation.Source: §1 · p. 2

2 RELATED WORK

Earlier deepfake datasets largely focused on coarse detection, with many centered on visual manipulations. AV-Deepfake1M builds on prior temporal-localization work with greater scale and quality, additional transcript-editing strategies, and a staged audio-visual generation pipeline.

2 RELATED WORK

  • Many earlier datasets focus on coarse-level detection, progressing from visual-only manipulation to audio and audiovisual modifications.The paper describes a range of visual techniques, including face swapping and reenactment.Source: §2 · p. 2
  • LAV-DF was the first content-driven dataset for temporal localization, but its quality and scale were limited and localization methods already performed strongly on it.AV-Deepfake1M is described as improving the quality, diversity, and scale of datasets for this task.Source: §2 · p. 2
  • The generation pipeline edits a transcript first, generates audio from the edits, then generates subject-dependent video from that audio.The audio generation stage supports both speaker-dependent and speaker-independent generation.Source: Figure 2
  • AV-Deepfake1M replaces LAV-DF’s rule-based antonym selection with an LLM to produce more diverse, context-consistent fake content.The comparison also identifies higher-quality audio and video generation methods as a difference from LAV-DF.Source: Transcript Manipulation · p. 3
  • Unlike LAV-DF, which uses replacement alone, AV-Deepfake1M also manipulates transcripts through insertion and deletion.Source: Transcript Manipulation · p. 3

3 AV-DEEPFAKE1M DATASET

AV-Deepfake1M combines large-scale, content-driven audio-visual manipulations with a generation pipeline evaluated for quality and diversity. Comparisons with LAV-DF report more diverse modifications and content that is more challenging for human viewers.

3 AV-DEEPFAKE1M DATASET

  • AV-Deepfake1M contains 1,886 hours of audio-visual data from 2,068 subjects, with content-driven modifications to audio, visuals, or both.Generated videos preserve the original background and identity.Source: §3 · p. 3

3.1 Data Generation Pipeline

  • The pipeline starts with VoxCeleb2 videos, extracts audio using FFmpeg, and generates transcripts with Whisper before manipulation and media generation.The paper describes this as a three-stage generation pipeline.Source: §3.1 · p. 3

3.1.1 Transcript Manipulation.

  • ChatGPT applies replace, delete, and insert transcript edits, set at ceil(video seconds/10); modified transcripts contain >27.7× more unique new words than LAV-DF.The authors report that this LLM-based strategy generates more diverse content than LAV-DF’s rule-based strategy.Source: §3.1.1 · p. 4, Figure 3 · p. 4

3.1.2 Audio Generation.

  • VITS and YourTTS provide identity-dependent and identity-independent speech; two new-word synthesis strategies add diversity, and audio quality exceeds that of other datasets.Deletion retains background noise, and fake audio segments are loudness-normalized to the original audio.Source: §3.1.2 · p. 4, Table 2 · p. 4

3.1.3 Video Generation.

  • TalkLip generates lip-synchronized visual frames from the original pose and fake audio; the authors report higher visual quality than existing datasets.Visual quality was evaluated using PSNR, SSIM, and FID.Source: §3.1.3 · pp. 4–5, Table 3 · p. 5

3.2 Dataset Statistics

  • Compared with LAV-DF, AV-Deepfake1M has more diversity in modifications, subjects, and segment and video lengths, with a lower average fake-segment proportion; its test set uses VITS-based identity-dependent audio.The dataset split uses 1,657 train subjects and 411 non-overlapping test subjects, with validation selected from the train set.Source: Figure 5 · p. 5, §3.2 · p. 5

3.3 Human Quality Assessment

  • In a study of 25 experienced participants classifying 200 videos, AV-Deepfake1M content was harder for humans to detect than LAV-DF content.Participants also marked potential fake-segment start and end points.Source: Table 5 · p. 6, §3.3 · p. 6

3.4 Computational Cost

  • Generating AV-Deepfake1M required approximately 3,000 GPU hours on NVIDIA RTX6000 GPUs.The total includes about 600 hours for speech recognition, 2,100 for VITS training, and 300 for data generation.Source: §3.4 · p. 6

4 BENCHMARKS AND METRICS

The benchmark evaluates detection and temporal localization of content-driven audio, visual, and audio-visual manipulations. It defines modality-specific test subsets, compares established localization and detection methods, and reports task-appropriate metrics.

4 BENCHMARKS AND METRICS

  • The benchmark targets detection and localization of content-driven audio, visual, and audio-visual manipulations.Source: §4 · p. 6

4.1 Data Partitioning

  • Subset V removes audio-only modifications from the full test set, while subset A removes visual-only modifications for unimodal-method comparisons.Source: §4.1 · p. 6

4.2 Implementation Details

  • Localization benchmarks include Pyannote, TriDet, ActionFormer, BA-TFD, BA-TFD+ [4], and UMMAFormer, alongside image-based and zero-shot detection methods.The benchmark also varies training label access among frame-, segment-, and video-level labels.Source: §4.2 · p. 6

4.3 Evaluation Metrics

  • Temporal localization is evaluated with average precision (AP) and average recall (AR), while video-level detection uses accuracy (Acc.) and AUC.Source: §4.3 · p. 6

5 RESULTS AND ANALYSIS

The benchmark finds lower temporal-localization performance than on previous datasets, while detection remains difficult even with fine-grained labels. Unimodal methods fare better on matching subsets, and pretraining on AV-Deepfake1M improves transfer to LAV-DF.

5.1 Audio-Visual Temporal Deepfake Localization

  • State-of-the-art temporal-localization methods perform significantly worse on AV-Deepfake1M than on previous datasets.The authors interpret the drop as evidence that existing methods are falling behind rapid advances in content generation.Source: §5.1 · p. 6, Table 6 · p. 7

5.2 Audio-Visual Deepfake Detection

  • Even with frame-level training labels, the best detection methods achieve AUC below 70; max aggregation is optimal among the tested strategies.Fine-grained labels improve performance over video-level labels and zero-shot models, with Face X-Ray and SBI exceptions among the latter groups.Source: §5.2 · p. 7

5.3 Unimodal Deepfake Detection and Localization

  • Visual-only methods perform better on subset V than on the full set, and audio-only methods show the same pattern on subset A.This pattern holds for both temporal localization and detection.Source: §5.3 · p. 7

5.4 Benchmark Comparison

  • Compared with LAV-DF, AV-Deepfake1M yields lower performance for localization and classification, while pretraining on it significantly improves transfer to LAV-DF.For BA-TFD+ and UMMAFormer, AP@0.5 falls from 96.30 to 44.42 and from 98.83 to 51.64, respectively.Source: §5.4 · p. 7

6 CONCLUSION

AV-Deepfake1M is presented as the largest audio-visual dataset for temporal deepfake localization, and its benchmark results are substantially lower than those on previous datasets. The authors identify fake/real video imbalance as a limitation and describe research-use restrictions addressing potential misuse.

6 CONCLUSION

  • AV-Deepfake1M is presented as the largest audio-visual dataset for temporal deepfake localization.Source: §6 · p. 7
  • The benchmark reports a significant performance drop compared with previous datasets, supporting AV-Deepfake1M's role as an important asset for future deepfake localization research.Source: §6 · pp. 7–8
  • The dataset has an imbalance in the numbers of fake and real videos, and its use is restricted to research under an end-user license agreement.The authors also acknowledge potential misuse, including creating deepfakes or other forms of exploitation.Source: §6 · p. 8

Supplementary Material

Figure 6 compares old and new transcript words for AV-Deepfake1M and LAV-DF, whose manipulation strategies are LLM-driven and rule-based, respectively.

Supplementary Material

  • Figure 6 pairs old and new transcript words for AV-Deepfake1M in panels (a–b) and LAV-DF in panels (c–d).The new words follow LLM-driven manipulation in AV-Deepfake1M and rule-based manipulation in LAV-DF.Source: Figure 6 · p. 11

A TRANSCRIPT MANIPULATION

The appendix compares transcript modifications across AV-Deepfake1M and LAV-DF and provides further details on a user study assessing human detection of realistic deepfakes.

A TRANSCRIPT MANIPULATION

  • LLM-driven transcript modifications are reported as more natural and diverse than LAV-DF's rule-based modifications.The qualitative comparison uses word clouds of old and new words for both datasets.Source: A TRANSCRIPT MANIPULATION · p. 11

B HUMAN QUALITY ASSESSMENT

B.1 Data

The user study sampled 200 test videos across AV-Deepfake1M and LAV-DF, with equal real/fake counts from each dataset and no repeated identity per participant. Participants classified each video and, when labeling it fake, marked the segment’s start and end.

B.1 Data

  • The 200-video sample included 50 real and 50 fake videos from each dataset, selected to maximize unique identities.For fair comparison with LAV-DF, fake videos contained one audio-visual replacement.Source: §B.1 · p. 11

B.2 Participants

  • Participants each evaluated 20 non-overlapping videos, first classifying them and then marking fake-segment start and end times when applicable.The interface presented the video with audio, task text, and controls for classification and segment selection.Source: §B.2 · p. 11, Figure 7 · p. 11

B.3 Evaluation and Analysis

Human performance was lower on AV-Deepfake1M than on LAV-DF and below state-of-the-art methods. The section also details three audio-visual manipulation setups and how word-level edits determine modality labels.

B.3 Evaluation and Analysis

  • Among 25 participants, binary detection accuracy was 64.84%; localization AP was 01.92 at 0.5 IoU and 15.32 at 0.1 IoU.Figure 8 illustrates qualitative user-study results alongside state-of-the-art temporal localization.Source: §B.3 · p. 11, Figure 8 · p. 12
  • AV-Deepfake1M was more challenging for humans than LAV-DF (Acc. 68.64 vs 84.03; AP@0.1 15.32 vs 36.80), while human performance was below state-of-the-art methods.Table 12 compares the user study with state-of-the-art temporal localization methods.Source: §B.3 · p. 12, Table 12 · p. 12

C AUDIO AND VIDEO GENERATION

  • The generation setups pair fake audio with fake video, fake audio with real video, or real audio with fake video.Deletion and insertion can cause slight visual changes while that modality is treated as real if words were not inserted or deleted in it.Source: C AUDIO AND VIDEO GENERATION · pp. 12–13, Figure 9 · p. 13

D LABEL ACCESS FOR TRAINING

Training label access varies by configuration: models can use frame-level labels directly, while segment- and video-level labels mark a unit fake if it contains any fake frames. Frame-based methods receive mapped labels for fair comparison.

D LABEL ACCESS FOR TRAINING

  • In the frame-level configuration, models train on ground-truth labels for each video frame.Figure 10 provides details on label access during training.Source: D LABEL ACCESS FOR TRAINING · p. 13
  • Segment- and video-level labels mark a unit fake when it contains any fake frames; frame-based Meso4 and MesoInception4 use mapped frame-level labels for fair comparison.MARLIN and MDS use segment-level labels during training.Source: D LABEL ACCESS FOR TRAINING · p. 13
Loading 2311.15308v2…