PaperlayerView exact revision on arXiv

Source-linked AI summary

Interpretable All-Type Audio Deepfake Detection with Audio LLMs via Frequency-Time Reinforcement Learning

Yuankun Xie, Xiaoxuan Guo, Jiayi Zhou, Tao Wang, Jian Liu, Ruibo Fu, Xiaopeng Wang, Haonan Cheng, Long Ye

arXiv:2601.02983v1cs.SDcs.AI

TL;DR

Audio deepfake detection needs all-type detectors that generalize across heterogeneous audio while providing interpretable decisions. The paper constructs Frequency-Time rationales and trains ALLMs with FT-GRPO, achieving state-of-the-art performance while producing interpretable, FT-grounded rationales.

  • Problem

    All-type audio deepfake detection requires models that generalize across speech, environmental sounds, singing voice, and music while supporting interpretable decisions.

  • Method

    The paper builds approximately 340K Frequency-Time chain-of-thought demonstrations and applies FT-GRPO, combining SFT initialization with GRPO under rule-based frequency-time constraints.

  • Results

    FT-GRPO achieves state-of-the-art all-type audio deepfake detection with interpretable rationales, reaching 99.75% accuracy on ASVspoof2019LA and 90.10% average accuracy across all test sets under co-training.

  • Takeaways & Limitations

    The results show that a 3B ALLM trained only on speech data can achieve state-of-the-art detection while producing interpretable rationales, with co-training further improving all-type performance.

  • Takeaways & Limitations

    The study covers a limited set of ALLM architectures, its automatically generated rationales may remain suboptimal, and its four-dataset evaluation does not exhaustively represent acoustic environments, synthesis methods, partial spoofs, or mixed audio types.

Abstract

from arXiv · show

Recent advances in audio large language models (ALLMs) have made high-quality synthetic audio widely accessible, increasing the risk of malicious audio deepfakes across speech, environmental sounds, singing voice, and music. Real-world audio deepfake detection (ADD) therefore requires all-type detectors that generalize across heterogeneous audio and provide interpretable decisions. Given the strong multi-task generalization ability of ALLMs, we first investigate their performance on all-type ADD under both supervised fine-tuning (SFT) and reinforcement fine-tuning (RFT). However, SFT using only binary real/fake labels tends to reduce the model to a black-box classifier, sacrificing interpretability. Meanwhile, vanilla RFT under sparse supervision is prone to reward hacking and can produce hallucinated, ungrounded rationales. To address this, we propose an automatic annotation and polishing pipeline that constructs Frequency-Time structured chain-of-thought (CoT) rationales, producing ~340K cold-start demonstrations. Building on CoT data, we propose Frequency Time-Group Relative Policy Optimization (FT-GRPO), a two-stage training paradigm that cold-starts ALLMs with SFT and then applies GRPO under rule-based frequency-time constraints. Experiments demonstrate that FT-GRPO achieves state-of-the-art performance on all-type ADD while producing interpretable, FT-grounded rationales. The data and code are available online.

1 Introduction

The paper frames all-type audio deepfake detection as a need for cross-type generalization and interpretable decisions, then introduces FT-grounded training for ALLMs. Its proposed FT-GRPO combines a rationale-producing cold start with constrained reinforcement learning and achieves strong reported performance.

  • ALLM-generated deepfakes span speech, environmental sounds, singing voices, and music, creating risks across media, cybersecurity, and political communication.
  • Binary-label SFT can improve accuracy but limits interpretability, motivating models that jointly generalize across audio types and produce decision rationales.
  • An automatic annotation-and-polishing pipeline constructs frequency-time CoT rationales for four public datasets, yielding ∼340K cold-start demonstrations.
  • FT-GRPO combines SFT initialization with GRPO under rule-based frequency-time constraints and uses non-think mismatch samples to improve accuracy and interpretability.
  • The authors establish an ALLM-based benchmark with SFT to measure current performance and cross-type generalization, then investigate reinforcement fine-tuning.
  • 99.75% accuracy is reported on ASVspoof2019LA with a 3B model trained only on speech data, while co-training reaches 90.10% average accuracy across all test sets.

2 Related Work

Prior ADD research has concentrated on speech, while sound, singing voice, and music have received fewer methodological investigations. The paper situates ALLMs as a newer approach that can classify deepfakes effectively but requires richer outputs for interpretability.

  • Speech: Speech deepfake detection has been extensively studied through ASVspoof-driven countermeasures, including AASIST and self-supervised-learning pipelines.
  • Other types: Research on environmental sound, singing voice, and music deepfake detection remains comparatively less methodologically developed and is often centered on datasets.
  • ALLMs for ADD: ALLMs have been applied to audio reasoning, captioning, recognition, generation, and quality assessment, providing broad task capabilities relevant to ADD.
  • ALLMs for ADD: A systematic ALLM ADD study achieved 99.40% ACC on the 19LA benchmark with Qwen-Audio SFT, but binary real/fake output severely limits interpretability.

3 Method

The method constructs polished frequency–time rationales and trains ALLMs with FT-GRPO, combining an SFT cold start with constrained GRPO to support accurate, interpretable all-type audio deepfake detection.

  • 3.1 Data Construction: The pipeline generates polished frequency–time rationales, retaining valid captions for reasoning data and mismatch cases as non-think samples.Raw captions are polished for concision, authenticity relevance, consistency, and format; contradictions are flagged as mismatches.
  • 3.2 FT-GRPO: FT-GRPO trains the ALLM in two stages: SFT on valid polished rationales followed by GRPO on reasoning and non-think data.The SFT stage teaches frequency- and time-domain reasoning and the target output schema before reinforcement optimization.
  • 3.2 FT-GRPO: The GRPO policy outputs a reasoning path and predicted label, while composite rewards encourage correctness, valid formatting, and complete frequency–time reasoning.The format reward checks the required wrappers, while the frequency–time reward checks declared frequency- and time-domain reasoning paths.
  • 3.2 FT-GRPO: The frequency–time reasoning reward assigns 0, 0.5, or 1.0 for missing, partial, or complete reasoning based on rule-based completeness checks.Completeness requires each frequency- or time-domain tag to be followed by at least one complete sentence.
  • 3.2 FT-GRPO: The evaluation uses all-type ADD datasets with original training, development, and evaluation splits.The supplied table caption identifies the split structure, while the dataset-selection description emphasizes controlled evaluation of cross-type generalization.
  • 3.2 FT-GRPO: GRPO samples response groups, computes group-relative advantages from their rewards, and updates the policy toward higher-advantage responses with KL regularization to the SFT reference.The advantage uses within-group mean and standard deviation with a small stability constant.

4 Experiments

The experiments evaluate ALLM-based countermeasures across speech, environmental sound, singing voice, and music using controlled datasets and an all-type protocol. The benchmark includes representative ALLMs, prior methods, and FT-GRPO settings.

  • 4.1 Dataset: The evaluation prioritizes relatively clean datasets and excludes partially spoofed cases to isolate deepfake artifacts from background noise and mixed genuine–spoof segments.
  • 4.1 Dataset: Speech, sound, singing, and music datasets use splits designed to test generalization to unseen spoofing or synthesis methods.Speech uses A01–A06 for training and A07–A19 for evaluation; sound uses G01–G04 for training or validation and G05–G07 for testing.
  • 4.2 Experimental Setup: The study evaluates Qwen2-Audio-Chat-7B, Qwen2.5-Omni-3B, and Qwen2.5-Omni-7B under an all-type training and evaluation protocol.Qwen2.5-Omni-3B is selected for FT-GRPO experiments to balance effectiveness and efficiency.
  • 4.3 Compared Methods: The comparison includes a prior small-model all-type ADD approach and a recent large-model approach for SOTA evaluation.
  • 4.2 Experimental Setup: SFT and GRPO use LoRA with rank r = 64, alpha = 16, and dropout = 0.05, while both the language model and audio encoder remain trainable.Experiments use 8 NVIDIA A100 GPUs and the ms-swift implementation.

5 Results and Analyse

Results show that ALLM performance varies substantially across audio types, while FT-GRPO improves accuracy and rationale quality over SFT-only or direct GRPO training. The strongest all-type result comes from co-training with FT-GRPO.

  • 5.1 ALLM Performance on All-Type ADD: Speech-trained Qwen2.5-Omni-3B reaches 99.04% in-domain accuracy and 82.45% average accuracy, whereas other training types lack consistently high in-domain performance.This indicates that generic all-purpose audio instruction tuning does not reliably solve ADD across every audio type.
  • 5.1 ALLM Performance on All-Type ADD: Direct GRPO improves average accuracy by 0.48% over SFT-only training but lowers ID Speech accuracy to 96.32% and often produces noisy, weakly constrained reasoning.The reward primarily targets prediction correctness and format compliance rather than the reasoning trajectory.
  • 5.2 Ablation study for FT-GRPO: The best ablation strategy uses thinking samples for SFT and mixed data for GRPO, reaching 99.75% ID Speech accuracy and 87.38% average accuracy.Injecting non-think samples during cold-start SFT substantially degrades token-level rationale-format accuracy.
  • 5.3 Compared with SOTA Models: FT-GRPO outperforms prior small- and large-model approaches, achieving 99.75% Speech accuracy and 87.38% average accuracy.The comparison notes that small-model results use a fixed 0.5 decision threshold, whereas ALLM4ADD matches responses directly to ground-truth labels.
  • 5.3 Compared with SOTA Models: Across all audio types, FT-GRPO improves average accuracy over SFT, including +5.00% for Speech-trained, +3.85% for Sound-trained, and +11.47% for Singing-trained models.Music-trained FT-GRPO gains only +1.55%, with drops on Speech and Sound offsetting gains on Singing and Music.
  • 5.3 Compared with SOTA Models: Co-trained FT-GRPO achieves 90.10% average accuracy, a +5.15% gain over SFT, with gains driven especially by Sound and Music.

6 Conclusion

The paper establishes an ALLM benchmark for all-type audio deepfake detection and introduces FT-GRPO to combine interpretable frequency-time reasoning with reinforcement fine-tuning. Experiments report SOTA accuracy with a 3B ALLM and further gains under co-training.

  • 6 Conclusion: The work establishes an all-type ADD benchmark and systematically evaluates representative ALLMs for cross-type performance and interpretable decisions.
  • 6 Conclusion: An automatic annotation-and-cleaning pipeline generates 340K FT chain-of-thought instances for training interpretable rationales.
  • 6 Conclusion: FT-GRPO combines an SFT cold start with GRPO using composite rewards for correctness, format compliance, and complete frequency-time reasoning.Non-think samples are incorporated where ALLMs tend to fail.
  • 6 Conclusion: Experiments show that FT-GRPO achieves SOTA accuracy with a 3B ALLM and improves all-type performance under co-training.

Limitations

The study identifies limitations in model coverage, automatically generated rationale quality, and test-condition diversity. These constraints bound how broadly its findings can be generalized.

  • The experiments cover mainly Qwen-Audio/Omni models, so conclusions may not generalize fully to other architectures.
  • Some automatically generated annotations may remain suboptimal because current ALLMs have limited audio-understanding capabilities.
  • Evaluation uses four public datasets but does not exhaustively cover acoustic environments, synthesis methods, partially spoofed audio, or mixed audio types.

A Details of Data Construction

The data-construction pipeline applies annotation to four datasets and divides each audio type into think and non-think subsets. Non-think rates are lowest for vocal content and higher for environmental sounds and music.

  • The annotation pipeline is applied to the training and development splits of 19LA, ESDD, CtrSVDD, and FakeMusicCaps.
  • Each audio type is partitioned into think and non-think subsets after annotation.
  • Around 8% non-think rates occur for speech and singing voice, versus 14.44% for environmental sounds and 37.47% for music.The non-think rate is used as a proxy for coherent FT-rationale production.
  • The rate gap indicates stronger current ALLM capability for semantic understanding of vocal-centric content than for environmental sounds and music.

B Details of Training Hyperparameters

This section reports the training settings used in the experiments, including hyperparameter studies, module-freezing comparisons, prompts, and the two-stage FT-GRPO pipeline.

  • The experiments first study learning rates and LoRA configurations before comparing freezing and fine-tuning across modules.
  • The reported settings include training prompts and full hyperparameters for the two-stage FT-GRPO pipeline.
  • The hyperparameter details cover cross-type training and co-training.

B.1 Learning Rate and LoRA Configuration

The study finds that larger LoRA capacity generally benefits audio deepfake detection, while learning-rate choice remains critical. A learning rate of 1 × 10−5 is selected as the best-performing moderate setting in the reported study.

  • A larger LoRA rank tends to yield higher accuracy for audio deepfake detection, indicating benefits from greater adapter capacity.
  • Overly large learning rates degrade performance, whereas 1 × 10−5 achieves the best results in the study.
  • The optimization study uses Qwen2.5-Omni-3B with the audio encoder, aligner, and LLM all trainable.

B.2 Freezing vs. Fine-tuning

The appendix compares configurable freezing choices for the audio encoder, aligner, and language model, finding that jointly fine-tuning these components performs best. It also documents the prompts, annotation examples, and FT-GRPO training settings used across audio types.

  • Configuration: The appendix exposes freeze_vit, freeze_aligner, and freeze_llm switches for controlling updates to the encoders, aligner, and language model.LoRA is used with an all-layer trainable setting, while module-level freezing remains configurable.
  • Freezing versus fine-tuning: Fine-tuning the audio encoder substantially outperforms freezing it on both development loss and evaluation accuracy.The comparison uses four training variants under the 19LA train/test protocol.
  • Freezing versus fine-tuning: Enabling the aligner further improves performance compared with keeping it frozen.The best results occur when the audio encoder, aligner, and language model are all fine-tuned.
  • Prompting and annotation: FT-guided prompts steer experiments toward frequency- and time-domain reasoning, with a simplified prompt used for binary real/fake SFT.FT-GRPO retains the construction-stage system prompt while changing the user prompt for verification.
  • Training settings: FT-GRPO uses separate SFT and GRPO stages, with two GRPO epochs for each audio type and fewer epochs in the larger co-training setting.Step 1 uses three epochs for speech and music, two for sound and singing, while co-training uses two cold-start and one GRPO epoch.
  • Prompting and annotation: The appendix provides annotation examples for speech, sound, singing voice, and music, alongside statistics for think and non-think samples.These examples illustrate the frequency-time rationale format used in data construction.
Loading 2601.02983v1…