PaperlayerView exact revision on arXiv

Source-linked AI summary

AV-FABLE: Audio-Visual Manipulation Forensics in Long-Form Videos

Yuesheng Huang, Hoi Fan Au, Jiaqi Hou, Ruicheng Zhang, Jiarui He, Haocheng Yang, Bin Hu, Kewei Zhang, Weijian Deng, Jun Zhou, Xiu Li

arXiv:2610.33413v1cs.CVcs.MM

TL;DR

Sparse edits in long-form videos require identifying both when a manipulation occurs and which modality it affects. AV-FABLE provides separate audio and visual timelines, while NFSR reads frozen AV-LLM features with separate temporal decoders. NFSR achieves the highest AV-mAP in-domain and across all three OOD settings.

  • Problem

    Video-level authenticity labels do not identify where sparse manipulations occur or which modality is affected, and existing benchmarks do not cover open-scene long-form audio-visual localization as a whole.

    Source: §1 · p. 1

  • Method

    AV-FABLE pairs pristine and manipulated long-form videos with separate modality timelines, and NFSR reads frozen AV-LLM features through separate audio and visual temporal decoders.

    Source: §1 · p. 2

  • Results

    NFSR achieves the highest AV-mAP in-domain and across all three OOD settings, including transfer to a source-disjoint subset of external TASLE.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Modality-separated temporal readouts are effective for long-form audio-visual forensics, and native-track evaluation exposes failures that joint-timeline evaluation can miss.

    Source: §1 · p. 1, §3.1 · p. 3, §6 · p. 9

    The authors identify broader real-world editing processes, more precise perceptual boundaries, media-corruption robustness, and evidence-grounded explanations as future work.

    Source: §6 · p. 9

Abstract

from arXiv · show

Localized AI manipulations in long-form videos often affect brief events within otherwise authentic content, making video-level classification insufficient for forensic analysis. Existing audio-visual deepfake benchmarks largely focus on short talking-head clips, while long-form temporal forensics remains predominantly visual. We introduce AV-FABLE, a dual-timeline benchmark for sparse manipulations in open-scene long-form audio-visual videos. AV-FABLE contains 16,650 pristine--manipulated twin pairs (998 hours) created with 27 editing configurations, with separate audio and visual timelines that enable modality-specific temporal localization and expose errors hidden by joint-timeline evaluation. On AV-FABLE, zero-shot prompting of audio-visual large language models (AV-LLMs) yields near-chance detection and poor temporal localization. In contrast, supervised readouts of their frozen perception features recover strong forensic signals, suggesting manipulation evidence is encoded in these representations but not effectively exposed through language outputs. Motivated by this observation, we propose NFSR(Native Forensics via Separated Readouts), which reads frozen AV-LLM features with separate audio and visual temporal decoders. NFSR achieves the best audio-visual localization performance in-domain and across all three out-of-distribution settings, including transfer to held-out videos from an external long-video forensic benchmark. AV-FABLE, code, and model weights will be publicly released.

1 INTRODUCTION

AV-FABLE addresses sparse manipulations in long-form videos by preserving separate audio and visual timelines. The paper reports that frozen AV-LLM features support strong forensic readouts, and NFSR achieves the highest AV-mAP in-domain and across three OOD settings.

1 INTRODUCTION

  • Audio and visual edits may overlap, occur asynchronously, or affect only one modality, so joint-timeline evaluation can reward the right time but wrong stream.The benchmark therefore preserves both temporal boundaries and modality identity.Source: §1 · pp. 1–2
  • AV-FABLE contains 16,650 pristine–manipulated twin pairs spanning 998 hours and 27 editing configurations, with separate timelines for modality-specific localization.Manipulations cover a median of 9.45% of each manipulated video.Source: §1 · pp. 2–3
  • Zero-shot Qwen2.5-Omni and Qwen3-Omni reach 51–52 detection AUROC and at most 0.30 mAP on either track, while frozen-feature probes reach 90.8 audio and 80.8 visual AUROC.The comparison suggests forensic evidence is present in perceptual representations but is not reliably exposed through zero-shot language outputs.Source: §1 · pp. 2–3
  • NFSR achieves 91.30 AV-mAP in-domain versus UMMAFormer’s 84.76 and the highest AV-mAP across all three OOD settings.The OOD settings include transfer to held-out videos from the external TASLE benchmark.Source: §1 · pp. 2–3

2 RELATED WORK

Prior work addresses pieces of temporal manipulation forensics, including speaker-centered audio-visual localization, broader visual settings, and long-form inconsistency grounding. The section positions AV-FABLE as combining open-scene long-form video with separate temporal annotations and native-track evaluation.

2 RELATED WORK

  • LAV-DF and the AV-Deepfake1M series extend audio-visual localization, predominantly in speaker-centered clips (Cai et al., 2022; 2023; 2024; 2025).TASLE and ActivityForensics embed sparse visual manipulations in untrimmed or activity-rich videos.Source: §2 · p. 3
  • AVID studies long-form audio-visual inconsistency grounding, but cross-modal compatibility and each stream’s authenticity are distinct targets (Chen et al., 2026b).Other localization approaches use cross-modal cues, adaptive features, contextual modeling, or modality-decoupled detectors.Source: §2 · p. 3
  • AV-FABLE combines open-scene long-form videos with separate temporal annotations and native-track evaluation, requiring both manipulation boundaries and the affected modality to be identified.This setting complements benchmarks focused on image- or video-level authenticity, interval-level single-modality supervision, or cross-modal inconsistency.Source: §2 · p. 3

3 AV-FABLE

AV-FABLE is a long-form, open-scene audio-visual benchmark designed to preserve modality-specific manipulation boundaries and provide controlled pristine–manipulated comparisons. It pairs event-grounded edits with separate timelines, annotations, and source- and generator-based evaluation splits.

3 AV-FABLE

  • 16,650 pristine–manipulated twin pairs span 998.1 hours and 27 editing configurations; AV-FABLE uniquely combines open-scene long-form videos, separate timelines, pristine twins, and event-grounded explanations.The benchmark includes 10,306 audio and 11,847 visual events.Source: §3 · p. 3

3.1 BENCHMARK DESIGN AND TASK

  • AV-FABLE localizes confidence-scored events separately by modality, while its primary native-track evaluation prevents a visual prediction from matching an audio event at the same time.Joint-mAP is an auxiliary view that matches across tracks, helping distinguish interval localization from track identification.Source: §3.1 · pp. 3–4

3.2 EVENT-DRIVEN CONSTRUCTION AND ANNOTATION

  • SEE-Forge selects natural host-video events for local edits, reinserts generated outputs at recorded coordinates, and pairs each manipulated video with a pristine twin processed through the same export steps.Execution-grounded labels record track, interval, operation, and target category; reviewers inspect edits against the pristine twin and edit instruction.Source: §3.2 · p. 4

3.3 DATASET CHARACTERISTICS AND EVALUATION SPLITS

  • Manipulations are sparse: their union covers a median 9.45% of manipulated-video duration, and 80% of videos last 33–207 seconds.All derivatives of a host video stay in one split; testing includes source transfer, held-out generators, and a held-out video-generation route.Source: §3.3 · p. 5

4 NFSR: NATIVE FORENSICS VIA SEPARATED READOUTS

NFSR reads frozen audio-visual perception features through host-conditioned representations and separate temporal decoders. Its design targets local evidence that depends on video context and may have different boundaries across modalities.

4 NFSR: NATIVE FORENSICS VIA SEPARATED READOUTS

  • NFSR combines frozen perception features with host conditioning and separate audio and visual decoders to localize manipulations on distinct timelines.Its components address evidence below the language interface, context-dependent interpretation, and modality-specific boundaries.Source: §4 · p. 5

4.1 READING FROZEN PERCEPTION FEATURES

  • Intermediate-layer features retain fine perceptual detail while interface features complement them with representations aligned to the language interface; NFSR reads both from frozen Qwen3-Omni encoders.The feature sequences are aligned at 0.25-second positions, and a modality-specific gated projection combines the two feature types.Source: §4.1 · p. 6

4.2 HOST-CONDITIONED FEATURE MODULATION

  • Host-conditioned modulation uses each input video’s channel-wise mean and standard deviation to reweight local features while preserving their absolute scale.The context-dependent scale and shift are shared across positions within a video and computed from the input video during training and inference.Source: §4.2 · p. 6

4.3 MODALITY-SEPARATED TEMPORAL LOCALIZATION

  • Independently parameterized audio and visual decoders predict event confidence and boundary distances from their own conditioned feature sequences.Each decoder uses a temporal pyramid with masked dilated convolutions and modality-specific annotations; proposals are combined across scales and clipped to video bounds.Source: §4.3 · p. 6
  • Video-level detection and four-way attribution use separately thresholded track scores calibrated on pristine source-validation videos, without changing the predicted event intervals.The accompanying in-domain results table organizes comparisons by modality block, output type, and metric.Source: §4.3 · p. 6, Table 2 · p. 7

5 EXPERIMENTS

Experiments test frozen AV-LLM feature readouts for sparse temporal localization in-domain and under distribution shift. NFSR leads in AV-mAP across the three OOD settings, while ablations show its components matter more under shift.

5 EXPERIMENTS

  • Evaluation uses source-disjoint splits, localization and detection metrics, and thresholds calibrated on 256 pristine validation videos at a nominal 5% FAR budget and fixed on ID and OOD data.Architecture and model choices are selected using source-validation mAP without access to OOD test results.Source: §5 · pp. 7–8

5.1 FORENSIC READOUT AND IN-DOMAIN RESULTS

  • NFSR reaches 91.30 AV-mAP versus UMMAFormer’s 84.76 on shared frozen features, with a +10.74 V-mAP gain; it also improves detection AUROC, Event-IoU, and attribution F1.Zero-shot AV-LLMs reach near-chance detection and at most 0.30 mAP, while supervised readouts attain 92.24 A-mAP and 90.36 V-mAP.Source: §5.1 · p. 8
  • Native-track evaluation reveals missing modalities: visual-only ActionFormer can outperform NFSR on the visual track, while joint-timeline scoring can obscure a detector’s absent audio output.NFSR’s visual predictions alone reach 64.04 Joint-mAP despite zero A-mAP.Source: §5.1 · p. 8

5.2 TRANSFER ACROSS OOD SETTINGS

  • NFSR achieves the highest AV-mAP in all three OOD settings and exceeds matched-feature UMMAFormer, mainly through higher V-mAP, though the two are on par on HDTF audio.NFSR also has higher attribution F1 and Event-IoU than UMMAFormer in all three settings; visual-only baselines lead on some individual track metrics.Source: §5.2 · p. 8, Table 3 · p. 8
  • Without recalibration, NFSR’s realized FAR stays within 2.97–4.67% across evaluation sets, but visual coverage under shift remains limited at 26.42–41.21 V-IoU.The OOD comparisons cover source transfer, held-out generators, and held-out TI2V; these settings change domain, modality composition, route, or generator.Source: §5.2 · pp. 8–9

5.3 ABLATION STUDIES

  • Removing any core component lowers OOD-All AV-mAP more than in-domain; intermediate-layer features contribute most, with drops of 6.55 points on ID and 8.72 on OOD-All.Removing interface features reduces OOD-All AV-mAP by 4.07 points and more than doubles OOD FAR; removing host conditioning lowers OOD-All AV-mAP by 4.73 points.Source: §5.3 · p. 9, Table 4 · p. 9
  • For audio-only speech replacements, the separated visual readout fires on 2.9% of edited videos and pristine twins, versus 4.6% of pristine twins and 11.8–15.1% of edited videos for cross-modal attention.The ablations use one-component-at-a-time removals under the same training and model-selection protocol.Source: §5.3 · p. 9

6 CONCLUSION

The paper presents AV-FABLE as a dual-timeline benchmark for sparse audio and visual manipulations in long-form videos, with NFSR performing best in-domain and across three OOD settings. It also describes the benchmark’s construction, release scope, and future work on broader edits, boundary annotations, corruption robustness, and evidence-grounded explanations.

6 CONCLUSION

  • AV-FABLE provides dual timelines for sparse audio and visual manipulations in open-scene long-form videos, and NFSR achieves the highest AV-mAP in-domain and across all three OOD settings.The benchmark preserves temporal boundaries and modality identity, exposing failures that joint-timeline evaluation can miss.Source: §6 · p. 9
  • Future directions include broader real-world editing processes, refined perceptual boundary annotations, improved media-corruption robustness, and more faithful evidence-grounded explanations..Source: §6 · p. 9
  • The planned release includes dataset manifests, split and calibration lists, an evaluator, code, model weights, annotations, metadata, generated assets, and reconstruction utilities, subject to source-data terms.Restricted source media and excerpts will not be redistributed.Source: REPRODUCIBILITY STATEMENT · p. 10, ETHICS STATEMENT · p. 10

A.1 SOURCES, COMPOSITION, AND LABELS

  • The release contains 16,650 twin pairs and 22,153 track-specific events; the manipulated videos include audio-only, visual-only, and both-track cases, with events sometimes asynchronous.The examples span indoor and outdoor settings, activities, objects, food preparation, and sports.Source: §A.1 · p. 19, Figure 3 · p. 19

A.2 EVENT-CONDITIONED CONSTRUCTION

  • SEE-Forge catalogs natural events, plans local edits, routes them to suitable generators, reinserts results while preserving non-target content, and exports matched pristine and manipulated twins.The pipeline draws on event captions or transcripts, acoustic onsets, motion, shot boundaries, and interactions.Source: §A.2 · p. 20

A.3 SPEECH REPLACEMENT

  • The speech-replacement subset contains 518 ActivityNet examples combining speech replacement with visual manipulation, using same-speaker references and generated replacement speech.The median replacement lasts 7.3 seconds.Source: §A.3 · p. 20

A.4 HUMAN REVIEW AND DECISION AGREEMENT

  • Two reviewers assessed all 22,153 events for edit validity, preservation of non-target content, boundary quality, and evidence support; unsupported evidence leaves events eligible for detection and localization, but not explanation.Disagreements were adjudicated by a third reviewer.Source: §A.4 · p. 21

A.5 METADATA-ONLY SHORTCUT PROBES

The metadata-only probes test whether matched-export file properties distinguish pristine from manipulated videos. Their results show little label information in those properties, while leaving decoded-signal traces outside the tested shortcut class.

A.5 METADATA-ONLY SHORTCUT PROBES

  • After matched export, multifield metadata probes remain near chance on ID and OOD data, and the strongest single field reaches only 51.9 ID AUROC.The probes use file properties without decoded frames or waveforms, and keep each twin pair in the same fold.Source: §A.5 · p. 21
  • Generator-specific texture, local resampling, and acoustic synthesis residue remain possible decoded-signal traces; held-out generators test unseen generators on seen routes and cannot separate these cues from legitimate evidence.The controls do not fully exclude pipeline-specific cues.Source: §A.5 · p. 21

A.6 EXTENDED BENCHMARK COMPARISON

  • Table 9 broadens the benchmark comparison across dataset scale, temporal structure, annotation coverage, and evaluation design.The supplied passage identifies the comparison dimensions but does not report table values.Source: §A.6 · p. 22

B.1 BENCHMARK SCOPE AND TEMPORAL SUPERVISION

This section situates AV-FABLE among forensic benchmarks and methods, emphasizing sparse edits in long-form video and modality-specific temporal targets. It also describes construction, evaluation scope, and scoring choices relevant to those targets.

B.1 BENCHMARK SCOPE AND TEMPORAL SUPERVISION

  • Earlier benchmark lines address facial authenticity, partial speech or acoustic-scene manipulation, speaker-centered audio-visual edits, and broader video or multimodal resources, but differ in content scope and temporal supervision.Sparse edits within extended authentic context are studied by visual-forensics benchmarks, while AVID focuses on audio-visual inconsistency understanding.Source: §B.1 · pp. 22–24
  • AV-FABLE assigns executed edits to separate audio and visual timelines, including asynchronous events; a temporally correct prediction on the wrong stream receives no credit.This distinguishes modality-specific manipulation localization from long-form inconsistency grounding, where a mismatch may not identify which stream was altered.Source: §B.1 · p. 24

B.2 GENERATION AND LOCALIZED EDITING

  • AV-FABLE uses executed edit coordinates as temporal ground truth and records observable changes within and around localized edits inserted into authentic hosts.The same editing generator can produce a full synthetic clip or a brief replacement that retains surrounding context.Source: §B.2 · p. 25

B.3 TEMPORAL LOCALIZATION AND FORENSIC REPRESENTATIONS

  • Temporal localization methods range from windowed classifier scores to explicit boundary prediction, while NFSR uses host-video feature statistics to condition local evidence before modality-specific interval prediction.Action boundaries need not coincide with manipulation boundaries, and the pristine twin is not required at inference.Source: §B.3 · p. 25

C.1 SPLITS AND OOD EVALUATION PROTOCOL

  • The evaluation uses source-disjoint splits and three OOD settings, but each setting changes multiple components and therefore does not isolate a single shift factor.Held-out TI2V jointly withholds a route and its generator; source transfer combines domains and modality compositions.Source: §C.1 · pp. 26–27, Table 10 · p. 26

C.2 SCORING AND OPERATING POINTS

  • Native-track mAP matches confidence-ranked intervals within the same video and modality, whereas fixed-threshold Event-IoU measures each event’s best overlap and assigns zero to misses.Unmatched and duplicate proposals count as false positives; source-calibrated thresholds are also used for video detection and attribution.Source: §C.2 · p. 27

C.3 NATIVE-TRACK AND JOINT-TIMELINE EVALUATION

This section distinguishes native-track localization from joint-timeline scoring and explains how merging can conceal modality-specific blind spots. It also outlines the readout and decoding implementation used for comparison.

C.3 NATIVE-TRACK AND JOINT-TIMELINE EVALUATION

  • A visual-only subset reaches 64.04 Joint-mAP and an audio-only subset reaches 57.36, each above every same-modality baseline despite producing no events on the discarded track.Native-track evaluation assigns zero mAP to the absent track, exposing a blind spot that joint-timeline scoring can hide.Source: §C.3 · p. 28, Table 11 · p. 28
  • In 727 of 818 ID both-track videos, overlapping audio and visual events merge into one reference interval, making the co-timed proposal from the second track a duplicate false positive under joint scoring.Without merging, the same predictions reach 91.28 mAP; native-track A-/V-metrics are therefore the primary localization measures.Source: §C.3 · p. 28

C.4 READOUT AND BASELINES

  • NFSR combines frozen intermediate-layer and interface features with host-conditioned evidence and separate temporal decoders; the readout uses fixed Qwen-Omni audio layer 16 and temporal proposals.The section describes implementation and baseline settings, but supplies no additional comparative result here.Source: §C.4 · p. 29

D FORENSIC EVIDENCE IN FROZEN REPRESENTATIONS

  • The section frames frozen-representation probes, encoder replacement, and feature-depth decomposition as analyses of the forensic evidence underlying NFSR.The supplied passage states the analysis scope rather than its findings.Source: D FORENSIC EVIDENCE IN FROZEN… · p. 30

D.1 PROBE DESIGN AND LAYER SELECTION

The analyses test how frozen representations expose forensic evidence, how readouts transfer, and how training data affects localization. They find that useful signals span representation components and that AV-FABLE supervision transfers to an external benchmark.

D.1 PROBE DESIGN AND LAYER SELECTION

  • Audio linear probes reach 90.8 AUROC on ID and 77.3 on OOD, outperforming unsupervised scores; temporal jumps remain near chance.The supervised probe recovers discriminative signals from frozen features that transfer to OOD without encoder updates.Source: §D.1 · pp. 30–31, Table 15 · p. 30
  • Qwen3-Omni visual block 6 peaks at 88.44±0.58 AUROC alone and 89.37±0.74 with interface features, while the final block reaches 80.55±1.16 alone.For audio, cross-validation selects layer 12 in both Qwen-Omni generations, but Qwen3’s 0.5-AUROC difference from layer 16 is within fold variation.Source: §D.1 · p. 31, Figure 6 · p. 31

D.2 ENCODER REPLACEMENT

  • Both Qwen-Omni generations support localization on held-out hosts and editing configurations, with complementary strengths across modality and domain; BEATs+CLIP leads on ID and generator-shifted audio.Qwen3 leads on HDTF audio, while Qwen2.5 leads on visual evaluations.Source: §D.2 · p. 31, Table 16 · p. 32

D.3 COMPLEMENTARY EVIDENCE ACROSS REPRESENTATION STAGES

  • The interface-predicted component and residual are both predictive, but intermediate-layer features outperform the residual alone across listed domains, and concatenation improves probe AUROC on ID, HDTF, and ANet-gen.Residual inputs reduce temporal-decoder OOD-pool mAP from 85.6 to 76.6, despite an invertible linear reparameterization with similar probe scores.Source: §D.3 · p. 32, Table 17 · p. 32

E SPARSE EVIDENCE IN LONG VIDEOS

  • Mean aggregation can dilute a fixed-length event as authentic context grows, whereas unrestricted local triggering can accumulate false alarms; NFSR instead uses temporal proposals and source-validation calibration.These are stylized cases motivating event-scale evidence with video-scale false-alarm evaluation.Source: E SPARSE EVIDENCE IN LONG VIDEOS · p. 33

F.1 SEED STABILITY

  • NFSR’s ID advantage over UMMAFormer is 6.54 AV-mAP points and its OOD localization advantage is 18.95 AV-mAP points, both exceeding observed standard deviations of at most 0.28 and 1.50, respectively.The TASLE-trained ActionFormer has stable detection AUROC but localization that varies substantially across seeds, particularly under distribution shift.Source: §F.1 · p. 33, Table 18 · p. 33

F.2 VALUE OF AV-FABLE SUPERVISION

  • AV-FABLE-trained visual readouts reach 91.0 visual mAP on AV-FABLE ID and 60.3 OOD-pool mAP, and transfer to TASLE at 59.0 mAP without target-dataset training.TASLE-trained models remain stronger on TASLE, and the localization benefit of AV-FABLE supervision depends on the prediction model.Source: §F.2 · pp. 33–34, Table 19 · p. 34

F.3 TRAINING-SET SCALE

The section tests training scale, cross-benchmark transfer, input robustness, cross-track specificity, and language-based verification. Results show data scale and modality separation matter, while transfer and robustness depend on the evaluation setting.

F.3 TRAINING-SET SCALE

  • Increasing training from 50% to the full set raises pooled OOD audio/visual mAP by 4.8/4.0 points, with both tracks improving throughout the sweep.The sweep retains the same epoch schedule, so sample count and optimizer updates vary together; results are single-run.Source: §F.3 · p. 34, Table 20 · p. 34

F.4 EXTERNAL COMPARISON WITH OFFICIAL MSLOC

  • NFSR transfers without TASLE training and leads on both metrics for the pure Kling/Jimeng subset and localization F1 on TVSum, while source-trained MSLoc-PG leads on the full test.The released refinement checkpoint does not improve aggregate TASLE scores in this evaluation.Source: §F.4 · p. 34

F.5 ROBUSTNESS TO INPUT DEGRADATION

  • A 360p input reduces visual mAP from 53.1 to 10.9 without changing audio mAP; audio noise similarly preserves visual localization.Removing one modality also preserves localization on the other track, while gain changes and A/V offsets are less damaging in the tested ranges.Source: §F.5 · pp. 34–35, Table 22 · p. 35

F.6 CROSS-TRACK FALSE ALARMS

  • The separated visual readout fires on 2.9% of both audio-edited videos and pristine twins, while the cross-modal readout fires on 4.6% of pristine twins and 11.8–15.1% of audio-only edits.The authors interpret this contrast as soundtrack sensitivity in cross-modal visual decisions; separate branches preserve visual specificity when labels co-occur during training.Source: §F.6 · p. 35

G LANGUAGE VERIFICATION AND TEMPORAL DECISIONS

  • The language-verification analyses test direct outputs, fixed-candidate verification, and score fusion, using event F1, classification AP, and candidate-set localization AP.They use an earlier training release without speech replacement, so HDTF combines source and manipulation-type shifts.Source: G LANGUAGE VERIFICATION AND TEMPORAL… · pp. 35–36

G.1 ZERO-SHOT AND DIRECT-TRANSFER RESULTS

The section compares direct language outputs, candidate verification, and score fusion for temporal prediction. Language-based judgments can help in specific settings, but do not consistently improve localization or ranking.

G.1 ZERO-SHOT AND DIRECT-TRANSFER RESULTS

  • The BEATs+CLIP readout recovers more events than direct language outputs, but audio transfer is weaker on unseen HDTF speech replacements than on generator-shifted event sounds.In this protocol, transfer varies with both the prediction path and training manipulation coverage.Source: §G.1 · p. 36, Table 23 · p. 36

G.2 FIXED-CANDIDATE VERIFICATION

  • SFT raises mean OOD candidate AP from 39.1 to 53.4, but complete-system mean mAP remains within 73.60–73.62.Improved candidate discrimination does not improve localization when candidate boundaries and coverage are fixed by the temporal readout.Source: §G.2 · p. 36, Table 24 · p. 36

G.3 LANGUAGE RESCORING AND SCORE FUSION

  • Standardized fusion improves pooled OOD audio AP from 42.8 to 51.4 but reduces both ID tracks and OOD visual AP; numerical readout scores outperform language scores on every slice.Equal-weight fusion lowers AP throughout, so language judgments do not provide a consistently better ranking rule.Source: §G.3 · pp. 36–37, Table 25 · p. 37

H EXPLANATION BASELINES

  • Event-conditioned explanations remain challenging: output validity and content correctness diverge, semantic identification is low, and the tuned model abstains on most manipulated inputs.The section evaluates zero-shot and LoRA-tuned Qwen3-Omni explanations conditioned on fixed events.Source: H EXPLANATION BASELINES · p. 37

H.1 TRAINING AND EVALUATION SETUP

The setup conditions a language model on NFSR events and evaluates structured explanations separately from temporal localization. Supervision improves format compliance and some ID semantic scores, but media-sensitive support decisions do not establish correct evidence.

H.1 TRAINING AND EVALUATION SETUP

  • Qwen3-Omni receives a modality, interval, confidence, and audio-visual crop around each conditioning event; generated text leaves NFSR intervals, confidence scores, detection decisions, and modality attribution unchanged.The context window is clipped to the video, both modalities remain available, and LoRA updates the language adapter independently of NFSR temporal readouts.Source: §H.1 · p. 37

H.2 STRUCTURED OUTPUT AND SEMANTIC IDENTIFICATION

  • SFT yields valid structured outputs on 99.22% of ID and 100.00% of OOD inputs, while ID operation/target F1 rises from 6.00/11.04 to 16.38/20.91.On OOD inputs, operation F1 increases slightly while target F1 decreases; training uses 4,000 examples and disjoint training and evaluation hosts.Source: §H.1 · p. 37, §H.2 · p. 38, Table 26 · p. 38
  • SFT raises pristine abstention to 87.50% on ID and 93.75% on OOD, but also abstains on 67.19% and 76.56% of manipulated inputs.With annotated intervals, boundary agreement is 100% by construction while operation/target F1 remains low and abstention frequent.Source: §H.2 · p. 38
  • Removing outer Markdown fences makes every zero-shot response schema-compliant without changing its content, while prompt clarification yields no reliable operation-F1 change.The clarification changes operation F1 by +3.44 on ID (interval [−0.84, 10.67]) and 0.00 on OOD (interval [−4.52, 4.43]).Source: §H.2 · p. 38

H.3 MEDIA DEPENDENCE OF SUPPORT DECISIONS

  • Replacing event crops with event-free windows reduces support by 14.06 percentage points on ID and 15.63 on OOD; masking the queried track eliminates support on both splits.Media sensitivity does not establish verdict or evidence correctness, and operation/target scores remain low.Source: §H.3 · pp. 38–39, Table 27 · p. 38

I QUALITATIVE EXAMPLES

The examples illustrate edits on audio and visual tracks, including co-timed and asynchronous events. NFSR localizes several examples but also misses events and produces a pristine-video false alarm; generated explanations are not verified evidence.

I QUALITATIVE EXAMPLES

  • Figure 7 shows visual object replacement and insertion, audio sound-event and speech replacement, and co-timed audio and visual events in one video.Rows show sampled event-boundary frames and local ground-truth timelines; audio rows also include the manipulated waveform.Source: I QUALITATIVE EXAMPLES · p. 39, Figure 7 · p. 39
  • In Figure 9, NFSR matches both audio events and the second visual event at tIoU 0.753–0.973, but misses the first visual event in a four-event asynchronous video.The four events comprise two on each track.Source: I QUALITATIVE EXAMPLES · p. 39, Figure 9 · p. 41
  • Figure 10 gives three selected ID examples where SFT supports the same high-tIoU NFSR candidates that zero-shot generation rejects, but the displayed evidence fields are unverified model outputs.Candidate tIoU is 0.857–0.991; in aggregate, SFT often abstains on manipulated inputs.Source: I QUALITATIVE EXAMPLES · p. 39, Figure 10 · p. 42
  • Figure 8 illustrates localized visual-only object replacement and audio-only speech replacement, alongside a missed visual insertion and a visual false alarm on pristine video.Predicted intervals use fixed track thresholds.Source: I QUALITATIVE EXAMPLES · p. 39, Figure 8 · p. 40

Related papers

Loading 2610.33413v1…