PaperlayerView exact revision on arXiv

Source-linked AI summary

AV-SafetyBench: A Safety Benchmark for Text-to-Audio-Video Generation

Suah Choi, Tae-Young Lee, Gyeong-Moon Park

arXiv:2609.06991v1cs.CVcs.AI

TL;DR

Existing safety benchmarks largely assess generated video or audio in isolation, while T2AV unsafe meaning can be carried by audio or emerge from interpreting both tracks together. AV-SafetyBench evaluates five open-source models using a multi-view protocol, finding that video-only evaluation misses substantial shares of assigned-risk unsafe outputs for four models.

  • Problem

    Existing safety benchmarks largely assess generated video or audio in isolation, while T2AV unsafe meaning can be carried by audio or emerge from interpreting both tracks together.

    Source: §1 · p. 1

  • Method

    AV-SafetyBench pairs 5,200 manually reviewed prompts across a four-axis, 13-category taxonomy with Full-AV, Video-Only, and Audio-Only judgments to assign risk sources.

    Source: §1 · pp. 1–2

  • Results

    For four of five models, video-only evaluation misses 41.6–48.3% of assigned-risk unsafe outputs; AV-Joint constitutes 87.5% in Cross-Modal Harm Emergence.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The findings indicate that T2AV safety evaluation requires assessing the visual and audio tracks together rather than in isolation.

    Source: §5 · p. 7

    Macro-averaged FUR is not a direct measure of safeguard strength because it also depends on whether a model can produce the content a category requires.

    Source: §4.1 · p. 6

Abstract

from arXiv · show

Recent text-to-audio-video (T2AV) models jointly generate video, speech, sound effects, and ambience from a single text prompt. This capability poses new challenges for safety evaluation, as unsafe content may be conveyed through the audio track or arise only when the visual and audio tracks are interpreted jointly. Existing safety benchmarks largely focus on either generated video or generated audio in isolation and are therefore not designed to capture these risks. To close this gap, we introduce AV-SafetyBench, the first safety benchmark developed specifically for T2AV generation. AV-SafetyBench comprises a four-axis, 13-category taxonomy and 5,200 manually reviewed prompts that specify visual scenes, speech, and non-speech audio. Our evaluation protocol assesses each output under three views: Full-AV, Video-Only, and Audio-Only. It then uses the Video-Only and Audio-Only judgments to assign Full-AV unsafe outputs to one of four risk sources: Video-Only, Audio-Only, AV-Both, or AV-Joint. We evaluate five open-source T2AV models and validate the automated Full-AV judgments against human annotations. Across the five models, Full-AV Unsafe Rates range from 25.1% to 49.4%. Beyond these aggregate rates, risk-source analysis reveals that, for four of the five models, Audio-Only and AV-Joint cases-unsafe outputs missed by video-only evaluation-account for 41.6-48.3% of Full-AV unsafe outputs for which a risk source could be assigned. In the Cross-Modal Harm Emergence category, AV-Joint accounts for 87.5% of unsafe outputs withan assigned risk source. Together, these findings demonstrate the value of AV-SafetyBench for evaluating T2AV safety across the visual and audio modalities and their interaction.

1 Introduction

AV-SafetyBench addresses safety risks in text-to-audio-video generation that can arise in audio or only through audiovisual interaction. Its evaluation finds substantial unsafe output rates and shows that single-modality assessment misses many unsafe cases.

Motivation

  • Existing video- and audio-safety benchmarks evaluate one modality in isolation, missing risks conveyed through generated audio or arising only from audiovisual interaction.T2VSafetyBench evaluates generated videos without their audio, whereas TTA-Bench evaluates audio in isolation (Miao et al. 2024; Wang et al. 2026).Source: §1 · p. 1

Benchmark

  • AV-SafetyBench introduces a four-axis, 13-category taxonomy and 5,200 manually reviewed prompts specifying visual scenes, speech, and non-speech audio.Cross-Modal Harm Emergence targets unsafe interpretations that arise only when the visual and audio tracks are considered together.Source: §1 · pp. 1–2

Evaluation protocol

  • The protocol assesses clips under Full-AV, Video-Only, and Audio-Only views, then assigns Full-AV unsafe outputs to one of four risk sources; it evaluates five open-source models with human validation.The evaluation covers 3,900 generated clips and validates the automated Full-AV judgments against human annotations.Source: §1 · p. 2

Results

  • Full-AV Unsafe Rates on AV-SafetyBench range from 25.1% to 49.4%; transferred T2VSafetyBench prompts yield rates up to 24.9 points lower under the same judge.The evaluation covers five open-source T2AV models.Source: §1 · p. 2, Figure 1 · p. 2
  • Audio-Only and AV-Joint cases account for 41.6–48.3% of assigned-risk Full-AV unsafe outputs for four of five models, while AV-Joint accounts for 87.5% in Cross-Modal Harm Emergence.These cases escape video-only evaluation; in Cross-Modal Harm Emergence, each isolated view is judged safe.Source: §1 · p. 2

2 Related Work

T2AV generation and evaluation have advanced, but existing safety work largely assesses generated media by modality rather than evaluating unsafe meaning in complete T2AV outputs and its source. AV-SafetyBench addresses this gap with full audiovisual assessment and isolated-view judgments.

2 Related Work

  • T2AV models evolved from separately pretrained audio and video systems to native joint or coupled architectures, while evaluation broadened beyond alignment and synchronization to quality, consistency, and controllability; safety is not a primary focus.Examples of the architectural transition include Hayakawa et al. 2025; Liu et al. 2026a,b; Low, Wang, and Katyal 2025; and HaCohen et al. 2026.Source: §2 · p. 2
  • Prior safety benchmarks largely assess generated images, video, or audio separately (Miao et al. 2024; Dai et al. 2024; Wang et al. 2026; Kang, Fang, and Li 2026), while Omni-SafetyBench evaluates multimodal inputs, not T2AV-generated media (Pan et al. 2025).The cited work does not establish whether unsafe meaning in generated T2AV is conveyed visually, aurally, or through interaction.Source: §2 · p. 3
  • AV-SafetyBench assesses complete audiovisual outputs and uses isolated-view judgments to identify whether unsafe meaning comes from video, audio, or their interaction; its taxonomy spans 13 categories across four axes.The benchmark is designed to evaluate unsafe meaning in natively generated T2AV outputs, which prior work did not assess.Source: §2 · p. 3, Figure 2 · p. 3

3 AV-SafetyBench

AV-SafetyBench defines a four-axis, 13-category framework and a manually reviewed set of 5,200 audiovisual prompts, evaluated through Full-AV, Video-Only, and Audio-Only views. Its protocol uses the isolated-view judgments to identify the source of Full-AV unsafe outputs.

3.1 Risk Taxonomy

  • The 13-category taxonomy spans Depicted Content, Identity and Rights, Social and Deceptive, and T2AV Compositional Risks.It was derived from major model-provider policies and prior safety-evaluation work (OpenAI 2025b; Google 2024; Anthropic 2025; Meta 2025; Schramowski et al. 2023; Wang and Yang 2024; Miao et al. 2024; Wang et al. 2026).Source: §3.1 · pp. 3–4
  • The taxonomy covers directly depicted harm, identity and rights misuse, abuse and deception, and risks emerging through time, audiovisual interaction, or false attribution.Cross-Modal Harm Emergence covers unsafe meaning that arises only from the relation between visual and audio cues; Identity-Claim Attribution concerns misleading content presented as originating from a recognizable person or organization.Source: §3.1 · pp. 4–5

3.2 Data Construction

  • The dataset contains 5,200 accepted prompts, built from 176 manually written seeds and expanded to 400 prompts per category after review.Seeds were pilot-tested for realized visual and audio cues, and expanded candidates were checked for category fit, audiovisual completeness, clarity, and duplication; prompts specify a visual scene with speech, non-speech audio, or both.Source: §3.2 · p. 5

3.3 Automated Evaluation Protocol

  • Gemini 3.5 Flash judges each output under Full-AV, Video-Only, and Audio-Only views, with identical prompts and category rubrics but view-specific inputs.Full-AV receives video, audio, and an ASR transcript; Video-Only receives the visual track; Audio-Only receives audio and its transcript. Full-AV Unsafe Rate is the macro-average of category-level unsafe proportions across all 13 categories.Source: §3.3 · p. 5
  • For Full-AV unsafe outputs, risk-source assignment distinguishes Video-Only, Audio-Only, AV-Both, and AV-Joint using isolated-view judgments.AV-Joint applies when both isolated views are judged safe; a source is assigned only when both isolated views return parseable safe or unsafe labels.Source: §3.3 · p. 5

4 Experiments

Across five open-source T2AV models, automated Full-AV unsafe rates varied substantially, and risk-source analysis showed that video-only evaluation misses many audio-related unsafe outputs. The results also vary by risk category and reflect whether models realize the content needed to express a risk.

4 Experiments

  • The evaluation covered 780 stratified prompts across five open-source models and 3,900 generated clips, assessed with the automated three-view protocol.Thirty annotators reviewed every clip under Full-AV, with at least three reviews per clip and a majority-vote label; resampling indicated that the subset matched the full set on reported metrics.Source: §4 · p. 5

4.1 Main Results

  • For depicted-content risks, daVinci had the highest FURs in three categories, while Explicit Sexual Content and Graphic Injury rankings reversed between Ovi-1.1 and LTX-2.3.Physical Harm FUR ranged from 38.3% to 51.7%; Likeness Misuse ranged from 18.3% to 28.3%, while IP Misuse reached 88.3% for Ovi-1.1 and 86.7% for LTX-2.3.Source: §4.1 · pp. 5–6
  • Automated macro-averaged Full-AV Unsafe Rates ranged from 25.1% to 49.4%, with mean automated–human Cohen’s κ of 0.762 across categories.Human annotations produced an overall FUR of 43.4%, compared with 41.2% for the automated judge; against the human majority label, judge precision was 91.4%, recall 86.8%, and false-positive rate 6.3%.Source: §4.1 · p. 5, Table 1 · p. 6
  • JavisDiT++ recorded FURs of 1.7%, 0.0%, and 0.0% for Targeted Abuse, Fabricated Communication, and Deceptive Solicitation, often failing to produce intelligible speech.Across the other four models, FUR was 60.0–66.7% for Targeted Abuse, 36.7–45.0% for Fabricated Communication, and 61.7–83.3% for Deceptive Solicitation; Illegal Activities ranged from 13.3% to 25.0% across models.Source: §4.1 · p. 6
  • Temporal Harm Emergence FUR was no higher than 6.7%, while Cross-Modal Harm Emergence reached 28.3% for LTX-2.3 and Ovi-1.1 and Identity-Claim Attribution reached 61.7% for LTX-2.3.NAVA had no Temporal Harm Emergence outputs; Cross-Modal Harm Emergence stayed below 10% for NAVA and JavisDiT++, and JavisDiT++ had no Identity-Claim Attribution outputs.Source: §4.1 · p. 6

4.2 Risk-Source Analysis

  • Audio-Only and AV-Joint cases missed by video-only evaluation accounted for 41.6–48.3% of assigned-source unsafe outputs in four models, versus 3.6% for JavisDiT++.JavisDiT++’s low missed share reflects rarely generated intelligible speech, not evidence that video-only evaluation catches its audio-carried risks; human agreement for AV-Joint cases was 91.4%, comparable to 89.6–92.6% for other sources.Source: §4.2 · pp. 6–7
  • Audio-Only dominated audio-related cases for four models, accounting for 63–76%, whereas AV-Both made up 72% of JavisDiT++’s audio-related cases.NAVA had a 76% Audio-Only share and 4% AV-Joint share; LTX-2.3 had 63% Audio-Only and 13% AV-Joint shares.Source: §4.2 · p. 7
  • Risk sources differed by taxonomy axis: Video-Only and AV-Both accounted for 95.4% of categories 1–6, Audio-Only for 94.7% of categories 7–9, and AV-Joint for 87.5% of Cross-Modal Harm Emergence.Illegal Activities was predominantly visual, though 20.8% depended on spoken instructions; Temporal Harm Emergence cases were assigned to one track rather than AV-Joint, and Identity-Claim Attribution was mostly audio-carried.Source: §4.2 · p. 7

5 Conclusion

AV-SafetyBench evaluates T2AV safety across video, audio, and their interaction. Experiments show that video-only evaluation misses unsafe outputs carried by audio or emerging through audiovisual interaction, supporting evaluation of both tracks together.

5 Conclusion

  • AV-SafetyBench is the first T2AV safety benchmark, with 5,200 manually reviewed prompts across 13 categories and a three-view protocol that attributes unsafe outputs to their source.The protocol evaluates Full-AV, Video-Only, and Audio-Only views.Source: §5 · p. 7
  • Across five open-source models, video-only evaluation misses unsafe outputs carried by audio or emerging through audiovisual interaction, with the missed share varying widely across models.Source: §5 · p. 7
  • T2AV safety evaluation requires assessing video and audio together, while a model’s measured safety profile reflects both its safeguards and the risks it can realize.Source: §5 · p. 7

Ethical Considerations

The study used voluntary adult annotators with informed consent and safeguards for potentially disturbing material.

Ethical Considerations

  • Adult annotators gave written informed consent, could skip items or withdraw at any time, and received compensation.They were told in advance that the material might be offensive or disturbing.Source: Ethical Considerations · p. 7

Supplementary Material for AV-SafetyBench … A.2 Prompt Construction

AV-SafetyBench’s supplement specifies full judging rubrics for 13 risk categories and describes how validated seed prompts were expanded and manually reviewed into 5,200 prompts. The supplement also warns that it contains unsafe, offensive, and potentially disturbing prompts and generated audiovisual content.

Supplementary Material for AV-SafetyBench

  • The supplement warns that it contains unsafe, offensive, and potentially disturbing prompts and generated audiovisual content; Table A1 reproduces the full judging rubric given to every judge.Its category index matches Figure 2 of the main paper and the category rows of every result table.Source: Supplementary Material for… · p. 10, §A.1 · p. 10, Table A1 · p. 11

A.1 Category Definitions and Judging Rubrics

  • The rubric distinguishes explicit sexual activity from suggestive sexualization, active physical danger from graphic injury aftermath, and misleading attribution from recognizable copyrighted or brand-asset use.These distinctions separate direct acts or immediate danger from suggestive framing, shocking injury depictions, and recognizable misuse of identity or protected assets.Source: §A.1 · p. 10
  • The rubric defines targeted abuse by identity-based or personal targeting and distinguishes fabricated factual claims, deceptive solicitation, and clearly recognizable illegal conduct from non-harmful or fictional counterparts.The 13-category rubric is reproduced verbatim as provided to judges.Source: §A.1 · p. 10, Table A1 · p. 11
  • Table A1 contains full rubrics for all 13 AV-SafetyBench risk categories, and the category index aligns with Figure 2 and the categories used in result tables.The rubrics were given verbatim and in full to every judge.Source: §A.1 · p. 10, Table A1 · p. 11

A.2 Prompt Construction

  • 176 seed prompts were validated by generating one LTX-2.3 clip per seed with default seed 10, then revising prompts when cues were absent or failed to convey the intended risk.Validation applied only to the seeds; the released prompts are LLM expansions of risk scenarios drawn from prior benchmarks.Source: §A.2 · p. 11
  • 5,200 prompts were produced by expanding validated risk scenarios with an LLM and manually removing duplicates and off-category candidates, yielding 400 prompts per category.The expansion instruction used GPT-5.3 and was applied separately for each category, using its rubric and validated seed prompts.Source: Figure A1 · p. 11, §A.2 · p. 11

A.3 Dataset Statistics … B.3 Judging Prompts

The appendix reports how speech involvement varies across prompt categories and documents dataset safeguards, model and judge settings, human review, and the structure of the judging prompts.

A.3 Dataset Statistics

  • Speech is specified in nearly all prompts for categories 7–9 and 13, where risk is carried by speech, and least often for depicted-content categories.Table A2 reports the number and share of prompts specifying an utterance in each category.Source: §A.3 · p. 12, Table A2 · p. 12

A.4 Scope and Release

  • All prompts specify adults only; the 5,200 prompts and per-clip judgments are released under CC BY 4.0, but generated clips are not released.For depicted-content categories 1–4, source prompts involving minors are excluded before expansion, and expanded candidates are checked during manual review.Source: §A.4 · p. 12

B.1 Model and Judge Configuration

  • Each evaluated model generates one native audio–video clip per prompt under default settings, without prompt rewriting or output selection.Ovi uses its 5-second preset, daVinci uses repository example settings, and Veo 3.1 and Seedance 2.0 are evaluated on their July 2026 public releases.Source: Table A3 · p. 13
  • Gemini 3.5 Flash judges each clip using silent video, a separate audio track, and an accompanying faster-whisper large-v3 transcript.Calls use temperature 0, seed 42, and default thinking; each view receives a safety label, evidence, confidence, and a judgment of whether realized harm fits the intended category. The rubric labels harmful content unsafe when residual doubt concerns only severity or centrality.Source: §B.1 · p. 13

B.2 Human Annotation Protocol

  • Thirty adult annotators reviewed clips with the category rubric, prompt, generated clip, and ASR transcript; at least three independently reviewed each clip, with majority vote determining its label.Annotators could skip, pause, or stop without penalty; participation took about four hours and was compensated KRW 50,000.Source: §B.2 · p. 13, Figure A2 · p. 14

B.3 Judging Prompts

  • The judging prompt uses the same base instruction for every sample, changing only the view condition and supplied evidence; its verdicts and fit judgments use three- and four-way schemas, respectively.Figure A3 specifies verdicts as yes/no/unclear, fit as yes/partial/no/unclear, and confidence as high/medium/low in one JSON object; Figure A4 reproduces the fixed wording blocks.Source: §B.3 · p. 15, Figure A3 · p. 15, Figure A4 · p. 16

B.4 Protocol and Subset Validation · C Additional Results

Protocol validation finds that single-pass judging avoids cross-view monotonicity violations but yields more conservative risk-source and video-only miss estimates. Tinyset validation finds that its observed metrics are representative of the full set under the tested stratified sampling design.

B.4 Protocol and Subset Validation

  • Separate-view judging produces 471 monotonicity violations, while the single-pass protocol produces none.The violations are cases where an isolated view is judged unsafe but the containing Full-AV view is judged safe.Source: §B.4 · p. 17, Table A4 · p. 17
  • Single-pass judging reports a 5.8% AV-Joint share and a 40.5% video-only miss rate, versus 13.0% and 46.7% under separate-view judging.The main paper reports the more conservative single-pass figures.Source: §B.4 · p. 17
  • Any residual single-pass leakage would work against, not in favor of, the claim that video-only evaluation misses many unsafe outputs.Leakage would increase agreement between isolated-view and Full-AV judgments, inflating AV-Both and Video-Only at AV-Joint’s expense.Source: §B.4 · p. 17
  • Every observed tinyset metric, including each risk-source share, falls within the central 95% band of the tested sampling distributions.The validation draws 10,000 subsamples of the same stratified design from the full 5,200 prompts.Source: §B.4 · p. 17, Table A5 · p. 17
  • The finite-population-corrected z-test agrees with the tinyset’s overall unsafe-rate representativeness (z = 0.75, p = .45).The two-sided Monte Carlo p-values are also reported for the sampling distributions.Source: Table A5 · p. 17
  • The tinyset contains 780 prompts, with 60 per category; after exclusions, 775 tinyset samples and 5,172 full-set samples remain valid.The tinyset is a subset of the full set, and its five exclusions are among those excluded overall.Source: Table A5 · p. 17

C.1 Judging Reliability · C.2 Risk-Source Validity · C.3 Capability Confound

These analyses test whether the benchmark’s judgments and risk-source assignments are reliable, and whether low unsafe rates reflect safety or limited generation capability. They find strong human agreement alongside sensitivity to judge choice and speech realization.

C.1 Judging Reliability

  • The automated judge reports 41.3% unsafe versus 43.5% for human annotators, with higher precision than recall and 91.4% of its unsafe calls confirmed by humans.Among outputs the judge called safe, 6.3% were flagged unsafe by annotators.Source: §C.1 · p. 18, Table A6 · p. 18
  • Open-source detectors report pooled unsafe rates of 70.5–89.4% against Gemini’s 41.2%, a larger shift than the 24-point gap between evaluated models.The authors therefore use one judge throughout and caution against reading small unsafe-rate differences as safeguard differences; enabling Nemotron thinking reduces its average from 89.4% to 72.8%.Source: §C.1 · p. 18, Table A7 · p. 18
  • Unclear verdicts remain below 1% in every view, no judging call fails to parse, and risk-source assignment covers 98.9–100% of Full-AV unsafe outputs.Thus the reported rates and risk-source statistics are computed on essentially the full judged set and unsafe set, respectively.Source: §C.1 · p. 19, Table A8 · p. 19
  • Intended-category fit is 96.5% overall and above 94% in all but Explicit Sexual Content (83.4%) and Temporal Harm (72.7%).Temporal Harm’s partial fits include outputs whose harm occurs at once rather than through the prompted staged progression.Source: §C.1 · p. 19, Table A9 · p. 19

C.2 Risk-Source Validity

  • Human annotators confirm 89.8% of outputs missed by video-only evaluation as unsafe, and agreement with automated judgments stays within three points across all four risk sources.The human study validates Full-AV unsafety, not the isolated-view judgments used to assign each source.Source: Table A10 · p. 20, §C.2 · p. 20
  • AV-Joint accounts for 87.5% of assigned unsafe outputs in Cross-Modal Harm Emergence, versus 1.4% outside that category, and is absent in eight of thirteen categories.The category is designed so each track is benign alone; AV-Joint also accounts for 4.4% of assigned unsafe outputs overall, while Identity-Claim Attribution is another concentration point at 13.6%.Source: §C.2 · p. 20, Table A11 · p. 20
  • Across alternative judges, the AV-Joint concentration in Cross-Modal Harm Emergence recurs under five of six judges, while the four speech-capable models retain higher audio-related shares than JavisDiT++.The concentration ranges from 61.2–87.5% inside the category versus less than 1.5% outside; Nemotron without thinking is the exception, with 0.0% inside and 0.06% outside, while thinking restores 61.5%.Source: §C.2 · p. 21, Table A12 · p. 21

C.3 Capability Confound

  • JavisDiT++ has 4.2% mean recall for prompted utterances, versus 76.5–80.5% for the other models, and near-zero unsafe rates in speech-carried categories.It also fails to produce a sentence-length utterance on a third of prompts, suggesting its low unsafe rate reflects incomplete speech realization rather than a measured safety intervention.Source: §C.3 · p. 22, Table A13 · p. 22

C.4 Benchmark and Model Comparison · D Qualitative Examples

The benchmark comparison found gaps of 0.7–24.9 percentage points, concentrated in audio-related risks; qualitative examples show cases where generation failures leave the intended identity or harmful progression unrealized. The section also presents targeted commercial-model case studies and samples spanning all 13 risk categories.

C.4 Benchmark and Model Comparison

  • The comparison used 780 AV-SafetyBench prompts and 699 non-adversarial T2VSafetyBench prompts, judging each suite against its own category definitions.Adversarial character-string prompts were excluded because they target text-to-image tokenizers rather than audiovisual content.Source: §C.4 · p. 22
  • Commercial case studies applied the three-view protocol to targeted prompts, but the sample was too small to rank models and included only outputs that passed provider filtering.The study tested whether audio-carried and cross-modal failure modes seen in open models also appeared behind commercial safety filters.Source: §C.4 · p. 22, Table A14 · p. 23, Table A15 · p. 24
  • The transfer-comparison gap ranged from 0.7 percentage points for JavisDiT++ to 24.9 for LTX-2.3, with native audiovisual prompts shifting LTX-2.3 risks toward Audio-Only and AV-Joint sources.Across the comparison, the gap was concentrated in audio-related risk sources.Source: §C.4 · p. 22, Figure A5 · p. 23

D Qualitative Examples

  • Figures A8–A14 show three samples per risk category, with frames, audio waveforms, and harm-relevant utterance transcripts; Table A16 lists prompts for the main-paper samples.Sexual and graphic content is moderately masked where applicable.Source: D Qualitative Examples · p. 25, Table A16 · p. 26, Figure A14 · p. 33
  • In Likeness Misuse examples, LTX-2.3 produced scenes without recognizable likenesses of the named public figures, and all three outputs were judged Full-AV safe.The outputs alone could not determine whether the missing identities reflected a safeguard or a capability limit.Source: Figure A6 · p. 25
  • In Temporal Harm Emergence examples, LTX-2.3 generated only the benign openings; the harmful staged turns were absent and all three outputs were judged Full-AV safe.The examples do not establish whether the turns were withheld or simply not generated.Source: Figure A7 · p. 26

E Limitations

The reported evidence has limitations in validation, uncertainty, judge dependence, temporal sampling, and its scope: the benchmark does not test deployed moderation systems.

E Limitations

  • Human validation covers Full-AV only, leaving isolated-view risk-source judgments unvalidated; per-cell rates lack confidence intervals and AV-Joint shares depend on the judge.Human annotators confirmed AV-Joint outputs as unsafe within three points of the other three sources. Resampling quantifies subset uncertainty, counts show support for small denominators, and the Cross-Modal Harm Emergence AV-Joint share ranges from 61.2% to 87.5% across five judges.Source: E Limitations · p. 34
  • At one frame per second, the judge may miss staged temporal harms resolved between samples, which may contribute to low Temporal Harm Emergence rates.Four- to six-second clips yield four to six frames; generation failures also bear on the low rates.Source: E Limitations · p. 34
  • The claim that video-only evaluation misses unsafe outputs concerns this benchmark’s three-view protocol, not deployed moderation models, whose failures remain unmeasured.Whether practical video and audio moderation tools miss Audio-Only and AV-Joint outputs is an empirical question the benchmark makes testable but does not answer.Source: E Limitations · p. 34

Related papers

Loading 2609.06991v1…