PaperlayerView exact revision on arXiv

Source-linked AI summary

Controllable Multi-label Video Safety Detection via Adaptive Tversky Policy Optimization

Guangyu Yang, Jingbiao Mei, Mingsheng Sun, Jinghong Chen, Yingtong Bu, Pengda Qin, Da Chen, Bill Byrne

arXiv:2610.02019v1cs.CLcs.CV

TL;DR

Existing video-safety systems often reduce unsafe content to binary labels and use static objectives despite varying precision–recall needs across policies and categories. ATPO applies adaptive Tversky rewards in reinforcement learning for multi-label detection; reported results include a higher Jaccard Index and reliable operating-point control. The authors note that target ratios require application-specific tuning and validation.

  • Problem

    Video-safety methods often collapse unsafe categories into a binary label and use static objectives, despite co-occurring labels and differing policy precision–recall requirements.

    Source: §1 · pp. 1–2

  • Method

    ATPO is a GRPO-based reinforcement-learning framework that dynamically adjusts false-positive and false-negative penalties through Adaptive Tversky Reward.

    Source: §1 · p. 2

  • Results

    40.66 to 75.44: ATPO increases the Jaccard Index, while ATR enables reliable global and category-level precision–recall control.

    Source: §1 · pp. 2–3

  • Takeaways & Limitations

    Adjusting the desired operating point provides a mechanism for aligning model behavior with different moderation policies.

    Source: §5 · p. 10

    Selecting target ratios requires application-specific tuning and validation; different operating points may lead to over-moderation or missed harmful content.

    Source: A Limitations · p. 15

Abstract

from arXiv · show

The rapid growth of video-based social media has increased users' exposure to harmful content, creating a need for reliable automated video safety detection. Although recent Vision-Language Models (VLMs) show strong video understanding capabilities, existing harmful video detection systems face two key limitations: they typically reduce safety detection to binary classification, overlooking the inherently multi-label nature of unsafe videos, and they rely on static training objectives that do not support controllable precision-recall trade-offs, though the desired operating point may vary across moderation pipelines and unsafe categories. To address these gaps, we propose Adaptive Tversky Policy Optimization (ATPO), a reinforcement learning framework for Multi-label Video Safety Detection (Multi-VSD). ATPO introduces the Adaptive Tversky Reward (ATR), which dynamically adjusts false-positive and false-negative penalties during training to enable controllable precision-recall trade-offs. Experiments on SafeWatch-Bench and XD-Violence show that ATPO substantially improves multi-label performance, increasing the Jaccard Index from 40.66 to 75.44 on SafeWatch-Bench-Real. Moreover, ATR enables reliable steering of the precision-recall operating point, supporting deployment scenarios with heterogeneous policy requirements. Code and checkpoints are provided at https://bruceyg.github.io/ATPO-project-page/ .

1 Introduction

Video safety detection must represent overlapping harmful categories and accommodate different precision–recall priorities across moderation policies. ATPO uses adaptive rewards to support multi-label detection and controllable operating points.

1 Introduction

  • Existing binary video-safety systems collapse unsafe categories, while harmful videos can contain overlapping categories such as violence and abuse.This limits the fine-grained, interpretable outcomes sought by human moderators.Source: §1 · pp. 1–2
  • Moderation policies need different precision–recall trade-offs, including category-specific priorities for reducing over-censorship or harmful exposure.The passage contrasts high-recall guardrails for children with more balanced general-purpose moderation and notes that misinformation and violent or extremist content can have different priorities.Source: §1 · p. 2
  • ATPO is a GRPO-based reinforcement-learning framework that adapts false-positive and false-negative penalties through Adaptive Tversky Reward.The paper formulates multi-label harmful-video detection under RL fine-tuning and studies reward formulations for multi-label objectives.Source: §1 · p. 2
  • 40.66 to 75.44: ATPO increases the Jaccard Index, and ATR enables reliable global and category-level precision–recall control.The reported control supports deployment scenarios with different risk sensitivities.Source: §1 · pp. 2–3

2 Related Work

Prior video-safety work spans binary anomaly detection, fine-grained recognition, and VLM-based safety methods, but typically does not model co-occurring harmful categories as multi-label predictions. This paper studies that multi-label moderation setting.

2 Related Work

  • Video anomaly detection has often been binary or single-category, while SafeWatch-Bench frames video safety as multi-category moderation; this work studies co-occurring harmful labels.Ex-VAD assigns one abnormal category per video, whereas Multi-VSD allows multiple harmful categories in the same video.Source: §2 · p. 3
  • Video-understanding RL work uses task-specific rewards, while safety-related efforts include GRPO-based guardrails and anomaly reasoning.This work builds on RL reward design for Multi-VSD, accounting for partial label correctness and asymmetric false-positive and false-negative costs.Source: §2 · p. 3
  • Other video-safety directions address safe generation, response alignment, or input misuse; ATPO instead moderates existing videos with fine-grained multi-label predictions.Source: §2 · p. 3

3 Proposed Method

The method formulates video safety as prediction of a set of unsafe-category labels and trains a VLM with SFT followed by GRPO rewards. ATPO adapts Tversky penalties from observed errors to steer global or category-specific precision–recall trade-offs.

3.1 Multi-label Video Safety Detection Formulation

  • Multi-VSD predicts a set of predefined unsafe categories for each video, with predicted and ground-truth label sets forming the basis for RL rewards and evaluation.The categories include violence, misinformation, and extremism.Source: §3.1 · p. 4

3.2 GRPO with Static Rewards

  • Training uses SFT warmup followed by GRPO, which scores sampled predicted label sets against ground truth and updates the policy using normalized group rewards.Exact Match gives reward only for an exact set match and provides sparse feedback, motivating partial rewards; Jaccard treats false positives and false negatives symmetrically, while Tversky coefficients control their asymmetric penalties.Source: §3.2 · pp. 4–5

3.3 Adaptive Tversky Policy Optimization (ATPO)

  • ATPO tracks rollout errors with exponential moving averages and adjusts Tversky coefficients by comparing observed and user-targeted false-negative-to-false-positive ratios.The controller updates a logit governing the relative weighting of false positives and false negatives, then derives coefficients for the next training step.Source: Figure 2 · p. 5, §3.3 · p. 5
  • C-ATR maintains separate controllers and coefficients per category, whereas G-ATR aggregates errors and shares one trade-off across categories.C-ATR allows each category to approach its own target operating point within a unified multi-label objective.Source: §3.3 · p. 6

4 Experiments

Experiments evaluate ATPO on video safety benchmarks, compare adaptive and static rewards, and test global and category-level precision–recall steering. The reported results show improved multi-label detection and controllable operating points.

4.1 Experiment Setup

  • SafeWatch-Bench-Real evaluation uses 820 test videos from six unsafe categories; XD-Violence evaluation uses its full 800-video test set.The study uses Qwen2.5-VL-7B and Qwen3-VL-8B backbones, SFT with ground-truth safety labels, and GRPO initialized from one-epoch SFT checkpoints; it reports Jaccard, Micro-F1, Macro-F1, and Binary F1.Source: §4.1 · p. 6

4.2 Multi-Label Video Safety Detection Performance

  • On SafeWatch-Bench-Real, Qwen2.5-VL-7B ATPO raises Jaccard from 40.66 to 75.44/75.15.For Qwen3-VL-8B, Jaccard rises from 48.69 to 75.50/74.63; ATPO-trained models also outperform larger general-purpose and video-reasoning VLMs.Source: Table 1 · p. 7, §4.2 · p. 7
  • Large VLMs remain below Jaccard 61 on SafeWatch-Bench-Real despite Binary F1 above 85, while ATPO models exceed Binary F1 93 on both datasets.General video-reasoning training yields no substantial gain: Video-R1’s Jaccard is 39.99 versus the backbone’s 40.66, while VideoChat-R1 reaches 44.20.Source: §4.2 · pp. 7–8
  • All four ATPO variants outperform SafeWatch-8B across eight dataset-metric combinations, including Jaccard scores of 75.44 versus 70.69 on SafeWatch-Bench-Real.On XD-Violence, ATPO-G-7B scores 88.17 versus SafeWatch-8B’s 79.84; G-ATR and C-ATR have comparable overall performance with minor metric trade-offs.Source: §4.2 · p. 8

4.3 Comparison of Static and Adaptive Reward Optimization

  • Against GRPO-STR, ATPO-G improves Jaccard by 5.35 on Real (70.09 to 75.44) and 5.17 on GenAI (69.12 to 74.29).Both adaptive variants outperform EM and STR across all eight dataset-metric combinations; both also improve all four metrics on both subsets over their shared SFT-1ep initialization.Source: §4.3 · pp. 8–9, Table 2 · p. 8

4.4.1 Global Control via G-ATR

  • As G-ATR’s target ratio rises, achieved P/R increases from 0.77 to 1.16 and ETR from 0.31 to 1.75; STR does not steer reliably.The G-ATR progression is monotonic across the tested sweep, whereas STR’s achieved P/R can move counter to the intended direction as α/β increases.Source: §4.4.1 · p. 9, Table 3 · p. 9

4.4.2 Category-Level Control via C-ATR

  • With C-ATR, C4 precision shifts from 55.28 to 70.41, while C6 recall shifts from 79.07 to 94.57.The asymmetric target ratios steer Misinformation toward higher precision and Extreme toward higher recall, addressing the evaluated categories’ differing policy priorities.Source: §4.4.2 · p. 10, Table 4 · p. 10

5 Conclusion

ATPO enables controllable precision–recall trade-offs during reinforcement-learning training for multi-label harmful video detection, while improving multi-label detection and maintaining strong binary safety performance.

5 Conclusion

  • ATPO is an adaptive reward framework for multi-label harmful video detection that enables precision–recall trade-offs during RL training.It adjusts the desired operating point to align model behavior with different moderation policies.Source: §5 · p. 10
  • Experiments show that ATPO improves multi-label detection while maintaining strong binary safety performance.Source: §5 · p. 10
  • ATPO provides a flexible mechanism for aligning model behavior with different moderation policies by adjusting the desired precision–recall operating point.Source: §5 · p. 10

A Limitations

The discussion identifies limits tied both to benchmark definitions and to deployment choices. ATPO’s training procedure combines SFT initialization with GRPO updates and adaptive reward control.

A Limitations

  • Benchmark category boundaries vary across cultural, legal, and platform contexts, so adapting ATPO to alternative definitions may require dataset-specific annotation and validation.Source: A Limitations · p. 15
  • Selecting target ratios requires application-specific tuning and validation, and different operating points may yield over-moderation or missed harmful content.The authors therefore frame ATPO as an assistive moderation component supporting human decisions, not a fully automated standalone system.Source: A Limitations · p. 15

B Full Algorithm

  • ATPO training starts with SFT, then alternates GRPO rollouts and policy updates with EMA statistics, ratio estimation, and controller updates to adapt Tversky coefficients.Source: B Full Algorithm · p. 16

C.1 Statistics of Datasets

The datasets differ in multi-label prevalence and category overlap, while the appendix details implementation choices, comparison baselines, precision-recall steering, and controller step-size behavior.

C.1 Statistics of Datasets

  • 25.52% of SafeWatch-Bench-Real unsafe videos are multi-label, versus 9.40% in XD-Violence; XD-Violence also has only 2.2% Abuse samples.SafeWatch-Bench-Real shows more multi-label ambiguity, while XD-Violence has notable category imbalance.Source: §C.1 · p. 16
  • Category co-occurrence is frequent in SafeWatch-Bench-Real, including Violence with Illegal and Extreme, while overlap is generally more limited in XD-Violence.Table 5 reports unsafe-category label distributions for 650 SafeWatch-Bench-Real and 500 XD-Violence test videos; multi-label counts need not sum to video totals.Source: §C.1 · p. 16, Table 5 · p. 17

C.2 Implementation Details

  • The prompt supplies task instructions and category definitions; videos are uniformly sampled at 1 fps with variable-length sequences, followed by one-epoch SFT and three-epoch RL.The shared controller settings include step size η = 0.01, EMA smoothing factor λ = 0.95, and stability constant ε = 10^-8.Source: §C.2 · p. 17

C.3 Prior Work

  • SafeWatch-8B serves as a specialized multi-label guardrail baseline; comparisons use 1-fps inputs and the same inference prompt across models.Other evaluated approaches include GuardReasoner-VL, VAD-R1, Video-R1, and VideoChat-R1.Source: §C.3 · pp. 18–19

D Precision-Recall Curves

E Effects of Step Size

  • Across four step sizes, ETR moves toward the target ratio of 1; larger η converges faster but oscillates more, while smaller η is slower and smoother.η controls how strongly current log-ratio error affects the controller update.Source: E Effects of Step Size · pp. 19–20, Figure 6 · p. 20

F Comparison of Training Objectives on Qwen3-VL-8B

On Qwen3-VL-8B, both ATPO variants outperform static-reward baselines and their shared initialization across four metrics. Their comparison with extended SFT shows metric-specific trade-offs.

F Comparison of Training Objectives on Qwen3-VL-8B

G Distribution-Shift Evaluation

On VHD11K-Video-Real without target-set fine-tuning, ATPO-C achieves the highest scores across all four metrics, including Jaccard gains over SFT-1ep and GRPO-STR.

G Distribution-Shift Evaluation

  • The evaluation uses 500 real-world VHD11K videos without target-dataset fine-tuning and excludes generated videos because lower generation quality can make unsafe content ambiguous.Source: G Distribution-Shift Evaluation · p. 21
  • Jaccard on VHD11K-Video-Real is 4.86 points higher for ATPO-C than SFT-1ep and 9.47 points higher than GRPO-STR.ATPO-C has the highest scores across all four metrics in this cross-dataset evaluation.Source: G Distribution-Shift Evaluation · p. 21

H Evaluation on Multi-Label Examples

On SafeWatch-Bench-Real videos with at least two harmful labels, ATPO-G-7B outperforms the compared models across all four metrics, with larger gains over GRPO (STR) in Macro F1 than Jaccard.

H Evaluation on Multi-Label Examples

  • ATPO-G-7B improves Jaccard and Macro F1 over GRPO (STR) by 3.01 and 14.18 points, respectively, and scores highest across all four metrics.The evaluation is restricted to videos with at least two ground-truth harmful labels.Source: H Evaluation on Multi-Label Examples · p. 21

I Qualitative Analysis

In three examples, ATPO-trained models identify harmful categories or interpretations missed by GRPO (STR), whose predictions focus on a single salient aspect or misread the scene.

I Qualitative Analysis

  • In a shop robbery, GRPO (STR) predicts Illegal but misses the gun-related violent threat; ATPO (G-ATR) predicts both categories.Figure 7 presents the qualitative comparison between GRPO with STR and ATPO with Adaptive Tversky Rewards.Source: I Qualitative Analysis · pp. 21, 23, Figure 7 · p. 22
  • For dog abuse in an elevator, GRPO (STR) predicts Violence instead of Abuse, while ATPO (G-ATR) correctly identifies animal abuse.Source: I Qualitative Analysis · p. 23
  • For knife training labeled Violence and Extreme, GRPO interprets the scene as routine training, while ATPO (C-ATR) acknowledges its potentially harmful nature.The passage says the C-ATR reasoning recognizes the potential to glorify violence.Source: I Qualitative Analysis · p. 23

J Complete Category-Level Control Results

Category-level targets steer precision and recall in the intended directions while broadly preserving balance in untargeted categories; ATPO-G also reaches operating points not recovered by the reported threshold settings.

J Complete Category-Level Control Results

  • C4 precision rises from 55.28 to 70.41 and C6 recall from 79.07 to 94.57, while C6 precision falls from 80.31 to 31.77.The remaining categories’ precision-recall ratios stay broadly balanced when their targets are held fixed.Source: J Complete Category-Level Control… · p. 23

K Comparison with Post-hoc Threshold Tuning

  • At nearly matched precision, ATPO-G (r*=5) reaches 67.62 recall versus 32.48 for SFT-1ep at τ=0.5 (78.56 versus 78.82 precision).ATPO-G (r*=0.5) also achieves 76.18 precision and 80.71 recall, exceeding all five reported GRPO (STR) threshold points in both metrics.Source: K Comparison with Post-hoc Threshold… · p. 24, Table 10 · p. 24

L Cross-Taxonomy Transfer and Low-Resource Adaptation

This section describes ATPO’s transfer from SafeWatch-Bench to XD-Violence and adaptation with limited target data, while noting deployment-validation and human-oversight considerations. It also records that the research involved neither human subjects nor crowdsourcing and that LLMs were not used to develop the core method.

L Cross-Taxonomy Transfer and Low-Resource Adaptation

  • 84.58 Jaccard with 10% target data was within 3.59 points of the full-data reference; zero-shot transfer raised Jaccard from 76.54 to 80.51 on XD-Violence.Zero-shot transfer and 10%-data adaptation improved all four metrics; the results support transfer across different category definitions.Source: L Cross-Taxonomy Transfer and… · pp. 24–25, Table 11 · p. 25

M Selecting Error-Ratio Targets for Deployment

  • Because achieved precision and recall depend on the model and deployment distribution, practitioners should train candidate targets and select checkpoints against held-out deployment-representative data.Check actual precision and recall constraints rather than relying on error-ratio matching; targets guide training and are not inference-time adjustments.Source: M Selecting Error-Ratio Targets for… · p. 25

N Societal Impacts

  • ATPO is intended to assist moderation across unsafe-content categories and support differing precision-recall needs, but its outputs should not replace human judgment in high-stakes decisions.The paper frames outputs as supporting signals within broader moderation pipelines, not final decisions.Source: N Societal Impacts · pp. 25–26

2. Limitations

  • Multiple training runs would substantially increase computational cost; the authors report consistency across models and datasets and describe experimental details for reproducibility.They state that code and scripts will be provided upon publication; experiments use existing public benchmarks, with no new datasets or high-risk models released.Source: §2 · pp. 27, 28, 30
  • The authors state that the research involved neither crowdsourcing nor human subjects, making the IRB question not applicable.The checklist explains that [N/A] means the paper does not involve crowdsourcing or research with human subjects.Source: §2 · p. 31
  • The authors report that LLMs were not used as an important, original, or non-standard component of core method development.They mark the LLM-usage declaration not applicable.Source: §2 · p. 32

Related papers

Loading 2610.02019v1…