PaperlayerView exact revision on arXiv

Source-linked AI summary

AnomalyCraft-700K: Component-Level Controllable and Verifiable Synthetic Anomalies for Fine-Grained Video Anomaly Understanding

Yuzhou Long, Haodong Zhang, Yunpeng Yang, Peng Wu, Guansong Pang

arXiv:2609.06978v1cs.CV

TL;DR

Fine-grained video anomaly understanding lacks component-level control and verification, and category-specific hard normal samples remain insufficient. AnomalyCraft-700K constructs component-controlled events, verifies component-to-video alignment, and reports effective supervision across anomaly understanding tasks.

  • Problem

    Synthetic anomaly generation is largely controlled at category or prompt level, lacks component-level video-text verification, and offers too few category-specific hard normals near the normal-anomaly boundary.

    Source: Introduction · p. 2

  • Method

    AnomalyCraft-700K uses semantic components and a progressive three-stage pipeline to generate anomalies, verify and correct component discrepancies, and construct category-specific hard normal counterparts.

    Source: Introduction · p. 2, Component-aware Post-generation… · p. 4

  • Results

    Evaluations across six tasks report effective semantic supervision from anomaly detection to fine-grained anomaly understanding.

    Source: Conclusion · p. 7

  • Takeaways & Limitations

    The dataset provides unified supervision spanning anomaly detection, retrieval and captioning, and fine-grained anomaly reasoning.

    Source: Introduction · p. 2

    Generic automatic annotation tends to focus on salient evidence and overlook subtle but distinctive details.

    Source: Introduction · p. 2

Abstract

from arXiv · show

Progress in video anomaly understanding (VAU) has long been limited by inherent deficiencies of real-world anomaly videos, which are hard to collect and offer little control over their content. Synthetic anomaly approaches partially alleviate data scarcity, yet their generation remains largely controlled at the category or prompt level. They also lack component-level verification of video-text consistency and provide insufficient hard normal samples near the normal-anomaly boundary. To address this, we present AnomalyCraft-700K, a component-controllable synthetic anomaly dataset for fine-grained VAU, containing over 40K videos and over 700K task-level textual annotations. From fine-grained semantic components and a progressive three-stage pipeline, we craft anomaly events that are richly detailed, semantically controlled, and temporally structured, and additionally construct per-category hard normal samples to prompt the model to discriminate based on anomaly semantics rather than surface visual cues. Moreover, using the components as verification units, AnomalyCraft-700K further performs component-wise correction of video-text discrepancies introduced during generation, providing reliable annotations with verified cross-modal alignment for six tasks that progress from anomaly detection, through anomaly retrieval and captioning, to fine-grained anomaly reasoning. Evaluations of widely used methods under both traditional and MLLM-based protocols demonstrate that AnomalyCraft-700K serves as an effective source of supervision, from anomaly detection to fine-grained anomaly understanding.

Introduction

Existing anomaly datasets and synthetic generation leave fine-grained semantic control, component-level verification, and category-specific hard normals insufficiently addressed. AnomalyCraft-700K combines component-based construction, verification, hard normal samples, and supervision across six tasks.

Introduction

  • Real-world benchmarks expanded scene and anomaly coverage, but supervision often compresses participants, interactions, and temporal processes into binary or category labels.Language annotations on fixed-content videos also cannot ensure that relevant participants, objects, interactions, and category-defining cues are systematically covered.Source: Introduction · p. 1
  • Synthetic datasets scale anomaly construction, yet typically control scenes, categories, or storylines rather than semantic constituents, and often leave component discrepancies unchecked.Normal samples generally enrich behavioral diversity rather than establish category-specific boundaries against visually similar anomalies.Source: Introduction · p. 2
  • AnomalyCraft-700K represents events with five core components and category-specific attributes, then progressively relaxes constraints while preserving defining anomaly semantics.Its components cover context, background, subject, object, and directed interaction; the dataset contains 40,100 videos and over 700K task-level multimodal supervision instances.Source: Introduction · p. 2
  • The dataset uses components to verify video-text alignment, adds visually similar but semantically normal category-specific samples, and supports six tasks from detection to fine-grained understanding.The verification process checks and corrects component-level discrepancies item by item, while hard normals encourage discrimination by anomaly semantics rather than surface cues.Source: Introduction · p. 2

Related Work

Prior work spans visual anomaly detection and language-based anomaly understanding, while retrieval and captioning align video events with text. Synthetic video-text work has expanded this area, but its task coverage remains bounded.

Video Anomaly Detection

  • Traditional VAD moved from one-class normality modeling to weakly supervised binary discrimination and CLIP-based adaptation, but remains centered on detection rather than fine-grained event semantics.VLM approaches instead interpret what happens and why it may be abnormal before judging whether an event is anomalous.Source: Video Anomaly Detection · p. 2

Video Anomaly Retrieval and Captioning

  • Retrieval localizes anomalous events from textual queries, while captioning generates descriptions of observed events.Cross-modal methods adapt pretrained models to capture correspondence between long videos and text.Source: Video Anomaly Retrieval and Captioning · pp. 2–3
  • SVTA uses generated anomaly videos paired with video-level descriptions for retrieval, while structured description, fine-grained reasoning, and question answering are beyond its primary scope.The work demonstrates the potential of generation models to scale anomaly-focused video-text alignment.Source: Video Anomaly Retrieval and Captioning · p. 3

Fine-grained Video Anomaly Understanding

Existing fine-grained VAU datasets enrich real-world videos with language supervision, while synthetic work has not provided component-level control and verification together with hard normal samples. AnomalyCraft-700K represents anomaly events through semantic components and their temporal evolution.

Fine-grained Video Anomaly Understanding

  • Real-world VAU datasets add descriptions, grounding, and reasoning tasks, but remain constrained by long-tailed data and uncontrollable video quality; synthetic methods verify at the narrative level and lack hard normals.AnomalyCraft-700K contrasts with this work by performing generation control and video-text verification at the component level and adding category-specific hard normals.Source: Fine-grained Video Anomaly Understanding · p. 3

Component-level Anomaly Representation

  • An anomalous event is modeled as semantic components C and temporal evolution T, with context, background, subject, object, and directed interaction specifying how the anomaly is instantiated.Category-specific pairings, admissible interactions, counts, relative positions, and interaction objects further constrain semantics and expand diversity.Source: Component-level Anomaly Representation · p. 3
  • The temporal representation organizes event progression into four stages, from an initial state through abnormal onset and escalation to the outcome.These stages guide event progression and underpin the progressive generation pipeline.Source: Component-level Anomaly Representation · p. 3

Progressive Component-level Generation

A single prompt containing every component can produce unstable generations, so the pipeline progressively relaxes constraints while preserving defining anomaly semantics. Its three stages balance controllability with event diversity.

Progressive Component-level Generation

  • Directly prompting all fine-grained components at once can cause missing components, incorrect interactions, or semantic drift, motivating progressive constraint relaxation.The pipeline aims to preserve defining anomaly semantics while allowing generation to become less constrained.Source: Progressive Component-level Generation · pp. 3–4
  • Each stage converts structured components into a generator-oriented description, synthesizes a video, and reviews results to refine the next stage.Descriptions place static components before dynamic interactions and append a continuation signal to stabilize semantics throughout the video and reduce end-of-generation drift.Source: Progressive Component-level Generation · pp. 3–4
  • Generation moves from fully specified components in Stage I, to relaxed auxiliary constraints in Stage II, then to core anomaly semantics in Stage III.The stages increasingly let the LLM instantiate unspecified details while maintaining category consistency and event plausibility.Source: Progressive Component-level Generation · p. 4

Component-aware Post-generation Verification

AnomalyCraft-700K verifies generated video content at the component level and corrects discrepancies before deriving downstream annotations. The process includes human review of ambiguous cases and is applied throughout all three construction stages.

Component-aware Post-generation Verification

  • Component-aware verification checks each predefined semantic component against the video and corrects discrepancies by updating values, revising categories, or discarding unclassifiable samples.Event-level screening comes first; MLLM observations also capture content beyond predefined components, and human reviewers decide ambiguous cases.Source: Component-aware Post-generation… · p. 4
  • Video retention rises from 81.56% to 90.05% and then 95.55% as the three construction stages progressively mature.The authors present the increasing rates as evidence of increasingly reliable component realization.Source: Component-aware Post-generation… · p. 4
  • Table 1 compares video anomaly understanding datasets.The supplied caption gives the comparison scope but no dataset-level values.Source: Table 1 · p. 5

Category-specific Hard Normal Construction

The dataset adds category-specific hard normal samples that retain anomaly-like visual cues while remaining semantically normal. Their construction varies semantic factors rather than the event’s overall appearance.

Category-specific Hard Normal Construction

  • Hard normal counterparts preserve each anomaly category’s characteristic visual cues while changing only semantic factors that determine abnormality.The design targets the visual shortcut that can arise when normal and anomalous samples look markedly different.Source: Category-specific Hard Normal… · p. 5
  • Construction alters one or more of five semantic dimensions: authorization, occupational purpose, rule constraint, controlled process, and harmless outcome.For example, staff-supervised bagging remains visually similar to shoplifting but differs in authorization.Source: Category-specific Hard Normal… · p. 5
  • Normal samples also include ordinary daily-activity videos and reclassified samples, and all undergo component-aware verification.The stated aim is to ground the normal–anomaly distinction in event semantics rather than superficial visual patterns.Source: Category-specific Hard Normal… · p. 5

Unified Multi-task Supervision

Six tasks are organized across detection, retrieval and captioning, and fine-grained reasoning. Their annotations derive from a shared verified event representation to maintain semantic consistency.

Unified Multi-task Supervision

  • The six tasks span anomaly detection, anomaly retrieval and captioning, and fine-grained anomaly reasoning, all grounded in a shared verified event representation.The representation provides a common semantic foundation rather than independently annotated task outputs.Source: Unified Multi-task Supervision · p. 5
  • T1 conventional VAD-oriented detection and T3 MLLM-oriented anomaly judgment use the verified anomaly category.These make up the anomaly detection capability dimension.Source: Unified Multi-task Supervision · p. 5
  • T2 organizes verified event components into retrieval texts, while T4 provides structured event captioning.Together they form the retrieval and captioning dimension.Source: Unified Multi-task Supervision · p. 5
  • T5 structured event reasoning and T6 fine-grained question answering use verified event semantics; T6 also tests boundary-sensitive choices across anomalous and hard normal samples.This dimension adds semantic discrimination near the normal–anomaly boundary.Source: Unified Multi-task Supervision · p. 5

Dataset Statistics

AnomalyCraft-700K contains 40,100 synthesized videos across 29 anomaly categories and one normal category. It provides task-level supervision derived from verified event representations.

Dataset Statistics

  • AnomalyCraft-700K contains 40,100 videos across 29 anomaly categories and one normal category, with approximately 3.01 million frames.The benchmark also provides approximately 743,350 language supervision instances across T3–T6; T1 and T2 have corresponding supervision.Source: Dataset Statistics · p. 5
  • Videos are synthesized using Wan2.2 at 1280×704 resolution and 16 FPS, with durations from 4 to 8 seconds.The stated duration range covers short-lived anomalies and more complex events with multiple interactions and evolving outcomes.Source: Dataset Statistics · p. 5
  • All task annotations derive from the same verified semantic representation to maintain consistency across evaluation tasks.Detailed annotation statistics for each task are provided in the Appendix.Source: Dataset Statistics · p. 5

Experiments

Across six tasks, the experiments evaluate conventional models and MLLMs under supervised and zero-shot settings. Results show strong binary anomaly detection, while fine-grained recognition, retrieval, captioning, and reasoning benefit from semantic supervision.

Experimental Setup

  • Samples are grouped by anomaly category, subject-object type, and generation source before 70:5:25 train-validation-test splitting; experiments use task-specific models and MLLMs up to 4B parameters.The split keeps samples from a generation source within one partition; MLLMs are evaluated with supervised fine-tuning and zero-shot protocols.Source: Experimental Setup · p. 5

Evaluation Metrics

  • Evaluation covers six tasks using task-specific metrics, including detection and classification scores, bidirectional Recall@1/5, captioning and slot scores, reasoning ratings, and QA measures.The reasoning judge is independent of the annotation process and evaluated models.Source: Evaluation Metrics · pp. 5–6

Results on Anomaly Detection

  • Conventional VAD methods achieve 92.43%–96.82% AUROC and 96.81%–98.72% AP, but the best 30-way result is 73.94% Top-1 accuracy and 69.81% macro-F1.For MLLM anomaly judgment, fine-tuning raises category accuracy from 27.36% to 82.97% for Qwen2.5-3B and from 28.48% to 87.13% for Qwen3-4B.Source: Table 2 · p. 6, Results on Anomaly Detection · p. 6, Table 3 · p. 6

Results on Anomaly Retrieval and Captioning

Results on Fine-grained Anomaly Reasoning

  • For structured event reasoning, fine-tuning raises Qwen3-4B’s total score from 29.93 to 41.22 and Qwen2.5-3B’s from 16.06 to 37.57.The largest improvements are in classification, key-event matching, informativeness, and factual consistency, rather than fluency.Source: Table 6 · p. 7, Results on Fine-grained Anomaly… · p. 7
  • Fine-grained QA correctness and hard-normal accuracy rise substantially after fine-tuning, with Qwen3 hard-normal accuracy increasing from 0.7455 to 0.9881.Qwen2.5 hard-normal accuracy increases from 0.7362 to 0.9858; the authors report these gains as evidence that hard normals test semantic boundary understanding.Source: Table 7 · p. 7, Results on Fine-grained Anomaly… · p. 7

Conclusion

AnomalyCraft-700K combines component-controllable synthetic anomaly generation, hard normal samples, and component-level annotation verification. Experiments across six tasks support its use as semantic supervision, while cross-dataset generalization remains future work.

Conclusion

  • The benchmark uses component-level anomaly representation and a progressively refined three-stage pipeline to generate fine-grained anomaly videos and visually close but semantically normal samples.Components are also used to verify and correct annotation discrepancies, supporting component-level accuracy and cross-modal consistency.Source: Conclusion · p. 7
  • Experiments across six tasks show that AnomalyCraft-700K provides effective semantic supervision from anomaly detection through fine-grained anomaly understanding.Source: Conclusion · p. 7
  • Future work will test whether pretraining or joint training on AnomalyCraft-700K improves cross-dataset generalization and complements real-world benchmarks across broader VAU tasks.Source: Conclusion · p. 7

Related papers

Loading 2609.06978v1…