PaperlayerView exact revision on arXiv

Source-linked AI summary

Probe-VAD: Ordinal Likelihood Probing for Training-Free Video Anomaly Detection

Jiawei Gu, Qilin Zhao, Tengkuo Guo, Zhiming Zhong, Shuangqing Zhang, Fan Lyu, Fang Zhao, Guo-Sen Xie, Caifeng Shan

arXiv:2609.17211v1cs.CV

TL;DR

Training-free VAD needs a way to translate frozen VLM visual understanding into continuous anomaly rankings without losing fine-grained preferences. Probe-VAD probes ordered severity thresholds using likelihoods, and its reported comparisons show gains across three VAD benchmarks. Isotonic projection makes threshold profiles ordinally consistent but does not change the scalar ranking statistic used for evaluation.

  • Problem

    Existing VLM-based VAD gives comparatively less attention to how visual understanding is converted into continuous anomaly-ranking signals.

    Source: §I · p. 1

  • Method

    Probe-VAD queries a frozen VLM with ordered severity thresholds, retains YES/NO likelihood preferences, and integrates cumulative evidence into continuous scores.

    Source: §I · p. 2

  • Results

    Across UCF-Crime, MSAD, and XD-Violence, direct visual conditioning outperforms matched caption input, likelihood scoring improves over direct numerical generation, and cumulative thresholds improve over flat 10-class likelihood scoring.

    Source: §VI · p. 13

  • Takeaways & Limitations

    Probe-VAD reports superior performance on three mainstream benchmarks, with 59.0% lower end-to-end wall-clock time and 87.3% smaller artifact size than the complete caption-based pipeline.

    Source: §I · p. 2

    Isotonic projection corrects threshold-profile violations but does not change the scalar ranking statistic used for evaluation.

    Source: §V · p. 12

Abstract

from arXiv · show

Video anomaly detection (VAD) aims to localize anomalous events in untrimmed videos. Vision-language models (VLMs) provide rich visual understanding for training-free VAD, but existing approaches impose restrictive interfaces between visual understanding and anomaly scoring. Caption-based pipelines compress visual evidence into text, potentially discarding subtle cues, while direct numerical generation forces the model to express its judgment through a small set of predefined scores. Such interfaces can obscure subtle differences in anomaly severity, causing visually distinct clips to receive similar representations or scores and thereby limiting the resolution of anomaly ranking. We propose \textbf{Probe-VAD}, an ordinal binary-probing framework that directly probes severity preferences from a frozen VLM. Given raw video clips, Probe-VAD queries ten ordered severity thresholds and extracts constrained \textit{YES}/\textit{NO} continuation likelihoods. Their normalized preferences form a cumulative severity profile, from which tail evidence is aggregated into a continuous anomaly score, with isotonic projection enforcing ordinal consistency. Experiments on public VAD benchmarks demonstrate superior performance with low computational cost. Probe-VAD provides a simple interface for translating frozen VLM visual understanding into continuous, rank-sensitive anomaly scores without task-specific training or caption-based compression. Code is available at: https://github.com/yvestine/COVAS-VAD.

I. INTRODUCTION

Training-free VAD still needs a faithful interface for converting frozen VLM understanding into continuous anomaly rankings. Probe-VAD uses ordered severity queries and pre-decoding likelihoods, with reported benchmark and efficiency gains.

I. INTRODUCTION

  • The underexplored problem is how to translate a frozen VLM’s rich visual understanding into continuous anomaly-ranking signals.Existing work emphasizes evidence representation, organization, or reasoning more than the score-extraction interface.Source: §I · pp. 1–2
  • Probe-VAD directly conditions a frozen VLM on video, retains YES/NO likelihood preferences over ordered severity thresholds, and integrates them into continuous anomaly scores.This avoids caption compression and single decoded numerical responses.Source: §I · p. 2
  • 59.0% lower end-to-end wall-clock time and 87.3% smaller artifact size than the complete caption-based pipeline accompany superior performance on three mainstream benchmarks.The reported benchmarks are UCF-Crime, MSAD, and XD-Violence.Source: §I · p. 2

A. Video Anomaly Detection

  • Prior VAD includes reconstruction and prediction,, weakly supervised methods,, and open-vocabulary methods, but these paradigms rely on task-specific learning.Probe-VAD instead converts frozen VLM preferences into continuous rankings without task-specific training.Source: A. Video Anomaly Detection · p. 2

B. Training-Free Video Anomaly Detection

Training-free VAD methods differ in how they describe, organize, and score visual evidence. Probe-VAD focuses on score extraction, combining ordinal severity propositions with likelihood preferences to produce continuous rankings.

B. Training-Free Video Anomaly Detection

  • Caption-based methods use textual descriptions for anomaly reasoning, temporal methods organize evidence hierarchically, and Probe-VAD instead targets the score-extraction interface of frozen VLMs.Related approaches also include direct VLM responses and pseudo-scene retrieval.Source: B. Training-Free Video Anomaly Detection · pp. 2–3

C. Ordinal Modeling and Likelihood-Based Scoring

  • Probe-VAD combines ordered severity thresholds with YES/NO continuation likelihoods, integrating cumulative evidence into continuous rankings without learning an ordinal prediction head.Related work includes ordinal regression, isotonic regression, likelihood scoring, and constrained VLM questions.Source: C. Ordinal Modeling and… · p. 3

A. Problem Formulation

  • The task is to produce continuous frame-level anomaly scores from a frozen VLM without target-domain optimization, parameter updates, prompt learning, or label-based calibration.Ground-truth annotations are used solely for evaluation.Source: A. Problem Formulation · p. 3

B. Framework Overview and Clip Construction

Probe-VAD processes untrimmed videos by constructing clips, directly encoding them with a frozen VLM, extracting ordinal evidence, and reconstructing frame-level predictions. Ordered YES/NO likelihood preferences over severity thresholds form a cumulative profile for continuous anomaly scoring.

B. Framework Overview and Clip Construction

  • Probe-VAD constructs overlapping clips, encodes each with a frozen VideoLLaMA3-7B, extracts cumulative ordinal evidence from constrained YES/NO likelihoods, and reconstructs clip scores as frame-level predictions.The framework's four stages are clip construction, visual encoding, ordinal evidence extraction, and frame-level reconstruction.Source: B. Framework Overview and Clip… · p. 3
  • Each clip uses a temporal window and sampled RGB frames, while clip centers are spaced by Δ frames.The actual number of sampled frames per clip is bounded by N.Source: B. Framework Overview and Clip… · p. 3

C. Cumulative Ordinal Visual Scoring

  • The method treats severity S_i as a conceptual, unobserved anomaly-evidence variable and queries K ordered thresholds, using K = 10 in the described setup.The threshold grid is {0.1, 0.2, . . . , 1.0}.Source: C. Cumulative Ordinal Visual Scoring · p. 3
  • At each threshold, the frozen VLM evaluates a constrained YES/NO query, and normalized YES preferences represent support for severity reaching that threshold.The preferences are described as VLM-induced tail-evidence proxies, not calibrated probabilities.Source: C. Cumulative Ordinal Visual Scoring · pp. 3–4
  • Because higher-threshold propositions are nested, their cumulative profile should preserve ordinal compatibility across increasing severity thresholds.The method treats different severity levels as ordered rather than independent categories.Source: C. Cumulative Ordinal Visual Scoring · p. 4

D. Monotonic Projection and Cumulative Tail Integration

Probe-VAD projects threshold-wise preferences onto a monotonic profile, then integrates the projected tail evidence into a continuous clip score. With uniformly spaced thresholds and equal-weight aggregation, the projection preserves the scalar score.

D. Monotonic Projection and Cumulative Tail Integration

E. Frame-Level Reconstruction and Efficient Inference

Probe-VAD smooths the clip-score sequence before assigning scores to frame intervals. It also reuses each clip's visual representation across threshold queries and batches compatible queries to reduce redundant computation.

E. Frame-Level Reconstruction and Efficient Inference

  • Gaussian smoothing suppresses local fluctuations in the regularly sampled clip-score sequence before frame-level reconstruction.The smoothing width σ is measured in units of the clip-score grid.Source: E. Frame-Level Reconstruction and… · p. 5
  • Each smoothed clip score is assigned to its corresponding Δ-frame interval, and the resulting sequence is cropped to the annotated video length.Source: E. Frame-Level Reconstruction and… · p. 5
  • Probe-VAD encodes each clip once and reuses its visual representation across thresholds; query batching and shared-prefix caching reduce redundant computation without changing scores.Threshold batch size affects computational efficiency and memory usage, not the resulting anomaly scores.Source: E. Frame-Level Reconstruction and… · p. 5

IV. EXPERIMENTS

The experiments evaluate Probe-VAD on three VAD benchmarks using method comparisons, controlled analyses, and robustness studies.

IV. EXPERIMENTS

  • The evaluation covers three VAD benchmarks and compares Probe-VAD with representative methods.Source: §IV · p. 5
  • The experiments include controlled analyses of Probe-VAD's core scoring mechanisms.Source: §IV · p. 5
  • The evaluation also studies robustness under different visual, linguistic, and temporal configurations.Source: §IV · p. 5

A. Experimental Setup

The evaluation uses three public VAD benchmarks, consistent frozen-model settings, and benchmark-specific metrics. Probe-VAD is tested with a fixed temporal sampling and severity-threshold configuration.

A. Experimental Setup

  • Experiments use UCF-Crime (290 test videos, 13 anomaly categories), MSAD (360 test videos), and XD-Violence (800 test videos, six event categories).The MSAD and XD-Violence descriptions cover diverse surveillance settings and, for XD-Violence, movies and online videos.Source: A. Experimental Setup · p. 5
  • Frame-level AUC is the primary metric; frame-level AP is also reported for MSAD and XD-Violence, with ground truth reserved for evaluation.VLM parameters remain frozen, and the same default configuration is used across datasets unless specified otherwise.Source: A. Experimental Setup · p. 5
  • VideoLLaMA3-7B is the default frozen VLM; clips span 10 seconds with at most 10 uniformly sampled frames, and Probe-VAD uses ten thresholds from 0.1 to 1.0.The setup uses likelihood temperature T = 1.0 and Gaussian smoothing with σ = 10; Qwen3-VL-8B-Instruct is also evaluated for backbone portability.Source: A. Experimental Setup · p. 5
  • The prompt figures cover configurations for Direct Generation and 10-class Likelihood, Probe-VAD, and the caption-controlled baseline.The Direct Generation and 10-class Likelihood prompt configuration is shared.Source: Figure 4 · p. 6, Figure 5 · p. 6, Figure 6 · p. 6

B. Comparison with Representative Methods

Probe-VAD achieves the best results among the compared zero-shot and training-free methods across all five directly comparable benchmark metrics. The comparison is system-level because methods differ in supervision, backbones, temporal processing, and preprocessing.

B. Comparison with Representative Methods

  • The comparison is system-level, not a controlled component-wise evaluation, because methods differ in supervision, backbone, temporal processing, and preprocessing.The authors note that the three benchmarks also differ in scene composition, anomaly duration, event complexity, and class imbalance.Source: B. Comparison with Representative… · p. 6
  • Probe-VAD leads the zero-shot and training-free methods on five metrics: 86.27% AUC (UCF-Crime), 92.11% AUC and 75.36% AP (XD-Violence), and 87.55% AUC and 78.43% AP (MSAD).Relative improvements over the strongest comparable baselines are 1.38 points on UCF-Crime, 0.77 AUC and 5.20 AP points on XD-Violence, and 1.65 AUC and 2.03 AP points on MSAD.Source: B. Comparison with Representative… · p. 6
  • Probe-VAD improves across three heterogeneous benchmarks without explicitly modeling their dataset-specific properties.The authors attribute the advantage to changing how visually conditioned evidence is exposed to the anomaly scorer.Source: B. Comparison with Representative… · p. 6
  • The larger XD-Violence AP gain suggests stronger concentration of anomalous frames near high scores, alongside improved global frame ordering.The authors relate this behavior to higher score resolution from likelihood-based inference, which reduces tied groups from discrete numerical generation.Source: B. Comparison with Representative… · p. 6

C. Ablation Studies

Controlled ablations test how visual conditioning, scoring interfaces, ordinal design, and reconstruction choices affect Probe-VAD. Across these experiments, retaining pre-decoding preferences and structuring them as cumulative severity judgments improves anomaly ranking.

Effect of Visual Conditioning

  • 5.57, 0.76, and 0.41 frame-level AUC points on UCF-Crime, MSAD, and XD-Violence favor direct visual conditioning over caption mediation.The aggregate results support the possibility that caption mediation removes ranking-relevant information, but do not isolate which visual factors account for the dataset-dependent differences.Source: Table II · p. 7, C. Ablation Studies · p. 7

Effect of the Scoring Interface

  • 14.23, 8.48, and 5.59 AUC points separate 10-class Likelihood from Direct Generation on UCF-Crime, MSAD, and XD-Violence; Probe-VAD adds 1.84, 0.78, and 0.34 points over 10-class Likelihood.The comparison attributes the larger gain to retaining pre-decoding preferences; the additional gain reflects the cumulative ordinal interface as a whole.Source: C. Ablation Studies · pp. 7–8, Table III · p. 8

Effect of Multi-frame Context and Temporal Order

  • On UCF-Crime, multi-frame input raises AUC from 83.20% to 86.27% over single-frame input, while ordered and shuffled frames differ by only 0.13 AUC points.The experiments suggest that multiple visual states contribute more than precise ordering; the method adds no new temporal encoder.Source: Table IV · p. 8, C. Ablation Studies · p. 8

Backbone, Candidate Semantics, and Prompt Formulation

  • The evaluated backbones achieve comparable AUC across three datasets, while candidate and prompt reformulations produce modest but nonzero changes on MSAD.Qwen3-VL raises MSAD AUC from 87.55% to 87.74%, while VideoLLaMA3 performs better on UCF-Crime and XD-Violence. Default YES/NO reaches 87.55% AUC; prompt-formulation AUC ranges from 86.95% to 87.76%.Source: C. Ablation Studies · pp. 8–9, Table V · p. 9

Severity Resolution and Threshold Placement

  • K = 10 improves AUC from 86.97% to 87.55% and AP from 76.77% to 78.43% versus K = 1; threshold-placement results vary narrowly across tested grids.K = 5 accounts for approximately 76% of the total AUC gain and 81% of the total AP gain from K = 1 to K = 10; across K = 5 grids, AUC ranges 87.28–87.49% and AP 77.76–78.15%.Source: Table VIII · p. 9, C. Ablation Studies · p. 10, Table IX · p. 10

Ordinal Severity Structure

  • Anomalous clips retain higher mean tail evidence across thresholds, but the largest group separation, AUC, and AP occur at different severity levels.The largest mean anomaly–normal difference is 0.3294 at τ = 0.1, while standalone AUC peaks at 87.54% at τ = 0.8 and AP at 78.39% at τ = 0.4; the 0.8–0.9 interval has the strongest standalone ranking signal, with 83.77% AUC and 74.79% AP.Source: C. Ablation Studies · pp. 10–11, Figure 7 · p. 10
  • The 0.4–0.5 interval has ∆A−N = −0.0116 and standalone AUC of 34.36%, so individual intervals are not interchangeable anomaly detectors.PAVA keeps each projected profile non-increasing; the negative group difference compares average interval evidence between anomalous and normal clips, not within-profile monotonicity.Source: C. Ablation Studies · p. 11

Frame Sampling and Temporal Smoothing

  • Uniform, Center-4s, and Center-2s sampling yield 87.55%, 87.60%, and 87.54% AUC, while excessive smoothing lowers AUC to 86.65% at σ = 20.Sampling differences are at most 0.06 percentage points. Moderate smoothing remains relatively stable: unsmoothed AUC is 88.13%, σ = 5 is 88.18%, and default σ = 10 is 87.55%.Source: C. Ablation Studies · p. 11, Table XI · p. 11, Table XII · p. 11

D. Efficiency Analysis

On a fixed 50-video MSAD subset, Probe-VAD is compared end to end with the complete caption-mediated pipeline using the same videos and sampled-frame budget.

End-to-end efficiency

  • 3.14 FPS versus 1.29 FPS gives Probe-VAD 2.44× the end-to-end throughput, while persistent intermediate storage falls from 6.06 MB to 0.77 MB, an 87.3% reduction.The fixed subset contains 2,235 clips and 22,350 sampled frames; throughput counts sampled RGB frames actually processed, and storage excludes raw videos and transient representations unless materialized.Source: D. Efficiency Analysis · p. 11

V. DISCUSSION AND LIMITATIONS

Ordinal consistency diagnostics find frequent but small threshold-order violations, while qualitative profiles illustrate temporal alignment and score-extraction failure cases. Probe-VAD still inherits perception errors from its frozen VLM, including missed brief, small, or occluded evidence.

Ordinal Consistency

  • 4.44 × 10−16 is the maximum observed scalar-score difference before and after PAVA, so projection is structural rather than performance-enhancing.PAVA enforces ordinal consistency without changing the scalar ranking statistic used for evaluation.Source: §V · pp. 11–12

Qualitative Analysis

  • Representative profiles show Probe-VAD and likelihood-based scoring peaking within annotated accident and explosion intervals, unlike less-aligned methods and delayed Direct Generation.Qualitative cases also show caption omissions, equal decoded scores for distinct clips, and errors involving salient benign motion or brief, small, partially occluded evidence.Source: Figure 8 · p. 12, §V · pp. 12–13

Ordinal Consistency

  • 75.33–87.99% of clips have adjacent ordering violations before PAVA, while mean element-wise corrections range from 0.002031 to 0.003827.The small corrections suggest most violations are local perturbations rather than large departures from the expected severity order.Source: §V · p. 12, Table XIV · p. 12

Limitations

  • Probe-VAD inherits frozen-VLM perception errors, which score extraction cannot resolve, especially for crowded, low-resolution, or ambiguous footage.Examples include mistaking salient benign motion for an anomaly and missing brief interactions, small objects, or partially occluded events.Source: §V · p. 13

VI. CONCLUSION

Probe-VAD converts frozen-VLM visual evidence into continuous anomaly rankings through direct visual conditioning and ordinal likelihood probing. Across three benchmarks it outperforms the compared interfaces and achieves strong frame-level AUC, with results also reported for another VLM backbone.

Conclusion

  • Probe-VAD directly conditions a frozen VLM on video and aggregates likelihood preferences over ordered severity propositions into continuous anomaly-ranking scores.This avoids caption compression and retaining only a single decoded numerical response.Source: §VI · p. 13
  • Across UCF-Crime, MSAD, and XD-Violence, direct visual conditioning consistently beats matched caption input, likelihood scoring improves over direct numerical generation, and cumulative thresholds improve over flat 10-class likelihood scoring.The reported frame-level AUC values are 86.27%, 87.55%, and 92.11%, respectively.Source: §VI · p. 13
  • 86.27%, 87.55%, and 92.11% are Probe-VAD's frame-level AUC values on UCF-Crime, MSAD, and XD-Violence, respectively.Source: §VI · p. 13
  • Experiments with Qwen3-VL-8B-Instruct show that the score-extraction interface is not tied to a single frozen VLM backbone.The paper concludes that direct evidence, likelihood preferences, and ordinal severity structure form an effective interface for continuous temporal rankings.Source: §VI · p. 13

Related papers

Loading 2609.17211v1…