ReactVAU: A Slow-Fast Decoupled Framework for Streaming Video Anomaly Understanding

Chia-Hui Chen; Shih-Ying Yeh; Fu-En Yang; Min-Hung Chen; Shang-Hong Lai

Source-linked AI summary

ReactVAU: A Slow-Fast Decoupled Framework for Streaming Video Anomaly Understanding

Chia-Hui Chen, Shih-Ying Yeh, Fu-En Yang, Min-Hung Chen, Shang-Hong Lai

arXiv:2609.07941v1Sep 7, 2026cs.CV

TL;DR

Existing VAU methods are not causal, while general streaming models can dilute transient anomalies and waste heavyweight computation on normal footage. ReactVAU combines fast detection, anomaly-aware persistent memory, and event-triggered slow reasoning, achieving competitive performance under strict streaming constraints while reducing redundant heavyweight invocations.

Problem
Offline VAU relies on future frames, while general streaming models dilute transient anomalies during memory compression and apply heavyweight reasoning across normal streams.
Method
ReactVAU uses a Fast Detection Module with SGF, AAPM for persistent anomaly evidence, and a Slow Reasoning Module awakened only for suspicious events.
Results
ReactVAU remains competitive with offline methods under strict causal streaming constraints while substantially reducing redundant heavyweight MLLM invocations.
Takeaways & Limitations
ReactVAU reconciles causal access, low-latency operation, and high-capacity semantic reasoning for streaming VAU.

Abstract

from arXiv · show

In this paper, we propose ReactVAU, a Slow-Fast Decoupled Framework for real-time streaming Video Anomaly Understanding (VAU). Existing VAU methods rely on offline inference with global temporal sampling, which violates causality and prevents deployment in live surveillance streams. Conversely, general streaming video models satisfy causal access but dilute rare transient anomalies during memory compression and often invoke heavyweight MLLMs uniformly over long normal intervals. React VAU addresses this gap with three synergistic components: a lightweight Fast Detection Module based on Spatial Grid Folding (SGF) for continuous anomaly filtering; an Anomaly-Aware Persistent Memory (AAPM) that protects critical visual cues from temporal decay; and a heavyweight Slow Reasoning Module that remains dormant during normal streams and is awakened only by suspicious events for semantic verification and causal description. Extensive experiments on multiple benchmarks demonstrate that ReactVAU operates under strict streaming constraints while simultaneously achieving competitive performance in both anomaly detection and causal reasoning, alongside significantly enhanced computational efficiency by minimizing heavyweight MLLM invocations. Project page is available at https://huiyuiui.github.io/React_VAU/

1 Introduction

ReactVAU targets streaming VAU's tension between causal access, transient anomaly preservation, and the cost of heavyweight reasoning. It combines fast anomaly filtering, persistent anomaly-aware memory, and event-triggered slow reasoning.

Offline VAU methods use future frames for global temporal sampling, breaking causality in live surveillance streams.
General streaming models dilute sparse anomaly features during memory updates and incur high cost by evaluating every frame with heavyweight MLLMs.
ReactVAU separates high-frequency anomaly filtering from low-frequency semantic reasoning through a lightweight Fast Module, AAPM, and event-triggered Slow Module.
Spatial Grid Folding converts short-term temporal anomaly detection into efficient 2D spatial reasoning without heavy temporal modeling.
AAPM uses anomaly-priority mechanisms to protect transient abnormal features during continuous memory compression.
ReactVAU achieves competitive performance against state-of-the-art offline models while improving computational efficiency under causal streaming inference.

2 Related Work

Related work spans offline anomaly detection, multimodal anomaly understanding, and causal streaming frameworks. ReactVAU addresses streaming VAU's combined need for low latency and semantic reasoning within these existing paradigms.

Traditional VAD commonly uses reconstruction-based one-class classification or 3D-feature multiple-instance learning, with many methods requiring offline feature buffering.
Recent online detectors process raw video segments directly to reduce latency and bypass offline feature extraction.
VAU methods use VLMs and MLLMs for open-world detection, caption-based reasoning, anomaly-sensitive attention, and supervised instruction tuning.
Streaming frameworks manage unbounded context through KV-cache compression, persistent memory, or organized visual histories.
Existing streaming VAU frameworks remain challenged by memory compression and the practical demands of real-time semantic anomaly understanding.

3 Method

ReactVAU combines causal streaming access with decoupled fast anomaly filtering, anomaly-aware memory protection, and event-triggered slow reasoning. Its memory hierarchy preserves long-term context while retaining dense evidence around suspicious events for semantic verification.

Framework overview: ReactVAU uses a Fast Detection Module, AAPM, and dormant Slow Reasoning Module within a causal streaming architecture.The Fast Module continuously filters frames, AAPM protects transient evidence, and the Slow Module activates only after a trigger.
Fast Detection Module: Spatial Grid Folding maps four frames from a one-second, 4 FPS window into a 2 × 2 grid for lightweight anomaly scoring.The visual encoder and projector produce binary Yes/No logits, from which Sdet is computed using localized Softmax.
Slow Reasoning Module: The Slow Reasoning Module wakes only when Sdet > τtrigger and retrieves a chronological, hierarchical visual memory sequence.The sequence combines PEMF, short-term memory, the anomaly pool, and dense real-time perception tokens.
Anomaly-Aware Persistent Memory: AAPM adds anomaly awareness at long-term, evidence-retention, and current-perception horizons to prevent transient anomalies from disappearing during compression.It supplements PEMF with anomaly-prioritized merging, an isolated pool, and dense perception after triggering.
Anomaly-Aware Persistent Memory: Anomaly-priority penalties make anomaly-rich node pairs less likely to merge while preserving native PEMF behavior for normal segments.The total penalty combines similarity, temporal distance, merge frequency, and an anomaly-protection term controlled by wa.
Anomaly-Aware Persistent Memory: The isolated anomaly pool stores eight frames at 128 tokens per frame outside the Cmax = 2048 PEMF quota and evicts the lowest-scoring frame when full.This preserves higher-density suspicious-frame features as evidence anchors for slow causal verification.

4 Experiments

ReactVAU is evaluated across streaming anomaly detection, multi-granular anomaly understanding, causal querying, and efficiency. Results show competitive causal performance while preserving anomaly evidence and reducing heavyweight inference overhead.

Video Anomaly Detection Performance: 88.44% AUC on UCF-Crime and 88.50% AP with 95.25% AUC on XD-Violence demonstrate competitive streaming anomaly detection without future information.ReactVAU outperforms existing online methods and remains competitive with offline fine-tuned methods that access future frames.
Video Anomaly Understanding Performance: ReactVAU achieves the best scores across reported long-range Event- and Video-level VAU metrics by preserving anomaly evidence in AAPM.Generic MLLMs degrade at longer temporal granularities, while ReactVAU’s causal memory retains sufficient evidence as context grows.
Efficiency and Scalability: ReactVAU reduces heavyweight 7B-parameter LLM inference queries from 34,670 to 15,955, achieving a 54.0% reduction in computational workload.The evaluation uses the UCF-Crime test set, where anomalies occur in approximately 43% of segments, and a single NVIDIA H100-80GB GPU.
Efficiency and Scalability: 98.3 ms/query is the empirical weighted average streaming latency, compared with 216.1 ms/query for StreamForest’s heavy backbone.ReactVAU operates at 22.1 ms/query during normal states by restricting heavyweight inference to event verification.
Causal Streaming VAU: Causal streaming queries use memory available at each timestamp, supporting clip-level queries at t = 20s and video-level summaries at t = 200s without question-specific global resampling.The continuously updated memory preserves transient localized actions and broader event narratives while processing streams sequentially.
Ablation Study: 86.90% AP is reached when the SGF-based Fast Module and Slow Module are both tuned, while integrating AAPM raises streaming performance to 88.50% AP on XD-Violence.The ablation attributes the improvement to rapid filtering, semantic verification, and protection of transient abnormal features.

5 Conclusion

ReactVAU combines causal streaming operation with lightweight detection, persistent memory, and event-triggered semantic reasoning. Its score-fusion design combines fast spatial sensitivity with slow semantic verification while reducing redundant heavyweight MLLM invocations.

Weighted fusion combines Fast Detection’s spatial sensitivity with Slow Reasoning’s semantic verification, whereas replacing the fused score with the Slow score fails to capture temporal dynamics.
ReactVAU reduces redundant heavyweight MLLM invocations while remaining competitive with offline methods that observe future frames.Evaluations on UCF-Crime, XD-Violence, and HIVAU-70K support this conclusion.
The framework uses Spatial Grid Folding for fast detection, Anomaly-Aware Persistent Memory for transient threat evidence, and event-triggered Slow Reasoning for semantic verification.The Slow Reasoning Module awakens only when semantic verification is needed.