VAU-R1: Advancing Video Anomaly Understanding via Reinforcement Fine-Tuning

VAU-R1 is a data-efficient framework for video anomaly reasoning that combines Multimodal Large Language Models (MLLMs) with Reinforcement Fine-Tuning (RFT).

This repository contains VAU-Bench, the first Chain-of-Thought (CoT) benchmark specifically designed for video anomaly understanding. It enables multimodal tasks such as multiple-choice question answering, temporal anomaly grounding, rationale-based reasoning, and anomaly classification.

📄 Paper: VAU-R1 on arXiv
🧠 Codebase: GVCLab/VAU-R1 GitHub Repository

📦 Note: Please download the original video files from UCF-Crime, ECVA, and MSAD separately to use with the provided annotations.

📁 Annotation Structure

🔀 Splits

• train
• val
• test (subdivided into three distinct datasets)

🏷️ Naming Convention

In both train and val splits, video names are prefixed to indicate the source dataset:

• msad_ → MSAD dataset
• ucf_ → UCF-Crime dataset
• ecva_ → ECVA dataset

✅ Data Availability

All data entries where fields are not equal to -1 are available and usable.

📄 Annotation Format

Each row in the annotation file contains the following fields:

🧠 Supported Tasks

VAU-Bench supports a variety of multimodal video understanding tasks, each mapped to specific columns in the annotation format:

• 🧩 Multiple-choice Question Answering
  → Uses columns: Question, Option 1, Option 2, Option 3, Option 4, Correct Option (Columns 2–7)

• 📍 Temporal Anomaly Grounding
  → Uses columns: Start Time, End Time (Columns 12–13)

• 🧠 Anomaly Reasoning (Chain-of-Thought)
  → Uses columns: Key Object, Description, Reasoning (Columns 8–10)

• 🏷️ Anomaly Classification
  → Uses column: Anomaly Class (Column 11)