MOVA Benchmark for Arena

This is the benchmark used for the subjective arena experiments of MOVA (MOVA: Towards Scalable and Synchronized Video–Audio Generation). All prompts are rewritten by the workflow introduced in the paper.

• Paper: MOVA: Towards Scalable and Synchronized Video–Audio Generation
• Code: https://github.com/OpenMOVA/MOVA

Overview

The benchmark contains 732 samples in total, organized into two subsets:

Each sample includes a first-frame image and a prompt for joint image-text to video-audio generation.

Files layout (img/ and videos/)

First-frame images and (optional) generated videos use the same two-bucket grouping as in data.json top-level keys:

If you ship generated videos under videos/, each model directory mirrors that layout:

videos/<model_name>/
├── MOVA-Bench/
│   ├── anime/
│   ├── games/
│   ├── movie/
│   ├── multi-speaker/
│   ├── others/
│   ├── shot-effect/
│   └── sports/
└── VerseBench_bilingual/
    ├── set1/
    ├── set2/
    └── set3/

Video files are stored as <id>.mp4 (filename is the sample id only, e.g. anime1.mp4 or 00000.mp4).

MOVA-Bench

MOVA-Bench is constructed from real-world videos by extracting the first-frame image and a corresponding prompt. Each prompt concisely describes key visual elements, such as the scene setting, characters, and environmental conditions. Depending on the scenario, audio-related information is incorporated to form a unified prompt for joint video-audio generation. All samples are adapted to ensure temporal consistency and logical coherence of the generated videos.

The benchmark consists of six scenario categories, each targeting specific challenges in joint video-audio generation:

• Multi-speaker (27 samples): Evaluates the model's ability to generate synchronized speech, facial expressions, and interactions among multiple characters.
• Movie (12 samples): Requires film-level narrative generation with plots referencing the background of the original films.
• Sports (20 samples): Focuses on athletes' performances, with some prompts including commentators' narration.
• Games (20 samples): Covers shooting games, 3D games, and competitive games.
• Shot-effect (30 samples): Designed to assess visual realism under camera panning, zooming, and rotation.
• Anime (20 samples): Includes both 2D anime and 3D animated content.
• Others (3 samples): Additional miscellaneous scenarios.

VerseBench (Bilingual)

This subset is adapted from VerseBench, where half of the originally English-only speech data was manually translated to construct a bilingual (English-Chinese) mix. It contains 600 samples split into three sets:

Data Format

All prompts and image paths are listed in data.json. Top-level keys are category groups (set1, set2, set3, movie, anime, …). Each entry contains:

{
  "prompt": "A detailed description of the scene, actions, and audio...",
  "first_frame_path": "img/MOVA-Bench/movie/movie1.png"
}

first_frame_path always lives under img/MOVA-Bench/... or img/VerseBench_bilingual/... as in the table above. Generated videos (if present) follow the same category under videos/<model_name>/MOVA-Bench/... or videos/<model_name>/VerseBench_bilingual/... with <id>.mp4 filenames.

Acknowledgments

We thank the authors of VerseBench for open-sourcing the benchmark.