Github

Project Page

Paper

📷 Dataset: SynCamVideo Dataset

• [2025.04.15]: Release a new version of the SynCamVideo Dataset with improved quality and greater diversity.
• [2025.04.15]: Please also check our MultiCamVideo Dataset.

1. Dataset Introduction

TL;DR: The SynCamVideo Dataset is a multi-camera synchronized video dataset rendered using Unreal Engine 5. It includes synchronized multi-camera videos and their corresponding camera poses. The SynCamVideo Dataset can be valuable in fields such as camera-controlled video generation, synchronized video production, and 3D/4D reconstruction. The camera is stationary in the SynCamVideo Dataset. If you require footage with moving cameras rather than stationary ones, please explore our MultiCamVideo Dataset.

The SynCamVideo Dataset is a multi-camera synchronized video dataset rendered using Unreal Engine 5. It includes synchronized multi-camera videos and their corresponding camera poses. It consists of 3.4K different dynamic scenes, each captured by 10 cameras, resulting in a total of 34K videos. Each dynamic scene is composed of four elements: {3D environment, character, animation, camera}. Specifically, we use animation to drive the character and position the animated character within the 3D environment. Then, Time-synchronized cameras are set up to render the multi-camera video data.

3D Environment: We collect 37 high-quality 3D environments assets from Fab. To minimize the domain gap between rendered data and real-world videos, we primarily select visually realistic 3D scenes, while choosing a few stylized or surreal 3D scenes as a supplement. To ensure data diversity, the selected scenes cover a variety of indoor and outdoor settings, such as city streets, shopping malls, cafes, office rooms, and the countryside.

Character: We collect 66 different human 3D models as characters from Fab and Mixamo.

Animation: We collect 93 different animations from Fab and Mixamo, including common actions such as waving, dancing, and cheering. We use these animations to drive the collected characters and create diverse datasets through various combinations.

Camera: To enhance the diversity of the dataset, each camera is randomly sampled on a hemispherical surface centered around the character.

2. Statistics and Configurations

Dataset Statistics:

Video Configurations:

Note: You can use 'center crop' to adjust the video's aspect ratio to fit your video generation model, such as 16:9, 9:16, 4:3, or 3:4.

Camera Configurations:

3. File Structure

SynCamVideo-Dataset
├── train
│   └── f24_aperture5
│       ├── scene1    # one dynamic scene
│       │   ├── videos
│       │   │   ├── cam01.mp4    # synchronized 81-frame videos at 1280x1280 resolution
│       │   │   ├── cam02.mp4
│       │   │   ├── ...
│       │   │   └── cam10.mp4
│       │   └── cameras
│       │       └── camera_extrinsics.json    # 81-frame camera extrinsics of the 10 cameras 
│       ├── ...
│       └── scene3400
└── val
    └── basic
        ├── videos
        │   ├── cam01.mp4    # example videos corresponding to the validation cameras
        │   ├── cam02.mp4
        │   ├── ...
        │   └── cam10.mp4
        └── cameras
            └── camera_extrinsics.json    # 10 cameras for validation

3. Useful scripts

• Data Extraction

tar -xzvf SynCamVideo-Dataset.tar.gz

• Camera Visualization

python vis_cam.py

Acknowledgments

We thank Jinwen Cao, Yisong Guo, Haowen Ji, Jichao Wang, and Yi Wang from Kuaishou Technology for their invaluable help in constructing the SynCamVideo-Dataset.

🌟 Citation

Please cite our paper if you find our dataset helpful.

@misc{bai2024syncammaster,
      title={SynCamMaster: Synchronizing Multi-Camera Video Generation from Diverse Viewpoints}, 
      author={Jianhong Bai and Menghan Xia and Xintao Wang and Ziyang Yuan and Xiao Fu and Zuozhu Liu and Haoji Hu and Pengfei Wan and Di Zhang},
      year={2024},
      eprint={2412.07760},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2412.07760}, 
}

Contact

Jianhong Bai