ReCo

🖥️ GitHub    ｜    🌐 Project Page    |   🤗 ReCo-Data   |    📈 ReCo-Bench   |    🤗 ReCo-Models    |    📖 Paper   

ReCo: Region-Constraint In-Context Generation for Instructional Video Editing

🔆 If you find ReCo useful, please give a ⭐ for this repo, which is important to Open-Source projects. Thanks!

Here, we will gradually release the following resources, including:

• ReCo training dataset: ReCo-Data
• Evaluation code: ReCo-Bench
• Model weights, inference code, and training code

Video Demos

🔥 Updates

• ✅ [2025.12.22] Upload Our arXiv Paper.
• ✅ [2025.12.23] Release ReCo-Data and Usage code.
• ✅ [2025.12.23] Release ReCo-Bench and evaluation code.
• ✅ [2026.01.16] Release ReCo Model weights(Preview) and inference code.
• ✅ [2026.01.16] Uploaded raw video object masks to ReCo-Data.
• ⬜ Release training code.

📊 ReCo-Data Preparation

ReCo-Data is a large-scale, high-quality video editing dataset consisting of 500K+ instruction–video pairs, covering four video editing tasks: object addition (add), object removal (remove), object replacement (replace), and video stylization (style).

Downloading ReCo-Data

Please download each task of ReCo-Data into the ./ReCo-Data directory by running:

bash ./tools/download_dataset.sh

Before downloading the full dataset, you may first browse the visualization examples.

These examples are collected by randomly sampling 50 instances from each task (add, remove, replace, and style), without any manual curation or cherry-picking, and are intended to help users quickly inspect and assess the overall data quality.

Note: The examples are formatted for visualization convenience and do not strictly follow the dataset format.

Directory Structure

After downloading, please ensure that the dataset follows the directory structure below:

ReCo-Data/
├── add/
│   ├── add_data_configs.json
│   ├── src_videos/
│   │   ├── video1.mp4
│   │   ├── video2.mp4
│   │   └── ...
│   └── tar_videos/
│       ├── video1.mp4
│       ├── video2.mp4
│       └── ...
├── remove/
│   ├── remove_data_configs.json
│   ├── src_videos/
│   └── tar_videos/
├── replace/
│   ├── replace_data_configs.json
│   ├── src_videos/
│   └── tar_videos/
└── style/
    ├── style_data_configs.json
    ├── src_videos/
    │   ├── video1.mp4
    │   └── ...
    └── tar_videos/
        ├── video1-a_Van_Gogh_style.mp4
        └── ...

Testing and Visualization

After downloading the dataset, you can directly test and visualize samples from any single task using the following script (taking the replace task as an example):

python reco_data_test_single.py \
  --json_path ./ReCo-Data/replace/replace_data_configs.json \
  --video_folder ./ReCo-Data \
  --debug

Mixed Task Loading

You can also load a mixed dataset composed of the four tasks (add, remove, replace, and style) with arbitrary ratios by running:

python reco_data_test_mix_data.py \
  --json_folder ./ReCo-Data \
  --video_folder ./ReCo-Data \
  --debug

Notes

• src_videos/ contains the original source videos.
• tar_videos/ contains the edited target videos corresponding to each instruction.
• *_data_configs.json stores the instruction–video mappings and metadata for each task.

📈 Evaluation

VLLM-based Evaluation Benchmark

Downloading ReCo-Bench

Please download ReCo-Bench into the ./ReCo-Bench directory by running:

bash ./tools/download_ReCo-Bench.sh

Usage

After downloading the benchmark, you can directly start the evaluation using:

cd tools
bash run_eval_via_gemini.sh

🏃 Inference

1. Environment Preparation

Create and activate the specialized Conda environment:

conda create -n reco python=3.11 -y
conda activate reco
pip install -r requirements.txt

2. Model Weights Setup

You need to prepare both the base model and our specific checkpoints.

Organize the files as follows:

.
├── Wan-AI/                      
├── all_ckpts/                   
│   └── 2026_01_16_v1_release_preview.ckpt  
├── assets/                      
└── inference_reco_single.py

3. Running Inference

We provide a bash script to automate the execution of different tasks (Replace, Remove, Style, Add and Propagation). Run the following command:

bash infer_server_single.sh

To run a specific task manually or customize the execution, use the python command directly:

python inference_reco_single.py \
    --task_name replace \
    --test_txt_file_name assets/replace_test.txt \
    --lora_ckpt all_ckpts/2026_01_16_v1_release_preview.ckpt

4. Key Arguments Explained

🚀 Training

Will be released soon.

🌟 Star and Citation

If you find our work helpful for your research, please consider giving a star⭐ on this repository and citing our work.

@article{zhang2025region,
  title={Region-Constraint In-Context Generation for Instructional Video Editing},
  author={Zhang, Zhongwei and Long, Fuchen and Li, Wei and Qiu, Zhaofan and Liu, Wu and Yao, Ting and Mei, Tao},
  journal={arXiv preprint arXiv:2512.17650},
  year={2025}
}

💖 Acknowledgement

Our code is inspired by several works, including WAN, ObjectClear--a strong object remover, VACE, Flux-Kontext-dev. Thanks to all the contributors!