yyang181/Vivid-VR

for ntire 2026

stars

commits

Python

primary language

Mar 22, 2026

updated

README

NTIRE 2026 SUGC-VR Challenge - Vivid-VR Solution

This repository contains the solution for the NTIRE 2026 SUGC-VR Challenge.

🔧 Dependencies and Installation

Create Conda Environment and Install Dependencies

# create new conda env
conda create -n Vivid-VR python=3.10
conda activate Vivid-VR

# install pytorch
pip install torch==2.2.1 torchvision==0.17.1 torchaudio==2.2.1 --index-url https://download.pytorch.org/whl/cu121

# install python dependencies
pip install -r requirements.txt

# install easyocr [Optional, for text fix]
pip install easyocr
pip install numpy==1.26.4  # numpy2.x maybe installed when installing easyocr, which will cause conflicts

Download Models

[Required] Download CogVideoX1.5-5B checkpoints from [huggingface].
[Required] Download cogvlm2-llama3-caption checkpoints from [huggingface].
[Required] Download Vivid-VR checkpoints from [huggingface].
[Optional, for text fix] Download easyocr checkpoints [english_g2] [zh_sim_g2] [craft_mlt_25k].
[Optional, for text fix] Download Real-ESRGAN checkpoints [RealESRGAN_x2plus].
Put them under the ./ckpts folder.

The ckpts directory structure should be arranged as:

├── ckpts
│   ├── CogVideoX1.5-5B
│   │   ├── ...
│   ├── cogvlm2-llama3-caption
│   │   ├── ...
│   ├── Vivid-VR
│   │   ├── controlnet
│   │       ├── config.json
│   │       ├── diffusion_pytorch_model.safetensors
│   │   ├── connectors.pt
│   │   ├── control_feat_proj.pt
│   │   ├── control_patch_embed.pt
│   ├── easyocr
│   │   ├── craft_mlt_25k.pth
│   │   ├── english_g2.pth
│   │   ├── zh_sim_g2.pth
│   ├── RealESRGAN
│   │   ├── RealESRGAN_x2plus.pth

☕️ Inference

Run the following commands to reproduce the results:

### NTIRE 2026 SUGC-VR final test
CUDA_VISIBLE_DEVICES=0 TORCH_CUDNN_SDPA_ENABLED=0 python VRDiT/inference_new.py --ckpt_dir=./ckpts --cogvideox_ckpt_path=./ckpts/CogVideoX1.5-5B --cogvlm2_ckpt_path=./ckpts/cogvlm2-llama3-caption --input_dir=data/input  --output_dir=data/output/vividvr --upscale=1 --textfix --num_temporal_process_frames 121 --gpu_rank 0 --gpu_num 1 --align_spatial

Similar repos

yyang181/CoDiVSR

Official code for CoDiVSR: Rethinking What to Condition and What to Disentangle in Diffusion-based…

same org

csbhr/Vivid-VR

Model

csbhr/Vivid-VR

The official repository of our ICLR 2026 paper "Vivid-VR: Distilling Concepts from Text-to-Video…

245

Jumpat/tigon

Official repository of Text-Image Conditioned 3D Generation (TIGON, CVPR 2026)

Video-Reason/VBVR-Wan2.2

Official training and inference code for VBVR (A Very Big Video Reasoning Suite)

AIDASLab/VIRST

[CVPR 2026] Official Implementation for "VIRST: Video-Instructed Reasoning Assistant for…

Eyeline-Labs/Vista4D

Official code, models, and data for Vista4D: Video Reshooting with 4D Point Clouds (CVPR 2026…

587

zhuxing0/Relit-LiVE

[SIGGRAPH 2026] Official code of the paper "Relit-LiVE: Relight Video by Jointly Learning…

121

Languages

Python

100.0%

yyang181/Vivid-VR

for ntire 2026

stars

commits

Python

primary language

Mar 22, 2026

updated

README

NTIRE 2026 SUGC-VR Challenge - Vivid-VR Solution

This repository contains the solution for the NTIRE 2026 SUGC-VR Challenge.

🔧 Dependencies and Installation

Create Conda Environment and Install Dependencies

# create new conda env
conda create -n Vivid-VR python=3.10
conda activate Vivid-VR

# install pytorch
pip install torch==2.2.1 torchvision==0.17.1 torchaudio==2.2.1 --index-url https://download.pytorch.org/whl/cu121

# install python dependencies
pip install -r requirements.txt

# install easyocr [Optional, for text fix]
pip install easyocr
pip install numpy==1.26.4  # numpy2.x maybe installed when installing easyocr, which will cause conflicts

Download Models

[Required] Download CogVideoX1.5-5B checkpoints from [huggingface].
[Required] Download cogvlm2-llama3-caption checkpoints from [huggingface].
[Required] Download Vivid-VR checkpoints from [huggingface].
[Optional, for text fix] Download easyocr checkpoints [english_g2] [zh_sim_g2] [craft_mlt_25k].
[Optional, for text fix] Download Real-ESRGAN checkpoints [RealESRGAN_x2plus].
Put them under the ./ckpts folder.

The ckpts directory structure should be arranged as:

├── ckpts
│   ├── CogVideoX1.5-5B
│   │   ├── ...
│   ├── cogvlm2-llama3-caption
│   │   ├── ...
│   ├── Vivid-VR
│   │   ├── controlnet
│   │       ├── config.json
│   │       ├── diffusion_pytorch_model.safetensors
│   │   ├── connectors.pt
│   │   ├── control_feat_proj.pt
│   │   ├── control_patch_embed.pt
│   ├── easyocr
│   │   ├── craft_mlt_25k.pth
│   │   ├── english_g2.pth
│   │   ├── zh_sim_g2.pth
│   ├── RealESRGAN
│   │   ├── RealESRGAN_x2plus.pth

☕️ Inference

Run the following commands to reproduce the results:

### NTIRE 2026 SUGC-VR final test
CUDA_VISIBLE_DEVICES=0 TORCH_CUDNN_SDPA_ENABLED=0 python VRDiT/inference_new.py --ckpt_dir=./ckpts --cogvideox_ckpt_path=./ckpts/CogVideoX1.5-5B --cogvlm2_ckpt_path=./ckpts/cogvlm2-llama3-caption --input_dir=data/input  --output_dir=data/output/vividvr --upscale=1 --textfix --num_temporal_process_frames 121 --gpu_rank 0 --gpu_num 1 --align_spatial

Similar repos

yyang181/CoDiVSR

Official code for CoDiVSR: Rethinking What to Condition and What to Disentangle in Diffusion-based…

same org

csbhr/Vivid-VR

Model

csbhr/Vivid-VR

The official repository of our ICLR 2026 paper "Vivid-VR: Distilling Concepts from Text-to-Video…

245

Jumpat/tigon

Official repository of Text-Image Conditioned 3D Generation (TIGON, CVPR 2026)

Video-Reason/VBVR-Wan2.2

Official training and inference code for VBVR (A Very Big Video Reasoning Suite)

AIDASLab/VIRST

[CVPR 2026] Official Implementation for "VIRST: Video-Instructed Reasoning Assistant for…

Eyeline-Labs/Vista4D

Official code, models, and data for Vista4D: Video Reshooting with 4D Point Clouds (CVPR 2026…

587

zhuxing0/Relit-LiVE

[SIGGRAPH 2026] Official code of the paper "Relit-LiVE: Relight Video by Jointly Learning…

121

Languages

Python

100.0%