Paper: https://arxiv.org/abs/2412.09262

Code: https://github.com/bytedance/LatentSync

LatentSync 1.6

Many people have reported that the teeth and lips generated by LatentSync 1.5 are blurry. To address this issue, we trained LatentSync 1.6 on 512 x 512 resolution videos.

Notably, we did not make any changes to the model structure or training strategy; the only modification was upgrading the training dataset to 512 x 512 videos. Therefore, the current code is compatible with both LatentSync 1.5 and 1.6. To switch between versions, you only need to load the corresponding checkpoint and modify the resolution parameter in the U-Net config file.

You can view the demo in LatentSync's official GitHub repo.