sberbank-ai/sber-vq-gan

Jupyter Notebook

8 commits

updated Jul 5, 2022

README

Tuned-VQ-GAN

This repository contains information about the Tuned VQ-GAN model.
We've improved the quality of image generation on some domains such as people and text.
We used COCO only validation images. The splitting you can get here.

original

Models and configs

VAE encoder - VAE decoder
VQ-GAN 16384 - Config 16384
VQ-GAN GUMBELF8 - Config GUMBELF8
Tuned VQ-GAN GUMBELF8 - Config GUMBELF8

Examples

Jupyter with model inference
Jupyter metrics calculation

Metrics calculation results

IS (More - better)

domain/model	VAE	16384	gumbelf8	Tuned-gumbelf8	Original
all	11.133	13.647	15.203	15.316	15.278
indoor	9.769	10.744	11.707	11.688	11.638
kitchen	9.726	11.354	12.333	12.152	11.813
appliance	5.705	6.024	6.154	6.199	5.890
electronic	7.830	9.509	9.712	9.606	9.497
furniture	10.861	13.346	14.500	14.531	14.592
outdoor	8.163	9.520	10.668	10.293	10.451
sports	7.467	8.544	8.814	8.841	8.962
food	7.954	8.725	9.390	9.434	9.191
vehicle	10.527	12.947	14.240	14.559	14.233
animal	11.933	14.249	15.999	15.879	15.857
accessory	9.399	11.687	13.117	13.388	13.228
person	13.752	17.794	20.048	20.420	20.600
face	11.903	14.987	16.986	17.489	17.584
text	14.902	18.457	21.396	21.292	21.131

FID (Less - better)

domain/model	VAE	16384	gumbelf8	Tuned-gumbelf8
all	59.753	38.912	30.304	30.136
indoor	74.734	57.925	45.432	44.686
kitchen	66.424	47.086	36.735	36.579
appliance	80.359	70.604	53.225	52.064
electronic	77.856	64.034	50.759	50.447
furniture	53.438	38.204	29.510	29.569
outdoor	91.932	58.877	46.309	45.287
sports	65.540	39.961	32.219	31.756
food	76.974	53.109	41.018	41.413
vehicle	60.318	34.259	26.721	26.463
animal	64.250	41.520	32.039	32.078
accessory	79.843	56.311	44.660	44.454
person	40.810	20.430	15.523	15.484
face	54.153	34.109	26.663	26.750
text	47.299	27.656	21.303	21.148

Images per domain (spliting)

domain	images count
food	702
person	2652
kitchen	904
sports	919
indoor	645
vehicle	1139
furniture	1239
appliance	316
accessory	718
animal	1003
electronic	589
outdoor	553
face	1488
text	1706

Authors

Denis Karachev
Alex Shonenkov
Denis Dmitrov

Similar repos

CompVis/taming-transformers

Taming Transformers for High-Resolution Image Synthesis

6,530

Victarry/awesome-image-synthesis

Collection of papers in image synthesis with paper graph.

ziqihuangg/Awesome-Evaluation-of-Visual-Generation

A list of works on evaluation of visual generation models, including evaluation metrics, models,…

464

kakaobrain/rq-vae-transformer

The official implementation of Autoregressive Image Generation using Residual Quantization (CVPR…

1,028

facebookresearch/ic_gan

Official repository for the paper "Instance-Conditioned GAN" by Arantxa Casanova, Marlene Careil,…

534

SerezD/vqvae-vqgan-pytorch-lightning

VQ-VAE/GAN implementation in pytorch-lightning

NVlabs/DDO

[ICML 2025 Spotlight] Direct Discriminative Optimization: Reinforcing Diffusion/Autoregressive with…

125

POSTECH-CVLab/PyTorch-StudioGAN

StudioGAN is a Pytorch library providing implementations of representative Generative Adversarial…

3,483