ruclip-vit-large-patch14-336

RuCLIP (Russian Contrastive Language–Image Pretraining) is a multimodal model for obtaining images and text similarities and rearranging captions and pictures. RuCLIP builds on a large body of work on zero-shot transfer, computer vision, natural language processing and multimodal learning.

Model was trained by Sber AI and SberDevices teams.

• Task: text ranking; image ranking; zero-shot image classification;
• Type: encoder
• Num Parameters: 430M
• Training Data Volume: 240 million text-image pairs
• Language: Russian
• Context Length: 77
• Transformer Layers: 12
• Transformer Width: 768
• Transformer Heads: 12
• Image Size: 336
• Vision Layers: 24
• Vision Width: 1024
• Vision Patch Size: 14

Usage Github

pip install ruclip

clip, processor = ruclip.load("ruclip-vit-large-patch14-336", device="cuda")

Performance

We have evaluated the performance on the following datasets:

Authors

• Alex Shonenkov: Github, Kaggle GM
• Daniil Chesakov: Github
• Denis Dimitrov: Github
• Igor Pavlov: Github