44 repos across 2 sub-areas
Multimodal AI systems that combine vision and language understanding to perform tasks like image captioning, visual question answering, and image-to-text generation. This cluster centers on transformer-based architectures (particularly variants of LLaVA and BLIP2) that bridge computer vision and natural language processing, enabling models to understand and describe visual content. Repositories here include pretrained model weights, fine-tuning implementations, and inference code built primarily with PyTorch.
Cluster 653492
38 repos
Document Understanding & OCR Models
6 repos
Deep learning models and frameworks for optical character recognition (OCR) and document understanding, particularly focused on vision transformer architectures for extracting text and structured data from scanned documents and images. The cluster centers on the Donut model family—a transformer-based approach to document understanding—along with supporting tools for multimodal document processing, layout analysis, and training on datasets like CORD (receipts) and RVL-CDIP (document classification).