Llama 2 Quantized Model Variants

18 repos

This cluster represents quantized versions of Meta's Llama 2 language models, specifically focusing on reduced-precision (2-bit, 3-bit, and 4-bit) variants optimized for efficient inference and deployment. The repositories contain model weights and configurations compatible with text generation inference frameworks, using the safetensors format for safe and efficient model serialization. Developers exploring this area will find different size variants (7B and 70B parameters) with various quantization schemes that enable running large language models on resource-constrained hardware while maintaining compatibility with standard inference endpoints.

Related clusters