Quantized LLM Model Variants
33 repos
4-bit quantized and compressed variants of large language models, primarily using AWQ (Activation-aware Weight Quantization) and related optimization techniques for efficient inference. This cluster contains dozens of fine-tuned and merged model checkpoints based on popular base models like Qwen, Mistral, and Airoboros, packaged with safetensors for compatibility with standard transformer frameworks. Developers exploring this area will find ready-to-use quantized model weights optimized for reduced memory footprint and faster inference without sacrificing significant model quality.
awq ·160
safetensors ·160
4-bit ·160
transformers ·149
endpoints_compatible ·111
conversational ·95
text-generation ·59
text-generation-inference ·58
multimodal ·54
image-text-to-text ·52
