LLM Inference and Quantization
2 repos
Libraries and tools for optimizing inference performance of large language models through techniques like quantization, pruning, and hardware acceleration. The cluster centers on frameworks like AutoGPTQ for post-training quantization, alongside educational resources and deployment utilities for running LLMs efficiently on resource-constrained hardware. Most repositories are Python-based implementations focused on the transformer inference pipeline.
Jupyter Notebook · 2
python ·2,378
quantization ·2,378
finetuning ·2,378
finetuning-llms ·2,378
inference ·2,378
large-language-models ·2,378
llm ·2,378
fine-tuning ·2,378
transformers ·0
mlm ·0
