Large Language Model Inference & Optimization
7 repos
Libraries and techniques for deploying and optimizing transformer-based language models in production, with a focus on quantization, inference acceleration, and text generation. The cluster centers on tools like AutoGPTQ for model compression and text-generation-inference for efficient serving, alongside supporting utilities for punctuation prediction and model patching. These repositories help practitioners reduce model size, improve latency, and integrate LLMs into applications without prohibitive computational costs.
Python · 6
Jupyter Notebook · 1
nlp ·866
transformer ·866
deep-learning ·486
diffusion ·359
cv ·359
text2image ·359
generative-art ·359
llm ·311
openai-api ·274
huggingface ·100
