Leosang-lx/LLM-Inference

Research and develop distributed LLM inference

Prathmesh234/simple-inference

Simple inference engine for llama from first principles

ziliangpeng/tinyllm

LLM inference in Python - GPT-2, OPT, LLaMA with KV cache, weight loader, benchmark CLI

Taylor-eOS/local-language-models

An inconsistent assortment of small LLM's for inference on home hardware