LLM inference in Python - GPT-2, OPT, LLaMA with KV cache, weight loader, benchmark CLI
0
stars
24
commits
Python
primary language
May 30, 2026
updated
24 commits
ziliangpeng/triton-llm
Pure Python + Triton GPT-2 inference (no PyTorch)
Prathmesh234/simple-inference
Simple inference engine for llama from first principles
kaengreg/mmlu_ru
Glanderlice/open_llm
2
LoselSpt/LLM
Lightning-AI/lit-gpt
20+ high-performance LLMs with recipes to pretrain, finetune and deploy at scale.
13,655
Lightning-AI/litgpt
13,654
Leosang-lx/LLM-Inference
Research and develop distributed LLM inference
95.9%
Starlark
4.1%
