Leosang-lx/LLM-Inference

Research and develop distributed LLM inference

alarmclock-kisser/ParallelOnnxDeltaForge

ONNX-powered LLM + LoRA inferencing for multi-accelerator setups

diagonalge/llm-scripts

LLM Models loaded directly with transformers

johnmai-dev/ANE-LM

LLM inference on Apple Neural Engine (ANE)

Taylor-eOS/local-language-models

An inconsistent assortment of small LLM's for inference on home hardware

Prathmesh234/simple-inference

Simple inference engine for llama from first principles

eldahl/VisioPetrae

A proof-of-concept system for scaleable deployment of VLM inference.