Shadowgraph
About
Sign in
← Explore by area/GGML and LLaMA C++ Inference

GGML inference and quantization

28 repos

C++ libraries and tools for efficient machine learning inference, centered on the GGML tensor library and GGUF quantization format. The cluster includes llama.cpp as its most prominent project—a portable inference engine for large language models—alongside related tooling for model quantization, speech-to-text processing, and cross-platform deployment. Most repos are low-level C++ implementations optimized for running ML models on consumer hardware with minimal dependencies.

C++ · 19
Python · 8
C · 1
ggml ·267,191
gguf ·63,531
cross-platform ·51,878
local-inference ·51,878
open-source-ai ·51,878
single-file-executable ·51,878
llama-cpp ·51,878
local-llm ·51,878
local-ai ·51,878
speech-to-text ·51,878

Related clusters

Local LLM Inference Optimization19Automatic Speech Recognition (ASR)32Mistral 7B Model Variants & GGUF Quantization11Whisper Speech Recognition & Deployment25Diffusion Models and Image Generation22MLX framework for Apple Silicon ML22

poolsideai/llama.cpp

LLM inference in C/C++

C++

6

8,486 commits

ggml-org/llama.cpp

LLM inference in C/C++

C++

127,769

8,823 commits

wendadawen/llama.cpp

LLM inference in C/C++

C++

0

8,391 commits

alainnothere/llama.cpp

LLM inference in C/C++

C++

1

8,043 commits

ggerganov/llama.cpp

LLM inference in C/C++

C++

127,769

8,993 commits

vagrillo/llama.cpp

LLM inference in C/C++

C++

1

8,956 commits

unslothai/llama.cpp

LLM inference in C/C++

C++

203

9,119 commits

gaetan-puleo/llama-cpp-strix-halo

No description

C++

13

8,880 commits

Veridian-Zenith/llama.cpp

No description

C++

0

9,049 commits

alin-o/llama.cpp-qwen38

No description

C++

0

9,044 commits

nathanmp/delphi.cpp

A hardware-aware fork of llama.cpp.

C++

0

9,039 commits

XHToken/llama.cpp

LLM inference in C/C++

C++

7

8,804 commits

chynggi/gigatoken-llama.cpp

LLM inference in C/C++ with Gigatoken / This Repo is Very Vibe-Coded-Experimental

C++

10

9,220 commits

LostRuins/koboldcpp

Run GGUF models easily with a KoboldAI UI. One File. Zero Install.

C++

11,653

13,404 commits

HimariO/llama.cpp.qwen2.5vl

Port of Facebook's LLaMA model in C/C++

C++

66

4,513 commits

ggerganov/ggml

Tensor library for machine learning

C++

15,294

4,140 commits

ggml-org/ggml

Tensor library for machine learning

C++

15,308

4,104 commits

amstaff666/llama-cpp-python

No description

Python

0

2,105 commits

abetlen/llama-cpp-python

Python bindings for llama.cpp

Python

10,612

2,118 commits

ali3alnasser/llama-cpp-python

No description

Python

0

1,955 commits

inference-sh/llama-cpp-python

No description

Python

12

2,017 commits

machaao/llama-cpp-python

Python bindings for llama.cpp

Python

0

1,969 commits

TmLev/llama-cpp-python

No description

Python

1

1,692 commits

jaepil/llama-cpp-python-deprecated

Python bindings for llama.cpp

Python

0

1,932 commits

mozilla-ai/llamafile

Distribute and run LLMs with a single file.

C++

25,939

859 commits

Mozilla-Ocho/llamafile

Distribute and run LLMs with a single file.

C++

25,939

859 commits

containers/ramalama

RamaLama is an open-source developer tool that simplifies the local serving of AI models from any…

Python

3,042

4,485 commits

tarruda/llama.cpp

Port of Facebook's LLaMA model in C/C++

C

13

189 commits