vitus9988/SLM

Raspberry Pi 5 base SLM Operation

stars

113

commits

Python

primary language

Dec 9, 2025

updated

README

SLM project

Raspberry Pi 5 base SLM Operation

CMake OpenBLAS

Quick setting / start

llamacpp

llamacpp-OpenBLAS Dockerfile

# docker image pull
docker pull vitus9988/llamacpp-openblas-lite-arm64:latest

# llama-cli local model
docker run --rm -v $(pwd)/models:/models \
vitus9988/llamacpp-openblas-lite-arm64:latest llama-cli \
-m /models/gemma-3n-E2B-it-Q4_K_M.gguf -p "세계에서 가장 높은산은?"

benchmark
- llamacpp python server & lm-eval Dockerfile
ollama
grafana & prometheus

Model Test

Model	Tps
lama3.2:1b	8
qwen2.5:0.5b	21
qwen2.5:0.5b-instruct-q4_K_S	21
qwen2.5:3b-instruct-q4_K_M	5
deepseek-r1:1.5b	9
gemma2:2b-instruct-q4_K_S	5
gemma-3-1b-it-Q4_K_M	13
gemma-3-4b-it-Q4_K_M	4
gemma-3n-E2B-it-Q4_K_M	6
gemma-3n-E4B-it-Q4_K_M	4
EXAONE-Deep-2.4B-Q4_K_M	6
HyperCLOVAX-SEED-Text-Instruct-1.5B-Q4_K_M	9
kanana-nano-2.1b-instruct-Q4_K_M	7
Midm-2.0-Mini-Instruct-Q4_K_M	7

Contributors

vitus9988

111 commits

github-actions[bot]

2 commits

Similar repos

gwauge/ml-for-smb

nosana-ci/nosana-llm-benchmarking

rcarmo/llama-cpp

SpacemiT K3 llama.cpp TurboQuant/IME2 server fork used on the Milk-V Jupiter 2

noonghunna/club-3090

Community recipes for serving LLMs on RTX 3090/4090/5090 CUDA gpus. Multi-engine (vLLM, llama.cpp,…

2,224

hec-ovi/llama-vulkan-strix

Docker Compose for llama.cpp GGUF servers on AMD Strix Halo: Qwen, Gemma, and Laguna packages…

turboderp/exllama

A more memory-efficient rewrite of the HF transformers implementation of Llama for use with…

2,942

vagrillo/llama.cpp

LLM inference in C/C++

unslothai/llama.cpp

LLM inference in C/C++

203

Languages

Python

87.0%

Dockerfile

13.0%

vitus9988/SLM

Raspberry Pi 5 base SLM Operation

stars

113

commits

Python

primary language

Dec 9, 2025

updated

README

SLM project

Raspberry Pi 5 base SLM Operation

CMake OpenBLAS

Quick setting / start

llamacpp

llamacpp-OpenBLAS Dockerfile

# docker image pull
docker pull vitus9988/llamacpp-openblas-lite-arm64:latest

# llama-cli local model
docker run --rm -v $(pwd)/models:/models \
vitus9988/llamacpp-openblas-lite-arm64:latest llama-cli \
-m /models/gemma-3n-E2B-it-Q4_K_M.gguf -p "세계에서 가장 높은산은?"

benchmark
- llamacpp python server & lm-eval Dockerfile
ollama
grafana & prometheus

Model Test

Model	Tps
lama3.2:1b	8
qwen2.5:0.5b	21
qwen2.5:0.5b-instruct-q4_K_S	21
qwen2.5:3b-instruct-q4_K_M	5
deepseek-r1:1.5b	9
gemma2:2b-instruct-q4_K_S	5
gemma-3-1b-it-Q4_K_M	13
gemma-3-4b-it-Q4_K_M	4
gemma-3n-E2B-it-Q4_K_M	6
gemma-3n-E4B-it-Q4_K_M	4
EXAONE-Deep-2.4B-Q4_K_M	6
HyperCLOVAX-SEED-Text-Instruct-1.5B-Q4_K_M	9
kanana-nano-2.1b-instruct-Q4_K_M	7
Midm-2.0-Mini-Instruct-Q4_K_M	7

Similar repos

gwauge/ml-for-smb

nosana-ci/nosana-llm-benchmarking

rcarmo/llama-cpp

SpacemiT K3 llama.cpp TurboQuant/IME2 server fork used on the Milk-V Jupiter 2

noonghunna/club-3090

Community recipes for serving LLMs on RTX 3090/4090/5090 CUDA gpus. Multi-engine (vLLM, llama.cpp,…

2,224

hec-ovi/llama-vulkan-strix

Docker Compose for llama.cpp GGUF servers on AMD Strix Halo: Qwen, Gemma, and Laguna packages…

turboderp/exllama

A more memory-efficient rewrite of the HF transformers implementation of Llama for use with…

2,942

vagrillo/llama.cpp

LLM inference in C/C++

unslothai/llama.cpp

LLM inference in C/C++

203

Contributors

vitus9988

111 commits

github-actions[bot]

2 commits

Languages

Python

87.0%

Dockerfile

13.0%