Catch local LLMs spilling into the CPU. One run shows GPU placement, prefill, decode, memory, power and a verdict.
18
stars
105
commits
Python
primary language
Aug 23, 2026
updated
README
Run it
curl -fsSL https://raw.githubusercontent.com/logxio/picchio/main/public/picchio.pyz -o picchio
chmod +x picchio
./picchio
On Windows, in PowerShell:
curl.exe -fsSL https://raw.githubusercontent.com/logxio/picchio/main/public/picchio.pyz -o picchio.pyz
python picchio.pyz
With no arguments, Picchio finds Ollama tags, local GGUF files and models in the Hugging Face and LM Studio caches. Pick one and it runs three passes.
You can also point it straight at a model or a running server:
./picchio model.gguf
./picchio qwen3.5:9b
./picchio http://127.0.0.1:8080
It runs on macOS, Linux and Windows with Python 3.9+. The download is one file and uses only the standard library.
Use the result
| I want to… | Run |
|---|---|
| catch CPU fallback and measure the run | ./picchio MODEL |
| know whether a model fits here before I download it | ./picchio plan https://huggingface.co/.../model-Q4_K_M.gguf · ./picchio plan qwen3.5:9b |
| check that fit at my own context and KV type | ./picchio plan MODEL --ctx 262144 --kv q8_0 |
| guard something already running and get told the moment it leaves the GPU: your own command, a loaded model, a server | ./picchio guard -- COMMAND · ./picchio guard ollama · ./picchio guard http://127.0.0.1:8080 |
| compare two runs and show the first changed setting | ./picchio compare before.txt after.txt |
| turn one run into a complete Ollama or llama.cpp Issue report | ./picchio MODEL --share bug-report |
Add --json when you want machine-readable output.
Add --ctx 262144 when the problem only appears at a larger context.
The fit check answers before the download, from the file header and the registry manifest, and it answers in one word and one move. In the browser: will it fit?
FITS. The whole model fits in this machine's memory.
5.3 GiB to download
What it gives you
- the exact number of model layers on the GPU
- separate prefill, decode and wall-clock speeds
- GPU activity, memory, power and energy per generated token
- the first setting that changed when you compare two runs
- a clear
HEALTHY,CPU FALLBACKorPARTIAL OFFLOADresult - a paste-ready GitHub Issue with your machine, model, GPU placement and measured rates
Compare machines
I measured these with Picchio:
| machine | model and engine | placement | prefill | decode | wall-clock |
|---|---|---|---|---|---|
| Apple M5 | Qwen3.5-9B, llama.cpp Metal | 33/33 | 588.0 | 21.1 | 15.5 |
| Apple M5 | same file, forced CPU | 0/33 | 26.8 | 12.2 | 3.0 |
| RTX 4090 | same file, llama.cpp CUDA | 33/33 | 6763.3 | 138.0 | 25.2 |
| RTX 5090 | same file, llama.cpp CUDA | 33/33 | 9135.9 | 226.4 | 57.3 |
| RTX 5090 | same file, llama.cpp Vulkan | 33/33 | 6206.3 | 198.4 | 51.0 |
| RTX 5090 | qwen3.5:9b, Ollama | 100% GPU | 8614.7 | 193.5 | 153.9 |
| RTX 4070 SUPER | same file, llama.cpp CUDA | 33/33 | 4187.8 | 78.2 | 28.9 |
| your machine |
Open every result or compare the outputs in examples/.
What Picchio reads
- llama.cpp: per-layer placement, applied sampling settings and timing
- Ollama: CPU/GPU weight split and timing
- macOS: Apple GPU activity, memory, power and energy per token
- NVIDIA on Linux and Windows: GPU activity, memory, power and energy per token through NVML
- AMD on Linux: GPU activity and memory through amdgpu sysfs
Point Picchio at a GGUF path, an Ollama tag or a running llama-server URL. The result tells you where the model ran and which number is safe to compare.
Send me your machine
./picchio MODEL --share row > result.md 2> picchio.txt
Add your result. If Picchio calls your run wrong, send me that one first.
License
Contributors
105 commits
See what people are saying
Similar repos
Languages
Python
99.4%
Catch local LLMs spilling into the CPU. One run shows GPU placement, prefill, decode, memory, power and a verdict.
18
stars
105
commits
Python
primary language
Aug 23, 2026
updated
README
Run it
curl -fsSL https://raw.githubusercontent.com/logxio/picchio/main/public/picchio.pyz -o picchio
chmod +x picchio
./picchio
On Windows, in PowerShell:
curl.exe -fsSL https://raw.githubusercontent.com/logxio/picchio/main/public/picchio.pyz -o picchio.pyz
python picchio.pyz
With no arguments, Picchio finds Ollama tags, local GGUF files and models in the Hugging Face and LM Studio caches. Pick one and it runs three passes.
You can also point it straight at a model or a running server:
./picchio model.gguf
./picchio qwen3.5:9b
./picchio http://127.0.0.1:8080
It runs on macOS, Linux and Windows with Python 3.9+. The download is one file and uses only the standard library.
Use the result
| I want to… | Run |
|---|---|
| catch CPU fallback and measure the run | ./picchio MODEL |
| know whether a model fits here before I download it | ./picchio plan https://huggingface.co/.../model-Q4_K_M.gguf · ./picchio plan qwen3.5:9b |
| check that fit at my own context and KV type | ./picchio plan MODEL --ctx 262144 --kv q8_0 |
| guard something already running and get told the moment it leaves the GPU: your own command, a loaded model, a server | ./picchio guard -- COMMAND · ./picchio guard ollama · ./picchio guard http://127.0.0.1:8080 |
| compare two runs and show the first changed setting | ./picchio compare before.txt after.txt |
| turn one run into a complete Ollama or llama.cpp Issue report | ./picchio MODEL --share bug-report |
Add --json when you want machine-readable output.
Add --ctx 262144 when the problem only appears at a larger context.
The fit check answers before the download, from the file header and the registry manifest, and it answers in one word and one move. In the browser: will it fit?
FITS. The whole model fits in this machine's memory.
5.3 GiB to download
What it gives you
- the exact number of model layers on the GPU
- separate prefill, decode and wall-clock speeds
- GPU activity, memory, power and energy per generated token
- the first setting that changed when you compare two runs
- a clear
HEALTHY,CPU FALLBACKorPARTIAL OFFLOADresult - a paste-ready GitHub Issue with your machine, model, GPU placement and measured rates
Compare machines
I measured these with Picchio:
| machine | model and engine | placement | prefill | decode | wall-clock |
|---|---|---|---|---|---|
| Apple M5 | Qwen3.5-9B, llama.cpp Metal | 33/33 | 588.0 | 21.1 | 15.5 |
| Apple M5 | same file, forced CPU | 0/33 | 26.8 | 12.2 | 3.0 |
| RTX 4090 | same file, llama.cpp CUDA | 33/33 | 6763.3 | 138.0 | 25.2 |
| RTX 5090 | same file, llama.cpp CUDA | 33/33 | 9135.9 | 226.4 | 57.3 |
| RTX 5090 | same file, llama.cpp Vulkan | 33/33 | 6206.3 | 198.4 | 51.0 |
| RTX 5090 | qwen3.5:9b, Ollama | 100% GPU | 8614.7 | 193.5 | 153.9 |
| RTX 4070 SUPER | same file, llama.cpp CUDA | 33/33 | 4187.8 | 78.2 | 28.9 |
| your machine |
Open every result or compare the outputs in examples/.
What Picchio reads
- llama.cpp: per-layer placement, applied sampling settings and timing
- Ollama: CPU/GPU weight split and timing
- macOS: Apple GPU activity, memory, power and energy per token
- NVIDIA on Linux and Windows: GPU activity, memory, power and energy per token through NVML
- AMD on Linux: GPU activity and memory through amdgpu sysfs
Point Picchio at a GGUF path, an Ollama tag or a running llama-server URL. The result tells you where the model ran and which number is safe to compare.
Send me your machine
./picchio MODEL --share row > result.md 2> picchio.txt
Add your result. If Picchio calls your run wrong, send me that one first.
License
See what people are saying
Similar repos
Contributors
105 commits
Languages
Python
99.4%