Multi-model LLM platform (15 models, 270M-12B) with OpenAI-compatible APIs. Testbed for gesture UI, AI-generated interfaces, and model collaboration modes.
See the codeREADME
LM Arena
Self-hosted LLM inference with GitHub Actions as the compute. Each model runs in a Docker container on a runner, exposes itself through a Cloudflare quick tunnel, and registers that URL in a tunnel-registry Worker. The frontend is a static React app on GitHub Pages that streams chat straight to whichever tunnels are live. No always-on server — compute exists only while a run is in flight.
┌──────────────────┐ ┌──────────────────────────────┐
│ GitHub Pages │ │ GitHub Actions runner │
│ static React │ │ Docker · llama.cpp server │
└──────────────────┘ └──────────────────────────────┘
│ │
│ ① fetch live tunnel URLs │ cloudflared quick tunnel
▼ ▼
┌─────────────────────────────────────────────────────────────────────────┐
│ tunnel-registry Worker · model name → active tunnel URL │
└─────────────────────────────────────────────────────────────────────────┘
│
└── ② browser streams chat directly to the inference tunnel
Why this shape
- No idle cost — compute is ephemeral Actions minutes, alive only during a run; nothing to keep warm.
- No infra to own — Pages serves the static app, a Worker holds the registry, Cloudflare opens the tunnel. There is no backend to deploy.
- One source of truth —
config/models.pydefines every model (ports, HF repo, quant, context, rank). The frontend'sservices.json, the workflow's model choices, and the table below are all generated from it.
Models
| # | Model | Class | Highlights |
|---|---|---|---|
| 1 | Nanbeige4.1-3B | reasoning | AIME 90.4%, GPQA 82.2%, outperforms Qwen3-32B on reasoning |
| 2 | Qwen3.5 4B | small | Multilingual (201 langs), 256K context, thinking mode, Apache 2.0 |
| 2 | Qwen3.5-27B Claude Distilled | medium | 27B reasoning distilled from Claude 4.6 Opus, preserved thinking mode, strong agentic capabilities |
| 3 | SmolLM3 3B | small | Hybrid reasoning (36.7% AIME), tool-calling (92.3% BFCL), 64K context |
| 3 | LFM2.5 8B-A1B | medium | Hybrid MoE, 8.3B total / 1.5B active, 128K context — largest model in the arena |
| 4 | LFM2.5 1.2B | small | Hybrid LFM2 model, 8 languages, edge-optimized with RL tuning |
| 4 | Phi-4 Mini Reasoning | reasoning | 3.8B matching o1-mini on math, outperforms R1-distill-7B, 128K context |
| 4 | Gemma 4 12B | medium | Gemma 4 unified 12B, 256K context, Apache 2.0 — largest dense model in the arena |
| 5 | LFM2.5 1.2B Thinking | small | Thinking variant of LFM2.5, hybrid architecture with reasoning traces |
| 5 | Gemma 3n E4B | medium | MatFormer: 8B params, 4B memory footprint, LMArena 1300+ (first sub-10B) |
| 6 | LFM2.5 350M | small | 255 tok/s on Jetson, 33ms TTFT, 229MB — fastest model in the arena |
| 6 | LFM2.5 VL 1.6B Extract | small | Vision model — image + field list → structured JSON. In-car, document scan, inspection |
| 7 | LFM2 350M StepGame | small | LFM2 350M fine-tuned on StepGame spatial reasoning (16% → 70% accuracy) |
| 7 | LFM2.5 VL 450M Extract | small | 450M vision model — image + field list → structured JSON, edge-tier extraction |
| 7 | Falcon H1R 7B | medium | Transformer-Mamba hybrid, AIME 88.1%, LiveCodeBench 68.6%, 256K context |
| 8 | LFM2 350M NK Risk | small | LFM2 350M fine-tuned on GDELT NK military events — geopolitical risk analyst |
| 8 | Jan-code 4B | small | Code-focused 4B model by Jan, 2.72GB Q4_K_M |
| 8 | Qwen3.5 7B | medium | Multilingual (201 langs), 256K context, thinking mode, Apache 2.0 |
| 13 | GPT-OSS 20B | reasoning | MoE (21B params / 3.6B active), function calling, agentic operations |
Live latency / throughput → benchmarks.html. Architecture notes for select models (growing set) → docs/models/.
Run it
make inference MODEL=qwen # launch one model (GitHub Actions → Cloudflare tunnel)
make down # cancel all in-flight inference runs
make build # build the frontend (regenerates services.json from config)
Launch / manage the whole fleet from the browser: open app/chat/frontend/public/admin.html. Local frontend dev: cd app/chat/frontend && npm install && npm run dev.
Internals
config/models.py is the single source of truth. Add a model = add one ModelConfig, then regenerate the derivatives — make build (services.json), make sync-workflow-choices (inference.yml), make sync-readme (this table). The CI build matrix reads the config directly, so new models are picked up automatically.
Deep architecture (workflows, data flow, layout) → docs/architecture.md. AI build-context → CLAUDE.md.
License
MIT
Contributors
Similar repos
Languages
TypeScript
68.5%
HTML
13.2%
Python
10.9%
CSS
3.1%
JavaScript
3.0%
Multi-model LLM platform (15 models, 270M-12B) with OpenAI-compatible APIs. Testbed for gesture UI, AI-generated interfaces, and model collaboration modes.
See the codeREADME
LM Arena
Self-hosted LLM inference with GitHub Actions as the compute. Each model runs in a Docker container on a runner, exposes itself through a Cloudflare quick tunnel, and registers that URL in a tunnel-registry Worker. The frontend is a static React app on GitHub Pages that streams chat straight to whichever tunnels are live. No always-on server — compute exists only while a run is in flight.
┌──────────────────┐ ┌──────────────────────────────┐
│ GitHub Pages │ │ GitHub Actions runner │
│ static React │ │ Docker · llama.cpp server │
└──────────────────┘ └──────────────────────────────┘
│ │
│ ① fetch live tunnel URLs │ cloudflared quick tunnel
▼ ▼
┌─────────────────────────────────────────────────────────────────────────┐
│ tunnel-registry Worker · model name → active tunnel URL │
└─────────────────────────────────────────────────────────────────────────┘
│
└── ② browser streams chat directly to the inference tunnel
Why this shape
- No idle cost — compute is ephemeral Actions minutes, alive only during a run; nothing to keep warm.
- No infra to own — Pages serves the static app, a Worker holds the registry, Cloudflare opens the tunnel. There is no backend to deploy.
- One source of truth —
config/models.pydefines every model (ports, HF repo, quant, context, rank). The frontend'sservices.json, the workflow's model choices, and the table below are all generated from it.
Models
| # | Model | Class | Highlights |
|---|---|---|---|
| 1 | Nanbeige4.1-3B | reasoning | AIME 90.4%, GPQA 82.2%, outperforms Qwen3-32B on reasoning |
| 2 | Qwen3.5 4B | small | Multilingual (201 langs), 256K context, thinking mode, Apache 2.0 |
| 2 | Qwen3.5-27B Claude Distilled | medium | 27B reasoning distilled from Claude 4.6 Opus, preserved thinking mode, strong agentic capabilities |
| 3 | SmolLM3 3B | small | Hybrid reasoning (36.7% AIME), tool-calling (92.3% BFCL), 64K context |
| 3 | LFM2.5 8B-A1B | medium | Hybrid MoE, 8.3B total / 1.5B active, 128K context — largest model in the arena |
| 4 | LFM2.5 1.2B | small | Hybrid LFM2 model, 8 languages, edge-optimized with RL tuning |
| 4 | Phi-4 Mini Reasoning | reasoning | 3.8B matching o1-mini on math, outperforms R1-distill-7B, 128K context |
| 4 | Gemma 4 12B | medium | Gemma 4 unified 12B, 256K context, Apache 2.0 — largest dense model in the arena |
| 5 | LFM2.5 1.2B Thinking | small | Thinking variant of LFM2.5, hybrid architecture with reasoning traces |
| 5 | Gemma 3n E4B | medium | MatFormer: 8B params, 4B memory footprint, LMArena 1300+ (first sub-10B) |
| 6 | LFM2.5 350M | small | 255 tok/s on Jetson, 33ms TTFT, 229MB — fastest model in the arena |
| 6 | LFM2.5 VL 1.6B Extract | small | Vision model — image + field list → structured JSON. In-car, document scan, inspection |
| 7 | LFM2 350M StepGame | small | LFM2 350M fine-tuned on StepGame spatial reasoning (16% → 70% accuracy) |
| 7 | LFM2.5 VL 450M Extract | small | 450M vision model — image + field list → structured JSON, edge-tier extraction |
| 7 | Falcon H1R 7B | medium | Transformer-Mamba hybrid, AIME 88.1%, LiveCodeBench 68.6%, 256K context |
| 8 | LFM2 350M NK Risk | small | LFM2 350M fine-tuned on GDELT NK military events — geopolitical risk analyst |
| 8 | Jan-code 4B | small | Code-focused 4B model by Jan, 2.72GB Q4_K_M |
| 8 | Qwen3.5 7B | medium | Multilingual (201 langs), 256K context, thinking mode, Apache 2.0 |
| 13 | GPT-OSS 20B | reasoning | MoE (21B params / 3.6B active), function calling, agentic operations |
Live latency / throughput → benchmarks.html. Architecture notes for select models (growing set) → docs/models/.
Run it
make inference MODEL=qwen # launch one model (GitHub Actions → Cloudflare tunnel)
make down # cancel all in-flight inference runs
make build # build the frontend (regenerates services.json from config)
Launch / manage the whole fleet from the browser: open app/chat/frontend/public/admin.html. Local frontend dev: cd app/chat/frontend && npm install && npm run dev.
Internals
config/models.py is the single source of truth. Add a model = add one ModelConfig, then regenerate the derivatives — make build (services.json), make sync-workflow-choices (inference.yml), make sync-readme (this table). The CI build matrix reads the config directly, so new models are picked up automatically.
Deep architecture (workflows, data flow, layout) → docs/architecture.md. AI build-context → CLAUDE.md.
License
MIT
Similar repos
Contributors
Languages
TypeScript
68.5%
HTML
13.2%
Python
10.9%
CSS
3.1%
JavaScript
3.0%