vLLM Playground

A modern web interface for managing and interacting with vLLM servers (www.github.com/vllm-project/vllm). Supports GPU and CPU modes, with special optimizations for macOS Apple Silicon and enterprise deployment on OpenShift/Kubernetes.

🆕 vLLM-Omni Multimodal Generation

Generate images, edit photos, create speech, and produce music - all with vLLM-Omni integration.

✨ Claude Code Integration

Run Claude Code with open-source models served by vLLM - your private, local coding assistant.

✨ Agentic-Ready with MCP Support

MCP (Model Context Protocol) integration enables models to use external tools with human-in-the-loop approval.

🖼️ VLM (Vision Language Model)

Upload images and chat with vision models like Qwen2.5-VL, LLaVA, and more.

🧩 Multiple instances & backends

Run subprocess, container, and remote vLLM servers side by side; switch tabs, save configs, and manage everything from Management → Instances. See Multi-Instance Guide for details.

🆕 What's New in v0.2.0

• Decision Models (Experimental) — A brand-new sandbox tab for structured-read decisions powered by DiffusionGemma and Jev / System One. Gallery of 12 use cases across 4 tabbed categories (Noul, Choice, Score, Combination), mock-by-default evaluation, and two interactive demos (Race Lane Decider, Rubric Grading Puzzle). Fully isolated from the main Server Config flow — see Decision Models Guide.
• Remote Jev endpoint — Connect to an external Jev API (e.g. https://api.typesafe.ai) with API key authentication. Credentials are validated at connect time with a probe request, not just a reachability check.
• 5 new use cases from TypeSafe cookbooks — LLM Guardrails, Content Moderation, RAG Passage Reranking, Citation Verification, and Confidence-Gated Model Routing, organized into a tabbed category gallery with count badges.
• Security hardening — Debug endpoints gated behind explicit opt-in (CWE-862); SQL injection template literal fix.

v0.1.9 added CI/CD pipeline, configurable container images, and remote mode fixes; v0.1.8 added multi-instance backends — see Changelog for the full history.

---

🚀 Quick Start

# Install from PyPI
pip install vllm-playground

# Pre-download container image (~10GB for GPU)
vllm-playground pull

# Start the playground
vllm-playground

Open http://localhost:7860 and click "Start Server" - that's it! 🎉

CLI Options

vllm-playground pull                # Pre-download GPU image (NVIDIA)
vllm-playground pull --nvidia       # Pre-download NVIDIA GPU image
vllm-playground pull --amd          # Pre-download AMD ROCm image
vllm-playground pull --tpu          # Pre-download Google TPU image
vllm-playground pull --cpu          # Pre-download CPU image
vllm-playground pull --all          # Pre-download all images
vllm-playground --port 8080         # Custom port
vllm-playground stop                # Stop running instance
vllm-playground status              # Check status

---

✨ Key Features

---

📦 Installation Options

📖 See Installation Guide for detailed instructions.

---

🔧 Configuration

Tool Calling

Enable in Server Configuration before starting:

1. Check "Enable Tool Calling"
2. Select parser (or "Auto-detect")
3. Start server
4. Define tools in the 🔧 toolbar panel

Supported Models:

• Llama 3.x (llama3_json)
• Mistral (mistral)
• Qwen (hermes)
• Hermes (hermes)

Claude Code Integration

Use vLLM to serve open-source models as a backend for Claude Code:

1. Go to Claude Code in the sidebar
2. Start vLLM with a recommended model (see tips on the page)
3. The embedded terminal connects automatically

Requirements:

• vLLM v0.12.0+ (for Anthropic Messages API)
• Model with native 65K+ context and tool calling support
• ttyd installed for web terminal

Recommended Model for most GPUs:

meta-llama/Llama-3.1-8B-Instruct
--max-model-len 65536 --enable-auto-tool-choice --tool-call-parser llama3_json

Note: This integration demonstrates using vLLM as a backend for Claude Code. Claude Code is a separate product by Anthropic - users must install it independently and comply with Anthropic's Commercial Terms of Service. vLLM Playground provides the terminal interface only.

MCP Servers

Connect to external tools via Model Context Protocol:

1. Go to MCP Servers in the sidebar
2. Add a server (presets available: Filesystem, Git, Fetch, Time)
3. Connect and enable in chat panel

⚠️ MCP requires Python 3.10+

CPU Mode (macOS)

Edit config/vllm_cpu.env:

export VLLM_CPU_KVCACHE_SPACE=40
export VLLM_CPU_OMP_THREADS_BIND=auto

Metal GPU Support (macOS Apple Silicon)

vLLM Playground supports Apple Silicon GPU acceleration:

1. Install vllm-metal following official instructions
2. Configure playground to use Metal:
   • Run Mode: Subprocess
   • Compute Mode: Metal
   • Venv Path: ~/.venv-vllm-metal (or your installation path)

See macOS Metal Guide for details.

Custom vLLM Installations

Use specific vLLM versions or custom builds:

1. Install vLLM in a virtual environment
2. Configure playground:
   • Run Mode: Subprocess
   • Venv Path: /path/to/your/venv

See Custom venv Guide for details.

---

📖 Documentation

Getting Started

• Installation Guide - All installation methods
• Quick Start - Get running in minutes
• macOS CPU Guide - Apple Silicon CPU setup
• macOS Metal Guide - Apple Silicon GPU acceleration
• Custom venv Guide - Using custom vLLM installations

Features

• Features Overview - Complete feature list
• Multi-Instance Guide - Multiple backends, tabs, and Instances management
• Gated Models Guide - Access Llama, Gemma, etc.

Deployment

• OpenShift/K8s Deployment - Enterprise deployment
• Architecture Overview - System design
• Container Variants - Container options

Reference

• Troubleshooting - Common issues
• Performance Metrics - Benchmarking
• Command Reference - CLI cheat sheet

Releases

• Changelog - Version history and changes
• v0.2.0 - Decision Models (Experimental), remote Jev endpoint, category gallery
• v0.1.9 - CI/CD pipeline, configurable container images, remote mode fixes
• v0.1.8 - Multi-instance backends & remote polish
• v0.1.7 - Hotfix & tutorials
• v0.1.6 - Observability dashboard, PagedAttention visualizer, token counter, logprobs
• v0.1.5 - Remote server, VLM vision support, markdown rendering
• v0.1.4 - vLLM-Omni multimodal, Studio UI
• v0.1.3 - Multi-accelerators, Claude Code, vLLM-Metal
• v0.1.2 - ModelScope integration, i18n improvements
• v0.1.1 - MCP integration, runtime detection
• v0.1.0 - First release, modern UI, tool calling

---

🏗️ Architecture

┌──────────────────┐
│   User Browser   │
└────────┬─────────┘
         │ http://localhost:7860
         ↓
┌──────────────────┐
│   Web UI (Host)  │  ← FastAPI + JavaScript
└────────┬─────────┘
         │
    ┌────┴────┐
    ↓         ↓
┌───────-─┐ ┌────────┐
│ vLLM    │ │  MCP   │  ← Containers / External Servers
│Container│ │Servers │
└────────-┘ └────────┘

📖 See Architecture Overview for details.

---

🆘 Quick Troubleshooting

📖 See Troubleshooting Guide for more.

---

🔗 Related Projects

• vLLM - High-throughput LLM serving
• Claude Code - Anthropic's agentic coding tool
• LLMCompressor Playground - Model compression & quantization
• GuideLLM - Performance benchmarking
• MCP Servers - Official MCP servers

---

📝 License

Apache 2.0 License - See LICENSE file for details.

🤝 Contributing

Contributions welcome! Please see CONTRIBUTING.md for setup instructions and guidelines.

---

Made with ❤️ for the vLLM community