Adventure Games with Local LLMs

Two text-adventure games inspired by the classic "Colossal Cave Adventure," each taking a fundamentally different approach to using AI.

Author: Jonathan M. Rothberg (@jmrothberg)

---

Browser demo (no install)

Run WebGPU Text-to-Image Compare in your browser — compare SD-Turbo, Janus-Pro-1B, and SD 1.5 multi-step side by side. Runs entirely in-browser via ONNX Runtime Web + WebGPU. Requires Chrome/Edge 113+ with WebGPU. First run downloads ~2 GB per model (cached).

---

Two Games, Two Philosophies

colossal_cave/ — Code-Driven Adventure

A traditional text adventure where the code drives the gameplay. The game has 25+ hand-designed rooms, monsters, treasures, riddles, and NPCs loaded from a JSON data file. MLX folders under ~/MLX_Models (or optional Ollama) parse commands and drive NPC/MB chat; local Diffusers models (FLUX, Z-Image, etc.) create artwork; Pyramid Flow generates optional room videos.

• Structured, predictable gameplay with rich pre-built content
• Apple Silicon Mac: load any ~/MLX_Models/<folder> directly via mlx-vlm / mlx-lm — Ollama not required
• Works on Apple Silicon Mac (MPS) or Linux with NVIDIA GPU (CUDA)
• Multi-GPU support for distributing LLM, diffusion, and video across GPUs
• See colossal_cave/README.md for setup and details

llm_adventure/ — LLM-Driven Adventure

A procedural adventure where the LLM is the game master. Instead of pre-defined rooms, the AI dynamically creates the entire world -- rooms, NPCs, items, puzzles, and narrative -- guided by a "World Bible" theme. The engine sends the LLM a curated game state each turn; the LLM responds with narration plus JSON directives (move_to, room_take, place_items, etc.) that the engine executes to update the game.

• Every playthrough is unique -- the LLM creates content on the fly
• As local LLMs improve (better JSON, smarter tool use), the game automatically gets better with no code changes
• Runs natively on Apple Silicon via MLX-LM (language) and MFLUX (images)
• See llm_adventure/README.md for setup and details

---

Quick Start

git clone https://github.com/jmrothberg/Collosol-Cave-with-local-LLM.git
cd Collosol-Cave-with-local-LLM
chmod +x scripts/startup.sh
./scripts/startup.sh --setup    # install deps (creates .venv, mlx-vlm on Apple Silicon)
./scripts/startup.sh            # install + launch Colossal Cave

Apple Silicon (recommended): put MLX weights in ~/MLX_Models/<folder-name>/ (must include config.json). Pick that folder name in the game UI — no Ollama needed. For gemma-4-12B-it-8bit, use mlx-vlm ≥ 0.6.1.

Optional Ollama: ollama serve in another terminal; pick ollama:<tag> in the dropdown.

Then follow the README in whichever game folder you want to play.

Tests (Colossal Cave)

./scripts/startup.sh --test
# or manually:
source .venv/bin/activate
cd colossal_cave && COLOSSAL_CAVE_SKIP_VECTOR_MODEL=1 COLOSSAL_CAVE_HEADLESS=1 pytest -m "not gpu and not ollama and not mlx"

See colossal_cave/README.md for architecture and full test docs. GitHub Actions runs the same CI-safe suite on push.

---

Project Structure

.
├── README.md                          # This file
├── scripts/
│   └── startup.sh                     # macOS/Linux setup + launch
├── colossal_cave/                     # Game 1: code-driven + LLM support
│   ├── README.md
│   ├── colossal_cave_July_5_26.py      # Main game (MLX/Ollama + Diffusers + Pyramid Flow)
│   ├── adventure_dataRA.json          # 25+ rooms, NPCs, monsters, riddles
│   ├── diffusion_manager.py           # Image generation interface
│   ├── complete_instruction.py        # Help system
│   └── ...                            # Video gen tools, downloaders, utilities
│
├── llm_adventure/                     # Game 2: LLM as game master
│   ├── README.md
│   ├── LMM_adventure_April_30.py     # Main game (MLX-LM + MFLUX)
│   ├── mflux_image_gen.py             # FLUX image generation (Apple Silicon)
│   └── deprecated_diffuser_server/    # Old diffusion server approach
│
├── .env.example
└── .gitignore

---

License

MIT License

---

Acknowledgments

• Inspired by the original "Colossal Cave Adventure" by Will Crowther and Don Woods
• MLX-LM, MLX-VLM, MFLUX, Ollama, Diffusers, Pyramid Flow, Gradio

---

Contact

• GitHub: @jmrothberg
• Project: Colossal-Cave-with-local-LLM