ClawedCommand

An isometric 2D RTS where you command armies through both traditional point-and-click micro and natural language instructions to a fine-tuned LLM that writes Lua strategy scripts in real-time. Built in Rust with Bevy 0.18.

The Premise

After the singularity, humanity uploaded themselves, chose animal forms, and promptly forgot why. Now six factions of cute-but-deadly critters wage war across a post-digital landscape, each guided by a rival AI. You play as catGPT — a cat army advised by the AI agent Le Chat — pursuing world domination one laser pointer at a time.

Light comedic tone. Mechanically serious.

What Makes This Different

Hybrid control. You can click-to-move like any RTS, or open Construct Mode and describe what you want in plain English. The AI generates a Lua combat script, binds it to a voice keyword, and you shout "focus fire!" during battle to execute it. Both input paths flow through the same deterministic command system — player commands always override AI suggestions.

Voice commands. Push-to-talk triggers an on-device keyword classifier (TC-ResNet8, 119 classes, runs in a background thread with zero frame impact). Recognized keywords execute bound Lua scripts instantly.

AI that learns. 49 generations of automated arena matches evolved the combat scripts from 0% to 95% win rate. The best strategies (centroid focus fire, conditional kiting, terrain-aware retreat) are baked into starter scripts that ship with the game.

Factions

Architecture

A Cargo workspace with six crates:

crates/
├── cc_core      Engine-agnostic types: components, commands, coords, map, terrain, fixed-point math
├── cc_sim       Bevy ECS simulation: 18-system FixedUpdate chain at 10Hz, deterministic lockstep
├── cc_client    Bevy app: isometric renderer, input handling, full HUD, camera, VFX
├── cc_voice     On-device voice recognition: Silero VAD + TC-ResNet8 classifier (ONNX)
├── cc_agent     AI layer: Lua runtime (mlua), ScriptContext API, LLM client, MCP tools, arena trainer
└── cc_harness   Headless sim wrapper + MCP server (35 tools via rmcp) for testing and AI training

Simulation

The simulation runs in FixedUpdate at 10 ticks/second with a strict system ordering:

tick → commands → abilities → status_effects → auras → stat_modifiers → production →
research → gathering → target_acquisition → combat → tower_combat → projectiles →
movement → builder → grid_sync → cleanup → victory

All math uses FixedI32<U16> for deterministic replay. Faction-aware A* pathfinding with terrain costs and elevation modifiers.

Rendering

Full isometric 2D renderer with:

• Zoom LOD: Tactical view (< 2.0x) shows full sprites, health bars, VFX. Strategic view (>= 2.0x) switches to colored-dot icons with inverse-scaled labels.
• Animation: 4-frame sprite sheets (idle/walk/attack) driven by ECS state
• VFX: Lightweight particle system (200 cap) with trail + impact bursts
• Fog of War: Per-tile visibility overlays
• Minimap: Click-to-jump overview with unit dots
• Autotile terrain: Borders, water animation, terrain atlas

AI Agent

The AI operates as a code generator above the runtime, not an in-loop decision maker:

1. Player describes intent in natural language
2. Local LLM (Qwen3-Coder-30B-A3B via Ollama) generates a Lua script
3. Script executes via ScriptContext with a 500-point compute budget
4. Available primitives: 25+ query methods, 15+ command methods, 20 composable behaviors
5. Scripts persist in a library, bindable to voice keywords

Voice Pipeline

Three-thread architecture (audio capture → VAD → classification), entirely on-device:

• VAD: Silero v5 (2.3MB ONNX) detects speech segments
• Classifier: TC-ResNet8 (262K params, 1MB ONNX), 99.8% validation accuracy
• Push-to-talk: V key, with visual feedback on the HUD

Campaign

A 23-mission narrative campaign across a Prologue and 5 Acts, with 4 branching endings.

Protagonist: Kelpie, a young otter who can simultaneously interface with all 6 faction AIs — making them the most dangerous individual alive.

Named heroes: Commander Felix Nine (catGPT), Marshal Thimble (The Clawed), Mother Granite (Seekers), Rex Solstice (The Murder), King Ringtail (LLAMA), The Eternal (Croak).

Mission definitions are RON files in assets/campaign/ with inline maps, hero spawns, wave definitions, objectives, and mutators (LavaRise, ToxicTide, VoiceOnlyControl, etc.).

Economy

Server Racks increase your AI action rate cap. Destroying enemy racks degrades their AI.

Running

# Standard game
cargo run -p cc_client

# Demo modes
cargo run -p cc_client -- --demo canyon       # Canyon battle scenario
cargo run -p cc_client -- --demo canyon 3     # Canyon with hero units
cargo run -p cc_client -- --demo showcase     # Building showcase
cargo run -p cc_client -- --demo cutscene 1   # Faction cutscene with dialogue
cargo run -p cc_client -- --demo voice        # Voice command demo
cargo run -p cc_client -- --demo match        # AI mirror match

Controls

AI Arena (training)

cargo run -p cc_agent --bin arena --features harness -- \
  --seeds 1,2,3 \
  --p0-scripts training/arena/gen_042/player_0/ \
  --shared-scripts training/arena/gen_042/player_1/

MCP Server (for LLM integration)

cargo run -p cc_harness

Exposes 35 tools (11 query, 10 command, 6 behavior, 8 sim-control) over the Model Context Protocol.

Test Suite

592+ tests across all crates:

cargo test --workspace

Training

Lua Script Evolution

The training/arena/ directory contains 49 generations of AI script evolution. Key discoveries:

• Group focus fire (centroid-based, all attackers target same enemy) is the single most impactful behavior
• Conditional kiting for ranged units when outnumbered prevents army loss without causing stalemates
• Terrain-aware retreat checks movement_cost and tries perpendicular escape routes when the flee path is blocked
• Closest-to-centroid targeting is critical — switching to weakest/lowest-HP targeting is catastrophic (20% win rate)

LLM Fine-Tuning

Previous fine-tuning was done on Devstral Small 2 (24B) with LoRA. Now using Qwen3-Coder-30B-A3B (MoE, 3B active params) via Ollama for faster inference. Historical adapter in training/lora_checkpoints/.

Voice Model

TC-ResNet8 trained via knowledge distillation from a larger teacher model. 2975 synthetic TTS samples, 119 keyword classes. Pipeline in training/voice/.

Asset Pipeline

tools/asset_pipeline/ — a Python pipeline that orchestrates sprite generation, post-processing (background removal, palette normalization, sheet slicing), and atlas manifest generation. Art style: Into the Breach meets Redwall — clean, minimal, readable, cute animals with tactical clarity.

Project Documentation

Tech Stack

• Engine: Bevy 0.18 (Rust)
• Fixed-point math: fixed crate (FixedI32<U16>)
• Scripting: mlua (Luau sandbox)
• AI inference: Qwen3-Coder-30B-A3B via Ollama (OpenAI-compatible API)
• Voice: ONNX Runtime (Silero VAD + TC-ResNet8)
• Audio capture: cpal
• MCP server: rmcp 0.17
• Map format: RON
• Asset processing: Python (Pillow, rembg, NumPy)