Pie runs small user-supplied WebAssembly programs, called inferlets, directly next to the model. Inferlets have direct access to the KV cache and forward pass, so agent loops, tool calls, custom samplers, and cache policies are customized per application without modifying the engine.

Note Pie is pre-release software under active development.

Quick Start

Pie is a standalone binary, no Python needed. For Windows, see the installation guide.

curl -fsSL https://pie-project.org/install.sh | bash

pie config init
pie model import Qwen/Qwen3.5-0.8B
pie serve

A checkpoint is matched against the catalog's import contracts at load and refused by name when none fits; pie model list prints the SKU beside every snapshot it can see.

pie serve holds the terminal. From another shell, submit an inferlet with the Python client (pip install pie-client):

pie-client submit text-completion -- --prompt "The capital of France is"

pie run is the same round trip without a server:

pie run --path ./target/wasm32-wasip2/debug/text_completion.wasm \
        --manifest ./Pie.toml -- --prompt "The capital of France is"

Backends

Four engines serve today, each a compile-time feature:

cargo build --release -p pie --bin pie --features cuda    # NVIDIA, Linux/Windows
cargo build --release -p pie --bin pie --features metal   # Apple silicon, macOS
cargo build --release -p pie --bin pie --features vulkan  # any Vulkan 1.2 device
cargo build --release -p pie --bin pie --features wgpu    # WebGPU: Vulkan, Metal or D3D12

vulkan compiles Slang to SPIR-V at build time and wants slangc on PATH (or PIE_SLANGC); wgpu needs no shader toolchain.

Building inferlets

Inferlets compile to the wasm32-wasip2 component target:

rustup target add wasm32-wasip2
cargo build --target wasm32-wasip2

Getting Help

GitHub Issues and GitHub Discussions.

License

Apache License 2.0. Third-party attributions: NOTICE.