Kairo

Evidence-driven Blackwell inference research: measure a workload, prove a result, and route only what was measured.

Phase 1 results · Experiment protocol · Phase 1 article · Reproduce Phase 1 · Blog · Contributing · Apache-2.0

Why Kairo

On a single RTX 5090, CUDA Graph replay can improve NVFP4 serving throughput by more than 2x for some workloads—and by far less for another valid workload. The useful conclusion is not “always enable Graphs.” Kairo captures the conditions behind a result, passes a correctness gate, and promotes only exact measured workload buckets into a fail-closed runtime policy.

Kairo is a research workbench and the beginning of a path toward native Blackwell inference components. It is not yet a general-purpose inference engine or a replacement for vLLM, CUTLASS, or cuBLAS.

Phase 1 at a glance

Local measurements on one RTX 5090 32GB, using pinned model and software revisions. Read the linked protocols before comparing values.

The 1.30x result is intentional context: CUDA Graphs are not promoted as a global default. Phase 1 results includes the scope, evidence links, and claims Kairo does not make.

What Kairo does

• Defines versioned workload and experiment protocols.
• Verifies declarative kernel blueprints before a build or launch.
• Records cache, correctness, artifact, and launch timing separately.
• Selects serving profiles only for exact measured workload buckets.
• Fails closed to manual when no measured route covers a request.
• Probes native SM120 data movement and matrix-compute paths, including correctness-aware negative results.

Quick start

Clone the repository:

git clone https://github.com/peter941221/Kairo.git
cd Kairo

The portable contracts do not require a GPU:

$env:PYTHONPATH = "src"
python -m unittest discover -s tests -q

Validate the sample kernel blueprint:

wsl.exe -d Ubuntu-24.04 -- bash -lc 'cd /mnt/c/Projects/Kairo && bash scripts/wsl/run_lab.sh validate-blueprint --file experiments/blueprints/fp16-tma-wmma.yaml'

Ask the runtime policy about a measured route:

wsl.exe -d Ubuntu-24.04 -- bash -lc 'cd /mnt/c/Projects/Kairo && bash scripts/wsl/run_lab.sh recommend-runtime --model qwen38 --concurrency 32 --prompt-tokens 512 --context-tokens 1024 --generation-tokens 128'

GPU-serving paths require a compatible RTX 5090 environment, local model weights, and the pinned runtime described in the environment contract. Set model and environment paths through the documented KAIRO_* variables; do not commit local paths or credentials.

For the full evidence-to-serving-route procedure, including the expected route and the exact c32/1K reproduction command, see Reproduce Phase 1.

Evidence and reproducibility

Every public result should answer: under which model, quantization, workload, hardware, software revision, and correctness tolerance did it win?

• Phase 1 results is the concise public result index.
• Experiment protocol defines controls and the promotion gate.
• Baseline status is the detailed chronological lab record and includes both promoted and rejected experiments.
• experiments/protocols/ contains machine-readable result and method records.
• evidence/ contains reviewed, redacted per-repeat public records; python scripts/render_phase1_results.py regenerates the published table and SVG chart in docs/generated/.
• Public release checklist lists the artifact, review, and repository gates before public visibility changes.

Repository map

blog/                 publication drafts and article sources
docs/                 result index, protocol, environment, and release guides
experiments/          versioned workloads, protocols, and kernel blueprints
scripts/wsl/          local RTX 5090 probes, runners, and analyzers
src/kairo_lab/        dependency-light validation, cache, dispatch, and policy
tests/                portable contract tests
.kairo-local/         ignored local credentials, logs, and machine facts

Roadmap

Phase 1 — complete research baseline. Controlled protocols, correctness gates, measured Graph/eager routes, route selection, and native capability probes.

Phase 2 — native components. Use Phase 1 evidence to choose narrowly scoped data movement, layout, subgraph, or execution-path work. A candidate is not promoted without correctness, boundary coverage, repeatability, and a fair end-to-end comparison.

Contributing

See CONTRIBUTING.md. In brief: do not extrapolate measured wins, do not commit credentials or private artifacts, and retain negative results that establish a meaningful boundary.

License

Copyright 2026 Kairo contributors. Licensed under the Apache License 2.0. The license grants broad reuse rights, including an express patent grant for contributed work; it does not grant trademark rights.