kime

kime (決め, "the decision") answers typed questions about text. You give it a state, which can be an email, a support ticket, a JSON document or what an agent sees on a web page, and a set of questions, each one a choice between options, a score on an ordered scale, or a yes or no statement. It returns a calibrated probability for every possible answer. It generates no text, so there is nothing to parse and nothing to hallucinate.

It is written in Rust. It covers every feature of TypeSafe's Jev and of Laya, speaks both of their wire formats, and is built to be ten times faster and cheaper than the best of them on every speed and cost benchmark while matching or beating them on accuracy and calibration.

The design is in spec/. spec/01-overview.md is the place to start.

Status

Early. kime runs the published Laya checkpoints on the CPU, CUDA and Apple GPUs with the same answers as Laya, and kime serve answers the System One API over HTTP. The native kime models, the router, the caches and the SDKs are still to come. The table below is what the project has to hit, not what it does today. When a row is measured, the number goes here with the command that produced it.

What it looks like

kime pull laya
kime serve --port 8000 &
curl -s localhost:8000/v1/systemone -H 'content-type: application/json' -d '{
  "model": "kime-latest",
  "state": "Hi, we were billed twice for March. Refund the duplicate today or we cancel.",
  "questions": {
    "team":   {"type": "choice", "instructions": "Which team should handle this",
               "criteria": {"billing": "Payment issues", "technical": "Bugs", "sales": "Plans", "other": "Anything else"}},
    "urgency": {"type": "score", "instructions": "How urgent is it",
               "criteria": ["not urgent", "this week", "today", "right now"]},
    "churn":  {"type": "noul", "instructions": "The customer threatens to leave"}
  }
}'

It answers {"model": "laya", "answers": {"team": {"type": "choice", "choice": "billing", "confidence": 0.95, "probabilities": {"billing": 0.96, ...}}, ...}, "usage": {"input_tokens": 150, "output_tokens": 0}}. A request that names a Laya model or no model at all gets laya-serve's response shape instead, so Laya clients work unchanged too. The same request works against Jev's API unchanged, and Laya's Python API works against the kime package with an import change. spec/03-api.md has the full wire format.

The bar

Ten times the best baseline on the same hardware for every speed and cost row, and equal or better for every quality row. The baselines are Laya 0.3.7 on its own hardware, laya-mlx and laya-coreml on Apple silicon, and Jev as hosted. The full scorecard, with the reason next to the one row that cannot reach 10x, is in spec/13-benchmarks.md.

How

The engine alone, running Laya's own weights, is worth three to five times: unpadded variable length attention, fused kernels, CUDA graphs per shape bucket, and no framework dispatch on the hot path. The rest comes from the model. Laya re-encodes the whole state once per question. kime-v1 encodes the state once, caches its keys and values, and answers every question with a small tower that cross attends into that cache, so ten questions cost little more than one. The options are permutation equivariant by construction, which is where the order flip rate comes from. spec/05-model.md has the architecture and spec/07-engine.md the runtime.

The repositories

The Python package lives in python/ for now. It and the TypeScript and Swift SDKs will get repositories of their own under tamnd/kime-* once they need their own release cadence.

Building

cargo build --release
./target/release/kime doctor
cargo xtask ci

Rust 1.90 or newer. The CUDA, Metal and Neural Engine backends need their toolkits and are off by default. CONTRIBUTING.md has the rest.

License

Apache-2.0. See LICENSE-APACHE.