Traces you can run, not just read.

Kitaru (来る, "to arrive") gives you replay-based evals for AI agents. Record or import production runs as sessions, replay them against your next model, prompt, or code change, and see what improved and what broke before you ship. Open source, self-hosted, Python and TypeScript. From the team behind ZenML.

Docs · Guided Tour · Get Started · Roadmap · Community

---

🎯 Why

Your agent has already been tested thousands of times in production. Most of that evidence is sitting in a trace store as something you can read but not run. Then you change a prompt, swap a model, or refactor a tool, and the first strong signal comes from a user who found the regression.

Kitaru turns that history into something you can test:

• Every run becomes a session. Wrap your agent once, or import the traces you already collect from Langfuse, LangSmith, Braintrust, Logfire, or Arize Phoenix. Your trace store stays your system of record.
• Replay re-executes your code. Your real agent runs again, with tool calls answered from the recording, so no card gets refunded twice. An unchanged replay gives you the faithful baseline; a forked replay shows the effect of one change.
• Evaluation starts with human judgment. Your coding assistant reviews the sessions that matter, interviews you against the evidence, and pins your answers to exact trace locations. Those judgments calibrate evaluators, cohorts freeze the population, and experiments replay the cohort against your change before it ships.

⚡ Get started

1. Install and log in. The local server is FastAPI + Postgres, and kitaru login --local provisions it with Docker:

uv add "kitaru[cli,worker,mcp]" kitaru-pydantic-ai    # or: pip install
kitaru login --local                                  # or: kitaru login <your-team-url>

2. Make your coding assistant Kitaru-capable. This is the intended way to drive Kitaru: skills teach the method, and the MCP server gives your assistant bounded operations.

npx skills add zenml-io/kitaru-skills

{
  "mcpServers": {
    "kitaru": {
      "command": "uv",
      "args": ["run", "kitaru-mcp", "--server", "http://localhost:8000", "--mode", "standard"]
    }
  }
}

3. Get an agent with traces. The fastest way to feel the loop is the template: a ready PydanticAI agent with checked-in Langfuse traces.

git clone https://github.com/zenml-io/kitaru-template
cd kitaru-template    # its README has the setup and import commands

Already have an agent in production? Import its traces from Langfuse, LangSmith, Braintrust, Logfire, or Arize Phoenix, or record with an adapter: one wrapper, no rewrite.

4. Let your assistant drive the loop. Open the repo in Claude Code, Codex, or Cursor. On the template, take the guided tour:

Use kitaru-guided-tour to walk me through Kitaru on the returns-agent template. I am new; explain each step as we go, and ask before anything paid or live.

On your own agent, run the investigation:

Use kitaru-investigation to investigate this agent and help me test one meaningful improvement. Show me the recorded evidence before asking for a judgment, and ask before creating resources or starting paid replay.

The assistant sweeps the sessions with built-in deterministic evaluators, interviews you over the ones that matter, drafts the evaluator, runs the experiment, and reports what improved and what regressed. You judge; it handles the investigation work. The tutorial walks the same loop on the template, step by step.

🔌 Languages and frameworks

Python and TypeScript SDKs talk to the same server. Adapters ship for:

Anything else still works: import your traces, write a one-page custom importer with help from an agent skill, or build a small adapter.

🔒 Self-hosted, by design

One server runs on your infrastructure, with Docker image and Helm chart included, and no user code executes on it. Replays, imports, and evaluations run on workers in your environment: your virtualenv, your credentials, your network. Traces never have to leave your systems. Apache 2.0, no mandatory SaaS control plane.

📚 Learn more

🤝 Contributing

We're happy to take contributions from outside the core team. Comment on an existing issue or open a new one before you write code; direct PRs are limited to collaborators, and a maintainer will add you once we've agreed on the change. CONTRIBUTING.md has the full flow, dev setup, and code style. The default branch is develop; all PRs should target it.

💬 Community and support

• Slack community: ask questions, share ideas
• Report a bug: goes straight to GitHub issues
• support@kitaru.ai: when email is easier

Hit something broken? Any of the three reaches a human. An issue with a session ID attached gets fixed fastest.

📄 License

Apache 2.0