Most AI kernel demos optimize a snippet. KDA-Pilot optimizes the parts that actually show up in SGLang diffusion and LLM serving workflows, then keeps the evidence needed to tell whether the agent really improved the production path.

If you care about autonomous CUDA/Triton/CuTe-DSL optimization that can be replayed, reviewed, and compared against real framework baselines, this is the repo to watch.

Why It Matters

• Real workloads, not toy shapes. Diffusion tasks were built from 20 real SGLang diffusion models and collapsed into per-kernel multi-shape workloads.
• Wall-time metrics. The headline numbers include Python, dispatch, wrappers, kernel launch, and cuda.synchronize() overhead, not just isolated device time.
• No reward-hacking path. Baseline and candidate use matching local ABIs; the task does not monkey-patch or import SGLang at runtime.
• Knowledge-guided iteration. Tasks can pull from KernelWiki and ncu-report-skill, so prior Blackwell/Hopper kernel work and NCU bottleneck evidence become part of the optimization loop.
• Agent loop with review. Candidate promotion is tied to correctness gates, run logs, and code review rather than "one fast row wins".

B200 Diffusion Results

These are B200 task outcomes against the corresponding SGLang/Triton/CuTe-DSL baselines where the task has a stable production-relative baseline. The measurements include dispatch and synchronization overheads, so they are closer to what a user sees from the public kernel path. Rows marked review are kept visible because they turned into SGLang-facing fast paths, but their original task headline changed after the baseline moved.

KernelWiki-Guided Highlights

The companion write-up records the benchmark interpretation, kernel-specific optimization paths, KernelWiki/reference links, and AKO4X comparison: KDA-Pilot optimizing SGLang Diffusion Kernel.

What Is Inside

diffusion/    SGLang diffusion-operator kernel tasks.
              Each task owns a copied baseline, optimized solution, benchmark,
              correctness contract, run logs, and result ledger.

llm/          SGLang autoregressive-model kernel-workflow campaign.
              Serve priority models on B200/H200, benchmark low/mid/high
              concurrency, profile forward passes, and turn >=1% kernel
              interfaces into flat task directories with per-task launchers.

external/     Optional shared knowledge submodules.
              KernelWiki/         Blackwell/Hopper kernel design references
              ncu-report-skill/   Nsight Compute profiling/report helper

Start with:

• diffusion/README.md for standalone diffusion kernel tasks and benchmark rules.
• llm/README.md for the LLM kernel-workflow campaign.
• diffusion/docs/standalone_diffusion_benchmark.md for the baseline/candidate benchmark contract.
• diffusion/docs/diffusion_kernel_rules.md for correctness, fallback, and promotion guardrails.

Task Lifecycle

Every diffusion kernel task follows the same shape:

prompt.md       task card for the agent
config.toml     benchmark/build defaults
baseline/       copied upstream SGLang baseline source
solution/       optimized candidate source
bench/          standalone benchmark and correctness harness
docs/           run logs, profile notes, source notes, decision ledger

The important rule is symmetry: the agent must compare the copied baseline and candidate through matching local interfaces, fixed workload rows, preallocated outputs, CUDA-event timing, interleaved A/B sampling, strict correctness checks, and full provenance.

Run A Task

Clone submodules when you want the optional knowledge references:

git submodule update --init --recursive

Launch a task from the repo root:

diffusion/scripts/launch_kernels/k03_b200_diffusion_qknorm_rope__multi_shape.sh

Useful environment switches:

KDA_NO_CLAUDE=1                 # prepare the worktree without launching an agent
KDA_BASE_BRANCH=<ref>           # launch from a specific committed ref
KDA_BASH_BIN=/opt/homebrew/bin/bash

macOS /bin/bash 3.2 is rejected by the launcher because nested Humanize/Codex hooks rely on modern Bash behavior.

How I Drive The Agents

These campaigns are run with coding agents in full-autonomy mode. For reproducibility, here is exactly how I launch them on a prepared task worktree.

Claude Code — the same combo the diffusion launcher uses (Opus 4.8 + max effort + auto/bypass-permission mode, i.e. the "Effort (Max)" + "Auto mode" toggles in the Claude Code UI):

claude --permission-mode bypassPermissions --model opus --effort max

diffusion/scripts/launch_kda_kernel_task.sh invokes exactly this (defaults CLAUDE_MODEL=opus, CLAUDE_EFFORT=max, --permission-mode bypassPermissions), so launching a task and running Claude by hand are equivalent. To iterate a long-running goal across model switches, resume the session:

claude --resume <session-uuid>

Ultracode mode — the maximum-thoroughness setting (the "Effort (Ultracode – xhigh + workflows)" entry in the Claude Code effort menu, paired with Auto mode). Ultracode is not a launch-flag effort value: claude --effort ultracode warns (Unknown --effort value 'ultracode') and falls back to the default — the valid --effort flag levels are low, medium, high, xhigh, max. Ultracode is a composite of xhigh effort + dynamic workflows enabled, so the agent reasons at xhigh and authors multi-agent workflows on substantive tasks. Enable dynamic workflows once — the Dynamic workflows toggle in /config (settings key enableWorkflows) — then launch at xhigh:

claude --permission-mode bypassPermissions --model opus --effort xhigh

Or do it in one self-contained command:

claude --permission-mode bypassPermissions --model opus --effort xhigh --settings '{"enableWorkflows": true}'

With workflows enabled, the in-session /effort menu shows "Ultracode" and selecting it pins xhigh. To opt a single prompt in instead of the whole session, include the keyword ultracode in that message.

Codex — full-access, no approval prompts:

codex --yolo --sandbox danger-full-access --ask-for-approval never

Both run unsandboxed / auto-approve because each task lives in an isolated prepared worktree with its own benchmark + correctness gates; an agent's wins only count when the correctness contract and run logs agree.

Current Campaigns

• Diffusion kernels: qk norm + RoPE, norm inference, rotary embedding, fused scale/shift, group norm + SiLU, attention concat/copy, causal Conv3D cat/pad, residual gate/add, CuTe-DSL norm/tanh/mul/add, and CuTe-DSL norm/scale/shift across B200 and H200 task folders.
• LLM kernel workflow: model-level serving commands, benchmark sweeps, kernel API logs, and 200+ flat task directories with per-task launchers for future optimization loops.
• Open frontier: compute-bound kernels such as FA4/MHA and GEMM-like paths remain harder; this repo keeps the failed and partial attempts visible so the next loop can start from evidence instead of folklore.

Acknowledgments

The "KDA" in KDA-Pilot stands for Kernel Design Agents — the agent methodology these campaigns build on. Credit to the original project from MIT Han Lab: mit-han-lab/kernel-design-agents. The SGLang serving-kernel tasks here are driven by a fork, BBuf/kernel-design-agents (wired into each task's prompt.md), and the bundled external/KernelWiki knowledge base likewise credits the original KDA project.

Star History