One consumer desktop - 32 GB RAM, one RTX 5080, a Gen5 NVMe - and three models it has no business holding in memory, the largest about thirteen times its RAM:

The grades mean what they say: OFFICIAL is the frozen release-gate protocol on the reference machine, with the verdict stated. PROBE is a deliberate measurement with a written protocol that is not a gate run - it promotes nothing, and the Kimi figure is one you should not plan interactive work around. The token-identity check is scoped - greedy decoding on gpt-oss-120b; on Qwen3.5-122B the separate check is run-to-run reproducibility, and no parity claim is made for sampled decoding or for K2.6. On the default packed path every expert byte is verified against your original GGUF before it is ever used (the opt-in virtual path copies nothing and verifies its manifest and addresses instead), and the server never bulk-loads experts up front. Which build and conditions every number was measured under: Measurements and each release's notes.

Unedited single take (2:49): cold boot to answer on the 447 GB Kimi K2.6 - server loaded in about 19 seconds, the NVMe sustaining multi-GB/s reads, system RAM under 32 GB the whole way - then the same run on Qwen3.5-122B. Task Manager stays on screen the whole time.

What this is, honestly:

• It is for MoE models only. Dense models gain nothing here, and a model that already fits in your RAM does not need this.
• It is a hands-on preview, not a one-click app. You bring your own GGUF; MoE-Direct never downloads weights. The rough edges are written down in the docs rather than hidden.
• Today it ships for Windows with an NVIDIA GPU (CUDA). Linux, macOS and more backends are where this is going - see the roadmap.

The exact boundary — what is changed and what is provably not touched (the math, the routing, the weights, your files) — is stated once, in How it works. Known rough edges live in Limitations and FAQ.

Quick start

You need Windows 10/11 x64, an NVMe SSD, disk space of about twice the model size on the default packed path (the opt-in virtual repack needs the model plus a manifest and its plan report, measured at about 6 MB for the 122B test model and about 16 MB for the 397B one), and a GGUF from the supported list.

1. Download the runtime zip (moe-direct-<version>-win-x64.zip) and SHA256SUMS.txt from Releases, right-click the zip, Properties, tick Unblock, then extract with Windows "Extract All" into a new, empty folder. Checking the download is one paste, not a hex comparison. In Explorer, open the folder that holds both downloaded files, right-click empty space > Open in Terminal (PowerShell), then paste:

   $e,$n=((Get-Content .\SHA256SUMS.txt -Raw).Trim() -split '\s+');$a=(Get-FileHash ".\$n" -Algorithm SHA256).Hash;if($a -eq $e){'OK: hash matches'}else{'MISMATCH: download again'}
   

   If you skip it, the launcher's own sealed-manifest check still catches files changed or corrupted inside the extracted bundle on every start - what it cannot do is check the zip you just downloaded against the published checksum; that is what this paste is for. A checksum from the same page proves the download is intact, not who published it - releases are unsigned for now, so publisher trust rests on the GitHub account and the HTTPS path.

2. Download every shard of one exact tested GGUF and revision from docs/models.md, and keep all shards in one folder. Place your GGUF under <drive>:\moe-models\<any-folder>\ and double-click Start-MoeDirect.cmd. Pick your model with the arrow keys.

3. Approve the one-time repack (the launcher shows the exact disk cost and time before writing anything), press Enter when the status screen appears, and connect any OpenAI-compatible client to the printed URL.

That is the whole loop. The first run repacks once (minutes to ~18 minutes here, with live progress); every later run goes straight to serving. A cold session's first conversation is the slowest by design; later turns reuse the prefix cache, and a restored exact prefix can skip that first-prefill entirely.

Prefer watching first? The setup walkthrough is a single real-time take with chapters, including the waits.

Windows SmartScreen will warn you: these preview builds are unsigned, so "Windows protected your PC" is the expected message, not a verdict on your download. Verify the SHA-256 and decide for yourself. We never ask you to disable Defender or SmartScreen. Details, including managed-PC cases, are in Getting started.

Full detail for every step, including what success looks like on screen: docs/getting-started.md. Connecting a chat client or an agent: docs/clients.md.

Supported models

reference-validated means both the byte-exact format gate and the frozen performance gate passed on the reference machine; format-validated means the byte-exact gate passed and any speed number is an observation. An unlisted GGUF of a known architecture can also be served through the experimental template path, clearly labelled.

Exact repositories, pinned revisions, minimum cache budgets, prefetch states and the template path: docs/models.md.

Measured results

Every number ships with the conditions it was measured under; these are the headlines.

Results scale primarily with NVMe read throughput; treat them as data points from the reference machine (32 GB RAM, one RTX 5080, Gen5 NVMe), not as promises for yours. Which build each number came from, the full protocols and the grading rules: docs/measured-results.md, TECHNICAL.md and each release's notes.

Roadmap

Work ships one piece per release, when it is measured, not on a schedule.

• v0.3 shipped as a preview; v0.3.1 is what we are building now. Up through v0.2.3, running a model here meant a one-time repack that wrote a second packed copy of the experts — your disk paid the model's size again, just to get the layout the engine wanted. From v0.3 the repack can be virtual: a small manifest, no data moved, space cost exactly 1.0x, reading experts straight out of the file you already have. And it ships with measured numbers, not promises — in a preregistered A/B, prefetch (on by default for the two catalog rows that carry it, the 122B test model among them) made in-place decode about 14% faster at under 2% extra bytes read. That number is a PROBE: it compares in-place with prefetch against in-place without it, not against the packed path, and it was measured on a working tree that predates the release binary rather than re-run on this zip's.
• v0.3 is a preview, on purpose. The honest part: the virtual path is still slower than the packed path today — the experts sit scattered through the original file, so fetching them costs more, and prefetch claws back only part of that. The packed path stays the default and gives up nothing. One rough edge belongs here too: on a virtual run the launcher's status screen still reports the packed path's rows, because it does not yet tell the two modes apart on that screen. copy integrity reads PASS although a virtual repack copies nothing; serving validation and the reference numbers are the packed profile's; and a -Repro or -Smoke run can go as far as printing performance gate : PASS, which is a packed verdict on a path that is not under that gate at all. What actually checks a virtual repack is a separate 8-item plan gate and the engine's own re-derivation, and that screen shows none of it. These gaps are known, measured, and are exactly what the next release exists to close.
• Next up, v0.3.1: the fixes land here. The codebase gets its cleanup pass first — readability and structural work, including the boundary contract the later pieces build on — then the speed recovery work on the in-place path, and adaptive prefetch: prefetch that derives its own starting point for any model family instead of shipping with fixed constants. The goal is an in-place repack you choose for the space, not one you tolerate.
• Then the engine gets its surgery: the engine-neutral expert-execution core — the parts this project owns (the expert store, cache, placement and prefetch) pulled behind a clean boundary, with llama.cpp as the first engine behind it.
• Separately, the speed work stays in the release queue, shipping in whatever order is ready first — a prefill path that reads each expert once per request instead of once per token (about 3x in an internal probe, not yet a published benchmark).
• And on that engine foundation, context. The question this project asked about weights — "does it fit my RAM?" becoming "does it fit my SSD?" — applies to the KV cache too. The goal is million-token context on the same consumer hardware, by tiering KV across VRAM, RAM and NVMe for the coming generation of sparse-attention models. Same contract as the experts: nothing is dropped or approximated, the memory is all there and read selectively. The first release will aim to prove it works and publish honest numbers; the finished feature set comes after, piece by piece.
• Code signing, wider hardware and OS support, further out.

What you are holding is the floor, not the ceiling.

Full detail on what has already shipped: docs/models.md, docs/measured-results.md, docs/warm-start.md and docs/disk-layout.md.

Troubleshooting

The status= line the launcher prints, every status code and its fix: docs/troubleshooting.md.

Documentation

And if you are curious how this actually gets built: some of my working records are up in docs/design-notes/, as-is. They are in Korean — I am a Korean developer, the design and the decisions are mine, and my AI assistants draft and cross-check under my direction — but each file opens with a short English TL;DR, and the notes preserve their round-by-round review history and references (the referenced private workspace files are not included). If that is your kind of thing, have a look.

Credit and license

Built on llama.cpp (MIT), base commit 0bd0ec6 (b10057) - upstream copyright and license preserved; source releases keep all upstream notices. See THIRD_PARTY_NOTICES.md.

MoE-Direct additions (c) 2026 tmxkzm1925-max, released under the MIT License. The MoE-Direct name identifies this project and its official builds - see TRADEMARKS.md. If you use this work, please cite it (CITATION.cff). Archived releases carry a DOI: 10.5281/zenodo.21739367.

Thanks. This project was built by one person with a great deal of machine help, and the help was not incidental. Anthropic's Claude and OpenAI's GPT models did design work, implementation and, just as usefully, adversarial review of each other's output, under human direction and with every change gated by the verification this project's documents describe. Having a second and a third reader who never got tired is most of the reason the checking here is as strict as it is.

Thanks are owed as well to the teams whose models this was measured against: Qwen, DeepSeek, Moonshot AI, OpenAI for gpt-oss, and Mistral. None of them are affiliated with this project and none of them have endorsed it. They published weights that one person with one desktop could actually study, and without that there would have been nothing here to run.