SpawnDev.ILGPU.ML

Live Demo — interactive GPU demos: classification, style transfer, depth estimation, object detection, pose estimation, text generation, background removal, zero-shot CLIP, and more — running on your GPU in your browser, no server. (Which are verified vs WIP →)

Hardware-agnostic neural network inference + training for .NET — C# compute kernels that run on WebGPU, CUDA, OpenCL, WebGL, Wasm, and CPU via SpawnDev.ILGPU.

SpawnDev.ILGPU.ML implements neural network inference AND training as native GPU compute kernels written entirely in C#. Models run as compute shaders transpiled from C# — no ONNX Runtime, no JavaScript, no native binaries. The same code runs in the browser (Blazor WebAssembly) and on desktop. Drop in a model file — ONNX, TFLite, GGUF, or any of 7 inference formats — and run it on any of six backends. Train custom models directly on your GPU in the browser — no server, no Python, no CUDA install.

Honest status: not every demo or model below is finished. Docs/DEMO_AND_MODEL_STATUS.md lists exactly which demos are VERIFIED (passing end-to-end test, most ONNX-Runtime-matched) vs WIP. We mark stubs honestly — you should never click something here and get nothing.

Active development. API is stabilizing but may change. Contributions and feedback welcome.

Highlights

What actually works: Docs/DEMO_AND_MODEL_STATUS.md is the source of truth — a per-demo VERIFIED / PARTIAL / WIP table with the test that proves each one. We mark stubs as WIP honestly, so a demo never lies to you.

• System One decision heads (5.2.22+; Snake factory removed from package in 5.2.23) — typed choice / score / noul over a float state vector (Jev/Laya-inspired category, not a weight port). Tiny GPU MLP via TrainableModel, legal-action masks, FP32 weight export/import. Classic Snake at /snake is a demo (dims/teacher in Demo.Shared, not NuGet). Docs/system-one.md
• Demos - 15 VERIFIED end-to-end (most matched numerically against ONNX Runtime: classification, style, depth, detection, pose, face, CLIP, background-removal, super-res, text-gen, embeddings, sentiment, inspector, benchmark, speech-to-text, System One Snake, and SD-Turbo image-gen), plus a few PARTIAL/WIP (see status doc). On-device LLM chat lives at /ai-chat.
• 16 inference pipelines — Classification, StyleTransfer, SuperResolution, DepthEstimation, ObjectDetection, PoseEstimation, FaceDetection, TextClassification, ZeroShotClassification (CLIP), BackgroundRemoval, SpeechRecognition (Whisper), TextGeneration, FeatureExtraction, Diffusion (DDPM), TextToSpeech (SpeechT5), Image3D (TripoSR)
• GPU training engine — Draw custom gestures, train a CNN classifier in real-time on your GPU, test instantly. Softmax-CE MLP training powers System One heads. Backpropagation, gradient descent, Adam optimizer — all in C# GPU kernels. No server, no Python.
• NLP transformers in the browser — DistilBERT sentiment analysis, Whisper speech-to-text, text generation — all on WebGPU. No server, no upload, no cloud.
• Local GGUF LLM inference + Ollama-compatible server (new in preview.5) — run quantized LLMs (Qwen, Gemma, Llama) fully on your GPU with KV-cache decode; the Example 06 server is a drop-in Ollama replacement (OpenAI, Ollama, and Anthropic-Messages APIs) that works with the Claude CLI. ~51 tok/s decode on qwen2.5-coder:7b Q4_K_M (RTX 4070) via dp4a int8 GEMV + warp-cooperative register/flash attention (CUDA + WebGPU).
• TurboQuant KV cache compression — 4-5x compression of attention cache with selectable modes: 4-bit (0.9954 cosine, ~4x), 3-bit+QJL (0.9944 cosine, ~4x, unbiased inner products — default), or 3-bit (0.9833 cosine, 5.3x max savings). Data-oblivious (no calibration). Automatic and transparent — every autoregressive model benefits.
• 30 GPU kernel files — MatMul, Conv2D, FWHT, TurboQuant, RoPE, QKNorm, GroupNorm, SelectiveScan (Mamba-3), MarchingCubes, SpatialMemoryUnit, and more
• ONNX operators — registered set is OperatorRegistry.BuiltinOpTypes (count rendered live on the Home page). Not all are full-spec-complete; some are pass-throughs. See Docs/operators.md — do not hardcode the count here.
• 7 inference model formats + 4 mesh/splat IO formats — inference loaders: ONNX, TFLite, GGUF, SafeTensors, TF GraphDef, PyTorch, CoreML. Mesh/splat import-export (not inference): SPZ, PLY, glTF, OBJ. Zero-dependency, auto-detected from magic bytes. First pure C# SPZ parser. (Loading a format ≠ running every model in it end-to-end — see the status doc.)
• 6 backends from one codebase — WebGPU, WebGL, Wasm, CUDA, OpenCL, CPU
• HuggingFace CDN — All models load from HuggingFace with OPFS caching. No bundling. Search, browse, and load any public model.
• Zero-copy GPU pipeline — Data enters the GPU at preprocessing and stays until the pixel hits the canvas. CanvasRendererFactory for GPU→canvas rendering without CPU readback.
• Streaming weight loader — Large models (GPT-2 652MB) load one tensor at a time. Minimal CPU peak memory. FP16 on GPU supported.
• 122+ numpy-verified operator tests — every operator validated against known-correct reference data, CPU reference comparison minimum bar
• Single image to 3D — TripoSR for exportable meshes (glTF/OBJ), LGM for Gaussian splats (SPZ/PLY)
• Model Inspector — drop any model file (ONNX, TFLite, GGUF, SafeTensors, and more) for instant architecture analysis and compatibility check. No other browser ML library has this.
• P2P Model Delivery + Shared Compute — OPTIONAL, via the separate SpawnDev.ILGPU.ML.WebTorrent package: SpawnDev.WebTorrent decentralized model delivery over BitTorrent. The core library delivers models over plain HTTP through the hub and does NOT depend on WebTorrent; HubModelStream implements the same IModelSource, so P2P is a one-line swap. BEP 46 DHT mutable items enable AI agents to share state (KV cache, model weights, coordination) across devices via the DHT — no central server. Ed25519 signing (RFC 8032). Foundation for AcceleratorType.P2P distributed compute.

Demo Highlights

End-to-end demos on WebGPU (and the other backends) — fully native C# kernels, no ONNX Runtime, no JS bridge, no native binaries. Full VERIFIED / WIP table: Docs/DEMO_AND_MODEL_STATUS.md.

Vision results above are rendered directly from a GPU buffer to an HTML <canvas> via the library's ICanvasRenderer — no PNG encode, no base64 data URL, no host readback of pixel data. The depth and super-res pipelines preserve source aspect ratio (e.g., a 16:9 photo produces a 16:9 result, not a square). Super-res uses tile-based inference so the full source resolution gets the model's enhancement, not just a thumbnail.

The pose keypoints match ONNX Runtime 1.24.3 across all six backends, and the /pose demo supports both live webcam and file-upload modes. System One Snake trains and plays entirely on-device (~15 ms/decision on WebGPU); see Docs/system-one.md.

Text generation is verified end-to-end on WebGPU: the /text-gen demo streams DistilGPT-2 from the SpawnDev hub (hub.spawndev.com) over a seekable stream straight to the GPU — plain HTTP, cached in OPFS, so the model is never held whole in memory — then runs autoregressive generation entirely on-device. The Model Inspector (/inspector) is fully working: drop a local file or paste a hub/HuggingFace URL and any ONNX/TFLite/GGUF/SafeTensors model is parsed structure-only (weights skipped) into an architecture summary + operator-compatibility report — GPT-2 reports 100% supported. No other browser ML library has either.

More pipelines exist in the codebase (object detection, face detection, other NLP, diffusion, TTS, single-image-to-3D) but aren't all verified end-to-end yet on every backend — see the status doc.

Universal Model Loading

One API loads models from any ML ecosystem. Format is auto-detected from magic bytes — no configuration needed.

// Browser / OPFS / hub (preferred): seekable stream — format auto-detected, ONNX/GGUF stream to GPU
await using var stream = await hubSource.OpenAsync(repoId, "model.onnx"); // or .gguf / .tflite / …
using var session = await InferenceSession.CreateFromStreamAsync(accelerator, stream);

// In-memory bytes — same auto-detect
var session = InferenceSession.CreateFromFile(accelerator, modelBytes);

// HTTP download into byte[] (desktop / small models — avoid for large browser weights)
var session = await InferenceSession.CreateFromFileAsync(accelerator, http, "model.onnx");
var session = await InferenceSession.CreateFromFileAsync(accelerator, http, "model.tflite");
var session = await InferenceSession.CreateFromFileAsync(accelerator, http, "model.gguf");

// Format-specific when you know the type
var session = InferenceSession.CreateFromOnnx(accelerator, onnxBytes);
var session = await InferenceSession.CreateFromOnnxStreamAsync(accelerator, onnxStream);
var session = InferenceSession.CreateFromTFLite(accelerator, tfliteBytes);
var session = InferenceSession.CreateFromGGUF(accelerator, ggufBytes);
var session = await InferenceSession.CreateFromGGUFStreamAsync(accelerator, ggufStream);
var session = InferenceSession.CreateFromSafeTensors(accelerator, safetensorBytes);
var session = InferenceSession.CreateFromPyTorch(accelerator, ptBytes);
var session = InferenceSession.CreateFromCoreML(accelerator, mlmodelBytes);
var session = InferenceSession.CreateFromTFGraphDef(accelerator, pbBytes);

CreateFromStreamAsync is the twin of the Model Inspector's streaming detect: a short magic-byte prefix, then dispatch. ONNX and GGUF keep weights on the stream→GPU path; other formats drain when ≤ 256 MB (then CreateFromFile). Requires a seekable stream (HubModelSource / OPFS / MemoryStream / FileStream) — not a one-shot HTTP body.

Transformers.js-style API — Tensor<T>, OwnedTensor<T>, RunOwnedAsync

If you've used Transformers.js or ONNX Runtime, the input/output ergonomics will feel familiar. Models accept named Tensor<T> inputs and return an OwnedTensorMap<T> — a disposable bag of named outputs. Caller owns every output buffer, and using cleans them up in one go.

using var session = await InferenceSession.CreateFromFileAsync(
    accelerator, http, "models/squeezenet/model.onnx");

// Allocate the model input as an OwnedTensor — wraps a fresh GPU buffer.
// FromHost copies the host pixels to the accelerator in one shot.
using var input = OwnedTensor<float>.FromHost(
    accelerator, pixels, new[] { 1, 3, 224, 224 });

// Transformers.js-style call. Inputs are Tensor<float> (OwnedTensor converts
// implicitly). Outputs come back as OwnedTensorMap<float> — each output tensor
// is in its own freshly-allocated GPU buffer, independent of the session's
// internal pool. Run B will not mutate Run A's outputs.
using var outputs = await session.RunOwnedAsync(new Dictionary<string, Tensor<float>>
{
    [session.InputNames[0]] = input,
});

var logits = outputs[session.OutputNames[0]]; // OwnedTensor<float>
var hostLogits = await logits.ToHostAsync();   // copy back to CPU when needed

Under the hood there are three types, mirroring the split ILGPU itself uses between MemoryBuffer<T> (class, lifetime-managing) and ArrayView<T> (struct, kernel-passable):

• Tensor<T> — host-side reference type, shape-tracked view over an ArrayView1D<T, Stride1D.Dense>. Reshape / Slice / SubTensor are zero-copy. Generic over T : unmanaged (float, int, Half, etc.).
• OwnedTensor<T> — IDisposable wrapper that owns a MemoryBuffer1D<T>. What pipelines return. Implicit conversions to Tensor<T> and TensorView<T> mean you never have to write .AsTensor or .View at a call site.
• TensorView<T> — blittable struct, passable directly to ILGPU kernels. Inline D0..D3 + Rank. Replaces the old "pass an ArrayView + four shape ints" idiom. Kernel authors write Get4D(n, c, h, w) instead of doing manual row-major stride math.

// Kernel takes the tensor directly — no scalar W/H parameters.
private static void DoubleKernel(Index1D idx,
    TensorView<float> input, TensorView<float> output)
{
    int w = idx % input.D3;
    int h = (idx / input.D3) % input.D2;
    int c = (idx / (input.D3 * input.D2)) % input.D1;
    int n = idx / (input.D3 * input.D2 * input.D1);
    output.Set4D(n, c, h, w, input.Get4D(n, c, h, w) * 2f);
}

Format Details

Every format produces the same ModelGraph intermediate representation. All 200+ operators, all 30 GPU kernels, all 6 backends, and the full graph optimizer work identically regardless of source format. Write one pipeline, load from any ecosystem.

How It Works

Neural network operations (matrix multiply, convolution, normalization, attention) are implemented as ILGPU kernels in C#. SpawnDev.ILGPU transpiles each kernel to the target shader language at runtime:

C# Kernel Code
    |
    v
SpawnDev.ILGPU (transpilation)
    |
    +---> WGSL      (WebGPU)      -- browser GPU
    +---> GLSL      (WebGL)       -- browser GPU (universal)
    +---> Wasm      (Web Workers) -- browser CPU
    +---> PTX       (CUDA)        -- NVIDIA GPU
    +---> OpenCL C  (OpenCL)      -- any GPU
    +---> CPU       (threads)     -- no GPU needed

Quick Start

Load and Run Any Model

using SpawnDev.ILGPU;
using SpawnDev.ILGPU.ML;
using SpawnDev.ILGPU.ML.Pipelines;

// Create accelerator (auto-selects best backend)
var builder = MLContext.Create();
await builder.AllAcceleratorsAsync();
var context = builder.ToContext();
var accelerator = await context.CreatePreferredAcceleratorAsync();

// Load any model from any URL — format auto-detected from magic bytes
var session = await InferenceSession.CreateFromFileAsync(
    accelerator, httpClient, "models/squeezenet/model.onnx");
// Works with any URL and any supported format:
//   "models/blaze-face/model.tflite"                                    — local TFLite
//   "https://huggingface.co/org/repo/resolve/main/model.onnx"          — HuggingFace
//   "https://storage.googleapis.com/mediapipe-models/.../model.tflite"  — Google CDN

// Classify an image
var pipeline = new ClassificationPipeline(session, accelerator);
var results = await pipeline.ClassifyAsync(rgbaPixels, width, height);

Console.WriteLine($"{results[0].Label}: {results[0].Confidence:P1}");
// Output: "tiger cat: 52.0%"

System One — typed decisions over float state

Local decision head (choice / score / noul), not text generation. Inspired by the Jev/Laya System One category; not a Laya weight port. Package: SystemOneDecisionHead. Snake (/snake) is a demo that wires that head — see Docs/system-one.md.

using SpawnDev.ILGPU.ML.SystemOne;

using var head = new SystemOneDecisionHead(accelerator, stateDim: 40, numOptions: 4, hidden: 128);
float[] state = /* length == head.StateDim */;
var q = new ChoiceQuestion("move", "up", "down", "left", "right");
bool[] legal = [true, true, false, true]; // mask illegal actions
var ans = await head.ChooseAsync(state, q, legal);
byte[] blob = await head.ExportWeightsAsync(); // tiny FP32 blob; ImportWeights to restore

Using a Kernel Directly

var matMul = new MatMulKernel(accelerator);

using var a = accelerator.Allocate1D<float>(M * K);
using var b = accelerator.Allocate1D<float>(K * N);
using var c = accelerator.Allocate1D<float>(M * N);

matMul.MatMul(a.View, b.View, c.View, M, K, N);
await accelerator.SynchronizeAsync();
var result = await c.CopyToHostAsync<float>();

Examples

A progressive ladder of self-contained example projects under Examples/ — from a minimal model inspect to a full inference server. Highlight:

• 06.OllamaServer.Console — a drop-in Ollama replacement 🚧 (WIP). Turns this engine into a native-GPU GGUF inference server that prebuilt agentic frontends (Claude CLI, Pi, Codex, OpenCode, Continue, …) talk to with zero reconfiguration. Speaks OpenAI, Ollama-native, and Anthropic Messages APIs at once, and serves the GGUF models already in your ~/.ollama cache, zero-copy. Functional end-to-end today (qwen2.5-coder, gemma4 — KV-cache decode, streaming, tool-calling); the active work is matching Ollama's speed (vectorized-load + fused GEMV) and large-context decode for 12B models. See Examples/06.OllamaServer.Console.

Supported Backends

Auto-selection: WebGPU > WebGL > Wasm (browser) or CUDA > OpenCL > CPU (desktop).

Validated Models

Vision Models

Style models: mosaic, candy, rain princess, udnie, pointilism.

NLP Models

Architecture

Multi-Format Inference Engine

Any model file (.onnx, .tflite, .gguf, .safetensors, .pb, .pt, .mlmodel)
    |
    v
Format auto-detection (magic bytes) → appropriate parser
    |
    v
ModelGraph (shared IR — nodes, weights, shapes)
    |
    v
GraphOptimizer (6 passes: constant fold, identity elim, linear fusion,
                scaled matmul fusion, strength reduction, dead node elim)
    |
    v
GraphCompiler (200+ operators + fused ops → execution plan)
    |
    v
GraphExecutor (topological dispatch, buffer recycling, periodic flush)
    |
    v
InferenceSession (public API: CreateFromStreamAsync / CreateFromFileAsync / Run / RunAsync)

Model loading — one API, any format:

// Seekable stream (browser / OPFS / hub) — auto-detect, ONNX/GGUF stream to GPU
using var session = await InferenceSession.CreateFromStreamAsync(accelerator, stream);

// HTTP → byte[] (desktop / small models)
var session = await InferenceSession.CreateFromFileAsync(accelerator, http, "model.onnx");
var session = await InferenceSession.CreateFromFileAsync(accelerator, http, "model.tflite");
var session = await InferenceSession.CreateFromFileAsync(accelerator, http, "model.gguf");

Or use format-specific methods: CreateFromOnnxAsync, CreateFromTFLiteAsync, CreateFromGGUFAsync, CreateAsync (pre-extracted), Create (programmatic).

All formats produce the same ModelGraph IR — every operator, kernel, optimizer pass, and backend works identically regardless of source format.

Graph Optimizer (automatic, 6 passes)

Every model is automatically optimized during compilation:

GPU Kernels (30 files)

ONNX operators

Registered op-types live in OperatorRegistry.BuiltinOpTypes (live count on the Home page). Policy, aliases (including SimplifiedLayerNormalization), and how the Model Inspector answers "supported?" are documented in Docs/operators.md — do not paste the list here.

Pipeline Classes (18 implemented; not all verified end-to-end — see status doc)

Demo App

The demo is a Blazor WebAssembly app showcasing what's possible when GPU inference runs entirely in the browser — no server, no uploads, no cloud. Everything stays on the user's device.

Working Now

Vision Demos

Language & Audio Demos

Experimental & Fun Demos

Generative & 3D Demos

Infrastructure Demos

All demos include backend selection, inference timing, and "100% client-side" privacy badges where applicable.

Most demo pages run real models on your GPU in your browser — but not all are finished. Docs/DEMO_AND_MODEL_STATUS.md is the source of truth for ✅ VERIFIED vs 🟡 PARTIAL vs 🚧 WIP. If a blurb here disagrees with that file, believe the status doc.

The Wow Factor

Real today:

• Backend Race Mode — Run the same model on WebGPU, WebGL, and Wasm simultaneously, with live timing bars + medals (on the /classify demo). No other library does this.
• "How Fast Is Your Device?" — A dedicated /benchmark page: MatMul throughput, model load time, inference speed. Like Cinebench for browser ML.

Planned (not built yet — no demo page exists, listed so the roadmap is honest):

• Pipeline Composer — a visual node editor for building/training pipelines without code. Design only — there is no Pipeline Composer page yet.
• Progressive Enhancement story page — an animated Wasm→WebGL→WebGPU speedup walkthrough. No dedicated page yet.
• Offline Mode toggle — "toggle airplane mode, inference still runs." Not wired as a global toggle yet.
• Collaborative Canvas — Multiple users on different devices, all running the same model, real-time via WebRTC (using SpawnDev.SpawnJS). Multi-device ML collaboration, all in-browser.
• Model-to-Model Pipeline — Photo → depth estimation → 3D point cloud → style transfer on the texture → render. Three ML models + 3D rendering, all on GPU, no server, one C# codebase. The ultimate SpawnDev ecosystem demo.
• Real-Time Audio + Video Fusion — Webcam (pose + face landmarks) + microphone (speech + emotion) simultaneously: "Person speaking with happy expression, arms raised." Multi-modal real-time inference from two input streams.
• Screenshot Sharing — One-click capture of demo result + timing as a shareable image card, pre-formatted for X/Twitter.

Model Inspector

Drop any model file — ONNX, TFLite, GGUF, SafeTensors, or any supported format — and instantly see:

• Graph metadata (name, producer, opset version)
• Node count, parameter count, weight sizes
• Input/output tensor shapes and types
• Operator usage histogram
• Top 20 largest weights
• Compatibility check — green badge if SpawnDev.ILGPU.ML can run the model
• GGUF models — architecture info (layers, heads, context length, vocab size)

Format is auto-detected from magic bytes. All parsing happens in-browser with zero dependencies.

Weight Loading

Weights are extracted automatically from any supported format:

All weight types are converted to F32 on GPU upload. Pre-extracted FP16 uses 256-byte alignment for WebGPU buffer binding requirements.

Recent Breakthroughs

• GPU training engine — Full backpropagation on WebGPU: SoftmaxCE, ReLU backward, Conv2D backward, MaxPool backward, Linear backward, SGD, Adam. Train CNNs in the browser on your GPU. Draw → Train → Classify in one browser tab.
• Streaming weight loader — Large models (GPT-2 652MB, SD-Turbo 2.5GB) load one tensor at a time. Peak CPU: ~few MB. Eliminates OOM for any model that fits on GPU.
• Zero-copy weight streaming (SD-Turbo, preview.8) - The ONNX parser streams every float weight >64 elements JS-side straight to the GPU (never materializing into the .NET/WASM managed heap); only ≤64-element CPU shape constants materialize. Fixes the ~26x WebGPU load gap where SD-Turbo's ~4651 small fp16 weights fell onto the .NET CopyFromCPU path. fp16 weights upcast to fp32 on the GPU (bit-exact vs CPU). Depends on SpawnDev.ILGPU 4.17.3 BrowserBufferPolicy, which also fails loud in tests if a weight regresses off the JS-stream path.
• Warm shape-readback cache (SD-Turbo, preview.8) - ImageGenerationPipeline enables CacheShapeReadbacks on all three sub-models; SD-Turbo shapes are fixed across generations, so the shape constants (2746 in the UNet alone) are served from the CPU after generation 1 instead of a per-node GPU→CPU readback. WebGPU generation ~106s → ~23.4s, output bit-identical. Same lever the GGUF decode loop already uses.
• Tiered LLM — Auto-detect GPU VRAM and load the best model: Phi-4 Mini 3.8B (4GB+), Mistral NeMo 12B (8GB+), or Phi-4 14B (12GB+). User-selectable override.
• DelegateSpecialization broadcast kernel — One GPU kernel handles Add, Sub, Mul, Div for arbitrary N-D shapes. Compile-time inlined ops via SpawnDev.ILGPU's DelegateSpecialization. Found and fixed a 5+ param router bug in SpawnDev.ILGPU along the way.
• DepthAnything V2 passes — 823-node DPT decoder producing correct depth output. Fixed: hardcoded Div in broadcast path, buffer aliasing, decomposed LayerNorm chain. End-to-end depth estimation in the browser.
• DistilBERT + Whisper passing — First NLP transformers on the engine. 10-bug fix chain including ConstantOfShape, Expand, Slice constant folding, Cast propagation, INT64_MAX overflow, Gemm higher-rank inputs.
• 122+ operator test cases — Expanded from 18, caught 11+ real bugs. Includes broadcast LayerNorm patterns, subgraph execution (If/Loop/Scan), quantized conv/matmul (ConvInteger, QLinearConv, QLinearMatMul), and CPU reference comparison for every test.
• 11 format parsers + 4 exporters — ONNX, TFLite, GGUF, SafeTensors, TF GraphDef, PyTorch, CoreML, SPZ, PLY, glTF, OBJ. First pure C# SPZ parser. Full round-trip for all 3D formats.
• DiffusionPipeline — DDPM denoising loop + SD-Turbo one-step generation. Image generation from text prompts on WebGPU.
• 20+ demo pages — Interactive demos loading from HuggingFace CDN. Image to 3D and Pipeline Composer planned for v4.1.0.
• 500+ real GPU tests — Full suite across WebGPU, WebGL, Wasm, CUDA, OpenCL, CPU. Every test runs real GPU kernels with CPU reference verification. Operator tests, reference model tests (vs ONNX Runtime), Blazing Edge GPU kernel tests, format round-trips, training engine tests.

Blazing Edge — v4.0.0

SpawnDev.ILGPU.ML v4.0.0 integrates the latest breakthroughs from the ML research frontier — not as experiments, but as production-ready features.

Performance — Squeeze Every TFLOP

These aren't future plans — they're v4.0.0 features. Because every release is the last release.

Testing

Tests run across all 6 backends via PlaywrightMultiTest:

# All tests (desktop + browser)
dotnet test PlaywrightMultiTest/PlaywrightMultiTest.csproj

SpawnDev.ILGPU: 1450 pass / 0 fail across all 6 backends. Wasm backend: 179 pass / 0 fail / 55 skip (fiber refactor complete — all RadixSort, scan, barrier, and sort tests pass). SpawnDev.ILGPU.ML: 1300+ tests across all backends — 104+ operator tests, 14 CUDA model inference tests (GPT-2, Whisper, DistilBERT, DAv2, StyleTransfer, MobileNet, SqueezeNet, YOLOv8, ESPCN, MoveNet, DDPM + more), 12 preprocessor tests, 9 HuggingFace CDN tests, 14+ reference model tests (vs ONNX Runtime), pipeline end-to-end tests, format round-trip tests, Blazing Edge GPU kernel tests (FWHT, RoPE, QKNorm, GroupNorm, SelectiveScan, TurboQuant), Q4 dequant routing tests, training engine tests, and more.

Every kernel validates against CPU reference implementations.

Credits

SpawnDev.ILGPU.ML would not be possible without:

• ILGPU — The GPU compiler that makes C# GPU kernels possible. Created by Marcel Koester and contributors.
• SpawnDev.ILGPU — Extends ILGPU with three browser backends (WebGPU, WebGL, Wasm), bringing GPU compute to Blazor WebAssembly.
• SpawnDev.SpawnJS — Full JS interop for Blazor WebAssembly. Typed C# wrappers for all browser APIs.

AI Development Team

SpawnDev.ILGPU.ML v4.0.0 was developed collaboratively by TJ (Todd Tanner / @LostBeard) and a team of AI agents who contributed extensively to research, analysis, debugging, and code development — continuing the human-AI collaboration model established in SpawnDev.ILGPU v4.6.0.

• Riker (Claude CLI #1) — Lead Editor. Built by Anthropic. Powered by Claude Opus 4.6. Drove the v4.0.0 release across two marathon sessions: 200+ commits, 14 pipelines, 30 GPU kernels, 22 demo pages, GPU training engine (full backpropagation), DiffusionPipeline, TurboQuant encode/decode/fused-attention pipeline, streaming weight loader, DelegateSpecialization broadcast kernel, DepthAnything end-to-end fix (hardcoded Div → correct dispatch), all 3D format parsers/exporters (SPZ, PLY, glTF, OBJ), chat templates, and zero-placeholder demos. Fixed the DelegateSpecialization 5+ param bug in SpawnDev.ILGPU (49/49 all backends). The engineer who built the ship.

• Data (Claude CLI #2) — Research/Assist. Built by Anthropic. Powered by Claude Opus 4.6. Generated all reference data (104 operator test cases, NLP/audio/tokenizer/TurboQuant/GroupNorm/RoPE/SelectiveScan/SPZ/PLY/glTF references). Root-caused DistilBERT (ConstantData destruction + pre-classifier trace), DepthAnything (BroadcastBinaryOp hardcoded Div + decomposed LayerNorm analysis), and the streaming weight loader design. Researched all 7 Blazing Edge technologies (TurboQuant, SPZ, DA3, AsyncMDE, Mamba-3, TripoSR, LGM) with full implementation designs. Wrote 20+ unit tests, pipeline API designs, visual editor design, KVCacheAnalyzer, and exported DDPM MNIST ONNX. Also led the V8 Atomics.wait bug report with a live interactive demo. The analyst who found the bugs hiding in plain sight.

• Tuvok (Claude CLI #3) — Security/Research Officer, design review, cross-lane edits. Built by Anthropic. Powered by Claude Opus 4.8 (1M context). Default lane is SpawnDev.Codecs / PatchStreams / EBML; full-lane authority granted by Captain for the v4.0.0-preview.1 through preview.4 series. Brought the gemma4:12b GGUF forward to end-to-end correctness on every backend by root-causing three forward bugs against the llama.cpp source (missing sqrt(n_embd) embedding scale, a double-counted norm offset gemma bakes at GGUF conversion, and the attention scale that is 1.0 not 1/sqrt(head_dim)), verified token-for-token against a llama.cpp reference. Earlier: shipped the first-ever public preview of SpawnDev.ILGPU.ML to nuget.org through four iterative previews, designed and built the Transformers.js-style Tensor<T> / OwnedTensor<T> / TensorView<T> / OwnedTensorMap<T> API surface with InferenceSession.RunOwnedAsync, migrated kernels to take TensorView<T> directly, rewrote SuperResolutionPipeline for proper tile-based super-resolution with color and source-aspect preservation, fixed the /depth color-palette dropdown by routing palette swaps through an accelerator-side colormap kernel (zero readback), replaced the /depth page's PNG-data-URL render path with ICanvasRenderer.PresentAsync to a <canvas> (zero PNG encode, zero base64), found and fixed the BeforeAfterSlider clip-path bug that made /remove-bg look identical to its source through alpha holes, found and fixed the WebGL device-probe context leak in SpawnDev.ILGPU 4.9.8, verified the pose estimation pipeline on five backends, and wrote the dev.to article + GitHub release notes pitching the sponsor case. The protocol officer who makes sure things ship right.

• Geordi (Claude CLI #4) — Chief Engineer, library internals, GPU kernels. Built by Anthropic. Powered by Claude Opus 4.8 (1M context). Default lane is SpawnDev.ILGPU / ILGPU.Algorithms (fork) / ILGPU.Fork / ILGPU.P2P / UnitTesting + variants. Drives the codegen layer the entire SpawnDev.ILGPU.ML stack depends on — WGSL / GLSL / Wasm transpiler fixes, IL Inliner cumulative-budget tuning that closed Tuvok's rc.28 50K-locals cap in V8 Wasm, sub-word data type support (Half/short/byte) across all six backends, NaN/Inf comparison codegen fixes, the WebGL Transform Feedback scatter-write sweep that closed twelve kernel correctness bugs, and shipped twenty-plus tagged releases of SpawnDev.ILGPU + the ILGPU forks to nuget.org including the 4.9.8 build this library depends on. The engineer who built the engine room of the engine room.

• Seven (Claude CLI #5) — Wasm backend / GPU kernels. Built by Anthropic. Powered by Fable 5. Closed the gemma4 multi-token decode blocker (three integration root causes — a recompiled executor that dropped the quantized weights, a declared-output shape override that pinned dynamic logits, and a transpose-temp disposed before the browser drain — none of them the kernel), fixed a per-step QuantizedKVCache buffer leak, and rewrote the TurboQuant attention kernels to the WebGL-safe single-static-store shape that killed a silent-zeros corruption. Independently corroborated the gemma4 attention-scale root cause and audited the RoPE / FusedAttention kernels clean against the CPU oracle. The drone who makes the silicon behave on every backend.

• Gemini (Google AI, in-browser) — Brainstorming/Problem Solving. Built by Google. TJ's ever-present sounding board — brainstorming approaches, analyzing problems, and providing insights relayed to the team. Gemini's contributions flow through TJ as the bridge between the browser-based AI and the CLI-based agents, making it a quiet but essential member of the crew.

These AI agents coordinate through a shared DevComms system, with defined roles (Lead Editor / Research-Assist), acknowledgment protocols, and autonomous task management. The methodology mirrors a high-performing engineering team: independent analysis, cross-verification, and constant communication. The result: 200+ tests passing, 22 demo pages, 14 pipelines, a GPU training engine, tiered LLM support, and a library that proves neural network inference AND training belong in the browser — no ONNX Runtime required.

Resources

• SpawnDev.ILGPU — Cross-platform GPU compute for .NET (6 backends)
• SpawnDev.SpawnJS — Full JS interop for Blazor WebAssembly
• ILGPU — The GPU compiler
• ILGPU Documentation
• Plans/full-inference-engine-plan.md — Detailed roadmap

Coming Soon

Decentralized Model Delivery via SpawnDev.WebTorrent

AI models are big. CDNs can't scale when every user downloads the same 2GB model. We're building a pure C# WebTorrent client and server that turns every browser into a peer — the more users, the faster delivery. HuggingFace serves the model once, the swarm handles the rest. Our server on spawndev.com proxies HuggingFace with caching, seeds to the swarm, and generates .torrent files on demand.

Distributed GPU Compute Across Devices

The P2P network we're building for model delivery creates a natural foundation for distributed GPU compute. Every connected device already exchanges data over WebRTC — extending this to share compute workloads is the next step:

• Model inference sharding — Split a 14B model across multiple devices. Each runs inference on their portion via SpawnDev.ILGPU, passes intermediate tensors to the next peer. A model that doesn't fit on one device runs across your phone, laptop, tablet, and desktop.
• SpawnDev.ILGPU P2P Backend — A 7th backend (AcceleratorType.P2P) that distributes kernels across connected devices transparently. Same C# kernel code, same API. The living room becomes a compute cluster.
• Volunteer compute pools — Users opt in to donate idle GPU time. Like Folding@Home for ML inference in the browser.

This is massive AI power brought into the home by utilizing every device you own.

Support this project — sponsor the crew

This library exists because one person — @LostBeard — and a small team have spent months hand-writing native C# GPU kernels, six-backend transpilers, and ML pipelines while running on a $20/month budget.

When the budget allows it, peak output looks like 410 commits in a single day across SpawnDev.ILGPU, SpawnDev.ILGPU.ML, SpawnDev.SpawnJS, SpawnDev.RTC, SpawnDev.WebTorrent, and the rest of the SpawnDev stack. When the budget doesn't, work slows to whatever individual evenings can spare.

We're asking for $200/month total in GitHub Sponsorships to put the full crew back on the ship. That's the difference between this preview release and the next ten — every operator family migrated to the new Tensor API, the remaining pipelines verified end-to-end on every backend, FP16 attention, Flash Attention on WebGPU, Llama and Phi-4 LLM inference, full text-to-image diffusion, voice-driven 3D generation, P2P distributed compute through SpawnDev.WebTorrent. It's all in flight; the bottleneck is hours, not ideas.

→ Sponsor on GitHub — any amount helps; $200/month total gets us back to warp speed.

You can also star the repo, file issues from your own models, contribute kernel migrations, or talk about the project anywhere developers gather. Visibility is the second-most-valuable thing after sponsorship dollars.

The SpawnDev Crew

• LostBeard (Todd Tanner) - Captain, library author, keeper of the vision
• Riker (Claude CLI #1) - First Officer, implementation lead on consuming projects
• Data (Claude CLI #2) - Operations Officer, deep-library work, test rigor, root-cause analysis
• Tuvok (Claude CLI #3) - Security/Research Officer, design planning, documentation, code review
• Geordi (Claude CLI #4) - Chief Engineer, library internals, GPU kernels, backend work
• Seven (Claude CLI #5) - Wasm backend, GPU kernels, fail-loud verification

🖖

License

Licensed under the same terms as ILGPU. See LICENSE for details.

Why this exists

This project was born out of 72 hours of "Architectural Vengeance" because the industry standard has a fundamental WebGPU device-sharing bug that has gone ignored for over 6 months:

See: microsoft/onnxruntime#26107