samnet

Run SAM's output through a VAE trained on human speech.

what

SAM (Software Automatic Mouth, 1982) is a tiny deterministic formant synth. It sounds like a robot: harsh 8-bit output from a handful of filters. We drive it through rustsam, a Rust reimplementation.

samnet takes that audio and runs it through a latent space trained only on human speech. The decoder can't produce anything else, so it reconstructs SAM's cadence and timbre out of parts that only know how people sound. The result lands somewhere between 1982 and now.

There's no training on our side. The effect comes from the projection.

text ──► SAM ──► 22050 ──► resample ──► encode ──► latents ──► decode ──► audio
                                        └──────── repeat N times ────────┘

Each round-trip pass strips away more of SAM's robotic character. Three is the default.

run

cargo run -p samnet-run --release -- 'the birch canoe slid on the smooth planks' -o canoe.wav
cargo run -p samnet-run --release -- -a 1 'hello world' | aplay
cargo run -p samnet-run --release                        # web UI on :8777

The first run builds assets/vae.safetensors (40MB) by stripping it out of pocket-tts's 219MB bundle, then never touches the network again. --assets <dir> does only that step, for use as a build hook.

The weights are gated: accept at https://huggingface.co/kyutai/pocket-tts and set HF_TOKEN. Without it the hub quietly serves a build whose encoder is all zeros, and Vae::load rejects those rather than emitting silence.

browser

The whole pipeline also runs client-side as a wasm build.

mise install                     # pins trunk, the build tool
cd crates/web
trunk build --release            # builds dist/ and stages weights/

The trunk here is trunk-rs, pinned in mise.toml as github:trunk-rs/trunk; mise's bare trunk registry is trunk.io, a linter, and the wrong tool. trunk-rs fetches its own matching wasm-bindgen and wasm-opt.

dist/ is the self-contained front end: SAM (wasm), the onnxruntime-web runtime, and the glue, about 2.5MB of wasm plus html and js. The VAE runs in a Web Worker so the tab stays responsive, on WebGPU where available and falling back to CPU, and it reports progress per pass.

The 40MB weights are staged separately into weights/ under a content-hashed name (with a small manifest.json). They do not ship in dist/: the file is over Cloudflare's 25 MiB per-file limit, so it lives in R2 instead. trunk serve is still fine for working on the UI, but a run that actually makes sound needs the weights, so use wrangler dev (below).

hosting

worker.ts is a small Cloudflare Worker. It serves the weights from an R2 bucket, and only the two object keys named in the manifest, so it never blind-proxies the bucket. Everything else comes from the static asset bundle. Same origin, so no CORS.

One-time:

cd crates/web
npm install
npx wrangler login                        # browser OAuth; creds live in ~/.config, not the repo
npx wrangler r2 bucket create samnet-weights

Deploy:

trunk build --release
for f in weights/*; do npx wrangler r2 object put "samnet-weights/$(basename "$f")" -f "$f" --remote; done
npx wrangler deploy

--remote matters: without it wrangler writes to a local simulated bucket and the deployed Worker sees nothing. The weights only need re-uploading when they change; their name changes with them. Nothing in the repo carries an account id or token, wrangler reads your login session, and CLOUDFLARE_ACCOUNT_ID can be set in the shell if you have more than one account.

For a full local run, seed a local R2 once, then use wrangler's dev server:

for f in weights/*; do npx wrangler r2 object put "samnet-weights/$(basename "$f")" -f "$f" --local; done
npx wrangler dev

shape

• crates/vae: SAM and the dsp that brackets it (resample, normalize, WAV). No server, no HF, no model weights, so it builds for wasm32-unknown-unknown.
• crates/run: CLI, web UI, the asset step, the ONNX export, and a static file server. Adds CUDA.
• crates/web: the wasm-bindgen wrapper and browser UI.

The VAE is Kyutai pocket-tts's Mimi VAE, hand-exported from the safetensors bundle to a stateful streaming ONNX graph (crates/run/src/onnx_export.rs) and run through ort natively or onnxruntime-web in the browser. It isn't a codec: its quantizer is a plain 32→512 projection, so latents stay continuous and nothing is lost to quantization noise. The export was diffed stage by stage against the candle reference, landing at cos 0.9999982 on latents and 0.9999548 on audio.

limits

• The native serve is a dev convenience: one ort::Session driven &mut behind single-threaded tiny_http, so requests don't overlap. The browser is the product; a one-worker queue would fix it if it ever mattered.
• The voice knobs (pitch/mouth/throat/speed/sing, full 0..=255) are exposed but untuned: which survive the VAE vs. wash back toward the speech prior is an open by-ear question.
• onnxruntime-web runs single-threaded (no COOP/COEP needed). The VAE is on the GPU, so wasm threads for SAM probably wouldn't move the needle.