Aloud

Aloud is a library-first Rust inference engine for Audio8 speech recognition and speech synthesis. It runs GGUF v3 directly with Aloud's own Rust kernels. It does not contain or invoke ggml, llama.cpp, C++, ONNX, PyTorch, or another inference runtime.

The CPU build has two direct runtime dependencies: memmap2 for resident model loading and the official hf-hub Rust client for model resolution. The optional GPU build adds wgpu and pollster. All direct dependencies are kept at their latest compatible releases.

Each TTS GGUF is self-contained. It includes the TTS backbone, codec decoder, codec encoder, tokenizer, configuration, and the embedded voices sky and aiden. Normal synthesis keeps the codec encoder out of resident memory. Voice creation loads the encoder-specific tensors instead.

CLI

Local paths and Hugging Face URIs use the same model-source API:

cargo run --release --bin aloud -- transcribe recording.wav \
  --model ./build/asr-q8.gguf --device auto

cargo run --release --bin aloud -- transcribe recording.wav \
  --model hf://isala404/aloud/audio8-asr-0.1b-q8_0.gguf

cargo run --release --bin aloud -- tts "Text to speak." -o out.wav \
  --model ./build/tts01-q8.gguf --voice sky --device cpu

cargo run --release --bin aloud -- tts "Text to speak." -o out.wav \
  --model ./build/tts06-q8.gguf --voice ./mine.aloudvoice

hf://owner/repo/path downloads through the maintained hf-hub Rust client into the standard Hugging Face cache. Cached files are reused, cached models work offline, and standard Hugging Face cache, token, and authentication environment variables are honored. Aloud never invokes the hf CLI during inference. Resolution is atomic under concurrent processes, then the library validates the GGUF architecture before loading it.

Create an external voice with the public Rust engine:

cargo run --release --bin aloud -- voice create reference.wav \
  --model ./build/tts01-q8.gguf \
  --text "Exact words spoken" \
  -o mine.aloudvoice

Audio8 needs the exact transcript. Aloud rejects missing or unencodable text rather than creating a mismatched prompt. The voice file records the codec identity and is validated against the selected model before synthesis. Existing version-one .aloudvoice files remain readable; newly created files include the codec identifier.

ASR input must be mono 16 kHz PCM WAV. Voice references and TTS output are mono 44.1 kHz PCM WAV.

Library

Model resolution, GGUF validation and loading, device selection, ASR, TTS, embedded and external voices, voice creation, caching, and WAV output are public library APIs. The aloud binary only parses arguments and calls them.

use aloud::engine::{Device, Model};

let model = Model::load_from("hf://isala404/aloud/audio8-asr-0.1b-q8_0.gguf")?;
let session = model.asr(Device::Auto)?;
let wav = aloud::wav::read("recording.wav")?;
let transcript = session.transcribe(&wav.samples, wav.sample_rate)?;
println!("{}", transcript.text);
# Ok::<(), String>(())

use aloud::engine::{Device, Model};

let model = Model::load_from("./build/tts01-q8.gguf")?;
let session = model.tts(Device::Cpu)?;
let voice = session.embedded_voice("aiden")?;
session.write_wav("out.wav", "Hello from Aloud.", Some(&voice), 512)?;
# Ok::<(), String>(())

Devices

Every inference API accepts auto, cpu, or gpu. cpu uses the resident Rust execution engine. gpu requires a build with --features gpu and a usable wgpu adapter; it returns an error instead of silently switching to CPU. auto currently selects CPU because the measured Q8 path is faster on the tested Apple M3 Pro.

The GPU backend keeps Q8 weights packed, runs model matrix and embedding work through wgpu, and uses the existing wgpu codec convolution stack. Tokenization, greedy control flow, and small fixed-shape operations remain host-side. GPU and CPU kernels have direct numerical parity tests; autoregressive TTS can choose a different residual code after a near-tie, so release verification also requires TTS-to-ASR round trips rather than pretending divergent waveforms are comparable.

Build and test GPU support with:

cargo test --release --features gpu --test gpu -- --nocapture

GGUF

Aloud accepts GGUF version 3 only. Tensor payloads are 64-byte aligned. Q8 weights use native GGUF Q8_0 blocks while retaining Aloud's tap-major codec and matrix layouts. A file with a valid GGUF header but an unimplemented general.architecture fails with a clear unsupported-architecture error.

There is no legacy .aloud model loader. Convert upstream checkpoints with:

tools/pull_weights.sh
reference/.venv/bin/python tools/convert.py asr tts01 tts06
reference/.venv/bin/python tools/convert.py --q8 asr tts01 tts06

Conversion is deterministic. Each output gets per-tensor dtype/shape/CRC checks and a whole-file SHA-256 sidecar. The f32 files are correctness references; Q8 files are separate distributable artifacts. TTS conversion includes both codec paths and both stable embedded voices in each file.

Correctness

The real-checkpoint suite checks more than final text or audible output:

• ASR preprocessing, encoder, adapter, decoder logits, short transcripts, and long-form WER.
• Exact f32 token grids for both TTS backbones.
• Codec stages and waveforms against PyTorch, including chunked decoding.
• Q8 tensor payload checks, ASR transcripts, TTS grids, and waveform SNR.
• Local and mock-Hub model resolution, concurrent downloads, cache reuse, and offline cache use.
• Public library and CLI flows for embedded voices, external voices, and Rust voice creation.
• CPU/GPU kernel and full-codec parity on a real wgpu adapter.
• TTS-to-ASR round trips with real models and audio.

Generate the reference tensors and run the suite:

HF_HOME=/tmp/aloud-hf-golden \
PYTHONPATH=/tmp/aloud-tf457/lib/python3.12/site-packages \
reference/.venv/bin/python tools/dump_golden_asr.py

HF_HOME=/tmp/aloud-hf-golden \
PYTHONPATH=/tmp/aloud-tf457/lib/python3.12/site-packages \
reference/.venv/bin/python tools/dump_golden_tts.py

HF_HOME=/tmp/aloud-hf-golden \
PYTHONPATH=/tmp/aloud-tf457/lib/python3.12/site-packages \
reference/.venv/bin/python tools/dump_golden_tts01.py

HF_HOME=/tmp/aloud-hf-golden \
PYTHONPATH=/tmp/aloud-tf457/lib/python3.12/site-packages \
reference/.venv/bin/python tools/dump_golden_codec.py

cargo test --release
cargo test --release --features gpu --test gpu -- --nocapture
tools/roundtrip.py

The no-checkpoint CI slice is:

cargo test --release --lib --test containers --test kernels --test batch --test model_source

Performance

Use tools/bench.sh for release measurements. It cools the machine, discards the model-faulting warm-up, and reports repeated runs instead of a thermally throttled anecdote.

tools/bench.sh asr testdata/speech_real.wav --model build/asr-q8.gguf
tools/bench.sh tts-stream "The quick brown fox jumps over the lazy dog." \
  --model build/tts01-q8.gguf
cargo run --release --bin bench -- kernels

Release measurements on an Apple M3 Pro use three cooled runs after a model-faulting warm-up. ASR takes 0.171 s for 2.66 s of audio, down from the pre-migration 0.188 s baseline. CPU TTS takes 1.024 s for 2.83 s of audio, down from 1.492 s. Explicit GPU ASR takes 1.80-1.98 s and a 60-frame TTS grid takes 22.79 s, so auto correctly selects CPU on this machine.

Resident Q8 storage is 329.2 MiB for ASR. TTS inference is 370.1 MiB for 0.1b and 855.5 MiB for 0.6b, down from the legacy paired-model baselines of 418 MiB and 918 MiB. Voice creation maps 210.8 MiB of encoder and quantizer tensors instead of loading the TTS backbone and codec decoder. The self-contained files are 581.6 MiB and 1068.6 MiB, but mutually exclusive resident views keep unused components out of memory.

The codec f32 path measures about 115 dB SNR against PyTorch. The current all-eligible-row Q8 codec is 37.6 dB on the short clip and 36.6 dB on the full clip; rebuilding the final legacy container produces the same values. The older 42.8/40.1 dB figure used a larger mixed-precision subset with eleven million fewer Q8 weights. GPU codec output is 115.1 dB against CPU and 117.3 dB against PyTorch. Real GPU synthesis with both an embedded voice and a GPU-created external voice round-trips through ASR to the exact source transcript.

Licensing and provenance

The engine source is separate from the model licenses. The release model repository must attribute every GGUF to its exact upstream checkpoint and retain the applicable terms:

• Audio8-ASR-0.1B: CC BY-NC 4.0.
• Audio8-TTS-Preview-0.1b: Audio8 Community License.
• Audio8-TTS-Preview-0.6b and its shared codec: Apache-2.0, including the upstream NOTICE.
• sky and aiden: conditioning grids made from short synthetic references for Aloud; the source audio is not distributed.

reference/ contains ignored upstream material used only for verification. It is not part of the source release or model upload.