[!IMPORTANT] Want faster improvements? This project is entirely self-funded on my minimum-wage salary, and every training run competes for a single consumer GPU. If you or your organisation can donate an RTX 5090, or the money to buy one second-hand, it would directly speed up the next versions of the dataset and the fine-tuned models. Reach out via olicorne.org or open a discussion on this page.
ONNX export of moondream/parakeet-redux, moondream's 1.58-bit parakeet-tdt-0.6b-v3: same architecture and tokenizer, but every encoder weight is -1, 0 or +1 times a per-128-block scale.
Built for Parakeet Web (transcription entirely client-side) and onnx-asr, with the same layout and pipeline as Olicorne/parakeet-tdt-0.6b-v3-optimized-onnx. Built with Claude Code.
For French medical audio see Olicorne/parakeet-tdt-0.6b-v3-redux-UltiMed-onnx: this repo's w2a8 encoder unchanged, with a decoder that carries 1.5 times the change the Olicorne/parakeet-tdt-0.6b-v3-UltiMed-onnx fine-tune made to its decoder (dictionary WER 8.78 % instead of 12.12 %, same 135 MB encoder).
| folder | contents | size | use |
|---|---|---|---|
w2a8/ | encoder | 190 MB (135 MB zstd) | MatMulNBits 2-bit, block 128, CPU / WASM |
int8/ | decoder | 18 MB | default decoder |
fp32/ | decoder | 73 MB | for batched decoding (see below) |
vocab.txt, config.json and the nemo128.onnx preprocessor sit at the root, next to parakeet-tdt-0.6b-v3-redux.nemo, the NeMo checkpoint the ONNX was exported from. *.zst files are precompressed copies for web servers.
Only the w2a8 encoder ships (since 2026-10-06). The fp32, fp16, int8 and w4a8 encoders were removed: they held the same ternary weights (see below), scored the same, and w2a8 is both the smallest and the fastest on CPU (1.46 s per clip, batch 1, against 1.54 to 1.81 s for int8 and 1.53 to 1.63 s for w4a8). They are in this repo's git history, and the .nemo rebuilds them with the ultra repo's build-from-nemo-export.sh --ternary.
CPU only. The 2-bit encoder crashes onnxruntime's CUDA provider (illegal memory access, onnxruntime-gpu 1.30.0, 2026-10-06); Parakeet Web runs it on WASM. On a GPU use parakeet-ultra int8, which is faster and more accurate there.
Because the weights are already ternary, 8, 4 and 2 bits hold exactly the same weights: each of the 264 ternary MatMuls is packed as code = sign + zero point, scale = block amplitude, which any width from 2 bits up represents without loss, and every packed tensor was decoded back and checked bit-exact against the source. They differed only in download size and kernel speed (and in the int8 activation quantization all three share), which is why only w2a8 is kept. The benchmark tables below still show the removed packings for that comparison. The subsampling projection is not ternary and stays fp32.
Batching. The encoder masks padded frames with a key-only attention bias, so onnx-asr's model.recognize([a, b, c]) works on clips of different lengths with no manual padding, and batch 1 is bit-identical to the previous unmasked export (the published numbers stay valid). On 1,000 UltiMed dictionary clips (UltiMed w4a8 encoder, fp32 decoder, CUDA, onnx-asr with a lockstep batched greedy TDT decode as in the ultra repo's scripts/drug-rules/transcribe_onnx.py), batches of 8, 16 and 32 took 0.06, 0.05 and 0.04 s per clip against 0.16 s one clip at a time, changed 3 to 4 transcripts out of 1,000, and kept WER at 3.26 % (3.26 % unbatched). Use the fp32 decoder when batching: the int8 decoder quantizes activations with one scale per batch, so a clip's transcript can depend on its batch neighbours. Stock onnx-asr decodes each clip on its own, so it gets the encoder speedup but not the decoder one.
scripts/hf-to-nemo.py (parakeet_web) maps the HF transformers checkpoint back onto the base v3 .nemo: a key rename only, every learned weight comes from moondream's safetensors, the VAD head is dropped.export_onnx.py --web-optimized, then scripts/build-from-nemo-export.sh of the ultra repo with --ternary: graph fold (bit-exact gate), 12-bit mantissa rounding, fp32 shards, decoder int8 + LSE/top-K surgery, fp16, pointwise Conv -> MatMul, MatMulNBits.Checked: the ONNX transcribes the JFK clip word for word like moondream's PyTorch model run in NeMo.
No <unk> output (since 2026-10-06). Both decoders have the joint's output bias for <unk> (id 0) set to -1e4, so greedy and beam search never emit it and take the next best token, or blank, instead. Parakeet emits <unk> for characters missing from its vocab, mostly ° ("37,8 degrés" came out as "37,8C"), + and some apostrophes; onnx-asr printed it raw and parakeet_web silently deleted it. The patch changes that one float and nothing else (ban-unk.py). On a paired check (parakeet-redux, int8) it lowered FLEURS fr WER from 6.20 % to 5.54 % and left the medical test slices unchanged. Benchmarks dated before 2026-10-06 ran without it. Done with Claude Code.
All numbers measured on CPU (onnxruntime, wer-quants.py from parakeet_web, with the int8 decoder for every encoder). Generated with Claude Code from the benchmark queue's output.
FLEURS, greedy, full validation split, 25 languages, w2a8 encoder (the only one shipped), rerun on 2026-10-07 with the <unk> ban:
| encoder | macro WER | macro CER | micro WER | micro CER | sets scored |
|---|---|---|---|---|---|
| w2a8 | 12.16 % | 3.62 % | 11.83 % | 3.56 % | 25/25 |
FLEURS, beam 5 (MAES), 60 clips per language, same setup:
| encoder | macro WER | macro CER | micro WER | micro CER | sets scored |
|---|---|---|---|---|---|
| w2a8 | 11.67 % | 3.34 % | 11.42 % | 3.31 % | 25/25 |
Before the ban (2026-10-05) w2a8 scored 12.64 % macro WER greedy and 11.71 % beam 5, and the fp32, int8 and w4a8 encoders that the repo shipped then all landed within 0.01 point of it: the weights are ternary, so 8 and 2 bits store them exactly.
Long audio (2026-10-05, before the <unk> ban; the fp32, int8 and w4a8 encoders are no longer shipped): eight speeches (six of 390 s, two of 60 s), each transcribed in ONE pass. The reference is this repo's fp32 decoding the same audio in independent 60 s sections, so these figures measure long-pass drift, not absolute accuracy. Means across the eight files:
| encoder | overall WER | overall CER | last chunk WER | worst chunk WER |
|---|---|---|---|---|
| fp32 | 5.3 % | 1.5 % | 8.2 % | 9.7 % |
| int8 | 5.4 % | 1.5 % | 8.2 % | 9.8 % |
| w4a8 | 5.5 % | 1.5 % | 8.2 % | 9.9 % |
| w2a8 | 5.5 % | 1.6 % | 8.2 % | 9.9 % |
Past 400 s: one 607 s French VoxPopuli clip in a single pass, same fp32-sectioned reference; it checks that a pass well past 400 s completes and stays close to sectioned decoding:
| encoder | WER | CER | proc/dur |
|---|---|---|---|
| int8 | 5.79 % | 2.33 % | 0.211 |
| w4a8 | 5.86 % | 2.34 % | 0.196 |
| w2a8 | 5.86 % | 2.39 % | 0.189 |
| fp32 | 5.71 % | 2.35 % | 0.211 |
CC-BY-4.0, like the original. Model by moondream, derived from NVIDIA's parakeet-tdt-0.6b-v3. This repo only converts and quantizes it.
[!IMPORTANT] Want faster improvements? This project is entirely self-funded on my minimum-wage salary, and every training run competes for a single consumer GPU. If you or your organisation can donate an RTX 5090, or the money to buy one second-hand, it would directly speed up the next versions of the dataset and the fine-tuned models. Reach out via olicorne.org or open a discussion on this page.
ONNX export of moondream/parakeet-redux, moondream's 1.58-bit parakeet-tdt-0.6b-v3: same architecture and tokenizer, but every encoder weight is -1, 0 or +1 times a per-128-block scale.
Built for Parakeet Web (transcription entirely client-side) and onnx-asr, with the same layout and pipeline as Olicorne/parakeet-tdt-0.6b-v3-optimized-onnx. Built with Claude Code.
For French medical audio see Olicorne/parakeet-tdt-0.6b-v3-redux-UltiMed-onnx: this repo's w2a8 encoder unchanged, with a decoder that carries 1.5 times the change the Olicorne/parakeet-tdt-0.6b-v3-UltiMed-onnx fine-tune made to its decoder (dictionary WER 8.78 % instead of 12.12 %, same 135 MB encoder).
| folder | contents | size | use |
|---|---|---|---|
w2a8/ | encoder | 190 MB (135 MB zstd) | MatMulNBits 2-bit, block 128, CPU / WASM |
int8/ | decoder | 18 MB | default decoder |
fp32/ | decoder | 73 MB | for batched decoding (see below) |
vocab.txt, config.json and the nemo128.onnx preprocessor sit at the root, next to parakeet-tdt-0.6b-v3-redux.nemo, the NeMo checkpoint the ONNX was exported from. *.zst files are precompressed copies for web servers.
Only the w2a8 encoder ships (since 2026-10-06). The fp32, fp16, int8 and w4a8 encoders were removed: they held the same ternary weights (see below), scored the same, and w2a8 is both the smallest and the fastest on CPU (1.46 s per clip, batch 1, against 1.54 to 1.81 s for int8 and 1.53 to 1.63 s for w4a8). They are in this repo's git history, and the .nemo rebuilds them with the ultra repo's build-from-nemo-export.sh --ternary.
CPU only. The 2-bit encoder crashes onnxruntime's CUDA provider (illegal memory access, onnxruntime-gpu 1.30.0, 2026-10-06); Parakeet Web runs it on WASM. On a GPU use parakeet-ultra int8, which is faster and more accurate there.
Because the weights are already ternary, 8, 4 and 2 bits hold exactly the same weights: each of the 264 ternary MatMuls is packed as code = sign + zero point, scale = block amplitude, which any width from 2 bits up represents without loss, and every packed tensor was decoded back and checked bit-exact against the source. They differed only in download size and kernel speed (and in the int8 activation quantization all three share), which is why only w2a8 is kept. The benchmark tables below still show the removed packings for that comparison. The subsampling projection is not ternary and stays fp32.
Batching. The encoder masks padded frames with a key-only attention bias, so onnx-asr's model.recognize([a, b, c]) works on clips of different lengths with no manual padding, and batch 1 is bit-identical to the previous unmasked export (the published numbers stay valid). On 1,000 UltiMed dictionary clips (UltiMed w4a8 encoder, fp32 decoder, CUDA, onnx-asr with a lockstep batched greedy TDT decode as in the ultra repo's scripts/drug-rules/transcribe_onnx.py), batches of 8, 16 and 32 took 0.06, 0.05 and 0.04 s per clip against 0.16 s one clip at a time, changed 3 to 4 transcripts out of 1,000, and kept WER at 3.26 % (3.26 % unbatched). Use the fp32 decoder when batching: the int8 decoder quantizes activations with one scale per batch, so a clip's transcript can depend on its batch neighbours. Stock onnx-asr decodes each clip on its own, so it gets the encoder speedup but not the decoder one.
scripts/hf-to-nemo.py (parakeet_web) maps the HF transformers checkpoint back onto the base v3 .nemo: a key rename only, every learned weight comes from moondream's safetensors, the VAD head is dropped.export_onnx.py --web-optimized, then scripts/build-from-nemo-export.sh of the ultra repo with --ternary: graph fold (bit-exact gate), 12-bit mantissa rounding, fp32 shards, decoder int8 + LSE/top-K surgery, fp16, pointwise Conv -> MatMul, MatMulNBits.Checked: the ONNX transcribes the JFK clip word for word like moondream's PyTorch model run in NeMo.
No <unk> output (since 2026-10-06). Both decoders have the joint's output bias for <unk> (id 0) set to -1e4, so greedy and beam search never emit it and take the next best token, or blank, instead. Parakeet emits <unk> for characters missing from its vocab, mostly ° ("37,8 degrés" came out as "37,8C"), + and some apostrophes; onnx-asr printed it raw and parakeet_web silently deleted it. The patch changes that one float and nothing else (ban-unk.py). On a paired check (parakeet-redux, int8) it lowered FLEURS fr WER from 6.20 % to 5.54 % and left the medical test slices unchanged. Benchmarks dated before 2026-10-06 ran without it. Done with Claude Code.
All numbers measured on CPU (onnxruntime, wer-quants.py from parakeet_web, with the int8 decoder for every encoder). Generated with Claude Code from the benchmark queue's output.
FLEURS, greedy, full validation split, 25 languages, w2a8 encoder (the only one shipped), rerun on 2026-10-07 with the <unk> ban:
| encoder | macro WER | macro CER | micro WER | micro CER | sets scored |
|---|---|---|---|---|---|
| w2a8 | 12.16 % | 3.62 % | 11.83 % | 3.56 % | 25/25 |
FLEURS, beam 5 (MAES), 60 clips per language, same setup:
| encoder | macro WER | macro CER | micro WER | micro CER | sets scored |
|---|---|---|---|---|---|
| w2a8 | 11.67 % | 3.34 % | 11.42 % | 3.31 % | 25/25 |
Before the ban (2026-10-05) w2a8 scored 12.64 % macro WER greedy and 11.71 % beam 5, and the fp32, int8 and w4a8 encoders that the repo shipped then all landed within 0.01 point of it: the weights are ternary, so 8 and 2 bits store them exactly.
Long audio (2026-10-05, before the <unk> ban; the fp32, int8 and w4a8 encoders are no longer shipped): eight speeches (six of 390 s, two of 60 s), each transcribed in ONE pass. The reference is this repo's fp32 decoding the same audio in independent 60 s sections, so these figures measure long-pass drift, not absolute accuracy. Means across the eight files:
| encoder | overall WER | overall CER | last chunk WER | worst chunk WER |
|---|---|---|---|---|
| fp32 | 5.3 % | 1.5 % | 8.2 % | 9.7 % |
| int8 | 5.4 % | 1.5 % | 8.2 % | 9.8 % |
| w4a8 | 5.5 % | 1.5 % | 8.2 % | 9.9 % |
| w2a8 | 5.5 % | 1.6 % | 8.2 % | 9.9 % |
Past 400 s: one 607 s French VoxPopuli clip in a single pass, same fp32-sectioned reference; it checks that a pass well past 400 s completes and stays close to sectioned decoding:
| encoder | WER | CER | proc/dur |
|---|---|---|---|
| int8 | 5.79 % | 2.33 % | 0.211 |
| w4a8 | 5.86 % | 2.34 % | 0.196 |
| w2a8 | 5.86 % | 2.39 % | 0.189 |
| fp32 | 5.71 % | 2.35 % | 0.211 |
CC-BY-4.0, like the original. Model by moondream, derived from NVIDIA's parakeet-tdt-0.6b-v3. This repo only converts and quantizes it.