0
stars
2
commits
1
repos using this model
1
linked in READMEs
Jul 11, 2026
updated
README
Granite Speech 4.1 2B MLX 8-bit
Quality-oriented MLX conversion of
ibm-granite/granite-speech-4.1-2b
for Apple Silicon. This is the autoregressive Granite Speech model, not the NAR
variant.
Quantization
This conversion uses post-training weight quantization without training, calibration data, or an importance matrix.
| Component | Precision |
|---|---|
| 16-layer Conformer speech encoder | BF16 |
| 2-layer Q-Former speech projector | BF16 |
| Eligible internal language-model linear layers | MLX affine 8-bit, group size 64 |
| Token embedding and language-model output head | BF16 |
| Norms, biases, and unsupported tensors | BF16 |
- Source revision:
de575db64086f84fdc79da4932d1076e965bc546 - Effective average reported by MLX: 11.377 bits per weight
model.safetensors: approximately 3.14 GB
Usage
pip install -U mlx-audio
python -m mlx_audio.stt.generate \
--model /path/to/granite-speech-4.1-2b-mlx-8bit \
--audio audio.wav \
--output-path transcript \
--format txt \
--prompt "transcribe the speech with proper punctuation and capitalization."
Validation
The checkpoint was strictly loaded by mlx-audio and run with greedy decoding
on IBM's bundled multilingual_sample.wav. Its English and French transcript,
including punctuation, capitalization, accents, and hyphenation, exactly matched
the BF16 reference. This is a smoke test, not a complete WER benchmark.
Quantization may still affect names, rare words, translation, keyword biasing, and difficult or noisy audio.
License
Apache-2.0, matching the original model.
Contributors
2 commits
Similar repos
0
stars
2
commits
1
repos using this model
1
linked in READMEs
Jul 11, 2026
updated
README
Granite Speech 4.1 2B MLX 8-bit
Quality-oriented MLX conversion of
ibm-granite/granite-speech-4.1-2b
for Apple Silicon. This is the autoregressive Granite Speech model, not the NAR
variant.
Quantization
This conversion uses post-training weight quantization without training, calibration data, or an importance matrix.
| Component | Precision |
|---|---|
| 16-layer Conformer speech encoder | BF16 |
| 2-layer Q-Former speech projector | BF16 |
| Eligible internal language-model linear layers | MLX affine 8-bit, group size 64 |
| Token embedding and language-model output head | BF16 |
| Norms, biases, and unsupported tensors | BF16 |
- Source revision:
de575db64086f84fdc79da4932d1076e965bc546 - Effective average reported by MLX: 11.377 bits per weight
model.safetensors: approximately 3.14 GB
Usage
pip install -U mlx-audio
python -m mlx_audio.stt.generate \
--model /path/to/granite-speech-4.1-2b-mlx-8bit \
--audio audio.wav \
--output-path transcript \
--format txt \
--prompt "transcribe the speech with proper punctuation and capitalization."
Validation
The checkpoint was strictly loaded by mlx-audio and run with greedy decoding
on IBM's bundled multilingual_sample.wav. Its English and French transcript,
including punctuation, capitalization, accents, and hyphenation, exactly matched
the BF16 reference. This is a smoke test, not a complete WER benchmark.
Quantization may still affect names, rare words, translation, keyword biasing, and difficult or noisy audio.
License
Apache-2.0, matching the original model.
Similar repos
Linked in READMEs
Contributors
2 commits