Granite Speech 4.0 1B — MLX (int8)
Pure-MLX selective-int8 conversion of IBM's
ibm-granite/granite-4.0-1b-speech
for local automatic speech recognition on Apple Silicon. The complete
waveform-to-text runtime uses mlx-speech; PyTorch, Transformers, mlx-audio,
vLLM, and cloud APIs are not runtime dependencies.
Model details
- Upstream developer: IBM
- MLX conversion and runtime: App Automaton
- Task: offline greedy automatic speech recognition
- Supported upstream speech inputs: English, French, German, Spanish,
Portuguese, and Japanese
- Input: mono audio, resampled to 16 kHz by the public file adapter
- Runtime: Apple Silicon with MLX
Precision
The Granite causal language model uses 8-bit affine weight quantization with
group size 64. The acoustic encoder, QFormer projector, normalization,
convolution, biases, activations, and KV cache retain BF16 precision. The exact
quantization contract is stored in config.json, and the runtime reconstructs
the saved quantized module set from checkpoint .scales tensors before strict
weight loading.
Release validation
BF16/int8 checks used two 12–13 second English samples.
The int8 transcripts differed from BF16 by one normalized word on each sample.
Hank WER was unchanged at 10.714%; Peggy WER changed from 3.571% to 7.143%, one
additional acronym error. On IBM's bundled multilingual sample, the first 64
tokens differed only at the first word and both retained the required
timothy was a spoiled cat phrase. These checks establish the local release
gate, not universal accuracy parity across all supported languages.
The released weight SHA-256 is
cf355a69e931ccac95d5cf942c3d540ba2456f06ad89c379c8132875b9098e6c.
It is byte-identical to the existing
mlx-community/granite-4.0-1b-speech-8bit
weight file. Full local methodology is recorded in the
mlx-speech quantization gate.
Usage
pip install "mlx-speech>=0.5.1"
import mlx_speech
asr = mlx_speech.asr.load("granite-speech-4.0-1b")
result = asr.generate("speech.wav", max_new_tokens=200)
print(result.text)
mlx-speech asr \
--model granite-speech-4.0-1b \
--audio speech.wav
For an explicit precision selector, granite-speech-4.0-1b-int8 resolves to
the same repository. A downloaded repository can also be loaded directly by
local path.
Limitations
- The
mlx-speech runtime currently exposes greedy transcription only.
- Mandarin speech transcription is not an upstream capability. IBM describes
English-to-Mandarin speech translation, which is a different task and is not
exposed by this runtime.
- No timestamps, beam search, or native long-audio chunking are included.
- Inference requires Apple Silicon and MLX.
Links
License
Apache 2.0, following the upstream IBM Granite Speech release. Refer to the
upstream model card for current terms and intended-use guidance.
