granite-4.2-8b-oQ6e is an open-weight model from Michael, released under Apache License 2.0. It has 8.8B parameters and a 131,072-token context. At 16-bit it needs about 21.1 GB of GPU memory, which fits on 1x MI300X from $1.85 an hour, at the lowest prices in the SAVRN Index.
This model was quantized using oQ (oMLX v0.7.0) mixed-precision quantization.
Runs On
What it takes to serve granite-4.2-8b-oQ6e (8.8B parameters): the memory its weights need at each precision, and the cheapest way to rent enough data-center GPUs to hold them.
| Precision | Weights | Memory needed | Cheapest setup | Per hour | Also fits |
|---|---|---|---|---|---|
| 16-bit | 17.6 GB | 21.1 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
| 8-bit | 8.8 GB | 10.5 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
| 4-bit | 4.4 GB | 5.3 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
Memory is the weights at that precision plus 20% for the runtime and a short context; a long context needs more. Prices are the lowest on-demand hourly rates in the SAVRN Index, read Oct 20, 2026.
granite-4.2-8b-oQ6e on every accelerator the SAVRN Index prices, at every precision
Model Card
By Michael, published under apache-2.0, revision c78fd1af2970.
This model was quantized using oQ (oMLX v0.7.0) mixed-precision quantization.
Read Michael's full model card
This model was quantized using oQ (oMLX v0.7.0) mixed-precision quantization.
Quantization details
- Model type: granite
- Bits: 6
- Group size: 64
- Format: MLX safetensors
Configuration
- Architecture
- GraniteForCausalLM
- Context length (tokens)
- 131,072
- Layers
- 40
- Hidden size
- 4,096
- Feed-forward size
- 12,800
- Attention heads
- 32
- Key/value heads
- 8
- Vocabulary size
- 100,352
- RoPE base
- 10,000,000
- Stored precision
- bfloat16
- Model type
- granite
Identity and Version
- Repository
- jk797/granite-4.2-8b-oQ6e
- Publisher
- Michael
- Task
- Not stated by the source
- Modality
- Other
- Library
- mlx
- Parameters
- 8.8B parameters
- Languages
- mlx, oq
- Revision
- c78fd1af29707990c2e5501ff94cdb17bae93a87
- First published
- 2026-10-05
- Last updated
- 2026-10-05
Files and Weights
15 files, 7.4 GB in total. The weights are 2 files totalling 7.4 GB in safetensors.
Every file
| File | Type | Size | SHA-256 |
|---|---|---|---|
| model-00001-of-00002.safetensors | Weights | 5.0 GB | 05b2e175cfb8 |
| model-00002-of-00002.safetensors | Weights | 2.4 GB | 8595c4b34939 |
| config.json | Configuration | 1.9 KB | — |
| generation_config.json | Configuration | 207 B | — |
| granite_thinking_parser.py | Configuration | 4.6 KB | — |
| model.safetensors.index.json | Configuration | 75.6 KB | — |
| oq_imatrix_report.json | Configuration | 13.8 KB | — |
| special_tokens_map.json | Configuration | 563 B | — |
| README.md | Documentation | 399 B | — |
| chat_template.jinja | Other | 9.0 KB | — |
| .gitattributes | Repository | 1.5 KB | — |
| merges.txt | Tokenizer | 916.6 KB | — |
| tokenizer.json | Tokenizer | 7.2 MB | — |
| tokenizer_config.json | Tokenizer | 17.7 KB | — |
| vocab.json | Tokenizer | 1.6 MB | — |
License and Download
- License
- apache-2.0
- Access
- Open weights, no gate
- Download size
- 7.4 GB
Released by Michael through its official repository on Hugging Face. Read the license.
Built From
- Derived from ibm-granite/granite-4.2-8b
- Quantized from ibm-granite/granite-4.2-8b
Memory Requirements
| Precision | Weights in memory |
|---|---|
| As published | 7.4 GB |
| 16-bit | 17.6 GB |
| 8-bit | 8.8 GB |
| 4-bit | 4.4 GB |
Weights only, from the published parameter count; the key-value cache and runtime add to this.
Questions About granite-4.2-8b-oQ6e
How much GPU memory does granite-4.2-8b-oQ6e need?
About 21.1 GB at 16-bit and 5.3 GB at 4-bit: the weights (8.8B parameters) plus a working margin. A long context needs more.
What is the cheapest GPU to run granite-4.2-8b-oQ6e on?
At 16-bit, 1x MI300X from $1.85 an hour; at 4-bit, 1x MI300X from $1.85 an hour, at the lowest on-demand prices the SAVRN Index lists.
Can I use granite-4.2-8b-oQ6e commercially?
Yes. granite-4.2-8b-oQ6e is released under Apache License 2.0. The Apache License 2.0 is a permissive open-source license. It permits commercial use, modification and redistribution. It requires keeping the license and copyright notices and any NOTICE file, stating significant changes, and it includes an express patent grant from contributors.
What is granite-4.2-8b-oQ6e's context length?
131,072 tokens, from the maximum position embeddings in its published configuration.