Open-weight model
gemma-4-31B-it-uncensored-heretic-exl3-4.50bpw-h8
by Sleepyj sjoe1244/gemma-4-31B-it-uncensored-heretic-exl3-4.50bpw-h8
gemma-4-31B-it-uncensored-heretic-exl3-4.50bpw-h8 is an open-weight model from Sleepyj, released under Apache License 2.0. It has 10.6B parameters and a 262,144-token context. At 16-bit it needs about 25.4 GB of GPU memory, which fits on 1x MI300X from $1.85 an hour, at the lowest prices in the SAVRN Index. It draws 52 downloads a month.
This is an EXL3 build of llmfan46/gemma-4-31B-it-uncensored-heretic for ExLlamaV3 and TabbyAPI. The vision tower is quantized to 6 bits. It loads only with ExLlamaV3 or TabbyAPI's exllamav3 loader, not with Transformers, vLLM or llama.cpp.
Runs On
What it takes to serve gemma-4-31B-it-uncensored-heretic-exl3-4.50bpw-h8 (10.6B parameters): the memory its weights need at each precision, and the cheapest way to rent enough data-center GPUs to hold them.
| Precision | Weights | Memory needed | Cheapest setup | Per hour | Also fits |
|---|---|---|---|---|---|
| 16-bit | 21.2 GB | 25.4 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
| 8-bit | 10.6 GB | 12.7 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
| 4-bit | 5.3 GB | 6.4 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
Memory is the weights at that precision plus 20% for the runtime and a short context; a long context needs more. Prices are the lowest on-demand hourly rates in the SAVRN Index, read Oct 9, 2026.
Model Card
By Sleepyj, published under apache-2.0, revision 593a63f3f4f5.
This is an EXL3 build of llmfan46/gemma-4-31B-it-uncensored-heretic
for ExLlamaV3 and TabbyAPI. The vision tower is quantized to 6 bits. It loads only with ExLlamaV3 or TabbyAPI's exllamav3 loader,
not with Transformers, vLLM or llama.cpp.
This is not the coder3101 heretic. That 4.00bpw quant is
sjoe1244/gemma-4-31B-it-heretic-exl3-4.00bpw-h6.
On a 24 GB RTX 4090 this quant tops out at about 67K prompt with an 8-bit KV cache, or about 118K with a 4-bit KV cache. It does not fit 139,264 context at all. If you need 128K+ on one 24 GB card, use the 4.15bpw-h6 or 4.00bpw-h6 quant instead. See RTX 4090 (24 GB) limits below for the measured numbers.
Quantization
| Method | EXL3, ExLlamaV3 1.5.3 |
| Weights | 4.50 bpw (measured 4.50 bpw over the decoder layers) |
Head (lm_head) |
8 bits |
| Vision tower | 6 bits |
| Codebook | mul1, out_scales always |
| Calibration | 250 rows x 2048 cols (ExLlamaV3 default set) |
| Source | llmfan46/gemma-4-31B-it-uncensored-heretic, BF16 (Gemma4ForConditionalGeneration) |
Configuration
- Architecture
- Gemma4ForConditionalGeneration
- Context length (tokens)
- 262,144
- Layers
- 60
- Hidden size
- 5,376
- Feed-forward size
- 21,504
- Attention heads
- 32
- Key/value heads
- 16
- Head dimension
- 256
- Vocabulary size
- 262,144
- Sliding window (tokens)
- 1,024
- Model type
- gemma4
- Quantization
- exl3
Identity and Version
- Repository
- sjoe1244/gemma-4-31B-it-uncensored-heretic-exl3-4.50bpw-h8
- Publisher
- Sleepyj
- Task
- Not stated by the source
- Modality
- Other
- Library
- exllamav3
- Parameters
- 10.6B parameters
- Languages
- Not stated by the source
- Revision
- 593a63f3f4f5b689fae1c49addcba2aecf2dd0e6
- First published
- 2026-09-29
- Last updated
- 2026-10-09
Files and Weights
15 files, 21.2 GB in total. The weights are 3 files totalling 21.2 GB in safetensors.
Every file
| File | Type | Size | SHA-256 |
|---|---|---|---|
| model-00001-of-00003.safetensors | Weights | 8.4 GB | 01261222a833 |
| model-00002-of-00003.safetensors | Weights | 8.4 GB | 1ca66f30e521 |
| model-00003-of-00003.safetensors | Weights | 4.3 GB | 91cabb5fdf5d |
| config.json | Configuration | 5.8 KB | — |
| generation_config.json | Configuration | 217 B | — |
| model.safetensors.index.json | Configuration | 309.0 KB | — |
| processor_config.json | Configuration | 1.7 KB | — |
| quantization_config.json | Configuration | 664.0 KB | — |
| README.md | Documentation | 9.6 KB | — |
| assets/kv-cache-quality.png | Other | 147.0 KB | 4b849dd6e2d9 |
| assets/rtx4090-fit.png | Other | 153.1 KB | afa1278702fd |
| chat_template.jinja | Other | 23.1 KB | — |
| .gitattributes | Repository | 1.7 KB | — |
| tokenizer.json | Tokenizer | 32.2 MB | a2619fe11b50 |
| tokenizer_config.json | Tokenizer | 2.1 KB | — |
License and Download
- License
- apache-2.0
- Access
- Open weights, no gate
- Download size
- 21.2 GB
Released by Sleepyj through its official repository on Hugging Face. Read the license.
Built From
- Derived from llmfan46/gemma-4-31B-it-uncensored-heretic
- Quantized from llmfan46/gemma-4-31B-it-uncensored-heretic
Memory Requirements
| Precision | Weights in memory |
|---|---|
| As published | 21.2 GB |
| 16-bit | 21.2 GB |
| 8-bit | 10.6 GB |
| 4-bit | 5.3 GB |
Weights only, from the published parameter count; the key-value cache and runtime add to this.
Questions About gemma-4-31B-it-uncensored-heretic-exl3-4.50bpw-h8
How much GPU memory does gemma-4-31B-it-uncensored-heretic-exl3-4.50bpw-h8 need?
About 25.4 GB at 16-bit and 6.4 GB at 4-bit: the weights (10.6B parameters) plus a working margin. A long context needs more.
What is the cheapest GPU to run gemma-4-31B-it-uncensored-heretic-exl3-4.50bpw-h8 on?
At 16-bit, 1x MI300X from $1.85 an hour; at 4-bit, 1x MI300X from $1.85 an hour, at the lowest on-demand prices the SAVRN Index lists.
Can I use gemma-4-31B-it-uncensored-heretic-exl3-4.50bpw-h8 commercially?
Yes. gemma-4-31B-it-uncensored-heretic-exl3-4.50bpw-h8 is released under Apache License 2.0. The Apache License 2.0 is a permissive open-source license. It permits commercial use, modification and redistribution. It requires keeping the license and copyright notices and any NOTICE file, stating significant changes, and it includes an express patent grant from contributors.
What is gemma-4-31B-it-uncensored-heretic-exl3-4.50bpw-h8's context length?
262,144 tokens, from the maximum position embeddings in its published configuration.