# gemma-4-31B-it-uncensored-heretic-exl3-4.50bpw-h8 by Sleepyj
Source: https://savrn.com/models/gemma-4-31b-it-uncensored-heretic-exl3-4-50bpw-h8
Markdown alternate of the page above; the site index is https://savrn.com/llms.txt

---

## Runs On

What it takes to serve gemma-4-31B-it-uncensored-heretic-exl3-4.50bpw-h8 (10.6B parameters): the memory its weights need at each precision, and the cheapest way to rent enough data-center GPUs to hold them.

| Precision | Weights | Memory needed | Cheapest setup | Per hour | Also fits |
| --- | --- | --- | --- | --- | --- |
| 16-bit | 21.2 GB | 25.4 GB | 1x [MI300X](https://savrn.com/ai-index/pricing/gpus/mi300x) (192 GB) Vultr | $1.85 | [1x H100](https://savrn.com/ai-index/pricing/gpus/h100) $1.99 · [1x MI325X](https://savrn.com/ai-index/pricing/gpus/mi325x) $2.00 |
| 8-bit | 10.6 GB | 12.7 GB | 1x [MI300X](https://savrn.com/ai-index/pricing/gpus/mi300x) (192 GB) Vultr | $1.85 | [1x H100](https://savrn.com/ai-index/pricing/gpus/h100) $1.99 · [1x MI325X](https://savrn.com/ai-index/pricing/gpus/mi325x) $2.00 |
| 4-bit | 5.3 GB | 6.4 GB | 1x [MI300X](https://savrn.com/ai-index/pricing/gpus/mi300x) (192 GB) Vultr | $1.85 | [1x H100](https://savrn.com/ai-index/pricing/gpus/h100) $1.99 · [1x MI325X](https://savrn.com/ai-index/pricing/gpus/mi325x) $2.00 |

Memory is the weights at that precision plus 20% for the runtime and a short context; a long context needs more. Prices are the lowest on-demand hourly rates in the [SAVRN Index](https://savrn.com/ai-index/pricing/gpus), read Oct 9, 2026.

[gemma-4-31B-it-uncensored-heretic-exl3-4.50bpw-h8 on every accelerator the SAVRN Index prices, at every precision](https://savrn.com/models/gemma-4-31b-it-uncensored-heretic-exl3-4-50bpw-h8/gpus)

## Model Card

By Sleepyj, published under apache-2.0, revision 593a63f3f4f5.

This is an EXL3 build of [llmfan46/gemma-4-31B-it-uncensored-heretic](https://huggingface.co/llmfan46/gemma-4-31B-it-uncensored-heretic) for ExLlamaV3 and TabbyAPI. The vision tower is quantized to 6 bits. It loads only with ExLlamaV3 or TabbyAPI's exllamav3 loader, not with Transformers, vLLM or llama.cpp.

This is not the coder3101 heretic. That 4.00bpw quant is [sjoe1244/gemma-4-31B-it-heretic-exl3-4.00bpw-h6](https://huggingface.co/sjoe1244/gemma-4-31B-it-heretic-exl3-4.00bpw-h6).

On a 24 GB RTX 4090 this quant tops out at about 67K prompt with an 8-bit KV cache, or about 118K with a 4-bit KV cache. It does not fit 139,264 context at all. If you need 128K+ on one 24 GB card, use the [4.15bpw-h6](https://huggingface.co/sjoe1244/gemma-4-31B-it-uncensored-heretic-exl3-4.15bpw-h6) or [4.00bpw-h6](https://huggingface.co/sjoe1244/gemma-4-31B-it-uncensored-heretic-exl3-4.00bpw-h6) quant instead. See RTX 4090 (24 GB) limits below for the measured numbers.

### Quantization

| Method | EXL3, ExLlamaV3 1.5.3 |
| --- | --- |
| Weights | 4.50 bpw (measured 4.50 bpw over the decoder layers) |
| Head (lm_head) | 8 bits |
| Vision tower | 6 bits |
| Codebook | mul1, out_scales always |
| Calibration | 250 rows x 2048 cols (ExLlamaV3 default set) |
| Source | llmfan46/gemma-4-31B-it-uncensored-heretic, BF16 (Gemma4ForConditionalGeneration) |

[Read the full model card (1,225 words)](https://savrn.com/models/gemma-4-31b-it-uncensored-heretic-exl3-4-50bpw-h8/card)

## Configuration

Architecture

Gemma4ForConditionalGeneration

Context length (tokens)

262,144

Layers

60

Hidden size

5,376

Feed-forward size

21,504

Attention heads

32

Key/value heads

16

Head dimension

256

Vocabulary size

262,144

Sliding window (tokens)

1,024

Model type

gemma4

Quantization

exl3

## Identity and Version

Repository

sjoe1244/gemma-4-31B-it-uncensored-heretic-exl3-4.50bpw-h8

Publisher

Sleepyj

Task

Not stated by the source

Modality

Other

Library

exllamav3

Parameters

10.6B parameters

Languages

Not stated by the source

Revision

593a63f3f4f5b689fae1c49addcba2aecf2dd0e6

First published

2026-09-29

Last updated

2026-10-09

## Files and Weights

15 files, 21.2 GB in total. The weights are 3 files totalling 21.2 GB in safetensors.

Weights3 files · 21.2 GB

Configuration5 files · 980.6 KB

Tokenizer2 files · 32.2 MB

Documentation1 file · 9.6 KB

Other3 files · 323.2 KB

Repository1 file · 1.7 KB

Every file

| File | Type | Size | SHA-256 |
| --- | --- | --- | --- |
| model-00001-of-00003.safetensors | Weights | 8.4 GB | 01261222a833 |
| model-00002-of-00003.safetensors | Weights | 8.4 GB | 1ca66f30e521 |
| model-00003-of-00003.safetensors | Weights | 4.3 GB | 91cabb5fdf5d |
| config.json | Configuration | 5.8 KB | — |
| generation_config.json | Configuration | 217 B | — |
| model.safetensors.index.json | Configuration | 309.0 KB | — |
| processor_config.json | Configuration | 1.7 KB | — |
| quantization_config.json | Configuration | 664.0 KB | — |
| README.md | Documentation | 9.6 KB | — |
| assets/kv-cache-quality.png | Other | 147.0 KB | 4b849dd6e2d9 |
| assets/rtx4090-fit.png | Other | 153.1 KB | afa1278702fd |
| chat_template.jinja | Other | 23.1 KB | — |
| .gitattributes | Repository | 1.7 KB | — |
| tokenizer.json | Tokenizer | 32.2 MB | a2619fe11b50 |
| tokenizer_config.json | Tokenizer | 2.1 KB | — |

## License and Download

License

apache-2.0

Access

Open weights, no gate

Download size

21.2 GB

[Download from Sleepyj](https://huggingface.co/sjoe1244/gemma-4-31B-it-uncensored-heretic-exl3-4.50bpw-h8)

Released by Sleepyj through its official repository on Hugging Face. [Read the license](https://www.apache.org/licenses/LICENSE-2.0).

## Built From

- Derived from llmfan46/gemma-4-31B-it-uncensored-heretic
- Quantized from llmfan46/gemma-4-31B-it-uncensored-heretic

## Memory Requirements

| Precision | Weights in memory |
| --- | --- |
| As published | 21.2 GB |
| 16-bit | 21.2 GB |
| 8-bit | 10.6 GB |
| 4-bit | 5.3 GB |

Weights only, from the published parameter count; the key-value cache and runtime add to this.

## Questions About gemma-4-31B-it-uncensored-heretic-exl3-4.50bpw-h8

### How much GPU memory does gemma-4-31B-it-uncensored-heretic-exl3-4.50bpw-h8 need?

About 25.4 GB at 16-bit and 6.4 GB at 4-bit: the weights (10.6B parameters) plus a working margin. A long context needs more.

### What is the cheapest GPU to run gemma-4-31B-it-uncensored-heretic-exl3-4.50bpw-h8 on?

At 16-bit, 1x MI300X from $1.85 an hour; at 4-bit, 1x MI300X from $1.85 an hour, at the lowest on-demand prices the SAVRN Index lists.

### Can I use gemma-4-31B-it-uncensored-heretic-exl3-4.50bpw-h8 commercially?

Yes. gemma-4-31B-it-uncensored-heretic-exl3-4.50bpw-h8 is released under Apache License 2.0. The Apache License 2.0 is a permissive open-source license. It permits commercial use, modification and redistribution. It requires keeping the license and copyright notices and any NOTICE file, stating significant changes, and it includes an express patent grant from contributors.

### What is gemma-4-31B-it-uncensored-heretic-exl3-4.50bpw-h8's context length?

262,144 tokens, from the maximum position embeddings in its published configuration.

## Sleepyj

[All models and datasets](https://savrn.com/model-publishers/sjoe1244)

## Versions

- [593a63f3f4f5](https://savrn.com/models/gemma-4-31b-it-uncensored-heretic-exl3-4-50bpw-h8/versions/593a63f3f4f5) · current 2026-10-09

## Explore More

- [All models under apache-2.0](https://savrn.com/models/licenses/apache-2-0)
- [Model comparisons](https://savrn.com/models/comparisons)
- [The model directory](https://savrn.com/models)
- [Open model prices by host](https://savrn.com/ai-index/pricing/open-models)

## Source

- Repository metadata, read 2026-10-09.
- [Hugging Face record](https://huggingface.co/sjoe1244/gemma-4-31B-it-uncensored-heretic-exl3-4.50bpw-h8)
- [How the hub is built](https://savrn.com/model-hub/methodology)