Open-weight model
gemma-4-e2b-it-OptiQ-4bit-decision
by MLX Community mlx-community/gemma-4-e2b-it-OptiQ-4bit-decision
gemma-4-e2b-it-OptiQ-4bit-decision is an open-weight model from MLX Community, released under Gemma Terms of Use. It has 4.6B parameters and a 131,072-token context. At 16-bit it needs about 11.1 GB of GPU memory, which fits on 1x MI300X from $1.85 an hour, at the lowest prices in the SAVRN Index.
gemma-4-e2b-it-OptiQ-4bit turned into a decision model with optiq lora train --decision: a LoRA on the backbone and a new joint schema head that scores noul, choice and score questions in one pass.
Runs On
What it takes to serve gemma-4-e2b-it-OptiQ-4bit-decision (4.6B parameters): the memory its weights need at each precision, and the cheapest way to rent enough data-center GPUs to hold them.
| Precision | Weights | Memory needed | Cheapest setup | Per hour | Also fits |
|---|---|---|---|---|---|
| 16-bit | 9.3 GB | 11.1 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
| 8-bit | 4.6 GB | 5.6 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
| 4-bit | 2.3 GB | 2.8 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
Memory is the weights at that precision plus 20% for the runtime and a short context; a long context needs more. Prices are the lowest on-demand hourly rates in the SAVRN Index, read Oct 12, 2026.
gemma-4-e2b-it-OptiQ-4bit-decision on every accelerator the SAVRN Index prices, at every precision
Model Card
gemma-4-e2b-it-OptiQ-4bit turned into a decision model with optiq lora train --decision: a LoRA on the backbone and a new joint schema head that scores noul, choice and score questions in one pass. It runs on Apple Silicon through MLX and OptiQ (optiq serve exposes /v1/systemone). Scored on the full 400-case test split (2,000 decisions) with the benchmark's own metrics. Latency is one request per case with all five questions, on an Apple M3 Max. This model was fitted on the benchmark's own train split and nothing else: 1,080 records, 3 epochs, LoRA rank 16. The test cases are disjoint from it, but they come from the same four workflows, so expect less on decisions of a different kind. The…
Excerpt from the card by MLX Community, licensed gemma.
Configuration
- Architecture
- Gemma4ForConditionalGeneration
- Context length (tokens)
- 131,072
- Layers
- 35
- Hidden size
- 1,536
- Feed-forward size
- 6,144
- Attention heads
- 8
- Key/value heads
- 1
- Head dimension
- 256
- Vocabulary size
- 262,144
- Sliding window (tokens)
- 512
- Model type
- gemma4
Identity and Version
- Repository
- mlx-community/gemma-4-e2b-it-OptiQ-4bit-decision
- Publisher
- MLX Community
- Task
- Not stated by the source
- Modality
- Other
- Library
- mlx
- Parameters
- 4.6B parameters
- Languages
- mlx
- Revision
- e8f725bb0a092ea078055a4c37690c9b8d6309f4
- First published
- 2026-10-11
- Last updated
- 2026-10-11
Files and Weights
18 files, 5.4 GB in total. The weights are 4 files totalling 5.4 GB in safetensors.
Every file
| File | Type | Size | SHA-256 |
|---|---|---|---|
| adapters.safetensors | Weights | 96.7 MB | 0a713e8aaf52 |
| joint_head.safetensors | Weights | 57.8 MB | 82095a5c7872 |
| model.safetensors | Weights | 4.3 GB | 088c5f8fed7c |
| optiq/optiq_vision.safetensors | Weights | 951.6 MB | 06968ae76236 |
| .eval_results/typed-decisions.yaml | Configuration | 2.2 KB | — |
| adapter_config.json | Configuration | 501 B | — |
| config.json | Configuration | 62.7 KB | — |
| decision_config.json | Configuration | 97 B | — |
| generation_config.json | Configuration | 208 B | — |
| joint_head_config.json | Configuration | 116 B | — |
| kv_config.json | Configuration | 982 B | — |
| model.safetensors.index.json | Configuration | 226.4 KB | — |
| optiq_metadata.json | Configuration | 28.8 KB | — |
| README.md | Documentation | 1.9 KB | — |
| chat_template.jinja | Other | 18.6 KB | — |
| .gitattributes | Repository | 1.6 KB | — |
| tokenizer.json | Tokenizer | 32.2 MB | cc8d3a0ce364 |
| tokenizer_config.json | Tokenizer | 2.7 KB | — |
License and Download
- License
- gemma
- Access
- Open weights, no gate
- Download size
- 5.4 GB
Released by MLX Community through its official repository on Hugging Face.
Built From
- Derived from mlx-community/gemma-4-e2b-it-OptiQ-4bit
- Quantized from mlx-community/gemma-4-e2b-it-OptiQ-4bit
Evaluations
Each result is shown as reported, with the conditions its reporter stated. None is a SAVRN measurement. A comparison lines two results up only when their configuration, unit and setup are all stated and identical.
| Benchmark | Conditions | Result | Reported by | Revision | Date |
|---|---|---|---|---|---|
| LocalLLaMA/typed-decisions | Task accuracyMetric accuracySetup Fitted by us on the benchmark's own train split only (1,080 records, LoRA rank 16 on the 4-bit backbone plus a new schema head, 3 epochs), then scored on the full 400-case test split (2,000 decisions): one request per case with all five questions, text only, the benchmark's own metrics, MLX on an…Comparison conditions not established | 0.8015 | Typed Decisions leaderboard (README) Reported by a third party |
Evaluated revision not stated | 2026-10-11 |
| LocalLLaMA/typed-decisions | Task brierMetric brierSetup Fitted by us on the benchmark's own train split only (1,080 records, LoRA rank 16 on the 4-bit backbone plus a new schema head, 3 epochs), then scored on the full 400-case test split (2,000 decisions): one request per case with all five questions, text only, the benchmark's own metrics, MLX on an…Comparison conditions not established | 0.0394 | Typed Decisions leaderboard (README) Reported by a third party |
Evaluated revision not stated | 2026-10-11 |
| LocalLLaMA/typed-decisions | Task eceMetric eceSetup Fitted by us on the benchmark's own train split only (1,080 records, LoRA rank 16 on the 4-bit backbone plus a new schema head, 3 epochs), then scored on the full 400-case test split (2,000 decisions): one request per case with all five questions, text only, the benchmark's own metrics, MLX on an…Comparison conditions not established | 0.181 | Typed Decisions leaderboard (README) Reported by a third party |
Evaluated revision not stated | 2026-10-11 |
| LocalLLaMA/typed-decisions | Task kl_from_goldMetric kl_from_goldSetup Fitted by us on the benchmark's own train split only (1,080 records, LoRA rank 16 on the 4-bit backbone plus a new schema head, 3 epochs), then scored on the full 400-case test split (2,000 decisions): one request per case with all five questions, text only, the benchmark's own metrics, MLX on an…Comparison conditions not established | 0.0723 | Typed Decisions leaderboard (README) Reported by a third party |
Evaluated revision not stated | 2026-10-11 |
Memory Requirements
| Precision | Weights in memory |
|---|---|
| As published | 5.4 GB |
| 16-bit | 9.3 GB |
| 8-bit | 4.6 GB |
| 4-bit | 2.3 GB |
Weights only, from the published parameter count; the key-value cache and runtime add to this.
Questions About gemma-4-e2b-it-OptiQ-4bit-decision
How much GPU memory does gemma-4-e2b-it-OptiQ-4bit-decision need?
About 11.1 GB at 16-bit and 2.8 GB at 4-bit: the weights (4.6B parameters) plus a working margin. A long context needs more.
What is the cheapest GPU to run gemma-4-e2b-it-OptiQ-4bit-decision on?
At 16-bit, 1x MI300X from $1.85 an hour; at 4-bit, 1x MI300X from $1.85 an hour, at the lowest on-demand prices the SAVRN Index lists.
Can I use gemma-4-e2b-it-OptiQ-4bit-decision commercially?
Yes, with conditions. gemma-4-e2b-it-OptiQ-4bit-decision is released under Gemma Terms of Use. Gemma models are released under Google's Gemma Terms of Use, which permit commercial use and redistribution subject to the Gemma Prohibited Use Policy, whose restrictions must be passed on to anyone the model is distributed to.
What is gemma-4-e2b-it-OptiQ-4bit-decision's context length?
131,072 tokens, from the maximum position embeddings in its published configuration.