FP8-dynamic quantized variant of gemma-4-12B-it.
Open-weight model · Any to any
gemma-4-12B-it-qat-w4a16-ct
by Google google/gemma-4-12B-it-qat-w4a16-ct
Gemma is a family of open models built by Google DeepMind. Gemma 4 models are multimodal, handling text and image input (with audio supported on E2B, E4B, and 12B) and generating text output.
Parameters13.3B
Context262,144
Weights10.3 GB
Licenseapache-2.0
AccessOpen weights
Monthly Downloads863.7k
Runs On
What it takes to serve gemma-4-12B-it-qat-w4a16-ct (13.3B parameters): the memory its weights need at each precision, and the cheapest way to rent enough data-center GPUs to hold them.
| Precision | Weights | Memory needed | Cheapest setup | Per hour | Also fits |
|---|---|---|---|---|---|
| 16-bit | 26.6 GB | 31.9 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
| 8-bit | 13.3 GB | 16.0 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
| 4-bit | 6.7 GB | 8.0 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
Memory is the weights at that precision plus 20% for the runtime and a short context; a long context needs more. Prices are the lowest on-demand hourly rates in the SAVRN Index, read Sep 18, 2026.
SAVRN's Notes on gemma-4-12B-it-qat-w4a16-ct
Eight gigabytes is the number that matters here. Google's 4-bit quantization holds 6.7 GB of weights and needs 8.0 GB to run, against 16.0 GB at 8-bit and 31.9 GB at 16-bit. The cheapest card in our table, one MI300X with 192 GB at $1.85 an hour on demand, has more room than the model asks for, so the hardware question becomes how many instances you stack on it, not whether it fits. Text and image go in, audio too on this 12B size, and text comes out across a 262,144-token window.
Apache 2.0 allows commercial use, modification and redistribution as long as the license, copyright and NOTICE files travel with the weights. Before committing, confirm your serving stack handles the Gemma4UnifiedForConditionalGeneration architecture, note that these weights derive from the q4_0-unquantized variant, and read the technical report at arXiv:2607.02770. We list no per-token host prices for it yet.
Model Card
By Google, published under apache-2.0, revision 1d2c2d7f2466.
Hugging Face |
GitHub |
Launch Blog |
Documentation |
Technical Report
License: Apache 2.0 | Authors: Google DeepMind
Configuration
- Architecture
- Gemma4UnifiedForConditionalGeneration
- Context length (tokens)
- 262,144
- Layers
- 48
- Hidden size
- 3,840
- Feed-forward size
- 15,360
- Attention heads
- 16
- Key/value heads
- 8
- Head dimension
- 256
- Vocabulary size
- 262,144
- Sliding window (tokens)
- 1,024
- Model type
- gemma4_unified
- Quantization
- compressed-tensors
Identity and Version
- Repository
- google/gemma-4-12B-it-qat-w4a16-ct
- Publisher
- Task
- Any to any
- Modality
- Multimodal
- Library
- transformers
- Parameters
- 13.3B parameters
- Languages
- Not stated by the source
- Revision
- 1d2c2d7f2466070e69d6fb3fd5ce9a7d75f2f6ee
- First published
- 2026-06-05
- Last updated
- 2026-07-20
Files and Weights
10 files, 10.3 GB in total. The weights are 1 file totalling 10.3 GB in safetensors.
Weights1 file · 10.3 GB
Configuration4 files · 8.7 KB
Tokenizer2 files · 32.2 MB
Documentation1 file · 29.4 KB
Other1 file · 18.7 KB
Repository1 file · 1.6 KB
Every file
| File | Type | Size | SHA-256 |
|---|---|---|---|
| model.safetensors | Weights | 10.3 GB | 60b6e3989502 |
| config.json | Configuration | 6.2 KB | — |
| generation_config.json | Configuration | 255 B | — |
| processor_config.json | Configuration | 1.4 KB | — |
| recipe.yaml | Configuration | 884 B | — |
| README.md | Documentation | 29.4 KB | — |
| chat_template.jinja | Other | 18.7 KB | — |
| .gitattributes | Repository | 1.6 KB | — |
| tokenizer.json | Tokenizer | 32.2 MB | cc8d3a0ce364 |
| tokenizer_config.json | Tokenizer | 3.8 KB | — |
License and Download
- License
- apache-2.0
- Access
- Open weights, no gate
- Download size
- 10.3 GB
Download from Google
Released by Google through Kaggle. Read the license.
Built From
- Derived from google/gemma-4-12B-it-qat-q4_0-unquantized
- Described by arXiv:2607.02770
- Quantized from google/gemma-4-12B-it-qat-q4_0-unquantized
Memory Requirements
| Precision | Weights in memory |
|---|---|
| As published | 10.3 GB |
| 16-bit | 26.6 GB |
| 8-bit | 13.3 GB |
| 4-bit | 6.7 GB |
Weights only, from the published parameter count; the key-value cache and runtime add to this.
Questions About gemma-4-12B-it-qat-w4a16-ct
How much GPU memory does gemma-4-12B-it-qat-w4a16-ct need?
About 31.9 GB at 16-bit and 8 GB at 4-bit: the weights (13.3B parameters) plus a working margin. A long context needs more.
What is the cheapest GPU to run gemma-4-12B-it-qat-w4a16-ct on?
At 16-bit, 1x MI300X from $1.85 an hour; at 4-bit, 1x MI300X from $1.85 an hour, at the lowest on-demand prices the SAVRN Index lists.
Can I use gemma-4-12B-it-qat-w4a16-ct commercially?
Yes. gemma-4-12B-it-qat-w4a16-ct is released under Apache License 2.0. The Apache License 2.0 is a permissive open-source license. It permits commercial use, modification and redistribution. It requires keeping the license and copyright notices and any NOTICE file, stating significant changes, and it includes an express patent grant from contributors.
What is gemma-4-12B-it-qat-w4a16-ct's context length?
262,144 tokens, from the maximum position embeddings in its published configuration.
Similar Models
Gemma is a family of open models built by Google DeepMind. Gemma 4 models are multimodal, handling text and image input (with audio supported on E2B, E4B, and 12B) and generating text output. This release includes open-weights models in both pre-trained and instruction-tuned variants. Gemma 4 features a context window of up to 256K tokens and maintains multilingual support in over 140 languages. Featuring both Dense and Mixture-of-Experts (MoE) architectures, Gemma 4 is well-suited for tasks like text generation, coding, and reasoning. The models are available in five distinct sizes: E2B, E4B, 12B, 26B A4B, and 31B. Their diverse sizes make them deployable in environments ranging from…
Gemma is a family of open models built by Google DeepMind. Gemma 4 models are multimodal, handling text and image input (with audio supported on E2B, E4B, and 12B) and generating text output. This release includes open-weights models in both pre-trained and instruction-tuned variants. Gemma 4 features a context window of up to 256K tokens and maintains multilingual support in over 140 languages. Featuring both Dense and Mixture-of-Experts (MoE) architectures, Gemma 4 is well-suited for tasks like text generation, coding, and reasoning. The models are available in five distinct sizes: E2B, E4B, 12B, 26B A4B, and 31B. Their diverse sizes make them deployable in environments ranging from…
Self-quantized FP8 (dynamic) of google/gemma-4-12B-it — Google's encoder-free omni model (text + image + audio + video). Quantized and benchmarked on an NVIDIA DGX Spark (GB10, sm121a). TL;DR: 13 GB on disk (from 23 GB BF16), 15.9 tok/s on a GB10 via vLLM, all four modalities intact. Data-free — no calibration needed. If you want the smallest + fastest build, see the sibling NVFP4 weight-only repo. FP8 is the conservative choice (dynamic activations, no calibration, widest kernel support). I scored all three formats on MMLU (English, 57 subjects) and TMMLU+ (Traditional Chinese, 66 subjects) with lm-evaluation-harness, 5-shot, chat template applied, limit=30 (N ≈ 1,710 EN / 1,980 TC, ±~1.0…
Gemma is a family of open models built by Google DeepMind. Gemma 4 models are multimodal, handling text and image input (with audio supported on E2B, E4B, and 12B) and generating text output. This release includes open-weights models in both pre-trained and instruction-tuned variants. Gemma 4 features a context window of up to 256K tokens and maintains multilingual support in over 140 languages. Featuring both Dense and Mixture-of-Experts (MoE) architectures, Gemma 4 is well-suited for tasks like text generation, coding, and reasoning. The models are available in five distinct sizes: E2B, E4B, 12B, 26B A4B, and 31B. Their diverse sizes make them deployable in environments ranging from…
LM Studio Community models highlights program. Highlighting new & noteworthy models by the community. Join the conversation on Discord. 4-bit quantized version of gemma-4-12B-it using MLX, optimized for Apple Silicon. Special thanks to the Apple Machine Learning Research team for creating MLX. LM Studio is not the creator, originator, or owner of any Model featured in the Community Model Program. Each Community Model is created and provided by third parties. LM Studio does not endorse, support, represent or guarantee the completeness, truthfulness, accuracy, or reliability of any Community Model. You understand that Community Models can produce content that might be offensive, harmful…