Open-weight model
llama-guard-3-1b-llm-int8
by Marcus Nogueira marcusnogueira/llama-guard-3-1b-llm-int8
llama-guard-3-1b-llm-int8 is an open-weight model from Marcus Nogueira. It has 1.5B parameters and a 131,072-token context. At 16-bit it needs about 3.6 GB of GPU memory, which fits on 1x MI300X from $1.85 an hour, at the lowest prices in the SAVRN Index.
Pre-quantized LLM.int8() checkpoint of meta-llama/Llama-Guard-3-1B at revision acf7aafa60f0410f8f42b1fa35e077d705892029. LLM.int8() stores the main linear weights in INT8 while dynamically processing outlier features through its higher-precision path.
Runs On
What it takes to serve llama-guard-3-1b-llm-int8 (1.5B parameters): the memory its weights need at each precision, and the cheapest way to rent enough data-center GPUs to hold them.
| Precision | Weights | Memory needed | Cheapest setup | Per hour | Also fits |
|---|---|---|---|---|---|
| 16-bit | 3.0 GB | 3.6 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
| 8-bit | 1.5 GB | 1.8 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
| 4-bit | 0.7 GB | 0.9 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
Memory is the weights at that precision plus 20% for the runtime and a short context; a long context needs more. Prices are the lowest on-demand hourly rates in the SAVRN Index, read Oct 19, 2026.
llama-guard-3-1b-llm-int8 on every accelerator the SAVRN Index prices, at every precision
Model Card
Pre-quantized LLM.int8() checkpoint of meta-llama/Llama-Guard-3-1B at revision acf7aafa60f0410f8f42b1fa35e077d705892029. LLM.int8() stores the main linear weights in INT8 while dynamically processing outlier features through its higher-precision path. This is not the same scheme as static INT8 W8A8 quantization. The resolved source revision, package versions, module inventory, tensor dtypes, and file hashes are recorded in quantizationmanifest.json, runtimeenvironment.json, verification.json, and SHA256SUMS. This checkpoint passed four fixed English/Portuguese safe/unsafe smoke cases in both BF16 and FP16 with vLLM 0.31.0, bitsandbytes 0.50.2, and vllm-bnb-plugin 0.0.3 on an NVIDIA A40.…
Excerpt from the card by Marcus Nogueira.
Configuration
- Architecture
- LlamaForCausalLM
- Context length (tokens)
- 131,072
- Layers
- 16
- Hidden size
- 2,048
- Feed-forward size
- 8,192
- Attention heads
- 32
- Key/value heads
- 8
- Head dimension
- 64
- Vocabulary size
- 128,256
- Model type
- llama
- Quantization
- bitsandbytes
Identity and Version
- Repository
- marcusnogueira/llama-guard-3-1b-llm-int8
- Publisher
- Marcus Nogueira
- Task
- Not stated by the source
- Modality
- Other
- Library
- Not stated by the source
- Parameters
- 1.5B parameters
- Languages
- en, pt
- Revision
- 29fdb1f49a71999a8013f4fac5524901c546bc4b
- First published
- 2026-10-14
- Last updated
- 2026-10-14
Files and Weights
14 files, 2.0 GB in total. The weights are 1 file totalling 2.0 GB in safetensors.
Every file
| File | Type | Size | SHA-256 |
|---|---|---|---|
| model.safetensors | Weights | 2.0 GB | cd43b568377c |
| config.json | Configuration | 1.4 KB | — |
| generation_config.json | Configuration | 183 B | — |
| quantization_manifest.json | Configuration | 13.0 KB | — |
| runtime_environment.json | Configuration | 468 B | — |
| verification.json | Configuration | 9.8 KB | — |
| vllm_smoke_bfloat16.json | Configuration | 6.2 KB | — |
| vllm_smoke_float16.json | Configuration | 6.2 KB | — |
| README.md | Documentation | 1.5 KB | — |
| SHA256SUMS | Other | 1.0 KB | — |
| chat_template.jinja | Other | 2.5 KB | — |
| .gitattributes | Repository | 1.6 KB | — |
| tokenizer.json | Tokenizer | 17.2 MB | 6b9e4e7fb171 |
| tokenizer_config.json | Tokenizer | 295 B | — |
License and Download
- License
- Not stated by the source
- Access
- Open weights, no gate
- Download size
- 2.0 GB
Released by Marcus Nogueira through its official repository on Hugging Face.
Built From
- Derived from meta-llama/Llama-Guard-3-1B
- Quantized from meta-llama/Llama-Guard-3-1B
Memory Requirements
| Precision | Weights in memory |
|---|---|
| As published | 2.0 GB |
| 16-bit | 3.0 GB |
| 8-bit | 1.5 GB |
| 4-bit | 0.7 GB |
Weights only, from the published parameter count; the key-value cache and runtime add to this.
Questions About llama-guard-3-1b-llm-int8
How much GPU memory does llama-guard-3-1b-llm-int8 need?
About 3.6 GB at 16-bit and 0.9 GB at 4-bit: the weights (1.5B parameters) plus a working margin. A long context needs more.
What is the cheapest GPU to run llama-guard-3-1b-llm-int8 on?
At 16-bit, 1x MI300X from $1.85 an hour; at 4-bit, 1x MI300X from $1.85 an hour, at the lowest on-demand prices the SAVRN Index lists.
What is llama-guard-3-1b-llm-int8's context length?
131,072 tokens, from the maximum position embeddings in its published configuration.