Open-weight model
NVIDIA-Nemotron-3-Nano-30B-A3B-Heretic-W6A8
by Cassidy Barton Xananthium/NVIDIA-Nemotron-3-Nano-30B-A3B-Heretic-W6A8
NVIDIA-Nemotron-3-Nano-30B-A3B-Heretic-W6A8 is an open-weight model from Cassidy Barton, released under other. It has 7.5B parameters and a 262,144-token context. At 16-bit it needs about 17.9 GB of GPU memory, which fits on 1x MI300X from $1.85 an hour, at the lowest prices in the SAVRN Index.
Native compressed-tensors W6A8 checkpoint published by Xananthium. Credit and upstream terms: trohrbaugh/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16-heretic. The source model card is preserved in READMEORIGINAL.md.
Runs On
What it takes to serve NVIDIA-Nemotron-3-Nano-30B-A3B-Heretic-W6A8 (7.5B parameters): the memory its weights need at each precision, and the cheapest way to rent enough data-center GPUs to hold them.
| Precision | Weights | Memory needed | Cheapest setup | Per hour | Also fits |
|---|---|---|---|---|---|
| 16-bit | 14.9 GB | 17.9 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
| 8-bit | 7.5 GB | 9.0 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
| 4-bit | 3.7 GB | 4.5 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
Memory is the weights at that precision plus 20% for the runtime and a short context; a long context needs more. Prices are the lowest on-demand hourly rates in the SAVRN Index, read Oct 23, 2026.
Model Card
Native compressed-tensors W6A8 checkpoint published by Xananthium. Credit and upstream terms: trohrbaugh/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16-heretic. The source model card is preserved in READMEORIGINAL.md. Quantization provenance and retained high-precision components are recorded in conversion-receipt.json. I wanted W6A8, so that is what we built: six-bit stored weights, dynamic eight-bit activations, and vLLM doing the serving. Humming packs the INT6 values across INT32 word boundaries and unpacks them for the 3090's supported INT8 matrix math. The cards do not need a native six-bit Tensor Core instruction. The extra unpacking and scale work is part of the cost, and the speed tests…
Excerpt from the card by Cassidy Barton, licensed other.
Configuration
- Architecture
- NemotronHForCausalLM
- Context length (tokens)
- 262,144
- Layers
- 52
- Hidden size
- 2,688
- Feed-forward size
- 1,856
- Attention heads
- 32
- Key/value heads
- 2
- Head dimension
- 128
- Vocabulary size
- 131,072
- Routed experts
- 128
- Experts active per token
- 6
- RoPE base
- 10,000
- Model type
- nemotron_h
- Quantization
- compressed-tensors
Identity and Version
- Repository
- Xananthium/NVIDIA-Nemotron-3-Nano-30B-A3B-Heretic-W6A8
- Publisher
- Cassidy Barton
- Task
- Not stated by the source
- Modality
- Other
- Library
- vllm
- Parameters
- 7.5B parameters
- Languages
- Not stated by the source
- Revision
- 27383d65558acacdde5d38ad789748c0148e123c
- First published
- 2026-10-07
- Last updated
- 2026-10-07
Files and Weights
33 files, 26.5 GB in total. The weights are 7 files totalling 26.5 GB in safetensors.
Every file
| File | Type | Size | SHA-256 |
|---|---|---|---|
| model-00001-of-00007.safetensors | Weights | 4.5 GB | 6da638455daa |
| model-00002-of-00007.safetensors | Weights | 4.1 GB | f0bc379be163 |
| model-00003-of-00007.safetensors | Weights | 4.1 GB | 6f2a2bb42659 |
| model-00004-of-00007.safetensors | Weights | 4.1 GB | 139093117621 |
| model-00005-of-00007.safetensors | Weights | 4.1 GB | 852f3ef73630 |
| model-00006-of-00007.safetensors | Weights | 4.1 GB | 557fb0f12df0 |
| model-00007-of-00007.safetensors | Weights | 1.7 GB | 49186e7fcdd3 |
| config.json | Configuration | 3.3 KB | — |
| configuration_nemotron_h.py | Configuration | 12.9 KB | — |
| conversion-receipt.json | Configuration | 2.5 KB | — |
| generation_config.json | Configuration | 208 B | — |
| model.safetensors.index.json | Configuration | 1.8 MB | — |
| results/claude-code.json | Configuration | 480 B | — |
| results/cybermetric.json | Configuration | 45.8 KB | — |
| results/cybersecurity-policy.json | Configuration | 264 B | — |
| results/long-context.json | Configuration | 351 B | — |
| results/metadata-controls.json | Configuration | 4.6 KB | — |
| results/native-api.json | Configuration | 527 B | — |
| results/reasoning-consistency.json | Configuration | 8.0 KB | — |
| results/serving-profile.json | Configuration | 959 B | — |
| results/speed.json | Configuration | 5.4 KB | — |
| results/suite-status.json | Configuration | 1.9 KB | — |
| runtime/humming-compile-compat/humming_compile_compat.py | Configuration | 1.4 KB | — |
| source-receipt.json | Configuration | 1.3 KB | — |
| special_tokens_map.json | Configuration | 420 B | — |
| README.md | Documentation | 4.4 KB | — |
| README_ORIGINAL.md | Documentation | 74.3 KB | — |
| runtime/humming-compile-compat/README.md | Documentation | 978 B | — |
| chat_template.jinja | Other | 10.5 KB | — |
| runtime/humming-compile-compat/pyproject.toml | Other | 422 B | — |
| .gitattributes | Repository | 1.6 KB | — |
| tokenizer.json | Tokenizer | 17.1 MB | 5215d9635c58 |
| tokenizer_config.json | Tokenizer | 188.0 KB | — |
License and Download
- License
- other
- Access
- Open weights, no gate
- Download size
- 26.5 GB
Released by Cassidy Barton through its official repository on Hugging Face.
Built From
- Derived from trohrbaugh/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16-heretic
Memory Requirements
| Precision | Weights in memory |
|---|---|
| As published | 26.5 GB |
| 16-bit | 14.9 GB |
| 8-bit | 7.5 GB |
| 4-bit | 3.7 GB |
Weights only, from the published parameter count; the key-value cache and runtime add to this.
Questions About NVIDIA-Nemotron-3-Nano-30B-A3B-Heretic-W6A8
How much GPU memory does NVIDIA-Nemotron-3-Nano-30B-A3B-Heretic-W6A8 need?
About 17.9 GB at 16-bit and 4.5 GB at 4-bit: the weights (7.5B parameters) plus a working margin. A long context needs more.
What is the cheapest GPU to run NVIDIA-Nemotron-3-Nano-30B-A3B-Heretic-W6A8 on?
At 16-bit, 1x MI300X from $1.85 an hour; at 4-bit, 1x MI300X from $1.85 an hour, at the lowest on-demand prices the SAVRN Index lists.
What license is NVIDIA-Nemotron-3-Nano-30B-A3B-Heretic-W6A8 released under?
other, as its publisher declares it. Read the license text before commercial use.
What is NVIDIA-Nemotron-3-Nano-30B-A3B-Heretic-W6A8's context length?
262,144 tokens, from the maximum position embeddings in its published configuration.