SAVRN
Search Contact SAVRN

Open-weight model · Text generation

nemotron-30b-a3b-gptq-component-jang-g64-lora-mtp

by Roman Romenskyi roman220220/nemotron-30b-a3b-gptq-component-jang-g64-lora-mtp

Parameters31.6B
Context262,144
Weights17.8 GB
License
AccessOpen weights
Monthly Downloads

Runs On

What it takes to serve nemotron-30b-a3b-gptq-component-jang-g64-lora-mtp (31.6B parameters): the memory its weights need at each precision, and the cheapest way to rent enough data-center GPUs to hold them.

PrecisionWeightsMemory neededCheapest setupPer hourAlso fits
16-bit 63.2 GB 75.8 GB 1x MI300X (192 GB)
Vultr
$1.85 1x H100 $1.99 · 1x MI325X $2.00
8-bit 31.6 GB 37.9 GB 1x MI300X (192 GB)
Vultr
$1.85 1x H100 $1.99 · 1x MI325X $2.00
4-bit 15.8 GB 18.9 GB 1x MI300X (192 GB)
Vultr
$1.85 1x H100 $1.99 · 1x MI325X $2.00

Memory is the weights at that precision plus 20% for the runtime and a short context; a long context needs more. Prices are the lowest on-demand hourly rates in the SAVRN Index, read Sep 18, 2026.

Model Card

The publisher has not written a card for this model.

Configuration

Architecture
NemotronHForCausalLM
Context length (tokens)
262,144
Layers
52
Hidden size
2,688
Feed-forward size
1,856
Attention heads
32
Key/value heads
2
Head dimension
128
Vocabulary size
131,072
Routed experts
128
Experts active per token
6
RoPE base
10,000
Model type
nemotron_h

Identity and Version

Repository
roman220220/nemotron-30b-a3b-gptq-component-jang-g64-lora-mtp
Publisher
Roman Romenskyi
Task
Text generation
Modality
Text
Library
mlx
Parameters
31.6B parameters
Languages
en
Revision
19cfc4f03d55446c3bbd49874ff4a67ef06c78a2
First published
2026-09-18
Last updated
2026-09-18

Files and Weights

13 files, 17.8 GB in total. The weights are 5 files totalling 17.8 GB in safetensors.

Weights5 files · 17.8 GB
Configuration3 files · 107.6 KB
Tokenizer2 files · 17.1 MB
Documentation1 file · 81 B
Other1 file · 9.9 KB
Repository1 file · 1.6 KB
Every file
FileTypeSizeSHA-256
model-00001-of-00004.safetensorsWeights5.2 GB 8f67bac5f6c7
model-00002-of-00004.safetensorsWeights5.3 GB 688dc21215da
model-00003-of-00004.safetensorsWeights5.2 GB 7b3fb273d57f
model-00004-of-00004.safetensorsWeights1.0 GB b6018d28b168
model-mtp.safetensorsWeights1.1 GB f3243fa4b90f
config.jsonConfiguration42.6 KB
generation_config.jsonConfiguration210 B
model.safetensors.index.jsonConfiguration64.8 KB
README.mdDocumentation81 B
chat_template.jinjaOther9.9 KB
.gitattributesRepository1.6 KB
tokenizer.jsonTokenizer17.1 MB 623c34567aeb
tokenizer_config.jsonTokenizer538 B

License and Download

License
Not stated by the source
Access
Open weights, no gate
Download size
17.8 GB
Download from Roman Romenskyi

Released by Roman Romenskyi through its official repository on Hugging Face.

Memory Requirements

PrecisionWeights in memory
As published17.8 GB
16-bit63.2 GB
8-bit31.6 GB
4-bit15.8 GB

Weights only, from the published parameter count; the key-value cache and runtime add to this.

Questions About nemotron-30b-a3b-gptq-component-jang-g64-lora-mtp

How much GPU memory does nemotron-30b-a3b-gptq-component-jang-g64-lora-mtp need?

About 75.8 GB at 16-bit and 18.9 GB at 4-bit: the weights (31.6B parameters) plus a working margin. A long context needs more.

What is the cheapest GPU to run nemotron-30b-a3b-gptq-component-jang-g64-lora-mtp on?

At 16-bit, 1x MI300X from $1.85 an hour; at 4-bit, 1x MI300X from $1.85 an hour, at the lowest on-demand prices the SAVRN Index lists.

What is nemotron-30b-a3b-gptq-component-jang-g64-lora-mtp's context length?

262,144 tokens, from the maximum position embeddings in its published configuration.

Similar Models

Model · Text generation

NVIDIA-Nemotron-3-Nano-30B-A3B-BF16

NVIDIA

September 2025 \- December 2025 The post-training data has a cutoff date of November 28, 2025\. The pre-training data has a cutoff date of June 25, 2025\. Nemotron-3-Nano-30B-A3B-BF16 is a large language model (LLM) trained from scratch by NVIDIA, and designed as a unified model for both reasoning and non-reasoning tasks. It responds to user queries and tasks by first generating a reasoning trace and then concluding with a final response. The model's reasoning capabilities can be configured through a flag in the chat template. If the user prefers the model to provide its final answer without intermediate reasoning traces, it can be configured to do so, albeit with a slight decrease in…

Open weights other 31.6B parameters 262,144 tokens transformers

Fastino-Nemotron-3.5-Lightning-Finance is a 30B-parameter, 3B-active mixture-of-experts model specialized for financial reasoning, extraction, and research fine-tuned on LoRA with the Fastino Fine-Tuning Agent. The model targets financial document reasoning, numerical question answering over filings and tables, numeric span extraction, financial entity recognition, conversational analysis, and source-grounded financial research. The evaluation suite includes FinQA, TAT-QA, SEC-Num, FinEntity, BizFinBench, BigFinanceBench, ConvFinQA, and FiQA. The published weights are BF16 and require about 66 GB before runtime overhead. An 80 GB or larger GPU, or tensor parallelism across multiple GPUs, is…

Open weights apache-2.0 31.6B parameters 262,144 tokens transformers

Model · Text generation

OTel-2.0-LLM-31B-IT

Farbod Tavakkoli

OTel-2.0-LLM-31B-IT is a telecom-specialized instruction model post-trained from Gemma 4 31B-IT on approximately 440 billion telecom training tokens. It is the first release in the OTel 2.0 family and is designed to support telco-grade AI workflows across network operations, standards interpretation, product development, network configuration assistance, RAG, and telecom-specific question answering. OTel 2.0 extends the original OTel effort from a RAG-oriented telecom fine-tuning release into a larger domain-adapted training program. The model was trained from a much larger standards and telecom corpus, with new data preparation coverage for direct telecom QnA, abstention, RAG…

Open weights apache-2.0 31.3B parameters 262,144 tokens transformers

Model · Text generation

GLM-4.7-Flash

Z.ai

Join our Discord community. Check out the GLM-4.7 technical blog, technical report(GLM-4.5). Use GLM-4.7-Flash API services on Z.ai API Platform. One click to GLM-4.7. GLM-4.7-Flash is a 30B-A3B MoE model. As the strongest model in the 30B class, GLM-4.7-Flash offers a new option for lightweight deployment that balances performance and efficiency. Default Settings (Most Tasks) For multi-turn agentic tasks (τ²-Bench and Terminal Bench 2), please turn on Preserved Thinking mode. Terminal Bench, SWE Bench Verified τ^2-Bench For τ^2-Bench evaluation, we added an additional prompt to the Retail and Telecom user interaction to avoid failure modes caused by users ending the interaction…

Open weights mit 31.2B parameters 202,752 tokens transformers

Model · Text generation

Qwen3-VL-30B-A3B-Instruct-AWQ

QuantTrio

As of 2025-10-08, create a fresh Python environment and run: For more details, refer to vLLM Official Qwen3-VL Guide Meet Qwen3-VL — the most powerful vision-language model in the Qwen series to date. This generation delivers comprehensive upgrades across the board: superior text understanding & generation, deeper visual perception & reasoning, extended context length, enhanced spatial and video dynamics comprehension, and stronger agent interaction capabilities. Available in Dense and MoE architectures that scale from edge to cloud, with Instruct and reasoning‑enhanced Thinking editions for flexible, on‑demand deployment. Text Understanding on par with pure LLMs: Seamless text–vision…

Open weights apache-2.0 31.1B parameters 262,144 tokens transformers

Model · Text generation

granite-4.0-h-small-w8a8-llmcompressor

AMD

ZenDNN v6.1.0 - ZenTorch v2.13.0.0 - PyTorch v2.13.0.0 - LLM Compressor v0.13.0 - vLLM v0.29.0 This is a quantized version of granite-4.0-h-small created by AMD using LLM Compressor (compressed-tensors) for ZenDNN-optimized CPU inference. The model was quantized from granite-4.0-h-small using LLM Compressor via the Round-to-Nearest (RTN) algorithm. This reduces the model weights from 60.0 GiB to 30.4 GiB on disk (~49% reduction). granite-4.0-h-small is a hybrid Mamba-MoE model: of its 40 layers, 4 are full-attention blocks and the other 36 are Mamba (linear-attention) blocks, and every layer carries a 72-expert MoE block (top-10 routing) alongside a shared MLP. The recipe only needs two…

Open weights apache-2.0 32.2B parameters 131,072 tokens transformers