SAVRN
Search Contact SAVRN

Open-weight model · Any to any

Qwen3-Omni-30B-A3B-FP8

by Markus marksverdhei/Qwen3-Omni-30B-A3B-FP8

Block-wise FP8 quantization of Qwen/Qwen3-Omni-30B-A3B-Instruct. - Vision encoder (thinker.visual) - Audio tower (thinker.audiotower) - Code2Wav decoder (code2wav) - vLLM >= 0.13.0 with Qwen3-Omni support - 2x 24GB GPUs (e.g., RTX 3090) or equivalent - ~35 GB…

Parameters35.3B
Context
Weights37.4 GB
Licenseother
AccessOpen weights
Monthly Downloads40.2k

Runs On

What it takes to serve Qwen3-Omni-30B-A3B-FP8 (35.3B parameters): the memory its weights need at each precision, and the cheapest way to rent enough data-center GPUs to hold them.

PrecisionWeightsMemory neededCheapest setupPer hourAlso fits
16-bit 70.5 GB 84.6 GB 1x MI300X (192 GB)
Vultr
$1.85 1x MI325X $2.00 · 1x MI355X $2.59
8-bit 35.3 GB 42.3 GB 1x MI300X (192 GB)
Vultr
$1.85 1x H100 $1.99 · 1x MI325X $2.00
4-bit 17.6 GB 21.2 GB 1x MI300X (192 GB)
Vultr
$1.85 1x H100 $1.99 · 1x MI325X $2.00

Memory is the weights at that precision plus 20% for the runtime and a short context; a long context needs more. Prices are the lowest on-demand hourly rates in the SAVRN Index, read Sep 18, 2026.

Model Card

Block-wise FP8 quantization of Qwen/Qwen3-Omni-30B-A3B-Instruct. - Vision encoder (thinker.visual) - Audio tower (thinker.audiotower) - Code2Wav decoder (code2wav) - vLLM >= 0.13.0 with Qwen3-Omni support - 2x 24GB GPUs (e.g., RTX 3090) or equivalent - ~35 GB disk space Block-wise quantization with 128x128 blocks provides better precision than per-tensor quantization while maintaining good compression. Each block has its own scale factor stored as weightscaleinv (inverse scale for efficient multiplication during inference). This is a quantized version of Qwen/Qwen3-Omni-30B-A3B-Instruct. Qwen3-Omni is a natively end-to-end multilingual omni-modal foundation model that processes text…

Excerpt from the card by Markus, licensed other.

Configuration

Architecture
Qwen3OmniMoeForConditionalGeneration
Model type
qwen3_omni_moe
Quantization
fp8

Identity and Version

Repository
marksverdhei/Qwen3-Omni-30B-A3B-FP8
Publisher
Markus
Task
Any to any
Modality
Multimodal
Library
transformers
Parameters
35.3B parameters
Languages
en, zh, ko, ja, de, ru, it, fr
Revision
042db39a102171d5994b72ab717fade92e80061a
First published
2026-01-20
Last updated
2026-01-20

Files and Weights

24 files, 37.4 GB in total. The weights are 15 files totalling 37.4 GB in safetensors.

Weights15 files · 37.4 GB
Configuration4 files · 5.6 MB
Tokenizer3 files · 4.5 MB
Documentation1 file · 2.4 KB
Repository1 file · 1.5 KB
Every file
FileTypeSizeSHA-256
model-00001-of-00015.safetensorsWeights4.0 GB f696ae48f6c8
model-00002-of-00015.safetensorsWeights2.5 GB 11c79a1bd08b
model-00003-of-00015.safetensorsWeights2.5 GB c13b856a3db8
model-00004-of-00015.safetensorsWeights2.5 GB 9c45e872565e
model-00005-of-00015.safetensorsWeights2.5 GB c0391162f6af
model-00006-of-00015.safetensorsWeights2.5 GB b01061196ad0
model-00007-of-00015.safetensorsWeights2.5 GB c1921d61d727
model-00008-of-00015.safetensorsWeights2.5 GB eff10a8e1107
model-00009-of-00015.safetensorsWeights2.5 GB 8b26b577d135
model-00010-of-00015.safetensorsWeights2.5 GB b2f768b082d2
model-00011-of-00015.safetensorsWeights2.5 GB 5aed3968f11a
model-00012-of-00015.safetensorsWeights2.5 GB 392ac7ccd657
model-00013-of-00015.safetensorsWeights2.8 GB 1fd08397db58
model-00014-of-00015.safetensorsWeights2.5 GB 3d2ae8a26493
model-00015-of-00015.safetensorsWeights553.7 MB f30142d3d5f7
config.jsonConfiguration14.1 KB
generation_config.jsonConfiguration146 B
model.safetensors.index.jsonConfiguration5.6 MB
preprocessor_config.jsonConfiguration603 B
README.mdDocumentation2.4 KB
.gitattributesRepository1.5 KB
merges.txtTokenizer1.7 MB
tokenizer_config.jsonTokenizer7.3 KB
vocab.jsonTokenizer2.8 MB

License and Download

License
other
Access
Open weights, no gate
Download size
37.4 GB
Download from Markus

Released by Markus through its official repository on Hugging Face. Read the license.

Built From

Memory Requirements

PrecisionWeights in memory
As published37.4 GB
16-bit70.5 GB
8-bit35.3 GB
4-bit17.6 GB

Weights only, from the published parameter count; the key-value cache and runtime add to this.

Questions About Qwen3-Omni-30B-A3B-FP8

How much GPU memory does Qwen3-Omni-30B-A3B-FP8 need?

About 84.6 GB at 16-bit and 21.2 GB at 4-bit: the weights (35.3B parameters) plus a working margin. A long context needs more.

What is the cheapest GPU to run Qwen3-Omni-30B-A3B-FP8 on?

At 16-bit, 1x MI300X from $1.85 an hour; at 4-bit, 1x MI300X from $1.85 an hour, at the lowest on-demand prices the SAVRN Index lists.

What license is Qwen3-Omni-30B-A3B-FP8 released under?

other, as its publisher declares it. Read the license text before commercial use.

Similar Models

Model · Any to any

Qwen3-Omni-30B-A3B-Instruct

Qwen

Qwen3-Omni is the natively end-to-end multilingual omni-modal foundation models. It processes text, images, audio, and video, and delivers real-time streaming responses in both text and natural speech. We introduce several architectural upgrades to improve performance and efficiency. Key features: Qwen3-Omni supports a wide range of multimodal application scenarios, covering various domain tasks involving audio, image, video, and audio-visual modalities. Below are several cookbooks demonstrating the usage cases of Qwen3-Omni and these cookbooks include our actual execution logs. You can first follow the QuickStart guide to download the model and install the necessary inference environment…

Open weights other 35.3B parameters transformers

NVIDIA Nemotron 3 Nano Omni is a multimodal large language model that unifies video, audio, image, and text understanding to support enterprise-grade Q&A, summarization, transcription, and document intelligence workflows. It extends the Nemotron Nano family with integrated video+speech comprehension, Graphical User Interface (GUI), Optical Character Recognition (OCR), and speech transcription capabilities, enabling end-to-end processing of rich enterprise content such as meeting recordings, M&E assets, training videos, and complex business documents. NVIDIA Nemotron 3 Nano Omni was developed by NVIDIA as part of the Nemotron model family. This model is available for commercial use. This…

Open weights other 33B parameters 262,144 tokens transformers

NVIDIA Nemotron 3 Nano Omni is a multimodal large language model that unifies video, audio, image, and text understanding to support enterprise-grade Q&A, summarization, transcription, and document intelligence workflows. It extends the Nemotron Nano family with integrated video+speech comprehension, Graphical User Interface (GUI), Optical Character Recognition (OCR), and speech transcription capabilities, enabling end-to-end processing of rich enterprise content such as meeting recordings, M&E assets, training videos, and complex business documents. NVIDIA Nemotron 3 Nano Omni was developed by NVIDIA as part of the Nemotron model family. This model is available for commercial use. This…

Open weights other 33B parameters 262,144 tokens transformers

Model · Any to any

Qwen3-Omni-30B-A3B-Thinking

Qwen

Qwen3-Omni is the natively end-to-end multilingual omni-modal foundation models. It processes text, images, audio, and video, and delivers real-time streaming responses in both text and natural speech. We introduce several architectural upgrades to improve performance and efficiency. Key features: Qwen3-Omni supports a wide range of multimodal application scenarios, covering various domain tasks involving audio, image, video, and audio-visual modalities. Below are several cookbooks demonstrating the usage cases of Qwen3-Omni and these cookbooks include our actual execution logs. You can first follow the QuickStart guide to download the model and install the necessary inference environment…

Open weights other 31.7B parameters transformers

NVIDIA Nemotron 3 Nano Omni is a multimodal large language model that unifies video, audio, image, and text understanding to support enterprise-grade Q&A, summarization, transcription, and document intelligence workflows. It extends the Nemotron Nano family with integrated video+speech comprehension, Graphical User Interface (GUI), Optical Character Recognition (OCR), and speech transcription capabilities, enabling end-to-end processing of rich enterprise content such as meeting recordings, M&E assets, training videos, and complex business documents. NVIDIA Nemotron 3 Nano Omni was developed by NVIDIA as part of the Nemotron model family. This model is available for commercial use. This…

Open weights other 18.3B parameters 262,144 tokens transformers

Model · Any to any

gemma-4-12B-it-qat-w4a16-ct

Google

Gemma is a family of open models built by Google DeepMind. Gemma 4 models are multimodal, handling text and image input (with audio supported on E2B, E4B, and 12B) and generating text output. This release includes open-weights models in both pre-trained and instruction-tuned variants. Gemma 4 features a context window of up to 256K tokens and maintains multilingual support in over 140 languages. Featuring both Dense and Mixture-of-Experts (MoE) architectures, Gemma 4 is well-suited for tasks like text generation, coding, and reasoning. The models are available in five distinct sizes: E2B, E4B, 12B, 26B A4B, and 31B. Their diverse sizes make them deployable in environments ranging from…

Open weights apache-2.0 13.3B parameters 262,144 tokens transformers