SAVRN
Search Contact SAVRN

Open-weight model · Any to any

Qwen2.5-Omni-7B-AWQ

by Qwen Qwen/Qwen2.5-Omni-7B-AWQ

Qwen2.5-Omni is an end-to-end multimodal model designed to perceive diverse modalities, including text, images, audio, and video, while simultaneously generating text and natural speech responses in a streaming manner.

Parameters10.7B
Context
Weights12.7 GB
Licenseother
AccessOpen weights
Monthly Downloads41.7k

Runs On

What it takes to serve Qwen2.5-Omni-7B-AWQ (10.7B parameters): the memory its weights need at each precision, and the cheapest way to rent enough data-center GPUs to hold them.

PrecisionWeightsMemory neededCheapest setupPer hourAlso fits
16-bit 21.5 GB 25.8 GB 1x MI300X (192 GB)
Vultr
$1.85 1x H100 $1.99 · 1x MI325X $2.00
8-bit 10.7 GB 12.9 GB 1x MI300X (192 GB)
Vultr
$1.85 1x H100 $1.99 · 1x MI325X $2.00
4-bit 5.4 GB 6.4 GB 1x MI300X (192 GB)
Vultr
$1.85 1x H100 $1.99 · 1x MI325X $2.00

Memory is the weights at that precision plus 20% for the runtime and a short context; a long context needs more. Prices are the lowest on-demand hourly rates in the SAVRN Index, read Sep 18, 2026.

Model Card

Qwen2.5-Omni is an end-to-end multimodal model designed to perceive diverse modalities, including text, images, audio, and video, while simultaneously generating text and natural speech responses in a streaming manner. This model card introduces a series of enhancements designed to improve the Qwen2.5-Omni-7B's operability on devices with constrained GPU memory. Key optimizations include: Implemented 4-bit quantization of the Thinker's weights using AWQ, effectively reducing GPU VRAM usage. Enhanced the inference pipeline to load model weights on-demand for each module and offload them to CPU memory once inference is complete, preventing peak VRAM usage from becoming excessive. Converted…

Excerpt from the card by Qwen, licensed other.

Configuration

Architecture
Qwen2_5OmniForConditionalGeneration
Stored precision
bfloat16
Model type
qwen2_5_omni
Quantization
awq

Identity and Version

Repository
Qwen/Qwen2.5-Omni-7B-AWQ
Publisher
Qwen
Task
Any to any
Modality
Multimodal
Library
transformers
Parameters
10.7B parameters
Languages
en
Revision
848615c28dfd87b0f624baea4be1f29bec0d1db1
First published
2025-05-14
Last updated
2025-05-15

Files and Weights

19 files, 12.7 GB in total. The weights are 5 files totalling 12.7 GB in pt, safetensors.

Weights5 files · 12.7 GB
Configuration6 files · 285.9 KB
Tokenizer4 files · 15.9 MB
Documentation2 files · 17.9 KB
Other1 file · 1.3 KB
Repository1 file · 1.5 KB
Every file
FileTypeSizeSHA-256
model-00001-of-00004.safetensorsWeights4.0 GB 27319a909f0d
model-00002-of-00004.safetensorsWeights3.1 GB 327b0afe4b87
model-00003-of-00004.safetensorsWeights4.0 GB 8c62153be9bf
model-00004-of-00004.safetensorsWeights1.6 GB d33733bedd85
spk_dict.ptWeights259.5 KB 6a05609b28f5
added_tokens.jsonConfiguration579 B
config.jsonConfiguration15.4 KB
generation_config.jsonConfiguration95 B
model.safetensors.index.jsonConfiguration268.3 KB
preprocessor_config.jsonConfiguration667 B
special_tokens_map.jsonConfiguration833 B
LICENSEDocumentation11.3 KB
README.mdDocumentation6.6 KB
chat_template.jinjaOther1.3 KB
.gitattributesRepository1.5 KB
merges.txtTokenizer1.7 MB
tokenizer.jsonTokenizer11.4 MB f9711e245647
tokenizer_config.jsonTokenizer5.2 KB
vocab.jsonTokenizer2.8 MB

License and Download

License
other
Access
Open weights, no gate
Download size
12.7 GB
Download from Qwen

Released by Qwen through ModelScope. Read the license.

Built From

Memory Requirements

PrecisionWeights in memory
As published12.7 GB
16-bit21.5 GB
8-bit10.7 GB
4-bit5.4 GB

Weights only, from the published parameter count; the key-value cache and runtime add to this.

Questions About Qwen2.5-Omni-7B-AWQ

How much GPU memory does Qwen2.5-Omni-7B-AWQ need?

About 25.8 GB at 16-bit and 6.4 GB at 4-bit: the weights (10.7B parameters) plus a working margin. A long context needs more.

What is the cheapest GPU to run Qwen2.5-Omni-7B-AWQ on?

At 16-bit, 1x MI300X from $1.85 an hour; at 4-bit, 1x MI300X from $1.85 an hour, at the lowest on-demand prices the SAVRN Index lists.

What license is Qwen2.5-Omni-7B-AWQ released under?

other, as its publisher declares it. Read the license text before commercial use.

Similar Models

Model · Any to any

Qwen2.5-Omni-7B

Qwen

Qwen2.5-Omni is an end-to-end multimodal model designed to perceive diverse modalities, including text, images, audio, and video, while simultaneously generating text and natural speech responses in a streaming manner. We conducted a comprehensive evaluation of Qwen2.5-Omni, which demonstrates strong performance across all modalities when compared to similarly sized single-modality models and closed-source models like Qwen2.5-VL-7B, Qwen2-Audio, and Gemini-1.5-pro. In tasks requiring the integration of multiple modalities, such as OmniBench, Qwen2.5-Omni achieves state-of-the-art performance. Furthermore, in single-modality tasks, it excels in areas including speech recognition (Common…

Open weights other 10.7B parameters transformers

Model · Any to any

gemma-4-12B-it

Google

Gemma is a family of open models built by Google DeepMind. Gemma 4 models are multimodal, handling text and image input (with audio supported on E2B, E4B, and 12B) and generating text output. This release includes open-weights models in both pre-trained and instruction-tuned variants. Gemma 4 features a context window of up to 256K tokens and maintains multilingual support in over 140 languages. Featuring both Dense and Mixture-of-Experts (MoE) architectures, Gemma 4 is well-suited for tasks like text generation, coding, and reasoning. The models are available in five distinct sizes: E2B, E4B, 12B, 26B A4B, and 31B. Their diverse sizes make them deployable in environments ranging from…

Open weights apache-2.0 12B parameters 262,144 tokens transformers

LM Studio Community models highlights program. Highlighting new & noteworthy models by the community. Join the conversation on Discord. 4-bit quantized version of gemma-4-12B-it using MLX, optimized for Apple Silicon. Special thanks to the Apple Machine Learning Research team for creating MLX. LM Studio is not the creator, originator, or owner of any Model featured in the Community Model Program. Each Community Model is created and provided by third parties. LM Studio does not endorse, support, represent or guarantee the completeness, truthfulness, accuracy, or reliability of any Community Model. You understand that Community Models can produce content that might be offensive, harmful…

Open weights apache-2.0 12B parameters 262,144 tokens transformers

LM Studio Community models highlights program. Highlighting new & noteworthy models by the community. Join the conversation on Discord. 8-bit quantized version of gemma-4-12B-it using MLX, optimized for Apple Silicon. Special thanks to the Apple Machine Learning Research team for creating MLX. LM Studio is not the creator, originator, or owner of any Model featured in the Community Model Program. Each Community Model is created and provided by third parties. LM Studio does not endorse, support, represent or guarantee the completeness, truthfulness, accuracy, or reliability of any Community Model. You understand that Community Models can produce content that might be offensive, harmful…

Open weights apache-2.0 12B parameters 262,144 tokens transformers

LM Studio Community models highlights program. Highlighting new & noteworthy models by the community. Join the conversation on Discord. 6-bit quantized version of gemma-4-12B-it using MLX, optimized for Apple Silicon. Special thanks to the Apple Machine Learning Research team for creating MLX. LM Studio is not the creator, originator, or owner of any Model featured in the Community Model Program. Each Community Model is created and provided by third parties. LM Studio does not endorse, support, represent or guarantee the completeness, truthfulness, accuracy, or reliability of any Community Model. You understand that Community Models can produce content that might be offensive, harmful…

Open weights apache-2.0 12B parameters 262,144 tokens transformers

LM Studio Community models highlights program. Highlighting new & noteworthy models by the community. Join the conversation on Discord. 5-bit quantized version of gemma-4-12B-it using MLX, optimized for Apple Silicon. Special thanks to the Apple Machine Learning Research team for creating MLX. LM Studio is not the creator, originator, or owner of any Model featured in the Community Model Program. Each Community Model is created and provided by third parties. LM Studio does not endorse, support, represent or guarantee the completeness, truthfulness, accuracy, or reliability of any Community Model. You understand that Community Models can produce content that might be offensive, harmful…

Open weights apache-2.0 12B parameters 262,144 tokens transformers