SAVRN
Search Contact SAVRN

Open-weight model · Text generation

Qwen1.5-MoE-A2.7B

by Qwen Qwen/Qwen1.5-MoE-A2.7B

Qwen1.5-MoE is a transformer-based MoE decoder-only language model pretrained on a large amount of data. For more details, please refer to our blog post and GitHub repo.

Parameters14.3B
Context8,192
Weights28.6 GB
Licenseother
AccessOpen weights
Monthly Downloads746.2k

Runs On

What it takes to serve Qwen1.5-MoE-A2.7B (14.3B parameters): the memory its weights need at each precision, and the cheapest way to rent enough data-center GPUs to hold them.

PrecisionWeightsMemory neededCheapest setupPer hourAlso fits
16-bit 28.6 GB 34.4 GB 1x MI300X (192 GB)
Vultr
$1.85 1x H100 $1.99 · 1x MI325X $2.00
8-bit 14.3 GB 17.2 GB 1x MI300X (192 GB)
Vultr
$1.85 1x H100 $1.99 · 1x MI325X $2.00
4-bit 7.2 GB 8.6 GB 1x MI300X (192 GB)
Vultr
$1.85 1x H100 $1.99 · 1x MI325X $2.00

Memory is the weights at that precision plus 20% for the runtime and a short context; a long context needs more. Prices are the lowest on-demand hourly rates in the SAVRN Index, read Sep 18, 2026.

SAVRN's Notes on Qwen1.5-MoE-A2.7B

Fourteen billion parameters sit in memory and 2.7 billion touch any given token, four of 60 experts. That split decides the hardware. Memory is sized for the whole 14.3B: 34.4 GB at 16-bit, 17.2 GB at 8-bit, 8.6 GB at 4-bit. Compute is sized for the 2.7B that fire. On the cheapest option in our table, one MI300X with 192 GB at $1.85 an hour, the 16-bit build takes under a fifth of the card, so plan on several copies per rental.

The license field says other with no summary, so the terms get read before anything commercial ships. The 8,192-token window is the number that limits use: short prompts and outputs suit it, long documents do not. The publisher upcycled it from Qwen-1.8B and places it near Qwen1.5-7B; the weights last changed April 18, 2024, safetensors only, and no host on our Index prices it by the token.

Model Card

Qwen1.5-MoE is a transformer-based MoE decoder-only language model pretrained on a large amount of data. For more details, please refer to our blog post and GitHub repo. Qwen1.5-MoE employs Mixture of Experts (MoE) architecture, where the models are upcycled from dense language models. For instance, Qwen1.5-MoE-A2.7B is upcycled from Qwen-1.8B. It has 14.3B parameters in total and 2.7B activated parameters during runtime, while achieving comparable performance to Qwen1.5-7B, it only requires 25% of the training resources. We also observed that the inference speed is 1.74 times that of Qwen1.5-7B. The code of Qwen1.5-MoE has been in the latest Hugging face transformers and we advise you to…

Excerpt from the card by Qwen, licensed other.

Configuration

Architecture
Qwen2MoeForCausalLM
Context length (tokens)
8,192
Layers
24
Hidden size
2,048
Feed-forward size
5,632
Attention heads
16
Key/value heads
16
Vocabulary size
151,936
Experts
60
Experts active per token
4
Sliding window (tokens)
32,768
RoPE base
1e+06
Stored precision
bfloat16
Model type
qwen2_moe

Identity and Version

Repository
Qwen/Qwen1.5-MoE-A2.7B
Publisher
Qwen
Task
Text generation
Modality
Text
Library
transformers
Parameters
14.3B parameters
Languages
en
Revision
1a758c50ecb6350748b9ce0a99d2352fd9fc11c9
First published
2024-02-29
Last updated
2024-04-18

Files and Weights

19 files, 28.6 GB in total. The weights are 8 files totalling 28.6 GB in safetensors.

Weights8 files · 28.6 GB
Configuration4 files · 417.6 KB
Tokenizer4 files · 11.5 MB
Documentation2 files · 8.3 KB
Repository1 file · 1.5 KB
Every file
FileTypeSizeSHA-256
model-00001-of-00008.safetensorsWeights4.0 GB 44f12f5b3d4e
model-00002-of-00008.safetensorsWeights4.0 GB 20a99873170c
model-00003-of-00008.safetensorsWeights4.0 GB e0b7cd920fc9
model-00004-of-00008.safetensorsWeights4.0 GB e2be2af186e5
model-00005-of-00008.safetensorsWeights4.0 GB 3b75e01d4df9
model-00006-of-00008.safetensorsWeights4.0 GB c2fc223a8818
model-00007-of-00008.safetensorsWeights4.0 GB 88b206c85ee0
model-00008-of-00008.safetensorsWeights668.5 MB 996c5057f123
config.jsonConfiguration919 B
configuration.jsonConfiguration81 B
generation_config.jsonConfiguration144 B
model.safetensors.index.jsonConfiguration416.5 KB
LICENSEDocumentation6.9 KB
README.mdDocumentation1.4 KB
.gitattributesRepository1.5 KB
merges.txtTokenizer1.7 MB
tokenizer.jsonTokenizer7.0 MB
tokenizer_config.jsonTokenizer1.3 KB
vocab.jsonTokenizer2.8 MB

License and Download

License
other
Access
Open weights, no gate
Download size
28.6 GB
Download from Qwen

Released by Qwen through ModelScope. Read the license.

Memory Requirements

PrecisionWeights in memory
As published28.6 GB
16-bit28.6 GB
8-bit14.3 GB
4-bit7.2 GB

Weights only, from the published parameter count; the key-value cache and runtime add to this.

Questions About Qwen1.5-MoE-A2.7B

How much GPU memory does Qwen1.5-MoE-A2.7B need?

About 34.4 GB at 16-bit and 8.6 GB at 4-bit: the weights (14.3B parameters) plus a working margin. A long context needs more.

What is the cheapest GPU to run Qwen1.5-MoE-A2.7B on?

At 16-bit, 1x MI300X from $1.85 an hour; at 4-bit, 1x MI300X from $1.85 an hour, at the lowest on-demand prices the SAVRN Index lists.

What license is Qwen1.5-MoE-A2.7B released under?

other, as its publisher declares it. Read the license text before commercial use.

What is Qwen1.5-MoE-A2.7B's context length?

8,192 tokens, from the maximum position embeddings in its published configuration.

Similar Models

Model · Text generation

Gemma-4-26B-A4B-NVFP4

NVIDIA

Gemma 4 26B IT is an open multimodal model built by Google DeepMind that handles text and image inputs, can process video as sequences of frames, and generates text output. It is designed to deliver frontier-level performance for reasoning, agentic workflows, coding, and multimodal understanding on consumer GPUs and workstations, with a 256K-token context window and support for over 140 languages. The model uses a hybrid attention mechanism that interleaves local sliding-window and full global attention, with unified Keys and Values in global layers and Proportional RoPE (p-RoPE) to support long-context performance. The NVIDIA Gemma 4 26B IT NVFP4 model is quantized with NVIDIA Model…

Open weights apache-2.0 14.4B parameters 262,144 tokens Model Optimizer

Model · Text generation

Qwen3-14B-AWQ

Qwen

Qwen3 is the latest generation of large language models in Qwen series, offering a comprehensive suite of dense and mixture-of-experts (MoE) models. Built upon extensive training, Qwen3 delivers groundbreaking advancements in reasoning, instruction-following, agent capabilities, and multilingual support, with the following key features: - Uniquely support of seamless switching between thinking mode (for complex logical reasoning, math, and coding) and non-thinking mode (for efficient, general-purpose dialogue) within single model, ensuring optimal performance across various scenarios. - Significantly enhancement in its reasoning capabilities, surpassing previous QwQ (in thinking mode) and…

Open weights apache-2.0 14.8B parameters 40,960 tokens transformers

Model · Text generation

Qwen3-14B

Qwen

Qwen3 is the latest generation of large language models in Qwen series, offering a comprehensive suite of dense and mixture-of-experts (MoE) models. Built upon extensive training, Qwen3 delivers groundbreaking advancements in reasoning, instruction-following, agent capabilities, and multilingual support, with the following key features: - Uniquely support of seamless switching between thinking mode (for complex logical reasoning, math, and coding) and non-thinking mode (for efficient, general-purpose dialogue) within single model, ensuring optimal performance across various scenarios. - Significantly enhancement in its reasoning capabilities, surpassing previous QwQ (in thinking mode) and…

Open weights apache-2.0 14.8B parameters 40,960 tokens transformers

Model · Text generation

Symiotic-14B

Convergent Intelligence

Purpose: Full-scale cognitive reasoning model with self-organizing memory and generative symbolic evolution SymbioticLM-14B is a 17.8-billion-parameter symbolic–transformer hybrid that couples high-capacity neural representation with structured symbolic cognition. It supports persistent memory, entropic recall, multi-stage symbolic routing, and self-organizing knowledge structures. This is an experimental research checkpoint — the capability claims below describe architectural intent, not benchmarked results (see Limitations). This model is ideal for advanced reasoning agents, research assistants, and symbolic math/code generation systems. - Long-form symbolic theorem generation and proof…

Open weights afl-3.0 14.8B parameters 40,960 tokens transformers

Model · Text generation

Qwen2.5-14B-Instruct

Qwen

Qwen2.5 is the latest series of Qwen large language models. For Qwen2.5, we release a number of base language models and instruction-tuned language models ranging from 0.5 to 72 billion parameters. Qwen2.5 brings the following improvements upon Qwen2: - Significantly more knowledge and has greatly improved capabilities in coding and mathematics, thanks to our specialized expert models in these domains. - Significant improvements in instruction following, generating long texts (over 8K tokens), understanding structured data (e.g, tables), and generating structured outputs especially JSON. More resilient to the diversity of system prompts, enhancing role-play implementation and…

Open weights apache-2.0 14.8B parameters 32,768 tokens transformers

Model · Text generation

Qwen2.5-14B-Instruct-AWQ

Qwen

Qwen2.5 is the latest series of Qwen large language models. For Qwen2.5, we release a number of base language models and instruction-tuned language models ranging from 0.5 to 72 billion parameters. Qwen2.5 brings the following improvements upon Qwen2: - Significantly more knowledge and has greatly improved capabilities in coding and mathematics, thanks to our specialized expert models in these domains. - Significant improvements in instruction following, generating long texts (over 8K tokens), understanding structured data (e.g, tables), and generating structured outputs especially JSON. More resilient to the diversity of system prompts, enhancing role-play implementation and…

Open weights apache-2.0 14.8B parameters 32,768 tokens transformers