SAVRN
Search Contact SAVRN

Open-weight model · Text generation

Qwen2-0.5B

by Qwen Qwen/Qwen2-0.5B

Qwen2 is the new series of Qwen large language models. For Qwen2, we release a number of base language models and instruction-tuned language models ranging from 0.5 to 72 billion parameters, including a Mixture-of-Experts model.

Parameters494M
Context131,072
Weights988.1 MB
Licenseapache-2.0
AccessOpen weights
Monthly Downloads752.5k

Runs On

What it takes to serve Qwen2-0.5B (494M parameters): the memory its weights need at each precision, and the cheapest way to rent enough data-center GPUs to hold them.

PrecisionWeightsMemory neededCheapest setupPer hourAlso fits
16-bit 1.0 GB 1.2 GB 1x MI300X (192 GB)
Vultr		 $1.85 1x H100 $1.99 · 1x MI325X $2.00
8-bit 0.5 GB 0.6 GB 1x MI300X (192 GB)
Vultr		 $1.85 1x H100 $1.99 · 1x MI325X $2.00
4-bit 0.2 GB 0.3 GB 1x MI300X (192 GB)
Vultr		 $1.85 1x H100 $1.99 · 1x MI325X $2.00

Memory is the weights at that precision plus 20% for the runtime and a short context; a long context needs more. Prices are the lowest on-demand hourly rates in the SAVRN Index, read Sep 18, 2026.

SAVRN's Notes on Qwen2-0.5B

At 16-bit this checkpoint needs 1.2 GB of memory, and at 4-bit only 0.3 GB. Our cheapest listed slot is still one MI300X with 192 GB at $1.85 an hour, far more than 494M parameters need, so the hardware decision is what else shares the card. This is the base model of the Qwen2 line, not the instruction-tuned one, and we read it as a starting point for your own fine-tune rather than a finished chat endpoint.

Apache 2.0 covers commercial use, modification and redistribution, with notices kept and changes stated, so a tuned derivative can ship in your product. Access is open. Two things to check: the 1.2 GB is weights plus overhead, and filling the 131,072-token context adds memory on top; and the one score in our file is 14.97 on MMLU-Pro from EvalEval, so run your own evaluation before committing.

SAVRN Research, 2026-09-18

Model Card

By Qwen, published under apache-2.0, revision 91d2aff3f957.

Introduction

Qwen2 is the new series of Qwen large language models. For Qwen2, we release a number of base language models and instruction-tuned language models ranging from 0.5 to 72 billion parameters, including a Mixture-of-Experts model. This repo contains the 0.5B Qwen2 base language model.

Compared with the state-of-the-art opensource language models, including the previous released Qwen1.5, Qwen2 has generally surpassed most opensource models and demonstrated competitiveness against proprietary models across a series of benchmarks targeting for language understanding, language generation, multilingual capability, coding, mathematics, reasoning, etc.

For more details, please refer to our blog, GitHub, and Documentation.

Model Details

Read the full model card (472 words)

Configuration

Architecture
Qwen2ForCausalLM
Context length (tokens)
131,072
Layers
24
Hidden size
896
Feed-forward size
4,864
Attention heads
14
Key/value heads
2
Vocabulary size
151,936
Sliding window (tokens)
131,072
RoPE base
1e+06
Stored precision
bfloat16
Model type
qwen2

Identity and Version

Repository
Qwen/Qwen2-0.5B
Publisher
Qwen
Task
Text generation
Modality
Text
Library
transformers
Parameters
494M parameters
Languages
en
Revision
91d2aff3f957f99e4c74c962f2f408dcc88a18d8
First published
2024-05-31
Last updated
2024-10-22

Files and Weights

10 files, 999.6 MB in total. The weights are 1 file totalling 988.1 MB in safetensors.

Weights1 file · 988.1 MB
Configuration2 files · 799 B
Tokenizer4 files · 11.5 MB
Documentation2 files · 15.4 KB
Repository1 file · 1.5 KB
Every file
FileTypeSizeSHA-256
model.safetensorsWeights988.1 MB 9cd8fc8c85a1
config.jsonConfiguration661 B —
generation_config.jsonConfiguration138 B —
LICENSEDocumentation11.3 KB —
README.mdDocumentation4.0 KB —
.gitattributesRepository1.5 KB —
merges.txtTokenizer1.7 MB —
tokenizer.jsonTokenizer7.0 MB —
tokenizer_config.jsonTokenizer1.3 KB —
vocab.jsonTokenizer2.8 MB —

License and Download

License
apache-2.0
Access
Open weights, no gate
Download size
988.1 MB
Download from Qwen

Released by Qwen through ModelScope. Read the license.

Evaluations

Each result is shown as reported, with the conditions its reporter stated. None is a SAVRN measurement. A comparison lines two results up only when their configuration, unit and setup are all stated and identical.

BenchmarkConditionsResultReported byRevisionDate
TIGER-Lab/MMLU-Pro Task mmlu_proMetric mmlu_proComparison conditions not established 14.97 EvalEval
Reported by a third party		 Evaluated revision not stated 2026-06-30

Memory Requirements

PrecisionWeights in memory
As published988.1 MB
16-bit1.0 GB
8-bit0.5 GB
4-bit0.2 GB

Weights only, from the published parameter count; the key-value cache and runtime add to this.

Compare Qwen2-0.5B

Questions About Qwen2-0.5B

How much GPU memory does Qwen2-0.5B need?

About 1.2 GB at 16-bit and 0.3 GB at 4-bit: the weights (494M parameters) plus a working margin. A long context needs more.

What is the cheapest GPU to run Qwen2-0.5B on?

At 16-bit, 1x MI300X from $1.85 an hour; at 4-bit, 1x MI300X from $1.85 an hour, at the lowest on-demand prices the SAVRN Index lists.

Can I use Qwen2-0.5B commercially?

Yes. Qwen2-0.5B is released under Apache License 2.0. The Apache License 2.0 is a permissive open-source license. It permits commercial use, modification and redistribution. It requires keeping the license and copyright notices and any NOTICE file, stating significant changes, and it includes an express patent grant from contributors.

What is Qwen2-0.5B's context length?

131,072 tokens, from the maximum position embeddings in its published configuration.

Similar Models

Model · Text generation

Qwen2.5-0.5B-Instruct

Qwen

Qwen2.5 is the latest series of Qwen large language models. For Qwen2.5, we release a number of base language models and instruction-tuned language models ranging from 0.5 to 72 billion parameters. Qwen2.5 brings the following improvements upon Qwen2: - Significantly more knowledge and has greatly improved capabilities in coding and mathematics, thanks to our specialized expert models in these domains. - Significant improvements in instruction following, generating long texts (over 8K tokens), understanding structured data (e.g, tables), and generating structured outputs especially JSON. More resilient to the diversity of system prompts, enhancing role-play implementation and…

Open weights apache-2.0 494M parameters 32,768 tokens transformers
View model

Model · Text generation

Qwen2.5-0.5B

Qwen

Qwen2.5 is the latest series of Qwen large language models. For Qwen2.5, we release a number of base language models and instruction-tuned language models ranging from 0.5 to 72 billion parameters. Qwen2.5 brings the following improvements upon Qwen2: - Significantly more knowledge and has greatly improved capabilities in coding and mathematics, thanks to our specialized expert models in these domains. - Significant improvements in instruction following, generating long texts (over 8K tokens), understanding structured data (e.g, tables), and generating structured outputs especially JSON. More resilient to the diversity of system prompts, enhancing role-play implementation and…

Open weights apache-2.0 494M parameters 32,768 tokens transformers
View model

Model · Text generation

DeepReasoning_1R

Convergent Intelligence

Part of the Standalone Models by Convergent Intelligence LLC: Research Division DistilQwen Collection — Our only BF16 series. Proof-weighted distillation from Qwen3-30B-A3B → 1.7B and 0.6B on H100. Three teacher variants (Instruct, Thinking, Coder), nine models, 2,788 combined downloads. The rest of the portfolio proves structure beats scale on CPU. This collection shows what happens when you give the methodology real hardware. This model is part of the Convergent Intelligence LLC: Research Division portfolio. All models in this portfolio are developed under the Discrepancy Calculus (DISC) framework — a measure-theoretic approach to understanding and controlling the gap between what a model…

Open weights 494M parameters 32,768 tokens transformers
View model

Model · Text generation

math-slm-qwen2.5-0.5b-v4

Mandavi Singh

A domain-specific small language model for step-by-step math problem solving, built by team03 (SLM Learners) for the Pramana SLM++ Bootcamp Round 2 submission. For an OpenAI-compatible endpoint, serve with servehf.py (stdlib + transformers only, no Ollama needed). Precision note: training ran in bf16 compute (QLoRA 4-bit NF4 base), but the merged checkpoint uploaded here is float16 (the merge step reloads the base in fp16). - Public Hugging Face datasets pulled via pulldata.py; licenses verified through the HF API on 2026-09-05 and recorded in datamanifest.md. - Held-out eval set built with buildheldouteval.py from raw ExamBench rows never used in training, with a final overlap check that…

Open weights apache-2.0 494M parameters 32,768 tokens
View model

Model · Text generation

Qwen3-0.6B-Base

Qwen

Qwen3 is the latest generation of large language models in Qwen series, offering a comprehensive suite of dense and mixture-of-experts (MoE) models. Building upon extensive advancements in training data, model architecture, and optimization techniques, Qwen3 delivers the following key improvements over the previously released Qwen2.5: Qwen3-0.6B-Base has the following features: For more details, including benchmark evaluation, hardware requirements, and inference performance, please refer to our blog, GitHub, and Documentation. The code of Qwen3 has been in the latest Hugging Face transformers and we advise you to use the latest version of transformers. With transformers<4.51.0, you will…

Open weights apache-2.0 596M parameters 32,768 tokens transformers
View model

Model · Text generation

Symbiotic-1B

Convergent Intelligence

Purpose: Lightweight, memory-augmented reasoning model for CPU and embedded inference SymbioticLM-1B is the compact version of the SymbioticAI architecture. It fuses Qwen’s rotary transformer design with a symbolic processing pipeline and a persistent episodic memory. Though smaller in parameter count, it retains the full cognitive engine: symbolic memory, dynamic thought evolution, and entropy-gated control. This model is ideal for symbolic reasoning in constrained environments — like research agents, lightweight assistants, and memory-efficient logical processing. - Procedural planning, math modeling, small-code generation - Less fluent in free-form language than larger variants…

Open weights afl-3.0 596M parameters 40,960 tokens transformers
View model