Skip to page content
SAVRN
Search Plan a project

Open-weight model · Text generation

Qwen2.5-1.5B-Instruct-gptq-4bit-int4-awq

by Domenic Federico dfed24/Qwen2.5-1.5B-Instruct-gptq-4bit-int4-awq

Qwen2.5-1.5B-Instruct-gptq-4bit-int4-awq is an open-weight model for text generation from Domenic Federico, released under Apache License 2.0. It has 1.5B parameters and a 32,768-token context. At 16-bit it needs about 3.7 GB of GPU memory, which fits on 1x MI300X from $1.85 an hour, at the lowest prices in the SAVRN Index.

Status: exported, not yet verified on a GPU. This checkpoint is in the AutoAWQ GEMM layout (4 bits, group size 64, asymmetric with integer zero points) so that vLLM's awq / awqmarlin kernels can load it.

Parameters1.5B
Context32,768
Weights1.2 GB
Licenseapache-2.0
AccessOpen weights
Monthly Downloads—

Runs On

What it takes to serve Qwen2.5-1.5B-Instruct-gptq-4bit-int4-awq (1.5B parameters): the memory its weights need at each precision, and the cheapest way to rent enough data-center GPUs to hold them.

PrecisionWeightsMemory neededCheapest setupPer hourAlso fits
16-bit 3.1 GB 3.7 GB 1x MI300X (192 GB)
Vultr		 $1.85 1x H100 $1.99 · 1x MI325X $2.00
8-bit 1.5 GB 1.9 GB 1x MI300X (192 GB)
Vultr		 $1.85 1x H100 $1.99 · 1x MI325X $2.00
4-bit 0.8 GB 0.9 GB 1x MI300X (192 GB)
Vultr		 $1.85 1x H100 $1.99 · 1x MI325X $2.00

Memory is the weights at that precision plus 20% for the runtime and a short context; a long context needs more. Prices are the lowest on-demand hourly rates in the SAVRN Index, read Oct 18, 2026.

Qwen2.5-1.5B-Instruct-gptq-4bit-int4-awq on every accelerator the SAVRN Index prices, at every precision

Model Card

By Domenic Federico, published under apache-2.0, revision a8e29f82c18d.

Status: exported, not yet verified on a GPU. This checkpoint is in the AutoAWQ GEMM layout (4 bits, group size 64, asymmetric with integer zero points) so that vLLM's awq / awqmarlin kernels can load it. The weights come from the recipe in github.com/dfed25/mlx-gptq run with --zero-point int: error-feedback rounding (GPTQ), an alternating grid fit, coordinate-descent refinement and a weighted refit, with every group's offset constrained to a whole number of steps from the start, so that conversion to this format changes the weights by 0.05% (mean relative), against about 10% when a free-offset model is converted afterwards. Measured on the MLX version of the same weights (MacBook Pro M4…

Read Domenic Federico's full model card

Qwen2.5-1.5B-Instruct, 4-bit, AWQ file format (integer zero points), for vLLM / int4 GPU kernels

Status: exported, not yet verified on a GPU. This checkpoint is in the AutoAWQ GEMM layout (4 bits, group size 64, asymmetric with integer zero points) so that vLLM's awq / awq_marlin kernels can load it. The weights come from the recipe in github.com/dfed25/mlx-gptq run with --zero-point int: error-feedback rounding (GPTQ), an alternating grid fit, coordinate-descent refinement and a weighted refit, with every group's offset constrained to a whole number of steps from the start, so that conversion to this format changes the weights by 0.05% (mean relative), against about 10% when a free-offset model is converted afterwards.

Measured on the MLX version of the same weights (MacBook Pro M4 Pro): WikiText-2 test perplexity 9.663 (fp16 9.379; the free-offset MLX model 9.586; Qwen's official AWQ 4-bit 10.16 and GPTQ-Int4 10.40 as measured independently on an A10G), HumanEval pass@1 33.5% (fp16 38.4%; standard error about 3.5 points). The vLLM numbers will be added here once run.

vllm serve dfed24/Qwen2.5-1.5B-Instruct-gptq-4bit-int4-awq --quantization awq_marlin --dtype float16

Calibration: 128 x 512 tokens of WikiText-2 train (in-domain for the perplexity above). Converter: mlx2hf.py --awq by M. Federico. Author: Domenic Federico (Cal Poly San Luis Obispo; [email protected]), built with Claude (Anthropic) as a coding and research assistant. Licence follows the base model (Apache 2.0).

Configuration

Architecture
Qwen2ForCausalLM
Context length (tokens)
32,768
Layers
28
Hidden size
1,536
Feed-forward size
8,960
Attention heads
12
Key/value heads
2
Vocabulary size
151,936
Sliding window (tokens)
32,768
RoPE base
1e+06
Stored precision
float16
Model type
qwen2
Quantization
awq

Identity and Version

Repository
dfed24/Qwen2.5-1.5B-Instruct-gptq-4bit-int4-awq
Publisher
Domenic Federico
Task
Text generation
Modality
Text
Library
Not stated by the source
Parameters
1.5B parameters
Languages
awq
Revision
a8e29f82c18d121c1e6d550efa028920f2d6edba
First published
2026-10-05
Last updated
2026-10-05

Files and Weights

9 files, 1.2 GB in total. The weights are 1 file totalling 1.2 GB in safetensors.

Weights1 file · 1.2 GB
Configuration3 files · 52.7 KB
Tokenizer2 files · 11.4 MB
Documentation1 file · 1.7 KB
Other1 file · 2.5 KB
Repository1 file · 1.6 KB
Every file
FileTypeSizeSHA-256
model.safetensorsWeights1.2 GB 7ce4f3161e39
config.jsonConfiguration880 B —
generation_config.jsonConfiguration242 B —
model.safetensors.index.jsonConfiguration51.6 KB —
README.mdDocumentation1.7 KB —
chat_template.jinjaOther2.5 KB —
.gitattributesRepository1.6 KB —
tokenizer.jsonTokenizer11.4 MB 3fd169731d2c
tokenizer_config.jsonTokenizer729 B —

License and Download

License
apache-2.0
Access
Open weights, no gate
Download size
1.2 GB
Download from Domenic Federico

Released by Domenic Federico through its official repository on Hugging Face. Read the license.

Built From

Memory Requirements

PrecisionWeights in memory
As published1.2 GB
16-bit3.1 GB
8-bit1.5 GB
4-bit0.8 GB

Weights only, from the published parameter count; the key-value cache and runtime add to this.

Questions About Qwen2.5-1.5B-Instruct-gptq-4bit-int4-awq

How much GPU memory does Qwen2.5-1.5B-Instruct-gptq-4bit-int4-awq need?

About 3.7 GB at 16-bit and 0.9 GB at 4-bit: the weights (1.5B parameters) plus a working margin. A long context needs more.

What is the cheapest GPU to run Qwen2.5-1.5B-Instruct-gptq-4bit-int4-awq on?

At 16-bit, 1x MI300X from $1.85 an hour; at 4-bit, 1x MI300X from $1.85 an hour, at the lowest on-demand prices the SAVRN Index lists.

Can I use Qwen2.5-1.5B-Instruct-gptq-4bit-int4-awq commercially?

Yes. Qwen2.5-1.5B-Instruct-gptq-4bit-int4-awq is released under Apache License 2.0. The Apache License 2.0 is a permissive open-source license. It permits commercial use, modification and redistribution. It requires keeping the license and copyright notices and any NOTICE file, stating significant changes, and it includes an express patent grant from contributors.

What is Qwen2.5-1.5B-Instruct-gptq-4bit-int4-awq's context length?

32,768 tokens, from the maximum position embeddings in its published configuration.

Similar Models

Model · Text generation

Qwen2.5-1.5B-Instruct

Qwen

Qwen2.5 is the latest series of Qwen large language models. For Qwen2.5, we release a number of base language models and instruction-tuned language models ranging from 0.5 to 72 billion parameters. Qwen2.5 brings the following improvements upon Qwen2: - Significantly more knowledge and has greatly improved capabilities in coding and mathematics, thanks to our specialized expert models in these domains. - Significant improvements in instruction following, generating long texts (over 8K tokens), understanding structured data (e.g, tables), and generating structured outputs especially JSON. More resilient to the diversity of system prompts, enhancing role-play implementation and…

Open weights apache-2.0 1.5B parameters 32,768 tokens transformers
View model

Model · Text generation

Qwen2.5-1.5B

Qwen

Qwen2.5 is the latest series of Qwen large language models. For Qwen2.5, we release a number of base language models and instruction-tuned language models ranging from 0.5 to 72 billion parameters. Qwen2.5 brings the following improvements upon Qwen2: - Significantly more knowledge and has greatly improved capabilities in coding and mathematics, thanks to our specialized expert models in these domains. - Significant improvements in instruction following, generating long texts (over 8K tokens), understanding structured data (e.g, tables), and generating structured outputs especially JSON. More resilient to the diversity of system prompts, enhancing role-play implementation and…

Open weights apache-2.0 1.5B parameters 131,072 tokens transformers
View model

Model · Text generation

logi-1.5b

Alves

This qwen2 model was trained 2x faster with Unsloth and Huggingface's TRL library.

Open weights apache-2.0 1.5B parameters 32,768 tokens transformers
View model

Model · Text generation

logi-1.5b-gguf

Alves

Versões quantizadas em GGUF do LOGI 1.5B, um modelo educacional em português brasileiro criado para o jogo Decifra.IA. O LOGI atua como tutor de: - lógica proposicional: AND, OR, NOT, IMPLICA, BICONDICIONAL, tabelas-verdade, equivalências, falácias e regras de inferência; - alfabetização em inteligência artificial: modelos, dados, treinamento, inferência, vieses, alucinação, RAG, privacidade e uso responsável; - segurança digital relacionada à IA: phishing, deepfakes, engenharia social, prompt injection e proteção de dados. Este repositório contém as quantizações da versão v5, treinada com 13.000 exemplos. Ela substitui a versão inicial do LOGI, que utilizava 1.183 exemplos. Para…

Open weights apache-2.0 1.5B parameters 32,768 tokens
View model

This repository contains an instruction-tuned causal language model for text generation and chat-style prompts. Use a recent version of transformers that supports this model architecture. The following example loads the model from this repository and generates a response.

Open weights apache-2.0 1.5B parameters 32,768 tokens transformers
View model