SAVRN
Search Contact SAVRN

Open-weight model · Text classification

Qwen3-Reranker-4B-W4A16-G128

by Mou Geren boboliu/Qwen3-Reranker-4B-W4A16-G128

GPTQ Quantized Qwen/Qwen3-Reranker-4B with Ultrachat, THUIR/T2Ranking and m-a-p/COIG-CQIA for calibration set. VRAM Usage: 17430M -> 11000M (w/o FA2, according to Embedding model's result). I think <5% accuracy, further evaluation on the way...

Parameters4.1B
Context40,960
Weights2.7 GB
Licenseapache-2.0
AccessOpen weights
Monthly Downloads583.7k

Runs On

What it takes to serve Qwen3-Reranker-4B-W4A16-G128 (4.1B parameters): the memory its weights need at each precision, and the cheapest way to rent enough data-center GPUs to hold them.

PrecisionWeightsMemory neededCheapest setupPer hourAlso fits
16-bit 8.1 GB 9.7 GB 1x MI300X (192 GB)
Vultr
$1.85 1x H100 $1.99 · 1x MI325X $2.00
8-bit 4.1 GB 4.9 GB 1x MI300X (192 GB)
Vultr
$1.85 1x H100 $1.99 · 1x MI325X $2.00
4-bit 2.0 GB 2.4 GB 1x MI300X (192 GB)
Vultr
$1.85 1x H100 $1.99 · 1x MI325X $2.00

Memory is the weights at that precision plus 20% for the runtime and a short context; a long context needs more. Prices are the lowest on-demand hourly rates in the SAVRN Index, read Sep 18, 2026.

SAVRN's Notes on Qwen3-Reranker-4B-W4A16-G128

A reranker scores retrieved passages before the generator sees them, and this is Mou Geren's 4-bit GPTQ cut of Qwen's Qwen3-Reranker-4B, calibrated on Ultrachat, THUIR T2Ranking and m-a-p COIG-CQIA. The W4A16 packaging is about memory: 2.0 GB of weights and 2.4 GB needed at 4-bit, against 9.7 GB at 16-bit; the publisher's own measurement is 11,000 MB of VRAM, down from 17,430 MB, without FA2. That fits beside the generator on one card, which is where we would put it; the Index's cheapest card is an MI300X at $1.85 an hour.

Apache 2.0 on this file allows commercial use, modification and redistribution with notices kept. The 40,960-token context means long candidate passages do not need trimming. Two checks: the publisher's accuracy loss estimate is under 5 percent and marked as awaiting further evaluation, and serving needs compressed-tensors, optimum and a GPTQ loader installed first.

Model Card

By Mou Geren, published under apache-2.0, revision 84d3701aa6a3.

GPTQ Quantized Qwen/Qwen3-Reranker-4B with Ultrachat, THUIR/T2Ranking and m-a-p/COIG-CQIA for calibration set. VRAM Usage: 17430M -> 11000M (w/o FA2, according to Embedding model's result). I think <5% accuracy, further evaluation on the way... The Embedding one shows ~0.7%. pip install compressed-tensors optimum and auto-gptq / gptqmodel, then goto the official usage guide.

Read Mou Geren's full model card

GPTQ Quantized Qwen/Qwen3-Reranker-4B with Ultrachat, THUIR/T2Ranking and m-a-p/COIG-CQIA for calibration set.

What's the benefit?

VRAM Usage: 17430M -> 11000M (w/o FA2, according to Embedding model's result).

What's the cost?

I think <5% accuracy, further evaluation on the way...

The Embedding one shows ~0.7%.

How to use it?

pip install compressed-tensors optimum and auto-gptq / gptqmodel, then goto the official usage guide.

Configuration

Architecture
Qwen3ForCausalLM
Context length (tokens)
40,960
Layers
36
Hidden size
2,560
Feed-forward size
9,728
Attention heads
32
Key/value heads
8
Head dimension
128
Vocabulary size
151,669
RoPE base
1,000,000
Stored precision
bfloat16
Model type
qwen3
Quantization
compressed-tensors

Identity and Version

Repository
boboliu/Qwen3-Reranker-4B-W4A16-G128
Publisher
Mou Geren
Task
Text classification
Modality
Text
Library
transformers
Parameters
4.1B parameters
Languages
Not stated by the source
Revision
84d3701aa6a3a7e581389c666ec340f91e7936c6
First published
2025-06-07
Last updated
2025-06-07

Files and Weights

10 files, 2.7 GB in total. The weights are 1 file totalling 2.7 GB in safetensors.

Weights1 file · 2.7 GB
Configuration3 files · 2.3 KB
Tokenizer4 files · 15.9 MB
Documentation1 file · 903 B
Repository1 file · 1.6 KB
Every file
FileTypeSizeSHA-256
model.safetensorsWeights2.7 GB d9372c3f5f2c
config.jsonConfiguration1.5 KB
generation_config.jsonConfiguration214 B
special_tokens_map.jsonConfiguration613 B
README.mdDocumentation903 B
.gitattributesRepository1.6 KB
merges.txtTokenizer1.7 MB
tokenizer.jsonTokenizer11.4 MB 5cfec206b0de
tokenizer_config.jsonTokenizer5.4 KB
vocab.jsonTokenizer2.8 MB

License and Download

License
apache-2.0
Access
Open weights, no gate
Download size
2.7 GB
Download from Mou Geren

Released by Mou Geren through its official repository on Hugging Face. Read the license.

Built From

Memory Requirements

PrecisionWeights in memory
As published2.7 GB
16-bit8.1 GB
8-bit4.1 GB
4-bit2.0 GB

Weights only, from the published parameter count; the key-value cache and runtime add to this.

Questions About Qwen3-Reranker-4B-W4A16-G128

How much GPU memory does Qwen3-Reranker-4B-W4A16-G128 need?

About 9.7 GB at 16-bit and 2.4 GB at 4-bit: the weights (4.1B parameters) plus a working margin. A long context needs more.

What is the cheapest GPU to run Qwen3-Reranker-4B-W4A16-G128 on?

At 16-bit, 1x MI300X from $1.85 an hour; at 4-bit, 1x MI300X from $1.85 an hour, at the lowest on-demand prices the SAVRN Index lists.

Can I use Qwen3-Reranker-4B-W4A16-G128 commercially?

Yes. Qwen3-Reranker-4B-W4A16-G128 is released under Apache License 2.0. The Apache License 2.0 is a permissive open-source license. It permits commercial use, modification and redistribution. It requires keeping the license and copyright notices and any NOTICE file, stating significant changes, and it includes an express patent grant from contributors.

What is Qwen3-Reranker-4B-W4A16-G128's context length?

40,960 tokens, from the maximum position embeddings in its published configuration.

Similar Models

Model · Text classification

jina-reranker-m0

Jina AI

pipelinetag: text-classification - sentence-transformers - vidore - reranker - qwen2vl - multilingual basemodel: libraryname: transformers jina-reranker-m0 is our new multilingual multimodal reranker model for ranking visual documents across multiple languages: it accepts a query alongside a collection of visually rich document images, including pages with text, figures, tables, infographics, and various layouts across multiple domains and over 29 languages. It outputs a ranked list of documents ordered by their relevance to the input query. Compared to jina-reranker-v2-base-multilingual, jina-reranker-m0 also improves text reranking for multilingual content, long documents, and code…

Open weights cc-by-nc-4.0 2.4B parameters 32,768 tokens transformers

More details please refer to our Github: FlagEmbedding. Different from embedding model, reranker uses question and document as input and directly output similarity instead of embedding. You can get a relevance score by inputting query and passage to the reranker. And the score can be mapped to a float value in [0,1] by sigmoid function. You can select the model according your senario and resource. - For multilingual, utilize BAAI/bge-reranker-v2-m3 and BAAI/bge-reranker-v2-gemma - For Chinese or English, utilize BAAI/bge-reranker-v2-m3 and BAAI/bge-reranker-v2-minicpm-layerwise. - For efficiency, utilize BAAI/bge-reranker-v2-m3 and the low layer of BAAI/bge-reranker-v2-minicpm-layerwise.…

Open weights apache-2.0 568M parameters 8,194 tokens sentence-transformers

Opir-multitask-large is the English, highest-accuracy multi-task checkpoint in the Opir family: an encoder-based GLiClass guardrail model for real-time LLM safety filtering. It supports binary safe/unsafe classification, toxicity detection, jailbreak and prompt-injection detection, and zero-shot harmful-content categorization over a hierarchical safety taxonomy. This card is for knowledgator/opir-multitask-large. The model is used through GLiClass zero-shot classification: pass text plus the candidate labels you want scored. Use single-label mode for binary safe/unsafe decisions and multi-label mode for taxonomy, toxicity, jailbreak, or custom policy labels. Use multi-label mode when you…

Open weights apache-2.0 439M parameters gliclass

Model · Text classification

roberta-large-mnli

Facebook AI community

roberta-large-mnli is the RoBERTa large model fine-tuned on the Multi-Genre Natural Language Inference (MNLI) corpus. The model is a pretrained model on English language text using a masked language modeling (MLM) objective. Use the code below to get started with the model. The model can be loaded with the zero-shot-classification pipeline like so: You can then use this pipeline to classify sequences into any of the class names you specify. For example: This fine-tuned model can be used for zero-shot classification tasks, including zero-shot sentence-pair classification (see the GitHub repo for examples) and zero-shot sequence classification. The model should not be used to intentionally…

Open weights mit 356M parameters 514 tokens transformers

Model · Text classification

Prompt-Guard-86M

Meta Llama

LLM-powered applications are susceptible to prompt attacks, which are prompts intentionally designed to subvert the developer’s intended behavior of the LLM. Categories of prompt attacks include prompt injection and jailbreaking: - Prompt Injections are inputs that exploit the concatenation of untrusted data from third parties and users into the context window of a model to get a model to execute unintended instructions. - Jailbreaks are malicious instructions designed to override the safety and security features built into a model. Prompt Guard is a classifier model trained on a large corpus of attacks, capable of detecting both explicitly malicious prompts as well as data that contains…

Access requested at publisher llama3.1 279M parameters transformers

This model is a fine-tuned version of xlm-roberta-base on the Language Identification dataset. This model is an XLM-RoBERTa transformer model with a classification head on top (i.e. a linear layer on top of the pooled output). For additional information please refer to the xlm-roberta-base model card or to the paper Unsupervised Cross-lingual Representation Learning at Scale by Conneau et al. You can directly use this model as a language detector, i.e. for sequence classification tasks. Currently, it supports the following 20 languages: arabic (ar), bulgarian (bg), german (de), modern greek (el), english (en), spanish (es), french (fr), hindi (hi), italian (it), japanese (ja), dutch (nl)…

Open weights mit 278M parameters 514 tokens transformers