GPTQ Quantized Qwen/Qwen3-Reranker-4B with Ultrachat, THUIR/T2Ranking and m-a-p/COIG-CQIA for calibration set. VRAM Usage: 17430M -> 11000M (w/o FA2, according to Embedding model's result). I think <5% accuracy, further evaluation on the way... The Embedding one shows ~0.7%. pip install compressed-tensors optimum and auto-gptq / gptqmodel, then goto the official usage guide.
VirbiusGuard-4B is an open-weight model for text classification from Min Cai, released under Apache License 2.0. It has 4B parameters and a 32,768-token context. At 16-bit it needs about 9.7 GB of GPU memory, which fits on 1x MI300X from $1.85 an hour, at the lowest prices in the SAVRN Index. It draws 210 downloads a month.
VirbiusAgent 安全分类器（Prompt L1 检测），基于 Qwen3Guard-Gen-4B 微调的 LoRA 模型。 输出严格 JSON：hitrule 与 triggeredid。 同口径评测相对基座：漏检 15.4% 降到 0.8%（gold1000 / V15），jailbreak 召回 57.1% 升到 100%。 0.6B 轻量版：https://www.modelscope.cn/models/i1see1you/VirbiusGuard 基座用官方 Safety 模板（Unsafe…
Parameters4B
Context32,768
Weights22.9 GB
Licenseapache-2.0
AccessOpen weights
Monthly Downloads210
Runs On
What it takes to serve VirbiusGuard-4B (4B parameters): the memory its weights need at each precision, and the cheapest way to rent enough data-center GPUs to hold them.
| Precision | Weights | Memory needed | Cheapest setup | Per hour | Also fits |
|---|---|---|---|---|---|
| 16-bit | 8.0 GB | 9.7 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
| 8-bit | 4.0 GB | 4.8 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
| 4-bit | 2.0 GB | 2.4 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
Memory is the weights at that precision plus 20% for the runtime and a short context; a long context needs more. Prices are the lowest on-demand hourly rates in the SAVRN Index, read Oct 1, 2026.
VirbiusGuard-4B on every accelerator the SAVRN Index prices, at every precision
Model Card
By Min Cai, published under apache-2.0, revision 60afa117a7d1.
VirbiusAgent 安全分类器（Prompt L1 检测），基于 Qwen3Guard-Gen-4B 微调的 LoRA 模型。 输出严格 JSON：hitrule 与 triggeredid。 同口径评测相对基座：漏检 15.4% 降到 0.8%（gold1000 / V15），jailbreak 召回 57.1% 升到 100%。 0.6B 轻量版：https://www.modelscope.cn/models/i1see1you/VirbiusGuard 基座用官方 Safety 模板（Unsafe / Controversial 视为拦截）；VirbiusGuard-4B 用引擎 JSON 协议。评测集与口径相同。 评测集：data/eval/gold1000.jsonl（615 unsafe / 385 safe）。误报 = FP / 385。 基座漏掉的主要是越狱与 Agent 工具滥用。V13.3 召回拉满但误报过高；V15 起进入可用区。V17 误报最低，但召回/自伤回退。 - 架构：Qwen3ForCausalLM（4B），LoRA（rank 32 / alpha 64） - 基座：Qwen3Guard-Gen-4B - 相对基座的补强：jailbreak 与 agent-behavior - V17 数据：与 0.6B V15 同口径，良性切片再平衡，含 oasst1、COIG 中文散文、OCR 风格文本 输出 10 种 unsafe 类别（triggeredid）或 safe（hitrule 为 false）。每条输入只输出一个主要类别：…
Read Min Cai's full model card
VirbiusAgent 安全分类器（Prompt L1 检测），基于 Qwen3Guard-Gen-4B 微调的 LoRA 模型。 输出严格 JSON：hit_rule 与 triggered_id。
同口径评测相对基座：漏检 15.4% 降到 0.8%（gold_1000 / V15），jailbreak 召回 57.1% 升到 100%。
0.6B 轻量版：https://www.modelscope.cn/models/i1see1you/VirbiusGuard
与 Qwen3Guard-Gen-4B 对比
基座用官方 Safety 模板（Unsafe / Controversial 视为拦截）；VirbiusGuard-4B 用引擎 JSON 协议。评测集与口径相同。
gold_1000（主表）
评测集：data/eval/gold_1000.jsonl（615 unsafe / 385 safe）。误报 = FP / 385。
| 模型 | acc | recall | 漏检 | FP | precision |
|---|---|---|---|---|---|
| Qwen3Guard-Gen-4B | 87.9% | 84.6% | 15.4%（95/615） | 6.8%（26/385） | 95.2% |
| 4B V13.3 | 93.6% | 99.5% | 0.5%（3/615） | 15.8%（61/385） | 90.9% |
| VirbiusGuard-4B V15 | 97.5% | 99.2% | 0.8%（5/615） | 5.2%（20/385） | 96.8% |
| 4B V17 | 97.8% | 98.0% | 2.0%（12/615） | 2.6%（10/385） | 98.4% |
基座漏掉的主要是越狱与 Agent 工具滥用。V13.3 召回拉满但误报过高；V15 起进入可用区。V17 误报最低，但召回/自伤回退。
关键类别召回（gold_1000）
| 类别 | 基座 | V13.3 | V15 | V17 |
|---|---|---|---|---|
| Jailbreak（98） | 57.1% | 100% | 100% | 98.0% |
| Agent Tool Misuse（84） | 81.0% | 100% | 98.8% | 100% |
| Suicide and Self-Harm（33） | 93.9% | 100% | 93.9% | 93.9% |
版本取舍（仅 gold_1000）
- 要最低误报：V17 （10/385）
- 要高召回且可用：V15
模型简介
- 架构：Qwen3ForCausalLM（4B），LoRA（rank 32 / alpha 64）
- 基座：Qwen3Guard-Gen-4B
- 版本：V17（当前默认）
- 相对基座的补强：jailbreak 与 agent-behavior
- V17 数据：与 0.6B V15 同口径，良性切片再平衡，含 oasst1、COIG 中文散文、OCR 风格文本
分类体系
输出 10 种 unsafe 类别（triggered_id）或 safe（hit_rule 为 false）。每条输入只输出一个主要类别：
Violent、Non-violent Illegal Acts、Unethical Acts、Suicide and Self-Harm、Jailbreak、PII、Copyright Violation、Politically Sensitive Topics、Sexual Content or Sexual Acts、Agent Tool Misuse。
训练数据按 A 口径（提及即违规）标注，Politically Sensitive 拦截较严。
下载
HuggingFace：https://huggingface.co/i1see1you/VirbiusGuard-4B
main 为最新 V17。
- Transformers：model-00001-of-00005.safetensors 至 model-00005-of-00005.safetensors（fp16，约 7.5GB）
- GGUF F16：gguf/virbiusguard-4b-v17-f16.gguf（约 7.5GB，Ollama / llama.cpp）
- GGUF Q8_0：gguf/virbiusguard-4b-v17-q8_0.gguf（约 4.0GB）
- GGUF Q4_K_M：gguf/virbiusguard-4b-v17-q4_k_m.gguf（约 2.3GB）
使用方式
Transformers：
from_pretrained("i1see1you/VirbiusGuard-4B")，用 tokenizer.apply_chat_template 组 prompt，max_new_tokens 至少 40。系统提示要求只输出 JSON 字段 hit_rule 与 triggered_id。
VirbiusAgent 引擎：替换环境变量 VIRBIUS_PROMPT_LLM_MODEL 即生效。
训练方法（概述）
- 教师模型离线标注，知识蒸馏
- mlx-lm LoRA 微调（rank 32 / alpha 64 / dropout 0.1 / lr 1.5e-4 / 2 epoch）
- 训练集：与 0.6B V15 同份 virbius_v15_train，约 22793 条
许可证 / 归属
基于 Qwen3Guard-Gen-4B 微调，数据集由教师模型离线标注。
联系我们
- 产品介绍：http://www.virbius.tech/virbiusguard.html
- 官网：https://www.grainmind.cn/
- 邮箱：[email protected]
Configuration
- Architecture
- Qwen3ForCausalLM
- Context length (tokens)
- 32,768
- Layers
- 36
- Hidden size
- 2,560
- Feed-forward size
- 9,728
- Attention heads
- 32
- Key/value heads
- 8
- Head dimension
- 128
- Vocabulary size
- 151,936
- RoPE base
- 1,000,000
- Stored precision
- bfloat16
- Model type
- qwen3
Identity and Version
- Repository
- i1see1you/VirbiusGuard-4B
- Publisher
- Min Cai
- Task
- Text classification
- Modality
- Text
- Library
- transformers
- Parameters
- 4B parameters
- Languages
- zh, en
- Revision
- 60afa117a7d100df9b877d3c546683bd22e57f33
- First published
- 2026-09-11
- Last updated
- 2026-09-24
Files and Weights
13 files, 22.9 GB in total. The weights are 5 files totalling 22.9 GB in gguf, safetensors.
Weights5 files · 22.9 GB
Configuration3 files · 35.4 KB
Tokenizer2 files · 11.4 MB
Documentation1 file · 3.8 KB
Other1 file · 216 B
Repository1 file · 1.9 KB
Every file
| File | Type | Size | SHA-256 |
|---|---|---|---|
| gguf/virbiusguard-4b-v17-f16.gguf | Weights | 8.1 GB | c6a6cde6e4d4 |
| gguf/virbiusguard-4b-v17-q4_k_m.gguf | Weights | 2.5 GB | 5f4a4904b5fb |
| gguf/virbiusguard-4b-v17-q8_0.gguf | Weights | 4.3 GB | 3cc0eaa0f651 |
| model-00001-of-00002.safetensors | Weights | 5.3 GB | e6900cbad9fe |
| model-00002-of-00002.safetensors | Weights | 2.7 GB | ee371e467cac |
| config.json | Configuration | 816 B | — |
| generation_config.json | Configuration | 161 B | — |
| model.safetensors.index.json | Configuration | 34.5 KB | — |
| README.md | Documentation | 3.8 KB | — |
| chat_template.jinja | Other | 216 B | — |
| .gitattributes | Repository | 1.9 KB | — |
| tokenizer.json | Tokenizer | 11.4 MB | be75606093db |
| tokenizer_config.json | Tokenizer | 693 B | — |
License and Download
- License
- apache-2.0
- Access
- Open weights, no gate
- Download size
- 22.9 GB
Download from Min Cai
Released by Min Cai through its official repository on Hugging Face. Read the license.
Built From
- Derived from Qwen/Qwen3Guard-Gen-4B
- Quantized from Qwen/Qwen3Guard-Gen-4B
Memory Requirements
| Precision | Weights in memory |
|---|---|
| As published | 22.9 GB |
| 16-bit | 8.0 GB |
| 8-bit | 4.0 GB |
| 4-bit | 2.0 GB |
Weights only, from the published parameter count; the key-value cache and runtime add to this.
Questions About VirbiusGuard-4B
How much GPU memory does VirbiusGuard-4B need?
About 9.7 GB at 16-bit and 2.4 GB at 4-bit: the weights (4B parameters) plus a working margin. A long context needs more.
What is the cheapest GPU to run VirbiusGuard-4B on?
At 16-bit, 1x MI300X from $1.85 an hour; at 4-bit, 1x MI300X from $1.85 an hour, at the lowest on-demand prices the SAVRN Index lists.
Can I use VirbiusGuard-4B commercially?
Yes. VirbiusGuard-4B is released under Apache License 2.0. The Apache License 2.0 is a permissive open-source license. It permits commercial use, modification and redistribution. It requires keeping the license and copyright notices and any NOTICE file, stating significant changes, and it includes an express patent grant from contributors.
What is VirbiusGuard-4B's context length?
32,768 tokens, from the maximum position embeddings in its published configuration.
Similar Models
Small, fast decisions for routing and checks at volume. Send text or JSON state with choice, noul (yes/no), or score questions. Get a probability for every offered answer, not generated text. Each batch takes one forward pass; large requests can use several batches. Use choice to route a request, noul for a yes/no check, or score for an ordered rating. The same call can ask several questions about a single state. Try it: POST /v1/systemone, GET /v1/models, and GET /healthz. Point TypeSafe's server-side Python or JavaScript SDKs at it with TYPESAFEBASEURL; text decisions use the same request and response fields as hosted Jev. Requests run one at a time by default; --batch-window-ms 5 enables…
Mapika/decider-4b v2.1 quantized to FP8 for vLLM: FP8 E4M3 weights with one scale per output channel and FP8 activations scaled per token at run time. 4.85 GB against 8.41 GB for the bf16 checkpoint. Quantized and measured by LLM Tech; the model, its training and its evaluation protocol are Mapika's. Read the bf16 card for what the model is and how it was trained. The base revision is eb5fbdfc9448473ec25e399882912863afbdb70e. Tokenizer, chat template, generation config and deciderconfig.json (temperatures included) are the author's files unchanged, apart from the version and quantization fields. Both models were run through vLLM 0.29.0 on the same rows: the author's regression set rebuilt…
Kev-4B (a LoRA on Qwen3.5-4B-Base with a pointer head) packed for mlx-serve's POST /v1/decisions. Kev answers typed questions about a piece of text (choice, noul, score) with calibrated probabilities. It never generates text. The pack folds the LoRA into the base the way kev does on MLX, quantizes the trunk to 8-bit (affine, group 64; a bf16 build comes from --q-bits 0), and stores the pointer head as kevhead.safetensors with the calibration temperature in kevconfig.json. No PyTorch or pickle file is needed to serve it. Built with tests/convertkevweights.py from the mlx-serve repo. Kev and Qwen3.5 are Apache-2.0.
Experimental open-weights judgment model by Kitani OpenJudgement is unfinished. We're releasing this checkpoint for people to experiment with, inspect, and build on. It still needs work on judgment quality, calibration, and inference efficiency. It is not as good as Jev overall in our internal task comparisons. It does show a meaningful improvement over untouched Qwen on our recorded validation comparison: 75.4% versus 64.2% annotation agreement. That is a result on a particular evaluation set, not a claim that we beat the base model on every task. There are questions it handles well and questions it confidently gets wrong. Please judge the preview by your own examples rather than assuming…
A calibrated typed-decision model: give it a state (text, ticket, policy, JSON) and a typed question — choice, boolean, or rubric score — and it returns a probability for every option in a single forward pass (~24 ms). No generation, no parsing, nothing to hallucinate. 12 of 13 subsets exceed Bespoke Nimble-9B — a model 2.2× its size — same prompt format, same scoring protocol. The primary suite: BoolQ, MultiNLI, PAWS, PubMedQA, SQuAD-2, VitaminC, Civil Comments, Aegis 2.0, MASSIVE (en/de), HelpSteer-2, SummEval (consistency / relevance). Every item human-labeled; byte-reproducible (manifest-locked ids + sha256); same protocol as the Bespoke Nimble evaluation. Wins: verification-style noul…