Kev-4B-MLX-Serve-8bit by Alin C Selea: Open-Weight Model

Runs On

What it takes to serve Kev-4B-MLX-Serve-8bit (4.2B parameters): the memory its weights need at each precision, and the cheapest way to rent enough data-center GPUs to hold them.

Precision	Weights	Memory needed	Cheapest setup	Per hour	Also fits
16-bit	8.4 GB	10.1 GB	1x MI300X (192 GB) Vultr	$1.85	1x H100 $1.99 · 1x MI325X $2.00
8-bit	4.2 GB	5.0 GB	1x MI300X (192 GB) Vultr	$1.85	1x H100 $1.99 · 1x MI325X $2.00
4-bit	2.1 GB	2.5 GB	1x MI300X (192 GB) Vultr	$1.85	1x H100 $1.99 · 1x MI325X $2.00

Memory is the weights at that precision plus 20% for the runtime and a short context; a long context needs more. Prices are the lowest on-demand hourly rates in the SAVRN Index, read Oct 1, 2026.

Kev-4B-MLX-Serve-8bit on every accelerator the SAVRN Index prices, at every precision

Model Card

By Alin C Selea, published under apache-2.0, revision 2fd4aab21005.

Kev-4B (a LoRA on Qwen3.5-4B-Base with a pointer head) packed for mlx-serve's POST /v1/decisions. Kev answers typed questions about a piece of text (choice, noul, score) with calibrated probabilities. It never generates text. The pack folds the LoRA into the base the way kev does on MLX, quantizes the trunk to 8-bit (affine, group 64; a bf16 build comes from --q-bits 0), and stores the pointer head as kevhead.safetensors with the calibration temperature in kevconfig.json. No PyTorch or pickle file is needed to serve it. Built with tests/convertkevweights.py from the mlx-serve repo. Kev and Qwen3.5 are Apache-2.0.

Read Alin C Selea's full model card

Kev-4B for mlx-serve

Kev-4B (a LoRA on Qwen3.5-4B-Base with a pointer head) packed for mlx-serve's POST /v1/decisions.

Kev answers typed questions about a piece of text (choice, noul, score) with calibrated probabilities. It never generates text.

The pack folds the LoRA into the base the way kev does on MLX, quantizes the trunk to 8-bit (affine, group 64; a bf16 build comes from --q-bits 0), and stores the pointer head as kev_head.safetensors with the calibration temperature in kev_config.json. No PyTorch or pickle file is needed to serve it.

curl -s localhost:11234/v1/decisions -H 'content-type: application/json' -d '{
  "model": "<this model id>",
  "state": {"subject": "Charged twice", "body": "Please refund the duplicate today."},
  "questions": {"refund": {"type": "noul", "instructions": "Does the customer ask for money back?"}}
}'

Built with tests/convert_kev_weights.py from the mlx-serve repo. Kev and Qwen3.5 are Apache-2.0.

Configuration

Architecture: Qwen3_5ForConditionalGeneration
Context length (tokens): 262,144
Layers: 32
Hidden size: 2,560
Feed-forward size: 9,216
Attention heads: 16
Key/value heads: 4
Head dimension: 256
Vocabulary size: 248,320
Model type: qwen3_5

Identity and Version

Repository: aselea/Kev-4B-MLX-Serve-8bit
Publisher: Alin C Selea
Task: Text classification
Modality: Text
Library: mlx-serve
Parameters: 4.2B parameters
Languages: mlx-serve, mlx, kev
Revision: 2fd4aab21005e4e88e065b964a03e9bf69195bfd
First published: 2026-09-28
Last updated: 2026-09-28

Files and Weights

10 files, 4.5 GB in total. The weights are 2 files totalling 4.5 GB in safetensors.

Weights2 files · 4.5 GB

Configuration3 files · 84.4 KB

Tokenizer2 files · 20.0 MB

Documentation1 file · 1.3 KB

Other1 file · 7.8 KB

Repository1 file · 1.6 KB

Every file

File	Type	Size	SHA-256
kev_head.safetensors	Weights	5.2 MB	f136fa6f6042
model.safetensors	Weights	4.5 GB	83adf34ef8f2
config.json	Configuration	2.9 KB	—
kev_config.json	Configuration	469 B	—
model.safetensors.index.json	Configuration	81.0 KB	—
README.md	Documentation	1.3 KB	—
chat_template.jinja	Other	7.8 KB	—
.gitattributes	Repository	1.6 KB	—
tokenizer.json	Tokenizer	20.0 MB	06b9509352d2
tokenizer_config.json	Tokenizer	1.1 KB	—

License and Download

License: apache-2.0
Access: Open weights, no gate
Download size: 4.5 GB

Download from Alin C Selea

Released by Alin C Selea through its official repository on Hugging Face. Read the license.

Built From

Derived from jaredpalmer/kev-4b
Quantized from jaredpalmer/kev-4b

Memory Requirements

Precision	Weights in memory
As published	4.5 GB
16-bit	8.4 GB
8-bit	4.2 GB
4-bit	2.1 GB

Weights only, from the published parameter count; the key-value cache and runtime add to this.

Questions About Kev-4B-MLX-Serve-8bit

How much GPU memory does Kev-4B-MLX-Serve-8bit need?

About 10.1 GB at 16-bit and 2.5 GB at 4-bit: the weights (4.2B parameters) plus a working margin. A long context needs more.

What is the cheapest GPU to run Kev-4B-MLX-Serve-8bit on?

At 16-bit, 1x MI300X from $1.85 an hour; at 4-bit, 1x MI300X from $1.85 an hour, at the lowest on-demand prices the SAVRN Index lists.

Can I use Kev-4B-MLX-Serve-8bit commercially?

Yes. Kev-4B-MLX-Serve-8bit is released under Apache License 2.0. The Apache License 2.0 is a permissive open-source license. It permits commercial use, modification and redistribution. It requires keeping the license and copyright notices and any NOTICE file, stating significant changes, and it includes an express patent grant from contributors.

What is Kev-4B-MLX-Serve-8bit's context length?

262,144 tokens, from the maximum position embeddings in its published configuration.

Similar Models

Model · Text classification

blink-4b

Govind Kamtamneni

Small, fast decisions for routing and checks at volume. Send text or JSON state with choice, noul (yes/no), or score questions. Get a probability for every offered answer, not generated text. Each batch takes one forward pass; large requests can use several batches. Use choice to route a request, noul for a yes/no check, or score for an ordered rating. The same call can ask several questions about a single state. Try it: POST /v1/systemone, GET /v1/models, and GET /healthz. Point TypeSafe's server-side Python or JavaScript SDKs at it with TYPESAFEBASEURL; text decisions use the same request and response fields as hosted Jev. Requests run one at a time by default; --batch-window-ms 5 enables…

Open weights other 4.2B parameters 262,144 tokens transformers

View model

Model · Text classification

decider-4b-fp8

LLM Tech

Mapika/decider-4b v2.1 quantized to FP8 for vLLM: FP8 E4M3 weights with one scale per output channel and FP8 activations scaled per token at run time. 4.85 GB against 8.41 GB for the bf16 checkpoint. Quantized and measured by LLM Tech; the model, its training and its evaluation protocol are Mapika's. Read the bf16 card for what the model is and how it was trained. The base revision is eb5fbdfc9448473ec25e399882912863afbdb70e. Tokenizer, chat template, generation config and deciderconfig.json (temperatures included) are the author's files unchanged, apart from the version and quantization fields. Both models were run through vLLM 0.29.0 on the same rows: the author's regression set rebuilt…

Open weights apache-2.0 4.2B parameters 262,144 tokens

View model

Model · Text classification

Qwen3-Reranker-4B-W4A16-G128

Mou Geren

GPTQ Quantized Qwen/Qwen3-Reranker-4B with Ultrachat, THUIR/T2Ranking and m-a-p/COIG-CQIA for calibration set. VRAM Usage: 17430M -> 11000M (w/o FA2, according to Embedding model's result). I think <5% accuracy, further evaluation on the way... The Embedding one shows ~0.7%. pip install compressed-tensors optimum and auto-gptq / gptqmodel, then goto the official usage guide.

Open weights apache-2.0 4.1B parameters 40,960 tokens transformers

View model

Model · Text classification

VirbiusGuard-4B

Min Cai

VirbiusAgent 安全分类器（Prompt L1 检测），基于 Qwen3Guard-Gen-4B 微调的 LoRA 模型。输出严格 JSON：hitrule 与 triggeredid。同口径评测相对基座：漏检 15.4% 降到 0.8%（gold1000 / V15），jailbreak 召回 57.1% 升到 100%。 0.6B 轻量版：https://www.modelscope.cn/models/i1see1you/VirbiusGuard 基座用官方 Safety 模板（Unsafe / Controversial 视为拦截）；VirbiusGuard-4B 用引擎 JSON 协议。评测集与口径相同。评测集：data/eval/gold1000.jsonl（615 unsafe / 385 safe）。误报 = FP / 385。基座漏掉的主要是越狱与 Agent 工具滥用。V13.3 召回拉满但误报过高；V15 起进入可用区。V17 误报最低，但召回/自伤回退。 - 架构：Qwen3ForCausalLM（4B），LoRA（rank 32 / alpha 64） - 基座：Qwen3Guard-Gen-4B - 相对基座的补强：jailbreak 与 agent-behavior - V17 数据：与 0.6B V15 同口径，良性切片再平衡，含 oasst1、COIG 中文散文、OCR 风格文本输出 10 种 unsafe 类别（triggeredid）或 safe（hitrule 为 false）。每条输入只输出一个主要类别：…

Open weights apache-2.0 4B parameters 32,768 tokens transformers

View model

Model · Text classification

OpenJudgement-4B-Preview

Kitani

Experimental open-weights judgment model by Kitani OpenJudgement is unfinished. We're releasing this checkpoint for people to experiment with, inspect, and build on. It still needs work on judgment quality, calibration, and inference efficiency. It is not as good as Jev overall in our internal task comparisons. It does show a meaningful improvement over untouched Qwen on our recorded validation comparison: 75.4% versus 64.2% annotation agreement. That is a result on a particular evaluation set, not a claim that we beat the base model on every task. There are questions it handles well and questions it confidently gets wrong. Please judge the preview by your own examples rather than assuming…

Open weights apache-2.0 4.5B parameters 262,144 tokens transformers

View model

Model · Text classification

metask-jev-4b-policy-mix

Raymond wei

A calibrated typed-decision model: give it a state (text, ticket, policy, JSON) and a typed question — choice, boolean, or rubric score — and it returns a probability for every option in a single forward pass (~24 ms). No generation, no parsing, nothing to hallucinate. 12 of 13 subsets exceed Bespoke Nimble-9B — a model 2.2× its size — same prompt format, same scoring protocol. The primary suite: BoolQ, MultiNLI, PAWS, PubMedQA, SQuAD-2, VitaminC, Civil Comments, Aegis 2.0, MASSIVE (en/de), HelpSteer-2, SummEval (consistency / relevance). Every item human-labeled; byte-reproducible (manifest-locked ids + sha256); same protocol as the Bespoke Nimble evaluation. Wins: verification-style noul…

Open weights apache-2.0 4.5B parameters 262,144 tokens transformers

View model