Jev_Qwen3.8-27B-NVFP4-FP8 is an open-weight model from Will Cheung, released under Apache License 2.0. It has 21.2B parameters and a 262,144-token context. At 16-bit it needs about 50.8 GB of GPU memory, which fits on 1x MI300X from $1.85 an hour, at the lowest prices in the SAVRN Index.
NVFP4 + FP8 mixed-precision quantization of SargeDev/JevQwen3.8-27B (itself a QLoRA merge of huihui-ai/Huihui-Qwen3.8-27B-abliterated trained on SargeDev/jev-distill-corpus-v3).
Runs On
What it takes to serve Jev_Qwen3.8-27B-NVFP4-FP8 (21.2B parameters): the memory its weights need at each precision, and the cheapest way to rent enough data-center GPUs to hold them.
| Precision | Weights | Memory needed | Cheapest setup | Per hour | Also fits |
|---|---|---|---|---|---|
| 16-bit | 42.3 GB | 50.8 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
| 8-bit | 21.2 GB | 25.4 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
| 4-bit | 10.6 GB | 12.7 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
Memory is the weights at that precision plus 20% for the runtime and a short context; a long context needs more. Prices are the lowest on-demand hourly rates in the SAVRN Index, read Oct 20, 2026.
Jev_Qwen3.8-27B-NVFP4-FP8 on every accelerator the SAVRN Index prices, at every precision
Model Card
By Will Cheung, published under apache-2.0, revision b0efca9253c6.
NVFP4 + FP8 mixed-precision quantization of SargeDev/JevQwen3.8-27B (itself a QLoRA merge of huihui-ai/Huihui-Qwen3.8-27B-abliterated trained on SargeDev/jev-distill-corpus-v3). - NVFP4 W4A4 (E2M1, block 16, FP8-E4M3 scales): all attention projections + gate/upproj - FP8DYNAMIC (per-channel weights, per-token activations): all downproj — the layer class most sensitive to 4-bit - 28 GB total (vs 51 GB bf16) — loads in vLLM with --quantization compressed-tensors 512 samples, 2048 max seq: 384 held-out jev-distill judge prompts in the exact eval-time format (state + question + options), plus 128 ultrachat200k general-chat rows to keep broad-language calibration honest. Calibrating on the…
Read Will Cheung's full model card
NVFP4 + FP8 mixed-precision quantization of SargeDev/Jev_Qwen3.8-27B (itself a QLoRA merge of huihui-ai/Huihui-Qwen3.8-27B-abliterated trained on SargeDev/jev-distill-corpus-v3).
Quant recipe (non-uniform)
- Tool: llm-compressor 0.14.0 (vllm-project),
QuantizationModifiermixed config groups - NVFP4 W4A4 (E2M1, block 16, FP8-E4M3 scales): all attention projections + gate/up_proj
- FP8_DYNAMIC (per-channel weights, per-token activations): all down_proj — the layer class most sensitive to 4-bit
- Ignored (kept high-precision):
lm_head,embed_tokens, MTP head - Format: compressed-tensors
mixed-precision(nvfp4-pack-quantized + float-quantized) - 28 GB total (vs 51 GB bf16) — loads in vLLM with
--quantization compressed-tensors
Calibration (domain-matched)
512 samples, 2048 max seq: 384 held-out jev-distill judge prompts in the exact eval-time format (state + question + options), plus 128 ultrachat_200k general-chat rows to keep broad-language calibration honest. Calibrating on the deployment domain, not generic text.
Verified on DGX Spark (GB10)
- vLLM 0.28-era serving stack,
--quantization compressed-tensors, GB10 sm_121a - Structured judge outputs correct post-quant: calibrated JSON distributions intact
- ~8 tok/s single-stream long-form generation (200+ token completions), judge-style JSON answers land in 2-5 s
- Chatty persona intact (identifies as Qwen; no behavioral regression observed)
Serve
vllm serve /path/to/Jev_Qwen3.8-27B-NVFP4-FP8 \
--quantization compressed-tensors \
--served-model-name jev-nvfp4-fp8 \
--max-model-len 8192 \
--gpu-memory-utilization 0.80
Trained with thinking disabled (enable_thinking=false); serve with thinking off for tuned behavior.
Apache-2.0. Quantized on NVIDIA DGX Spark (GB10).
Configuration
- Architecture
- Qwen3_5ForCausalLM
- Context length (tokens)
- 262,144
- Layers
- 64
- Hidden size
- 5,120
- Feed-forward size
- 17,408
- Attention heads
- 24
- Key/value heads
- 4
- Head dimension
- 256
- Vocabulary size
- 248,320
- Model type
- qwen3_5_text
- Quantization
- compressed-tensors
Identity and Version
- Repository
- SargeDev/Jev_Qwen3.8-27B-NVFP4-FP8
- Publisher
- Will Cheung
- Task
- Not stated by the source
- Modality
- Other
- Library
- Not stated by the source
- Parameters
- 21.2B parameters
- Languages
- en
- Revision
- b0efca9253c6f5898cc62098756707c27323107a
- First published
- 2026-09-24
- Last updated
- 2026-09-24
Files and Weights
11 files, 29.3 GB in total. The weights are 2 files totalling 29.3 GB in safetensors.
Every file
| File | Type | Size | SHA-256 |
|---|---|---|---|
| model-00001-of-00002.safetensors | Weights | 19.9 GB | 6411c75a4a02 |
| model-00002-of-00002.safetensors | Weights | 9.3 GB | 9c2ab5406cd5 |
| config.json | Configuration | 20.1 KB | — |
| generation_config.json | Configuration | 214 B | — |
| model.safetensors.index.json | Configuration | 152.5 KB | — |
| recipe.yaml | Configuration | 2.1 KB | — |
| README.md | Documentation | 2.2 KB | — |
| chat_template.jinja | Other | 9.0 KB | — |
| .gitattributes | Repository | 1.6 KB | — |
| tokenizer.json | Tokenizer | 20.0 MB | f399b3cd12fa |
| tokenizer_config.json | Tokenizer | 1.1 KB | — |
License and Download
- License
- apache-2.0
- Access
- Open weights, no gate
- Download size
- 29.3 GB
Released by Will Cheung through its official repository on Hugging Face. Read the license.
Built From
- Derived from SargeDev/Jev_Qwen3.8-27B
- Quantized from SargeDev/Jev_Qwen3.8-27B
- Trained on (disclosed) SargeDev/jev-distill-corpus-v3
Memory Requirements
| Precision | Weights in memory |
|---|---|
| As published | 29.3 GB |
| 16-bit | 42.3 GB |
| 8-bit | 21.2 GB |
| 4-bit | 10.6 GB |
Weights only, from the published parameter count; the key-value cache and runtime add to this.
Questions About Jev_Qwen3.8-27B-NVFP4-FP8
How much GPU memory does Jev_Qwen3.8-27B-NVFP4-FP8 need?
About 50.8 GB at 16-bit and 12.7 GB at 4-bit: the weights (21.2B parameters) plus a working margin. A long context needs more.
What is the cheapest GPU to run Jev_Qwen3.8-27B-NVFP4-FP8 on?
At 16-bit, 1x MI300X from $1.85 an hour; at 4-bit, 1x MI300X from $1.85 an hour, at the lowest on-demand prices the SAVRN Index lists.
Can I use Jev_Qwen3.8-27B-NVFP4-FP8 commercially?
Yes. Jev_Qwen3.8-27B-NVFP4-FP8 is released under Apache License 2.0. The Apache License 2.0 is a permissive open-source license. It permits commercial use, modification and redistribution. It requires keeping the license and copyright notices and any NOTICE file, stating significant changes, and it includes an express patent grant from contributors.
What is Jev_Qwen3.8-27B-NVFP4-FP8's context length?
262,144 tokens, from the maximum position embeddings in its published configuration.