2.5x faster throughput than other NVFP4 quants. This is an Unsloth NVFP4 quantized checkpoint calibrated on a mixture of our Unsloth dataset + UltraChat dataset. Works on a 24GB VRAM GPU. Benchmarks on 1xB200 128 concurrency. For accuracy benchmarks, we conducted MMLU-Pro, AIME 2025, GPQA for FP8, BF16, NVIDIA's NVFP4 and our NVFP4s - we show our faster quants do similarly on all: Read all benchmarks in our NVFP4 blog To install vLLM in a separate venv: Then to serve the 27B NVFP4 quant: Also do NOT use the Marlin backend since it's 2x slower - use the native vLLM or cute-DSL / CUTLASS / flashinfertrtllm backends! You must use the below or you will get 2x slower inference! This checkpoint…
Open weights
apache-2.0
21.2B parameters
262,144 tokens
transformers
This model is an updated quantized version of Qwen/Qwen3.8-27B, using a mixed-precision FP4/FP8 scheme with an unquantized language-model head and updated quantization scales. See Evaluation for accuracy results. This model was produced by applying mixed-precision quantization to Qwen/Qwen3.8-27B. MLP projections are quantized to FP4, attention projections and the final MLP layers are quantized to FP8, and the KV cache is quantized to FP8, while the language-model head is kept in full precision to preserve output quality. The quantization scales were updated by calibrating on a 512-sample subset of the perfectblend dataset with a recipe that combines AWQ and GPTQ. Only the weights and…
Open weights
apache-2.0
20.3B parameters
262,144 tokens
transformers
EXL3 quantization (2.25 bits per weight on average) of UkisAI's Swift 1.5 Qwen3.8-Flash-Next (MoE, 125B total / 6B active, plus a 51B n-gram "PLE" table, MTP head and vision), made to serve the whole model on one 48 GB GPU with vLLM (continuous batching): 262,144 tokens per session without YaRN, 4 concurrent sessions sharing a 420k-token KV cache, vision and MTP. Validated on a real NVIDIA RTX PRO 5000 Blackwell (48 GB). All Swift 1.5 Qwen3.8-Flash-Next (125B-A6B) variants we publish, measured on 2026-10-14 under the same conditions (see Family comparison for every number): Download size: 66.7 GB (40.5 GB of EXL3 weights + 26.2 GB EXL3 n-gram table). vLLM users can skip the n-gram table…
Open weights
other
20.2B parameters
262,144 tokens
exllamav3
EXL3 quantization (2.25 bits per weight on average) of Qwen3.8-Flash-Next (MoE, 125B total / 6B active, plus a 51B n-gram "PLE" table, MTP head and vision), made to serve the whole model on one 48 GB GPU with vLLM (continuous batching): 262,144 tokens per session without YaRN, 4 concurrent sessions sharing a 420k-token KV cache, vision and MTP. Same layout and memory footprint as our Swift 1.5 Flash-Next 2.25 bpw, which was validated on a real NVIDIA RTX PRO 5000 Blackwell (48 GB). All Qwen3.8-Flash-Next (125B-A6B) variants we publish, measured on 2026-10-14 under the same conditions (see Family comparison for every number): Download size: 66.7 GB (40.5 GB of EXL3 weights + 26.2 GB EXL3…
Open weights
other
20.2B parameters
262,144 tokens
exllamav3
Solstice-AI/Qwopus3.8-27B-Flash-NVFP4-1M provides the official, production-grade NVIDIA NVFP4 release of Qwopus3.8-27B-Flash with a native-behaving 1,048,576-token (1M) context window. NVIDIA Blackwell-native NVFP4 Tensor Core quantization with baked 1M YaRN scaling, running without runtime flag overhead. Evaluated under the standardized benchmark harness: 1M YaRN Scaling & Quantization Suite: Solstice-AI
Open weights
apache-2.0
19.2B parameters
1,048,576 tokens
WORK IN PROGRESS A mixed NVFP4/MXFP8 quantization-aware distillation of Qwen3.8-27B, trained for one epoch. The student learns from the original BF16 teacher while its MLP weights are quantized in the forward pass. Distillation updates the MLP weights and text normalization weights to account for quantization error. This is a trained distillation checkpoint, not a post-training conversion of the original weights. Attention/GDN projections and the LM head were frozen in their MXFP8 representations during distillation. Packed NVFP4 and MXFP8 weights reconstruct to the same BF16 weight values used by the student during training. The tokenizer, chat template, generation configuration and…
Open weights
apache-2.0
19.2B parameters
262,144 tokens
transformers
