WORK IN PROGRESS A mixed NVFP4/MXFP8 quantization-aware distillation of Qwen3.8-27B, trained for one epoch. The student learns from the original BF16 teacher while its MLP weights are quantized in the forward pass. Distillation updates the MLP weights and text normalization weights to account for quantization error. This is a trained distillation checkpoint, not a post-training conversion of the original weights. Attention/GDN projections and the LM head were frozen in their MXFP8 representations during distillation. Packed NVFP4 and MXFP8 weights reconstruct to the same BF16 weight values used by the student during training. The tokenizer, chat template, generation configuration and…
Open weights
apache-2.0
19.2B parameters
262,144 tokens
transformers
4-bit quantization of google/gemma-4-31B-it (commit 842da37), produced for efficient vLLM serving. Quantized by axteam. - Scheme — W4A16 — FP4 (E2M1) weights group 16 (FP8 E4M3 scale + per-tensor FP32 global scale), BF16 activations, stored as compressed-tensors nvfp4-pack-quantized. - Kept in BF16 — vision tower, vision projection, token embeddings (tied with lmhead), MoE router, and all norms. These are left unquantized to preserve quality (a small fraction of total parameters). - Tokenizer / chattemplate / processorconfig / generationconfig are byte-identical to the original, including Google's July-2026 corrected chattemplate. config.json is the original plus a quantizationconfig block.…
Open weights
apache-2.0
18.5B parameters
262,144 tokens
transformers
EXL3 quantization (2.25 bits per weight on average) of UkisAI's Swift 1.5 Qwen3.8-Flash-Next (MoE, 125B total / 6B active, plus a 51B n-gram "PLE" table, MTP head and vision), made to serve the whole model on one 48 GB GPU with vLLM (continuous batching): 262,144 tokens per session without YaRN, 4 concurrent sessions sharing a 420k-token KV cache, vision and MTP. Validated on a real NVIDIA RTX PRO 5000 Blackwell (48 GB). All Swift 1.5 Qwen3.8-Flash-Next (125B-A6B) variants we publish, measured on 2026-10-14 under the same conditions (see Family comparison for every number): Download size: 66.7 GB (40.5 GB of EXL3 weights + 26.2 GB EXL3 n-gram table). vLLM users can skip the n-gram table…
Open weights
other
20.2B parameters
262,144 tokens
exllamav3
EXL3 quantization (2.25 bits per weight on average) of Qwen3.8-Flash-Next (MoE, 125B total / 6B active, plus a 51B n-gram "PLE" table, MTP head and vision), made to serve the whole model on one 48 GB GPU with vLLM (continuous batching): 262,144 tokens per session without YaRN, 4 concurrent sessions sharing a 420k-token KV cache, vision and MTP. Same layout and memory footprint as our Swift 1.5 Flash-Next 2.25 bpw, which was validated on a real NVIDIA RTX PRO 5000 Blackwell (48 GB). All Qwen3.8-Flash-Next (125B-A6B) variants we publish, measured on 2026-10-14 under the same conditions (see Family comparison for every number): Download size: 66.7 GB (40.5 GB of EXL3 weights + 26.2 GB EXL3…
Open weights
other
20.2B parameters
262,144 tokens
exllamav3
The RadixArk Qwen3.8-27B-NVFP4 model is the quantized version of Qwen/Qwen3.8-27B. The quantization was produced at RadixArk using NVIDIA Model Optimizer, following a mixed NVFP4 W4A4 recipe. Run on SGLang: launch command and per-platform recipes in the Qwen3.8-27B cookbook. This model is not owned or developed by RadixArk. It is a quantized derivative of Qwen's model; see the upstream Qwen3.8-27B model card for the source model's capabilities, training information, limitations, and license. Global Developers looking to deploy an off-the-shelf, pre-quantized model in AI agent systems, chatbots, RAG systems, and other AI-powered applications. Hugging Face 08/14/2026 via…
Open weights
apache-2.0
18.2B parameters
262,144 tokens
Model Optimizer
NVFP4 checkpoint of an abliterated Swift-Qwen3.8-27B (UkisAI's reasoning-efficient fine-tune of Qwen3.8-27B). For vLLM and SGLang. GGUFs for llama.cpp: The Swift 1.5 version is source). - Swift's own NVFP4 recipe, unmodified, from ukisai/Swift-Qwen3.8-27B-NVFP4, calibrated with NVIDIA ModelOpt. - MTP head and vision tower in BF16, bit-identical to the source. 21.9 GB, NVIDIA ModelOpt mixed-precision format. Needs a vLLM with ModelOpt mixed-precision support (tested on 0.29.0). No --quantization flag. Sampling, as for Swift and Qwen: temperature 1.0, topp 0.95, topk 20, minp 0. The model thinks before answering by default. Tested on an RTX 5090 (32 GB) with vLLM 0.29.0: NVFP4 layers on…
Open weights
other
18.2B parameters
262,144 tokens
vllm
