Qwen3.8-27B-NVFP4

Model · Image and text to text

Swift-Qwen3.8-27B-Uncensored-NVFP4

AJ Gazin

NVFP4 checkpoint of an abliterated Swift-Qwen3.8-27B (UkisAI's reasoning-efficient fine-tune of Qwen3.8-27B). For vLLM and SGLang. GGUFs for llama.cpp: The Swift 1.5 version is source). - Swift's own NVFP4 recipe, unmodified, from ukisai/Swift-Qwen3.8-27B-NVFP4, calibrated with NVIDIA ModelOpt. - MTP head and vision tower in BF16, bit-identical to the source. 21.9 GB, NVIDIA ModelOpt mixed-precision format. Needs a vLLM with ModelOpt mixed-precision support (tested on 0.29.0). No --quantization flag. Sampling, as for Swift and Qwen: temperature 1.0, topp 0.95, topk 20, minp 0. The model thinks before answering by default. Tested on an RTX 5090 (32 GB) with vLLM 0.29.0: NVFP4 layers on…

Model · Image and text to text

Swift-1.5-Qwen3.8-27B-Uncensored-NVFP4

AJ Gazin

NVFP4 checkpoint of an abliterated Swift 1.5 Qwen3.8-27B (UkisAI's reasoning-efficient fine-tune of Qwen3.8-27B). For vLLM and SGLang. GGUFs for llama.cpp: (measured on the BF16 source). - Swift's own NVFP4 recipe, unmodified, from ukisai/Swift-Qwen3.8-27B-NVFP4, calibrated with NVIDIA ModelOpt. The module split matches UkisAI's Swift 1.5 NVFP4 exactly. - MTP head and vision tower in BF16, bit-identical to the source. 21.9 GB, NVIDIA ModelOpt mixed-precision format. Needs a vLLM with ModelOpt mixed-precision support. No --quantization flag. Sampling, as for Swift and Qwen: temperature 1.0, topp 0.95, topk 20, minp 0. The model thinks before answering by default. Same format, recipe, module…

Model · Image and text to text

Qwen3.8-27B-Uncensored-NVFP4-v100-skinny

Tim Eastwood

Mixed-precision quantization of prepared for dnv2003/v100-skinny. - MLP gateproj, upproj, downproj, and lmhead: NVFP4, group size 16 87c9f8cf83021957d1a1a575c90c9a4eaaf7ef0c See quantization-audit.json and hfquantconfig.json for the complete machine-readable layout. This model has had safety alignment substantially removed. Use it only for lawful, controlled research and add appropriate safeguards before deployment.

Model · Image and text to text

Qwen3.8-27B-NVFP4

Inferact Inc.

Quantized version of https://huggingface.co/Qwen/Qwen3.8-27B

Model · Image and text to text

Qwen3.8-27B-NVFP4-QAD

Local Inference Lab

WORK IN PROGRESS A mixed NVFP4/MXFP8 quantization-aware distillation of Qwen3.8-27B, trained for one epoch. The student learns from the original BF16 teacher while its MLP weights are quantized in the forward pass. Distillation updates the MLP weights and text normalization weights to account for quantization error. This is a trained distillation checkpoint, not a post-training conversion of the original weights. Attention/GDN projections and the LM head were frozen in their MXFP8 representations during distillation. Packed NVFP4 and MXFP8 weights reconstruct to the same BF16 weight values used by the student during training. The tokenizer, chat template, generation configuration and…

GLM-5.3-Flash with its routed experts quantized to NestQuant, a nested two-level format. Each expert has a 1.5-bit base plus a residual plane that lifts it to 4 bits. At serving time an expert moves from 1.5 bit to 4 bit by loading its residual plane on top of the base, and the base bytes stay the same. This build targets a 128 GB Mac with 96 GB usable for weights, KV cache and runtime. The native vision tower is included, so the model still takes images. Status: TODO. Encoding in progress. This repo cannot be loaded with stock transformers, vLLM or MLX. The serving kernel and loader will be published separately. - 67 of 288 experts per layer (23%) run at 4 bit. The rest run at 1.5 bit.…