Qwen3.6-27B-NVFP4

Model · Image and text to text

Qwen3.8-27B-NVFP4-QAD

Local Inference Lab

WORK IN PROGRESS A mixed NVFP4/MXFP8 quantization-aware distillation of Qwen3.8-27B, trained for one epoch. The student learns from the original BF16 teacher while its MLP weights are quantized in the forward pass. Distillation updates the MLP weights and text normalization weights to account for quantization error. This is a trained distillation checkpoint, not a post-training conversion of the original weights. Attention/GDN projections and the LM head were frozen in their MXFP8 representations during distillation. Packed NVFP4 and MXFP8 weights reconstruct to the same BF16 weight values used by the student during training. The tokenizer, chat template, generation configuration and…

Model · Image and text to text

Qwen3.8-27B-NVFP4

RadixArk

The RadixArk Qwen3.8-27B-NVFP4 model is the quantized version of Qwen/Qwen3.8-27B. The quantization was produced at RadixArk using NVIDIA Model Optimizer, following a mixed NVFP4 W4A4 recipe. Run on SGLang: launch command and per-platform recipes in the Qwen3.8-27B cookbook. This model is not owned or developed by RadixArk. It is a quantized derivative of Qwen's model; see the upstream Qwen3.8-27B model card for the source model's capabilities, training information, limitations, and license. Global Developers looking to deploy an off-the-shelf, pre-quantized model in AI agent systems, chatbots, RAG systems, and other AI-powered applications. Hugging Face 08/14/2026 via…

Model · Image and text to text

Qwen3.6-35B-A3B-NVFP4

Unsloth AI

1.56x faster throughput than other NVFP4 quants. This is an Unsloth NVFP4 quantized checkpoint calibrated on a mixture of our Unsloth dataset + UltraChat dataset. Works on a 32GB VRAM GPU. Benchmarks on 1xB200 128 concurrency. Use the 35B NVFP4 Fast version for 1.79x faster at a little less accuracy For accuracy benchmarks, we conducted MMLU-Pro, AIME 2025, GPQA for FP8, BF16, NVIDIA's NVFP4 and our NVFP4s - we show our faster quants do similarly on all: Read all benchmarks in our NVFP4 blog To install vLLM in a separate venv: Then to serve the 35B variant: You must use the below or you will get 2x slower inference! Also do NOT use the Marlin backend since it's 2x slower - use the native…

Model · Image and text to text

Qwen3.8-27B-NVFP4

Inferact Inc.

Quantized version of https://huggingface.co/Qwen/Qwen3.8-27B

Model · Image and text to text

gemma-4-26B-A4B-it

Google

Gemma is a family of open models built by Google DeepMind. Gemma 4 models are multimodal, handling text and image input (with audio supported on E2B, E4B, and 12B) and generating text output. This release includes open-weights models in both pre-trained and instruction-tuned variants. Gemma 4 features a context window of up to 256K tokens and maintains multilingual support in over 140 languages. Featuring both Dense and Mixture-of-Experts (MoE) architectures, Gemma 4 is well-suited for tasks like text generation, coding, and reasoning. The models are available in five distinct sizes: E2B, E4B, 12B, 26B A4B, and 31B. Their diverse sizes make them deployable in environments ranging from…

Model · Image and text to text

gemma-4-26B-A4B-it-AWQ-4bit

Cyankiwi

Gemma is a family of open models built by Google DeepMind. Gemma 4 models are multimodal, handling text and image input (with audio supported on small models) and generating text output. This release includes open-weights models in both pre-trained and instruction-tuned variants. Gemma 4 features a context window of up to 256K tokens and maintains multilingual support in over 140 languages. Featuring both Dense and Mixture-of-Experts (MoE) architectures, Gemma 4 is well-suited for tasks like text generation, coding, and reasoning. The models are available in four distinct sizes: E2B, E4B, 26B A4B, and 31B. Their diverse sizes make them deployable in environments ranging from high-end phones…