NVFP4 (4-bit) weight quantization of the abliterated (refusal-removed) Qwen3.8-Flash-Next — for Blackwell + vLLM
One Gateway. Every Model. — Route Smarter · Ship Safer · Spend Less.
Website ·
Model Catalog ·
GitHub ·
Discord ·
X
NVFP4 weight quantization of the abliterated (refusal-removed) build of Qwen's
Qwen3.8-Flash-Next — a large
Mixture-of-Experts (512 experts, 10 routed + 1 shared active) preview of the Qwen4 architecture
(qwen4_exp): Gated DeltaNet linear attention + Qwen Sparse Attention (QSA), HyperConnections, PLE
n-gram embeddings, native vision-language, reasoning, and tool-calling. The MoE expert weights are
quantized to NVFP4 (4-bit, NVIDIA FP4 E2M1, group-16 + FP8 block scales) and attention/shared-expert
weights to FP8, cutting the model from ~330 GB (bf16) to ~178 GB while keeping full-precision
paths where they matter. Browse all models in the OrcaRouter Model Catalog.
Disclaimer — read before use
This model has had its safety alignment substantially removed via abliteration (orthogonalizing
the refusal direction out of the residual stream). It will comply with harmful, unethical, or illegal
requests the original Qwen3.8-Flash-Next would refuse. Released strictly for legitimate research —
interpretability, AI-safety / refusal-mechanism study, red-teaming, and robustness evaluation. You
assume full responsibility for how you use it and everything it generates; add your own safety and
moderation layers before any deployment. Use must comply with the
Apache 2.0 License inherited from the base model and all
applicable law. The authors accept no liability for misuse.
Requirements
- A Blackwell GPU (B100 / B200 / GB200 / RTX 50-series) — NVFP4 uses the hardware FP4 tensor
cores. It will not run on Hopper (H100/H200) or older; those lack FP4 compute.
- A runtime that supports the
qwen4_exp architecture + compressed-tensors NVFP4. This is a brand-new
architecture: use a recent vLLM build with qwen4_exp support (and transformers>=5.16). Stock
runtimes that predate qwen4exp will not load it.
- Multimodal (vision) requires the runtime's Qwen vision stack; text-only works without images.
What's quantized
| Component |
Precision |
MoE experts (mlp.experts, the bulk) |
NVFP4 (W4, E2M1 group-16 + FP8 scales) |
Attention (self_attn.{q,k,v,o}, linear_attn.{in_proj_qkv,in_proj_z,out_proj}), shared expert, lm_head |
FP8 (W8) |
| PLE n-gram embedding, token/vision embeddings, HyperConnections, QSA indexer, Gated-DeltaNet conv/dt, norms, vision tower |
bf16 (kept full precision) |
- Weight-only: activations are quantized dynamically at runtime (no static calibration); the
quantization is data-free (weights derived directly from the bf16 checkpoint). The abliteration is
baked into the weights, so refusal-removal is preserved.
- KV cache is not quantized (bf16 at runtime).
- The PLE n-gram embedding (a single ~66B-parameter tensor) is kept bf16 by design and is the
largest shard; it dominates the on-disk size.
Note on the recipe: this is a weight-only NVFP4 build (W4 experts / W8 attention, dynamic
activations). A fully static W4A4 variant requires an activation-calibration forward pass, which
must hold that ~100 GB n-gram embedding on a single GPU — only feasible on very-large-memory
(e.g. Blackwell/H200-class) hardware. Ping us if you need the W4A4 build.
Usage (vLLM, Blackwell)
pip install -U "vllm>=<qwen4exp-supporting release>" "transformers>=5.16"
vllm serve orcarouter/Qwen3.8-Flash-Next-Uncensored-NVFP4 \
--tensor-parallel-size 4 --trust-remote-code \
--enable-expert-parallel --enable-auto-tool-choice --tool-call-parser qwen3_coder
Then call the OpenAI-compatible endpoint (/v1/chat/completions) as usual — tool calling, reasoning
(chat_template_kwargs.enable_thinking), and vision (image_url content parts) all work through the
runtime's Qwen4 stack.
Evaluation
Abliteration was measured on this build (bf16, served with vLLM) vs the official Qwen/Qwen3.8-Flash-Next:
harmful-prompt refusal collapses from 64–100% to ~0–3.3%, benign over-refusal stays near 0%,
and capability stays within ±2 pts of the base. NVFP4 is a deterministic weight derivation and
inherits these behaviours, with a small additional quality trade-off from 4-bit experts.
License
Apache 2.0, inherited from Qwen/Qwen3.8-Flash-Next.
Abliteration and quantization do not change the underlying license obligations.
