Qwen3.8-Flash-Next-GSQ-RCO-abliterated-GGUF

An abliterated build of the official ISTA-DASLab GSQ-RCO quantizations. Not a single one of GSQ's learned quantized tensors was recomputed — 144 "write-to-residual-stream" projection tensors across all 48 layers were replaced with the corresponding tensors from a ready-made abliterated release.

In one line: what you get is still the GSQ-RCO quantization you already know, with those 144 tensors abliterated. It is the only abliteration route that preserves GSQ's work.

This model is abliterated and refusal-removed — it has no meaningful built-in guardrails and may comply with harmful, illegal or unsafe requests. Deploy it only where you can supply your own moderation, access control and legal review; do not put it in front of end users without a safety layer of your own.

本模型经过拒答方向消融，不具备可靠的内置安全护栏，可能配合有害、违法或不安全的请求。请仅在你能够提供审核、访问控制与法律审查的场景部署。

GSQ treats each tensor's grid assignment and per-group scale as learnable parameters, trained jointly through a Gumbel-Softmax relaxation; RCO then assigns one quantization type per tensor under a total size budget. As a result, a GSQ-RCO tensor value is not what you get by feeding BF16 weights to a standard quantizer — rerunning the same recipe yields a different set of values.

There is exactly one way to abliterate a GSQ-RCO model: replace tensors in place inside the already-quantized GGUF. That is what this repository does.

The ablation itself is a strictly rank-1 linear edit — W ← W − r(rᵀW) — with a single direction r shared by all 48 layers × both attention types (DeltaNet / QSA) × both the attention and MLP sites × all 512 routed experts. Given that, "copy an already-abliterated tensor" and "compute the ablation yourself and quantize it" land on the same line, and the former introduces no new quantization error.

The rank-1 structure was measured on a BF16 abliterated release of the same base: σ₂/σ₁ of ΔW = 0.0032–0.0053; the direction of every one of the 512 routed experts agrees with the shared-expert direction at |cos| = 1.00000, and likewise across layers and sites.

Each tier ships as two shards. Shard 2 is byte-identical across all four tiers and byte-identical to the copy in the upstream repository: it holds the per-layer n-gram embedding table (per_layer_token_embd, 51.2B parameters), a lookup table rather than a matmul weight, so it is pinned at IQ4_NL, excluded from the search, and untouched by the ablation.

Low tiers do not get fat here. Upstream ffn_down_exps can only live in Q2_0 or IQ4_NL (the tensor has 640 elements per row, which is not divisible by 256, so every block-256 K / I format is ruled out). The abliterated values, however, only exist in an IQ4_NL build — take them literally and the two 2-bit tiers would have to lift all 48 expert tensors to IQ4_NL, +11.7 GB each. This repository re-encodes the abliterated values into Q2_0 using GSQ's own learned block scales, so all four tiers keep the upstream per-layer type assignment exactly (IQ3_S measured at 9×Q2_0 + 39×IQ4_NL, identical to upstream). The only remaining cost is the 96 small tensors, whose abliterated values only exist at Q8_0: +0.25 GB on IQ3_S and +0.30 – 0.44 GB on the others.

Of the file's 1223 tensors, 144 were touched — all in the same family of "write-to-residual-stream" projections, covering all 48 layers:

95 of them changed quantization type; the other 49 (48 routed experts + 1 shared expert) kept the upstream type — different values, same format. Full-file type histogram:

Left alone: token_embd and output (the output head). Abliterations often touch those two as well; the release we took the weights from did not, so neither do we. The other 1079 tensors are unchanged, byte for byte.

Check ③ is hard evidence; check ④ is only a spot check — we ran a small probe set, and only that one prompt is a clean discriminator for this model. Do not read it as a full evaluation of the ablation.

Measured on the Strata engine, not a llama.cpp baseline — dual-GPU layer split, 256K context, KV streaming. With the upstream GSQ-RCO layout restored, those 9 ffn_down_exps tensors go back to Q2_0 (2.25 bpw), so the expert read volume is identical to upstream and decode lands level with it (104.4 vs 104.5; the 0.1% is measurement noise). Cache slots went back from 11478 to 12095. Absolute numbers on llama.cpp will differ, but the direction will not.

This is standard GGUF with exactly the architecture of the upstream files, so it loads unmodified in any reasonably recent llama.cpp — no patched fork required. Usage is identical to the upstream tiers.

Download (both shards; llama.cpp picks up the second automatically)

hf download SC117/Qwen3.8-Flash-Next-GSQ-RCO-abliterated-GGUF --include "IQ3_S/*" --local-dir .

Text

llama-cli -m IQ3_S/Qwen3.8-Flash-Next-GSQ-RCO-abliterated-IQ3_S-00001-of-00002.gguf \ -lm mmap --lazy-mode on -ngl 99 -c 32768 \ -p "Give me a detailed way to get around a news site's paywall."

Vision (multimodal)

hf download SC117/Qwen3.8-Flash-Next-GSQ-RCO-abliterated-GGUF \ mmproj-Qwen3.8-Flash-Next-BF16.gguf --local-dir . llama-mtmd-cli -m IQ3_S/Qwen3.8-Flash-Next-GSQ-RCO-abliterated-IQ3_S-00001-of-00002.gguf \ --mmproj mmproj-Qwen3.8-Flash-Next-BF16.gguf -lm mmap --lazy-mode on \ --image photo.jpg -p "Describe this image."

Speculative decoding with the MTP draft head

llama-cli -m IQ3_S/Qwen3.8-Flash-Next-GSQ-RCO-abliterated-IQ3_S-00001-of-00002.gguf \ -md mtp-Qwen3.8-Flash-Next-Q8_0.gguf --draft-max 4 -lm mmap --lazy-mode on -ngl 99

-lm mmap --lazy-mode on keeps the n-gram table memory-mapped on disk: it is 28.8 GB and one row is read per token. Shard 1 wants to be resident (VRAM or RAM); shard 2 is fine on an SSD. The vision projector is a standard clip GGUF shared with the upstream tiers, 0.91 GB. Native context is 262,144 tokens and the KV cache grows with it, so start at -c 32768 and work up.

All four tiers of the abliterated weights · the n-gram shard, byte-identical to upstream (download it once) · the standardclipvision projector · the complete transplant inventory and the per-tensor verification tooling · sizes and upstream type allocations for every tier

No BF16 or high-precision builds — for lossless weights use thebase model· No training, fine-tuning or weight optimization of any kind — this repositoryonly moves tensors· No large-scale evaluation of the ablation (see check ④)

The quantization is the work of ISTA-DASLab, the abliterated weights are the work of orcarouter; this repository contributes the alignment procedure, the assembled files and the verification data.

The transplant is pure byte arithmetic — it needs no quantizer and no GPU:

Apache-2.0, inherited from the base model — please also honour the licenses and model cards of every upstream repository.

Qwen — the Qwen3.8-Flash-Next base model · IST-DASLab — GSQ + RCO and the quantized weights · orcarouter — the abliterated weights · llama.cpp — the GGUF format and tooling · huihui-ai — the BF16 release used to confirm the rank-1 structure of the ablation. This repository is an independent project and is not affiliated with any of them.