Qwen3.8-27B-24GB-128k-GGUF

Qwen3.8-27B mixed-precision GGUF for 24 GB GPUs at 128k context

Three GGUF quantizations of Qwen/Qwen3.8-27B that spend bits where quantization error hurts most: the 16 full-attention layers and the recurrent (linear-attention) layers get more precision, the bulky feed-forward up/gate weights get less. Built for running the model at a 128k context entirely on one 24 GB GPU (measured on an RX 7900 XTX with llama.cpp/ROCm).

All are text-only (no vision projector). Unsloth's mmproj-F16.gguf from unsloth/Qwen3.8-27B-GGUF is built from the same base model and should work with them, but this was not tested. R1 and R2 keep the multi-token-prediction (MTP) layer, quantized with the same recipe; R3 drops it (64 blocks, nextn_predict_layers = 0), so it cannot be used for MTP speculative decoding.

Quality: KL divergence against Q8_0

Reference: a Q8_0 quantization of the same BF16 weights. Text: 24,576 held-out tokens (6 × 4,096) of technical text (C source, decompiler output, CPU-manual prose, technical notes). Measured with llama-perplexity --kl-divergence. Lower KLD is closer to the reference.

R1-mix has about 2.3× lower mean KLD than the Ollama library's qwen3.8:27b Q4_K_M (similar size), with a much smaller worst-case tail; R2-mix about 2.1× lower while 1.3 GB smaller. Only that one Q4_K_M was measured; Q4_K_M files built with an imatrix (as most community uploads are) were not tested and would likely land closer. Unsloth's UD-IQ4_XS has the best perplexity ratio but a higher KLD; KLD compares the whole output distribution and is the better quantization metric. Caveats: one evaluation text of modest size, technical domain only (not chat or tool use), single run. R3-mix was measured later on a larger evaluation text; its results are in the next section, with R1, R2 and UD-IQ4_XS re-measured on that text.

R3-mix: a recipe from per-tensor measurements

R1 and R2 were designed by hand. For R3 each tensor group was measured instead: starting from a base recipe (R2's types, MTP layer removed, token_embd at Q8_0 since it stays in system RAM), one group at a time was raised one quantization step, the file rebuilt and its KL divergence against Q8_0 measured, giving the quality gained per GiB spent. Median KLD gain per GiB, highest first: full-attention attn_output 0.00080, recurrent ssm_out 0.00055, attn_q 0.00043, recurrent attn_gate 0.00032, attn_qkv 0.00026, ffn_down 0.00023, ffn_up 0.00022, ffn_gate 0.00018. Raising the output layer to Q6_K made things slightly worse; spending the bits on one band of layers instead of all layers did not help. R3 takes the best-value upgrades until the file fills a 24 GB card at 128k context with a 4-bit KV cache.

Evaluation: 65,536 tokens (16 × 4,096) of technical text (decompiler output, C, notes) interleaved with transcripts of a coding agent working in that material; reference Q8_0 of the BF16 weights; llama-perplexity --kl-divergence. Not comparable with the table above (different text). The R1-mix and R2-mix rows are the files in this repo.

R3-mix is closest to Q8_0 on every measure: about 30% lower mean KLD and 44% lower median KLD than R2-mix, one more token in a hundred matching Q8_0's top choice, and a smaller worst-case tail. It is also the largest file. Single run, one evaluation text; mean KLD varies by about ±0.002 between texts of this size, so the median and top-token columns are the more stable comparison.

Speed and memory (same machine and llama.cpp build as below, 128k context, KV cache q4_0, batch 2048, three-turn 26.5k + 5.7k + 5.2k-token workload): prompt 961 / 805 / 760 tok/s, generation about 34 tok/s (R2-mix: about 39), 20.07 GB dedicated VRAM, no spill to system memory. That is all the VRAM Windows gave one process on this card, so close other GPU-heavy applications first. An 8-bit KV cache at 128k does not fit with R3.

Long-context recall (three planted lines at 10/50/90% depth in decompiler output): 3/3 at 57,710 and at 115,936 tokens, with the 4-bit KV cache.

Agent tasks. Through Qwen Code, on the eleven tasks described below: K1-K7 and E1-E3 all correct, K8 stopped at the turn cap after over-researching (10/11 solved, 16.7 min); in five reruns of K8, four were fully correct. Through Hermes Agent on a four-task subset (K1, K2, K4, E1, one run each, a full agent setup with persona, skills and project rules), against two hosted models through a ChatGPT subscription:

gpt-6-luna never opened K1's file (twice; it searched elsewhere and declined to answer). R3-mix's K4 miss named the helper that performs the directory creation rather than the function the answer key expects. Same caps for all (30 turns, 600 s per task); local replies were also capped at 8,192 tokens. Four tasks and single runs: this shows a local 27B model in the same range as hosted models on this kind of work, not a ranking.

R2-mix vs Unsloth UD-IQ4_XS and Ollama's default

The closest same-model alternative is Unsloth's UD-IQ4_XS, which is smaller; Ollama's library qwen3.8:27b (Q4_K_M) is what most Ollama users get. Neither alternative is worse on everything:

In short: R2-mix stays closest to the full-precision model on average and in the worst cases; UD-IQ4_XS is ~1.1 GB smaller, ~3% faster at generation and has the best perplexity ratio; the Ollama Q4_K_M is the largest and furthest from Q8_0 here. KLD/PPL from the quality table above. All three were served the same way for the speed rows: the mean over a complete agentic test session (next section) with llama-server b1343, ROCm, RX 7900 XTX, 128k context, KV cache q4_0; prompt speed over chunks of 2k tokens or more. The Ollama column measures those weights under llama-server, not Ollama's own runtime.

Agentic test through Qwen Code

Eleven tasks run headless through Qwen Code 0.25.1 (qwen -o json), one run each, identical settings for every quant (llama-server as above, --reasoning-effort medium, temperature 0.3, caps of 30 turns / 10 minutes / 80 tool calls). K1-K8 are code-reading tasks on a reverse-engineering workspace (Ghidra decompiles, Wine sources, notes), answered as KEY=value facts and graded against answers checked by hand; E1-E3 are editing tasks graded by building and running tests (E1: add an operator to a MASM program; E2: fix two bugs in a Python module; E3: add a CLI option to spec).

What this does and does not show: with one run per task, a difference of one task (or a single run that thinks until the cap, as jakeatx did on K8) is within run-to-run noise, so R2-mix, R1-mix, jakeatx, UD-IQ4_XS and the Ollama Q4_K_M should be read as roughly equal on this suite. The consistent signal is that the two quants with the highest KLD (vmarcelo IQ4-MIX, Ollama Q4_K_M) are the ones that lost facts on K2. The tasks come from one workspace and one domain. E1 for R2-mix and R1-mix was re-run after a prompt fix (the original prompt suggested cmd /c build.cmd, which hangs under Qwen Code's Git Bash shell on Windows); the other quants ran with the fixed prompt only.

Speed and memory at 128k (RX 7900 XTX)

Setup: Windows 11, RX 7900 XTX 24 GB, Ryzen 7 7800X3D, 32 GB RAM; llama.cpp build 11558 (c83f3058) with ROCm 10.2.0a (lemonade-sdk/llamacpp-rocm b1343). Workload: a three-turn conversation (26.5k-token first prompt, then 5.7k and 5.2k new tokens per turn, reusing the cache), like an agent loop.

Long-context recall (three planted lines at 10/50/90% depth in ~58k and ~116k tokens of decompiler output): 3/3 at both lengths for R2 with the 8-bit KV cache. The 4-bit cache was recall-tested only with the Ollama library Q4_K_M of the same model (also 3/3 at both lengths), not with R2.

The speed and recall figures were measured on builds of these exact recipes made with a different imatrix (same tensor types and file sizes, so the same memory traffic); the KLD table above is for the files in this repo.

Recommended llama-server command (24 GB GPU, 128k)

llama-server -m Qwen3.8-27B-R2-mix.gguf -ngl 99 -c 131072 -np 1 -fa on -b 2048 -ub 2048 \
  -ctk q8_0 -ctv q8_0 --cache-ram 2048 --ctx-checkpoints 8 --reasoning-effort medium \
  --temp 0.6 --top-k 20 --top-p 0.95 --min-p 0

For R3-mix use -m Qwen3.8-27B-R3-mix.gguf -ctk q4_0 -ctv q4_0 -lm none with the same other flags (-lm none reads the file once instead of memory-mapping it, keeping system RAM free).

Notes from tuning on Windows/ROCm: - The embedded chat template defaults to reasoning_effort = xhigh when the request does not set one (Ollama sends medium). At xhigh one code-review test spent its whole 8,192-token budget thinking without answering; at medium it answered in 96 s. Set --reasoning-effort (or chat_template_kwargs) explicitly. - On Windows a single process got about 20 GB of the card's 24 GB as dedicated VRAM; anything beyond silently spills to shared system memory and prompt processing drops several-fold (4-5× with llama-server at 128k, about 15× in an Ollama setup with MTP drafting in these tests). If other applications hold VRAM, use -ctk q4_0 -ctv q4_0. - Large physical batches (-ub 2048) are much faster on ROCm than the default 512. - MTP speculative decoding (--spec-type draft-mtp) needs a second context; at 98k (with the earlier Q4_K_M weights) it spilled out of VRAM and slowed prompt processing about 8×. It may help at shorter contexts. - If the system has an AMD CPU with integrated graphics, ROCm may list it as a second device; pin the GPU with -dev ROCm0. - Recent llama.cpp applies a Hadamard rotation to quantized K caches automatically, which is why a 4-bit cache held up.

Recipe

Base type IQ4_XS; per-tensor overrides via llama-quantize --tensor-type-file (files in recipes/), imatrix from unsloth/Qwen3.8-27B-GGUF (imatrix_unsloth.gguf), source BF16/Qwen3.8-27B-BF16-*.gguf from the same repo.

llama-quantize --imatrix imatrix_unsloth.gguf --tensor-type-file recipes/R2.txt \
  --output-tensor-type q5_k --token-embedding-type q4_k Qwen3.8-27B-BF16-00001-of-00002.gguf out.gguf IQ4_XS

(R1: recipes/R1.txt, --output-tensor-type q6_k.) R3: recipes/R3.txt, --output-tensor-type q5_k --token-embedding-type q8_0 --prune-layers 64, then gguf-set-metadata out.gguf qwen35.nextn_predict_layers 0 (without it llama.cpp still expects the removed MTP layer and runs 63 layers).

Related quants

The idea here (IQ4_XS for the bulk feed-forward weights, more bits for attention and the recurrent layers) is not new; several earlier Qwen3.8-27B releases do the same or more:

• jakeatx/Qwen3.8-27B-ATX-IQ4_XS-M-GGUF: IQ4_XS base with Q4_K_M-style upgrades, built for a 24 GB RTX 3090 at 200K context with an 8-bit K cache and MTP.
• vmarcelo/Qwen3.8-27B-MIX_GGUF: custom per-tensor mixes with an imatrix, built from the FP8 checkpoint for 16 GB cards.
• utautako/Qwen3.8-27B-NVFP4-MTP-Q8attn-GGUF: attention and DeltaNet projections at Q8_0 with NVFP4 feed-forward weights.
• tooltd/Qwen3.8-27B-IQ4-XS-16GB-VRAM-GGUF: automatic rate-distortion bit allocation with an imatrix.

Measured with the same KLD setup as above:

R2-mix and ATX-IQ4_XS-M are effectively tied. IQ4-MIX is smaller and was made from the FP8 checkpoint, while the reference here is a Q8_0 of the BF16 weights, so part of its distance is the FP8 source rather than its recipe. What this repo adds is mainly the measurements and the Windows/ROCm 128k configuration notes above.

License and credits

Apache 2.0, as the base model; see LICENSE. Base model by the Qwen team. Hosted-model comparison run through a ChatGPT subscription; model names as listed by that service on 2026-10-12. BF16 GGUF conversion and imatrix by Unsloth. Quantization recipe and measurements by jdominick05.