Qwen3.6-35B-A3B · lododo-Distill-v1.0 (GGUF)
A distilled fine-tune of the Qwen3.6-35B-A3B base (MoE, 3B active), trained on data generated by Qwen3.8-27B as the teacher — bringing the base model to (and beyond) the teacher's capability while keeping the MoE speed advantage.
Put simply: 27B-level capability at 35B-A3B speed.
| Metric |
This model |
Teacher: Qwen3.8-27B |
Base: Qwen3.6-35B-A3B (original) |
| LCB code (AtCoder 112) |
69.6% |
52.7% (+16.9) |
50.9% (+18.7) |
| BFCL tool calling (100) |
67.0% |
53.0% (+14.0) |
62.0% (+5.0) |
| GSM8K math (500) |
96.0% |
95.2% (+0.8) |
94.4% (+1.6) |
| GPQA science (198) |
53.5% |
40.9% (+12.6) |
43.4% (+10.1) |
| IFEval instruction (150) |
33.3% |
36.7% (−3.4) |
24.0% (+9.3) |
| JSON Schema (50) |
100% |
100% |
100% |
| Chinese writing (chars) |
140 |
159 |
150 |
Running this model on a small GPU? Usemoefast
moefast is a small launcher that configures llama.cpp for your machine. It reads the model's GGUF
metadata, measures your free VRAM, computes the expert-cache size, context and batch parameters,
writes them into an editable config file next to the model, and starts llama-server.
The expert cache itself — keeping expert weights in host RAM and caching the hot ones in VRAM — is
llama.cpp's own (--moe-cache-mib, upstream PRs #29887/#30112). moefast makes sure it is
enabled, correctly sized, and remembered per model — the cache defaults to off in llama.cpp,
which roughly halves throughput.
git clone https://github.com/conlododo/moefast && cd moefast
./install.sh
./bin/moefast run /path/to/this-model.gguf
No patch, no fork — stock llama.cpp only.
Repo: https://github.com/conlododo/moefast
Files
| File |
Size |
Notes |
Qwen3.6-35B-A3B-lododo-Distill-v1.0-UD-Q4_K_XL.gguf |
22.85 GB |
Recommended: merged model (includes the MTP head), single file |
Quick start
llama-server -m Qwen3.6-35B-A3B-lododo-Distill-v1.0-UD-Q4_K_XL.gguf \
-ngl 99 --jinja -c 32768 --port 8080
Sampling parameters (only relevant for MTP / Strata)
The model has no special sampling requirement — set temperature / top_p as your task needs.
Only when you enable MTP speculative decoding in an engine that injects model-card sampling defaults
(e.g. Strata) must the request carry:
{"temperature": 0, "presence_penalty": 0, "frequency_penalty": 0, "repeat_penalty": 1.0}
Why: Strata injects temperature=1.0, top_p=0.95, top_k=20, presence_penalty=1.5 for qwen35moe,
and its MTP gate requires temperature==0 && repeat==1 && freq==0 && present==0 — the defaults shut the
drafter off entirely (drafts accepted 0 of 0 in our tests).
llama.cpp's own --spec-type draft-mtp is not affected.
Evaluation
Settings: thinking disabled, Q4 quant, identical llama.cpp flags, LCB reported on the AtCoder 112-question subset (LeetCode 63 listed separately).
| Metric |
35B original |
27B original |
empero |
This model |
| LCB AtCoder (112) |
50.9% |
52.7% |
50.0% |
69.6% (78/112) |
| GPQA Diamond (198) |
43.4% |
40.9% |
59.6% |
53.5% (106/198) |
| IFEval strict (150) |
24.0% |
36.7% |
23.3% |
33.3% |
| BFCL tool calling (100) |
62.0% |
53.0% |
59.0% |
67.0% (67/100) |
| GSM8K (500) |
94.4% |
95.2% |
96.4% |
96.0% (480/500) |
| JSON Schema (50) |
100% |
100% |
100% |
100% (50/50) |
| Chinese writing (chars) |
150 |
159 |
136 |
140 |
| LeetCode (63, supplementary) |
6.3% |
6.3% |
9.5% |
7.9% |
Versus the original 35B base: better on 7/7 metrics (LCB +18.7, GPQA +10.1, IFEval +9.3, BFCL +5.0, GSM8K +1.6).
Three deployment forms (same weights)
| Form |
Size |
LCB AtCoder |
| Merged (this repo) |
22.85 GB |
69.6% |
| LoRA f16 mount |
40 MB |
67.9% |
| LoRA f32 mount |
79 MB |
66.1% |
| LoRA q8_0 mount (not recommended) |
31 MB |
62.5% |
A LoRA delta is a small-magnitude tensor; q8_0 block quantization amplifies its relative error. If you want a small file, use f16.
Training summary
| Item |
Value |
| Teacher → base |
Distillation data from Qwen3.8-27B → Qwen3.6-35B-A3B |
| Data |
48.73M tokens (~52k samples) |
| Method |
QLoRA r=16 / α=32 (attention layers + shared experts) |
| Training |
0.5 epoch (747 steps), single uninterrupted run |
| Stability |
Equal-weight average of two independent runs (reduces random-init variance) |
Reproducibility (public benchmarks)
| Metric |
Dataset |
Link |
| LCB |
LiveCodeBench code_generation_lite (release_v6) |
https://huggingface.co/datasets/livecodebench/code_generation_lite |
| GPQA |
GPQA Diamond |
https://huggingface.co/datasets/Idavidrein/gpqa |
| IFEval |
Google IFEval (lm-eval v0.4.13, limit 150) |
https://huggingface.co/datasets/google/IFEval |
| BFCL |
Berkeley Function Calling Leaderboard |
https://huggingface.co/datasets/gorilla-llm/Berkeley-Function-Calling-Leaderboard |
| GSM8K |
OpenAI GSM8K (first 500 of test) |
https://huggingface.co/datasets/openai/gsm8k |
| JSON Schema |
In-house probe (50 items, Draft-7 validation) |
not a public benchmark |
| Chinese writing style |
In-house probe (31 prompts) |
not a public benchmark |
Determinism: --workers 1 + temperature 0; repeated runs on the same weights match item by item.
Known limitations
| Item |
Notes |
| LeetCode format |
7.9% (training data is AtCoder stdin/stdout style; weak at class Solution style) |
| IFEval |
33.3%, 3.4pp below the 27B teacher (mixing instruction data cost 11-14pp of coding ability in our experiments, so it was left out) |
| Long context |
Supports up to the base model's native 256K context; the usable length depends on available VRAM (KV cache) |
| Languages |
Strong in English and Chinese; other languages not systematically evaluated |
License & acknowledgements
- Released under Apache-2.0, inherited from Qwen/Qwen3.6-35B-A3B.
- The base GGUF quantization comes from unsloth/Qwen3.6-35B-A3B-MTP-GGUF (Apache-2.0); this merged model was produced by tensor-level merging into that UD-Q4_K_XL file.
- Changes: supervised fine-tuning (distillation data) and weight averaging relative to the base, then tensor-level merging into GGUF.
- Not affiliated with or endorsed by the Qwen team; "Qwen" is used only to describe the base model.
- Use at your own risk; no warranty of any kind.
