lododo-Distill-v1.0 · LoRA adapter (GGUF)
This is the LoRA adapter form (40 MB) of lododo-Distill-v1.0.
The model itself: a distilled fine-tune of the Qwen3.6-35B-A3B base (MoE, 3B active), trained on data generated by Qwen3.8-27B as the teacher — bringing the base to (and beyond) the teacher's capability while keeping the MoE speed advantage.
| Metric |
Teacher: Qwen3.8-27B |
Base: Qwen3.6-35B-A3B |
This LoRA (f16) |
Merged build |
| LCB code (AtCoder 112) |
52.7% |
50.9% |
67.9% |
69.6% |
| BFCL tool calling (100) |
53.0% |
62.0% |
66.0% |
67.0% |
| GSM8K math (500) |
95.2% |
94.4% |
96.0% |
96.0% |
| IFEval instruction (150) |
36.7% |
24.0% |
33.3% |
33.3% |
Which repo should you use?
| Your situation |
Pick |
You already have Qwen3.6-35B-A3B-UD-Q4_K_XL.gguf (or plan to reuse it) |
Yesthis LoRA (40 MB) — only 40 MB on top |
| Starting from scratch and want the best result |
merged build: lododoai/Qwen3.6-35B-A3B-Distill-Qwen3.8-27B-GGUF (69.6%, 22.85 GB) |
Mounting the LoRA is ~1.7pp below the merged build (the delta is applied at runtime instead of being baked into the weights), in exchange for 22 GB of disk/traffic.
Running this model on a small GPU? Usemoefast
moefast is a small launcher that configures llama.cpp for your machine. It reads the model's GGUF
metadata, measures your free VRAM, computes the expert-cache size, context and batch parameters,
writes them into an editable config file next to the model, and starts llama-server.
The expert cache itself — keeping expert weights in host RAM and caching the hot ones in VRAM — is
llama.cpp's own (--moe-cache-mib, upstream PRs #29887/#30112). moefast makes sure it is
enabled, correctly sized, and remembered per model — the cache defaults to off in llama.cpp,
which roughly halves throughput.
git clone https://github.com/conlododo/moefast && cd moefast
./install.sh
./bin/moefast run /path/to/this-model.gguf
No patch, no fork — stock llama.cpp only.
Repo: https://github.com/conlododo/moefast
Files
| File |
Size |
Notes |
lododo-Distill-v1.0-lora-f16.gguf |
40 MB |
Recommended (f16, essentially lossless) |
Usage
# 1) download the base (must be the MTP-enabled build)
hf download unsloth/Qwen3.6-35B-A3B-MTP-GGUF Qwen3.6-35B-A3B-UD-Q4_K_XL.gguf --local-dir .
# 2) start with the LoRA mounted
llama-server -m Qwen3.6-35B-A3B-UD-Q4_K_XL.gguf \
--lora lododo-Distill-v1.0-lora-f16.gguf \
-ngl 99 --jinja -c 32768 --port 8080
Sampling parameters (only relevant for MTP / Strata)
The model has no special sampling requirement — set temperature / top_p as your task needs.
Only when you enable MTP speculative decoding in an engine that injects model-card sampling defaults
(e.g. Strata) must the request carry:
{"temperature": 0, "presence_penalty": 0, "frequency_penalty": 0, "repeat_penalty": 1.0}
Why: Strata injects temperature=1.0, top_p=0.95, top_k=20, presence_penalty=1.5 for qwen35moe,
and its MTP gate requires temperature==0 && repeat==1 && freq==0 && present==0 — the defaults shut the
drafter off entirely (drafts accepted 0 of 0 in our tests).
llama.cpp's own --spec-type draft-mtp is not affected.
Training summary
| Item |
Value |
| Teacher → base |
Distillation data from Qwen3.8-27B → Qwen3.6-35B-A3B |
| Data |
48.73M tokens (~52k samples) |
| Method |
QLoRA r=16 / α=32 (attention layers + shared experts) |
| Training |
0.5 epoch (747 steps), single uninterrupted run |
| Stability |
Equal-weight average of two independent runs (reduces random-init variance) |
Known limitations
| Item |
Notes |
| LeetCode format |
7.9% (training data is AtCoder stdin/stdout style) |
| IFEval |
33.3%, 3.4pp below the 27B teacher |
| Long context |
Supports up to the base's native 256K context; usable length depends on VRAM |
License & acknowledgements
- Released under Apache-2.0, inherited from Qwen/Qwen3.6-35B-A3B.
- The base GGUF quantization comes from unsloth/Qwen3.6-35B-A3B-MTP-GGUF (Apache-2.0).
- Not affiliated with or endorsed by the Qwen team; "Qwen" is used only to describe the base model.
- Use at your own risk; no warranty of any kind.
