qwen38-dspark-moe-2layer is an open-weight model from Yusheng Su, released under Apache License 2.0. Its published files total 3.6 KB.
2-layer MoE DSpark draft models for Qwen3.8-27B: an ablation of Kan Wu's 5-layer Qwen3.8-27B DSpark MoE drafter (SpecForge branch kan/moe-3-qwen38) with the draft reduced from 5 to 2 decoder layers and nothing else changed (512 routed experts x 512 wide…
Model Card
By Yusheng Su, published under apache-2.0, revision 71a5dc98d839.
2-layer MoE DSpark draft models for Qwen3.8-27B: an ablation of Kan Wu's 5-layer Qwen3.8-27B DSpark MoE drafter (SpecForge branch kan/moe-3-qwen38) with the draft reduced from 5 to 2 decoder layers and nothing else changed (512 routed experts x 512 wide, top-10 softmax routing, sigmoid-gated shared expert of width 2048, router centering; 8.48B total / about 1.08B active parameters). One servable export per training epoch, each in its own folder (config.json, model.safetensors, tokenizer and a README with the training metrics of that checkpoint): Serving needs an SGLang build with the Qwen3MoeDSparkModel draft class (upstream SGLang 0.5.18 only ships the dense Qwen3DSparkModel). Download one…
Read Yusheng Su's full model card
2-layer MoE DSpark draft models for Qwen3.8-27B: an ablation of Kan Wu's 5-layer Qwen3.8-27B DSpark MoE drafter
(SpecForge branch kan/moe-3-qwen38) with the draft reduced from 5 to 2 decoder layers and nothing else changed
(512 routed experts x 512 wide, top-10 softmax routing, sigmoid-gated shared expert of width 2048, router centering;
8.48B total / about 1.08B active parameters).
One servable export per training epoch, each in its own folder (config.json, model.safetensors, tokenizer and a README with the training metrics of that checkpoint):
| folder | phase / checkpoint | status |
|---|---|---|
1ep/ |
phase A (from scratch), step 4958 = end of epoch 1 | pending (training in progress) |
2ep/ |
phase B (weights-only warm start from the epoch-1 checkpoint), run-local step 5000 = end of epoch 2 | pending (training in progress) |
3ep/ |
phase B, run-local step 9916 = end of epoch 3 | pending (training in progress) |
- Training recipes and code: https://github.com/yushengsu-thu/SpecForge/tree/exp/qwen3.8-27b-dspark-moe-2layer
- Live training metrics: https://wandb.ai/miles_training/specforge-2%20layer%20moe
- Data:
RadixArk/Qwen3.8-27B-Regen-Mixture-v1; target served in bf16 (Qwen/Qwen3.8-27B) on 8x H200.
Serving needs an SGLang build with the Qwen3MoeDSparkModel draft class (upstream SGLang 0.5.18 only ships the dense
Qwen3DSparkModel). Download one folder, e.g. hf download yushengsu/qwen38-dspark-moe-2layer --include "1ep/*", and pass
it as --speculative-draft-model-path together with --speculative-algorithm DSPARK --speculative-dspark-block-size 7
--speculative-draft-model-quantization unquant --attention-backend triton --speculative-draft-attention-backend triton.
Accept-length evaluations (GSM8K / MATH500 / MT-Bench, accepted drafts per verify step) will be added as they are measured.
Identity and Version
- Repository
- yushengsu/qwen38-dspark-moe-2layer
- Publisher
- Yusheng Su
- Task
- Not stated by the source
- Modality
- Other
- Library
- Not stated by the source
- Parameters
- Not stated by the source
- Languages
- moe
- Revision
- 71a5dc98d839e6b86b20987e3c9ddb6e14b8034d
- First published
- 2026-10-08
- Last updated
- 2026-10-08
Files and Weights
2 files, 3.6 KB in total.
Every file
| File | Type | Size | SHA-256 |
|---|---|---|---|
| README.md | Documentation | 2.0 KB | — |
| .gitattributes | Repository | 1.5 KB | — |
License and Download
- License
- apache-2.0
- Access
- Open weights, no gate
Released by Yusheng Su through its official repository on Hugging Face. Read the license.
Built From
- Derived from Qwen/Qwen3.8-27B
Questions About qwen38-dspark-moe-2layer
Can I use qwen38-dspark-moe-2layer commercially?
Yes. qwen38-dspark-moe-2layer is released under Apache License 2.0. The Apache License 2.0 is a permissive open-source license. It permits commercial use, modification and redistribution. It requires keeping the license and copyright notices and any NOTICE file, stating significant changes, and it includes an express patent grant from contributors.