2-layer MoE DSpark draft models for Qwen3.8-27B: an ablation of Kan Wu's 5-layer Qwen3.8-27B DSpark MoE drafter (SpecForge branch kan/moe-3-qwen38) with the draft reduced from 5 to 2 decoder layers and nothing else changed (512 routed experts x 512 wide, top-10 softmax routing, sigmoid-gated shared expert of width 2048, router centering; 8.48B total / about 1.08B active parameters). One servable export per training epoch, each in its own folder (config.json, model.safetensors, tokenizer and a README with the training metrics of that checkpoint): Serving needs an SGLang build with the Qwen3MoeDSparkModel draft class (upstream SGLang 0.5.18 only ships the dense Qwen3DSparkModel). Download one…
Independent publisher
Yusheng Su
yushengsu
Models in Library5
Datasets in Library0
Models on Hugging Face11
Followers1
Compare the published evidence
This profile connects the 5 model and dataset records shown below to their upstream cards and sources. The library counts describe SAVRN's coverage, rather than this publisher's entire output.
Reported licenses: apache-2.0. Open a record to review its specific license and conditions.
Compare model specifications, source dates, and hardware estimates on the individual records. Publisher identity verification is separate from a model's quality, deployment readiness, or suitability for your work.
Models
5-layer DSpark MoE drafter for Qwen3.8-27B with 256 routed experts (width 512, top-10) and one shared expert (2048): Kan Wu's kan/moe-3-qwen38 SpecForge recipe with half the experts, trained from scratch for 3 epochs on RadixArk/Qwen3.8-27B-Regen-Mixture-v1 (global batch 256, lr 5e-4 constant) on one 4x GB300 node. Each finished epoch is a complete servable SGLang export in its own folder: Training curves: https://wandb.ai/milestraining/specforge-5layer-moe-0.5%2Aexperts/runs/qwen38-dspark-moe-256exp-3ep-gb300 Recipe and config: branch exp/qwen38-dspark-moe-256exp-3ep-gb300 of https://github.com/yushengsu-thu/SpecForge
Placeholder. The servable export of the 2-layer Qwen3.8-27B DSpark MoE drafter after epoch 3 will be uploaded here automatically when that checkpoint is written. Live training metrics: https://wandb.ai/milestraining/specforge-2%20layer%20moe Code and recipes: https://github.com/yushengsu-thu/SpecForge/tree/exp/qwen3.8-27b-dspark-moe-2layer
Placeholder. The servable export of the 2-layer Qwen3.8-27B DSpark MoE drafter after epoch 2 will be uploaded here automatically when that checkpoint is written. Live training metrics: https://wandb.ai/milestraining/specforge-2%20layer%20moe Code and recipes: https://github.com/yushengsu-thu/SpecForge/tree/exp/qwen3.8-27b-dspark-moe-2layer
Placeholder. The servable export of the 2-layer Qwen3.8-27B DSpark MoE drafter after epoch 1 will be uploaded here automatically when that checkpoint is written. Live training metrics: https://wandb.ai/milestraining/specforge-2%20layer%20moe Code and recipes: https://github.com/yushengsu-thu/SpecForge/tree/exp/qwen3.8-27b-dspark-moe-2layer