A DSpark draft model for speculative decoding with unsloth/Llama-3.2-3B-Instruct as the verifier, trained with speculators. The drafter proposes 8 tokens at a time and the verifier checks them in one forward pass, so output is identical to running the verifier alone — a lossless speedup. Mean acceptance length is 3.27 tokens committed per verification step, up to 4.92 on mathreasoning. vLLM loads the verifier automatically from the config — don't pass it separately. Then query the OpenAI-compatible endpoint at http://localhost:8000/v1. 5 Qwen3 layers (hidden size 3072, intermediate size 8192, 24 attention heads over 8 KV heads, sliding-window attention with a 2048-token window), ~0.9B…
Independent publisher
Yosef Worku Alemneh
rasyosef
Pretraining, Supervised Fine Tuning, Direct Preference Optimization, Retrieval Augmented Generation (RAG), Function Calling
Models in Library1
Datasets in Library0
Models on Hugging Face82
Followers40