SAVRN
Search Contact SAVRN

Open-weight model · Text generation

Qwen3.8-4B-Distill-GGUF

by Empero empero-ai/Qwen3.8-4B-Distill-GGUF

Developed by Empero GGUF quantizations of empero-ai/Qwen3.8-4B — a full-parameter distillation of Qwen3.8 2.4T A95B into the Qwen3.5-4B architecture — for llama.cpp, Ollama, LM Studio, Jan, KoboldCpp, and other stock GGUF runtimes.

Parameters—
Context—
Weights22.8 GB
Licenseapache-2.0
AccessOpen weights
Monthly Downloads774.1k

Model Card

By Empero, published under apache-2.0, revision 391fc7d103e3.

Developed by Empero GGUF quantizations of empero-ai/Qwen3.8-4B — a full-parameter distillation of Qwen3.8 2.4T A95B into the Qwen3.5-4B architecture — for llama.cpp, Ollama, LM Studio, Jan, KoboldCpp, and other stock GGUF runtimes. This card is about choosing a file and running it. The capability writeup, full benchmark results, and best practices live on the main model card. Headline results for the source model (CoT protocols, lm-evaluation-harness, identical settings base vs. student): Sizes are exact decimal GB from the uploaded files (1 GB = 1,000,000,000 bytes). Practical weight-size-based guidance at modest context — the KV cache is the dominant cost at long context and may require…

Read Empero's full model card

Qwen3.8-4B — GGUF

Developed by Empero

GGUF quantizations of empero-ai/Qwen3.8-4B — a full-parameter distillation of Qwen3.8 2.4T A95B into the Qwen3.5-4B architecture — for llama.cpp, Ollama, LM Studio, Jan, KoboldCpp, and other stock GGUF runtimes.

This card is about choosing a file and running it. The capability writeup, full benchmark results, and best practices live on the main model card.

Headline results for the source model (CoT protocols, lm-evaluation-harness, identical settings base vs. student):

Task Qwen3.5-4B (base) Qwen3.8-4B Δ
mmlu (CoT, 57 subjects) 0.354 0.553 +0.199
gsm8k_cot 0.850 0.785 −0.065

[!Note] Qwen3.5-class models are hybrids: three Gated DeltaNet layers for every full-attention layer. A recent llama.cpp build with Qwen3.5 / Gated DeltaNet support is required — older builds will fail to load the architecture.

Files

File Quant Size Notes
Qwen3.8-4B-Q4_K_M.gguf Q4_K_M 2.783 GB Recommended. Best quality/size balance for most users.
Qwen3.8-4B-Q5_K_M.gguf Q5_K_M 3.161 GB Higher quality at a modest size increase.
Qwen3.8-4B-Q6_K.gguf Q6_K 3.563 GB Near-lossless.
Qwen3.8-4B-Q8_0.gguf Q8_0 4.611 GB Highest-quality quantization.
Qwen3.8-4B-BF16.gguf BF16 8.666 GB Full precision reference.

Sizes are exact decimal GB from the uploaded files (1 GB = 1,000,000,000 bytes).

What fits on a GPU?

Practical weight-size-based guidance at modest context — the KV cache is the dominant cost at long context and may require offload regardless of weight quant:

Quant Guidance
Q4_K_M / Q5_K_M Comfortable on 4–6 GB cards; strong CPU-only option as well.
Q6_K / Q8_0 6–8 GB recommended.
BF16 12 GB+.

Usage

llama.cpp

llama-cli -m Qwen3.8-4B-Q4_K_M.gguf \
  --temp 0.6 --top-p 0.95 --top-k 20 \
  -n 16384 -cnv

Use the built-in chat template (-cnv). The model is a reasoning model: every answer opens with a <think> block, so allow a generous -n and strip the <think>...</think> span for end users.

Ollama / LM Studio / Jan / KoboldCpp

Download the GGUF of your choice and load it directly; the chat template is embedded in the file. Recommended sampling: temperature=0.6, top_p=0.95, top_k=20.

Provenance & licensing

Quantizations of empero-ai/Qwen3.8-4B, a distillation of Qwen3.8 2.4T A95B into Qwen/Qwen3.5-4B trained on ~45,000 curated teacher traces from our internal Qwen3.8 distillation datasets. Weights are Apache-2.0, inherited from the Qwen base, shared as-is.

Stay in the loop

Sign up for the Empero newsletter at empero.org for releases, evals, and research notes.

Support / Donate

If this model helped you, consider supporting the project:

  • BTC: bc1qx6zepu6sfkvshgdmc4ewu6pk6rpadvpgffpp7v
  • LTC: ltc1qv2mefzps2vtjcpwfx8xxdrpplrcvltswm68r7x

Acknowledgements

Identity and Version

Repository
empero-ai/Qwen3.8-4B-Distill-GGUF
Publisher
Empero
Task
Text generation
Modality
Text
Library
gguf
Parameters
Not stated by the source
Languages
en
Revision
391fc7d103e3942a408def3e4f51c2f85d464417
First published
2026-08-15
Last updated
2026-08-16

Files and Weights

8 files, 22.8 GB in total. The weights are 5 files totalling 22.8 GB in gguf.

Weights5 files · 22.8 GB
Documentation1 file · 3.8 KB
Other1 file · 449 B
Repository1 file · 1.8 KB
Every file
FileTypeSizeSHA-256
Qwen3.8-4B-BF16.ggufWeights8.7 GB 448616595da5
Qwen3.8-4B-Q4_K_M.ggufWeights2.8 GB dec96e8cf2e1
Qwen3.8-4B-Q5_K_M.ggufWeights3.2 GB 735cd00b154f
Qwen3.8-4B-Q6_K.ggufWeights3.6 GB 529393d9f785
Qwen3.8-4B-Q8_0.ggufWeights4.6 GB 770b780d6754
README.mdDocumentation3.8 KB —
SHA256SUMSOther449 B —
.gitattributesRepository1.8 KB —

License and Download

License
apache-2.0
Access
Open weights, no gate
Download size
22.8 GB
Download from Empero

Released by Empero through its official repository on Hugging Face. Read the license.

Built From

  • Derived from empero-ai/Qwen3.8-4B-Distill
  • Quantized from empero-ai/Qwen3.8-4B-Distill

Memory Requirements

PrecisionWeights in memory
As published22.8 GB

Weights only, from the published parameter count; the key-value cache and runtime add to this.

Questions About Qwen3.8-4B-Distill-GGUF

Can I use Qwen3.8-4B-Distill-GGUF commercially?

Yes. Qwen3.8-4B-Distill-GGUF is released under Apache License 2.0. The Apache License 2.0 is a permissive open-source license. It permits commercial use, modification and redistribution. It requires keeping the license and copyright notices and any NOTICE file, stating significant changes, and it includes an express patent grant from contributors.

Similar Models

Fine-tune Qwen3 (14B) for free using our Google Colab notebook! - Read our Blog about Qwen3 support: unsloth.ai/blog/qwen3 - View the rest of our notebooks in our docs here. Qwen3-Coder is available in multiple sizes. Today, we're excited to introduce Qwen3-Coder-30B-A3B-Instruct. This streamlined model maintains impressive performance and efficiency, featuring the following key enhancements: - Significant Performance among open models on Agentic Coding, Agentic Browser-Use, and other foundational coding tasks. - Long-context Capabilities with native support for 256K tokens, extendable up to 1M tokens using Yarn, optimized for repository-scale understanding. - Agentic Coding supporting for…

Open weights apache-2.0 transformers
View model

Model · Text generation

opt-125m

AI at Meta

OPT was first introduced in Open Pre-trained Transformer Language Models and first released in metaseq's repository on May 3rd 2022 by Meta AI. Disclaimer: The team releasing OPT wrote an official model card, which is available in Appendix D of the paper. Content from this model card has been written by the Hugging Face team. To quote the first two paragraphs of the official paper OPT was predominantly pretrained with English text, but a small amount of non-English data is still present within the training corpus via CommonCrawl. The model was pretrained using a causal language modeling (CLM) objective. OPT belongs to the same family of decoder-only models like GPT-3. As such, it was…

Open weights other 2,048 tokens transformers
View model

Model · Text generation

Ornith-1.5-9B-GGUF

Ornith

Chirp Chirp! We are introducing Ornith-1.5, a major step toward building foundation models through end-to-end self-improvement. Ornith-1.5 extends Ornith-1.0 (which was developed on top of Qwen3.5 and Gemma4 with additional continued pretraining, mid-training, and post-training) by expanding the self-improvement loop from scaffold and rollout optimization to jointly optimizing task generation, scaffold construction, and solution rollouts. Rather than relying on a fixed set of human-curated tasks and manually designed harnesses, Ornith-1.5 continuously generates new training tasks, discovers effective strategies for solving them, and improves the policy through reinforcement learning. For…

Open weights mit transformers
View model

Model · Text generation

Ornith-1.5-35B-A3B-GGUF

Ornith

Chirp Chirp! We are introducing Ornith-1.5, a major step toward building foundation models through end-to-end self-improvement. Ornith-1.5 extends Ornith-1.0 (which was developed on top of Qwen3.5 and Gemma4 with additional continued pretraining, mid-training, and post-training) by expanding the self-improvement loop from scaffold and rollout optimization to jointly optimizing task generation, scaffold construction, and solution rollouts. Rather than relying on a fixed set of human-curated tasks and manually designed harnesses, Ornith-1.5 continuously generates new training tasks, discovers effective strategies for solving them, and improves the policy through reinforcement learning. For…

Open weights mit transformers
View model

Model · Text generation

Ornith-1.0-9B-GGUF

Ornith

Aloha! Today, we are releasing Ornith-1.0, a self-improving family of open-source models for agentic coding. This model card documents Ornith-1.0-9B, the most lightweight member of the Ornith family, designed for efficient single-GPU deployment. Ornith-1.0-9B is a dense ~9B model (≈19 GB in bf16), so it serves comfortably on a single 80GB GPU. The recipes below stand up an OpenAI-compatible server; add --tensor-parallel-size / --tp if you want to shard across more GPUs. For a quick local test (or to script offline generation), load the model directly with Transformers. Make sure you have a recent release installed — see the Transformers installation guide; Ornith-1.0-9B requires…

Open weights mit transformers
View model

Uncensored Qwen3.8-27B, published as GGUF quantizations with the multi token prediction (MTP) head retained and verified. Refusal behaviour has been substantially reduced, not eliminated. See Measured behaviour for the numbers. Capabilities, training data, and architecture are otherwise unchanged. - Refusal directions removed with Heretic, which co minimizes refusal count against KL divergence from the base model. No handwritten refusal removal code, no finetuning, no additional training data. - Abliteration runs at bf16 (no 4 bit quantization). the resulting LoRA is merged into the bf16 base, so the published weights are not a quantized round trip. - mtp. tensors are copied verbatim from…

Open weights apache-2.0 llama.cpp
View model