qwen3-8b-moe-merge-merged is an open-weight model from Cokoro, released under Apache License 2.0. Its published files total 16.9 GB.
Base: Qwen/Qwen3-8B @ b968826d9c46dd6066d109eabc6255188de91218 (bf16, non-thinking chat template qwen3nothink.jinja from the experiment repo).
Model Card
By Cokoro, published under apache-2.0, revision 321f8df747e0.
Base: Qwen/Qwen3-8B @ b968826d9c46dd6066d109eabc6255188de91218 (bf16, non-thinking chat template qwen3nothink.jinja from the experiment repo). Part of a multi-expert training + merging experiment on SWE-bench Pro v2-hard, HLE, ASI-Bench and Terminal-Bench 2.0. - mergedties/ - mopd/ - mergedlinearlora64/ - mergedlinear/ Recipes, data sources/licenses, evaluation protocol and results: see the experiment repository (models/README.md, data/README.md, configs/eval/benchmarks.yaml, results/summary.csv). Full-weight subfolders load with AutoModelForCausalLM.frompretrained("cokoro/qwen3-8b-moe-merge-merged", subfolder=" "). Limitations: experts were trained on small/proxy data (GSM8K for HLE…
Read Cokoro's full model card
Merged models (linear, TIES) and MOPD student
Base: Qwen/Qwen3-8B @ b968826d9c46dd6066d109eabc6255188de91218 (bf16, non-thinking chat template
qwen3_nothink.jinja from the experiment repo). Part of a multi-expert training + merging experiment on
SWE-bench Pro v2-hard, HLE, ASI-Bench and Terminal-Bench 2.0.
Contents:
- merged_ties/
- mopd/
- merged_linear_lora64/
- merged_linear/
Recipes, data sources/licenses, evaluation protocol and results: see the experiment repository (models/README.md, data/README.md, configs/eval/benchmarks.yaml, results/summary.csv).
Load a LoRA subfolder:
from transformers import AutoModelForCausalLM
from peft import PeftModel
m = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-8B", revision="b968826d9c46dd6066d109eabc6255188de91218", dtype="bfloat16")
m = PeftModel.from_pretrained(m, "cokoro/qwen3-8b-moe-merge-merged", subfolder="<subfolder>")
Full-weight subfolders load with AutoModelForCausalLM.from_pretrained("cokoro/qwen3-8b-moe-merge-merged", subfolder="<subfolder>").
Limitations: experts were trained on small/proxy data (GSM8K for HLE, CommonsenseQA templates for ASI); scores on all four benchmarks are low for this 8B base. Not for production use.
Identity and Version
- Repository
- cokoro/qwen3-8b-moe-merge-merged
- Publisher
- Cokoro
- Task
- Not stated by the source
- Modality
- Other
- Library
- transformers
- Parameters
- Not stated by the source
- Languages
- Not stated by the source
- Revision
- 321f8df747e05d182dee8a299b4cc2c660bf237f
- First published
- 2026-10-18
- Last updated
- 2026-10-18
Files and Weights
23 files, 16.9 GB in total. The weights are 6 files totalling 16.9 GB in safetensors.
Every file
| File | Type | Size | SHA-256 |
|---|---|---|---|
| merged_linear_lora64/adapter_model.safetensors | Weights | 349.2 MB | e24f1e6b6be2 |
| merged_ties/model-00001-of-00004.safetensors | Weights | 4.9 GB | 7a06b38ed6b2 |
| merged_ties/model-00002-of-00004.safetensors | Weights | 4.9 GB | 7f02d52b9ea9 |
| merged_ties/model-00003-of-00004.safetensors | Weights | 5.0 GB | 839d4b7342c5 |
| merged_ties/model-00004-of-00004.safetensors | Weights | 1.6 GB | ca4371c808b7 |
| mopd/adapter_model.safetensors | Weights | 174.7 MB | ce3241634f99 |
| merged_linear_lora64/adapter_config.json | Configuration | 1.1 KB | — |
| merged_linear_lora64/merge_manifest.json | Configuration | 323 B | — |
| merged_ties/config.json | Configuration | 1.6 KB | — |
| merged_ties/generation_config.json | Configuration | 214 B | — |
| merged_ties/merge_manifest.json | Configuration | 549 B | — |
| merged_ties/model.safetensors.index.json | Configuration | 32.9 KB | — |
| mopd/adapter_config.json | Configuration | 1.2 KB | — |
| mopd/training_metrics.json | Configuration | 453 B | — |
| README.md | Documentation | 1.4 KB | — |
| merged_ties/chat_template.jinja | Other | 4.2 KB | — |
| mopd/chat_template.jinja | Other | 4.2 KB | — |
| mopd/mopd_log.jsonl | Other | 31.7 KB | — |
| .gitattributes | Repository | 1.6 KB | — |
| merged_ties/tokenizer.json | Tokenizer | 11.4 MB | be75606093db |
| merged_ties/tokenizer_config.json | Tokenizer | 693 B | — |
| mopd/tokenizer.json | Tokenizer | 11.4 MB | 79cb3c783570 |
| mopd/tokenizer_config.json | Tokenizer | 693 B | — |
License and Download
- License
- apache-2.0
- Access
- Open weights, no gate
- Download size
- 16.9 GB
Released by Cokoro through its official repository on Hugging Face. Read the license.
Built From
- Adapter of Qwen/Qwen3-8B
- Derived from Qwen/Qwen3-8B
Memory Requirements
| Precision | Weights in memory |
|---|---|
| As published | 16.9 GB |
Weights only, from the published parameter count; the key-value cache and runtime add to this.
Questions About qwen3-8b-moe-merge-merged
Can I use qwen3-8b-moe-merge-merged commercially?
Yes. qwen3-8b-moe-merge-merged is released under Apache License 2.0. The Apache License 2.0 is a permissive open-source license. It permits commercial use, modification and redistribution. It requires keeping the license and copyright notices and any NOTICE file, stating significant changes, and it includes an express patent grant from contributors.