Skip to page content
SAVRN
Search Plan a project

Open-weight model · Text generation

ModujoMoE

by Modujo Modujo-AI/ModujoMoE

ModujoMoE is an open-weight model for text generation from Modujo. It has 32,768-token context. Its published files total 159.2 GB. It draws 297 downloads a month.

This repository keeps model variants in self-contained subdirectories. The repository name is retained for compatibility, but the repository root no longer contains model weights and must not be loaded directly.

Parameters—
Context32,768
Weights159.1 GB
License—
AccessOpen weights
Monthly Downloads297

Model Card

This repository keeps model variants in self-contained subdirectories. The repository name is retained for compatibility, but the repository root no longer contains model weights and must not be loaded directly. Separate 4B PT and experimental SFT snapshots are available in named directories. The SFT has not completed its teacher-distillation stage. The PT is a later continuation snapshot than the SFT initialization source; see each model card for lineage and limitations. pretrain/Modujo-1B-A0.75B/ is the released pretraining artifact. - 36 layers with 8 routed experts per layer, top-2 routing, and one shared expert - Not instruction-tuned and not intended to be treated as a chat model…

Excerpt from the card by Modujo.

Configuration

Architecture
Qwen4ExpForCausalLM
Context length (tokens)
32,768
Layers
32
Hidden size
1,536
Attention heads
12
Key/value heads
2
Head dimension
128
Vocabulary size
248,320
Experts
64
Experts active per token
2
Model type
qwen4_exp_text

Identity and Version

Repository
Modujo-AI/ModujoMoE
Publisher
Modujo
Task
Text generation
Modality
Text
Library
transformers
Parameters
Not stated by the source
Languages
pt, sft
Revision
a7603e817ee972039c646c85f8a3c8eef9e484f7
First published
2026-09-13
Last updated
2026-10-20

Files and Weights

214 files, 159.2 GB in total. The weights are 160 files totalling 159.1 GB in safetensors.

Weights160 files · 159.1 GB
Configuration29 files · 6.5 MB
Tokenizer10 files · 92.8 MB
Documentation5 files · 24.8 KB
Other9 files · 94.8 KB
Repository1 file · 1.6 KB
Every file
FileTypeSizeSHA-256
pretrain/Modujo-1B-A0.75B/model.safetensorsWeights2.0 GB 36bd33866a41
pt/Modujo-4B-A0.66B/model.safetensorsWeights16.9 GB 52ff09415352
sft/Modujo-4B-A0.66B/model-00001-of-00004.safetensorsWeights4.9 GB b96fb9a00dda
sft/Modujo-4B-A0.66B/model-00002-of-00004.safetensorsWeights4.8 GB b792e3b40218
sft/Modujo-4B-A0.66B/model-00003-of-00004.safetensorsWeights4.8 GB 3ed5584b1161
sft/Modujo-4B-A0.66B/model-00004-of-00004.safetensorsWeights1.4 GB e4bfa93d2bf8
sft/qwen38-distill-step8032/frozen-model-00027.safetensorsWeights800.0 MB f5a1513adbcd
sft/qwen38-distill-step8032/frozen-model-00028.safetensorsWeights800.0 MB 7cf8581d776c
sft/qwen38-distill-step8032/frozen-model-00029.safetensorsWeights800.0 MB c33d0cf164a8
sft/qwen38-distill-step8032/frozen-model-00030.safetensorsWeights800.0 MB 70885808995d
sft/qwen38-distill-step8032/frozen-model-00031.safetensorsWeights800.0 MB 4f9181e54f90
sft/qwen38-distill-step8032/frozen-model-00032.safetensorsWeights800.0 MB 93a3221e8109
sft/qwen38-distill-step8032/frozen-model-00033.safetensorsWeights800.0 MB af5d0023a429
sft/qwen38-distill-step8032/frozen-model-00034.safetensorsWeights800.0 MB c446c3af6076
sft/qwen38-distill-step8032/frozen-model-00035.safetensorsWeights800.0 MB 55c080f04882
sft/qwen38-distill-step8032/frozen-model-00036.safetensorsWeights800.0 MB 7ac8f72e51ac
sft/qwen38-distill-step8032/frozen-model-00037.safetensorsWeights800.0 MB fd6deecd49b7
sft/qwen38-distill-step8032/frozen-model-00038.safetensorsWeights800.0 MB 0020a3ae064f
sft/qwen38-distill-step8032/frozen-model-00039.safetensorsWeights800.0 MB c51b84fcd68d
sft/qwen38-distill-step8032/frozen-model-00040.safetensorsWeights800.0 MB b5c92f45a0d9
sft/qwen38-distill-step8032/frozen-model-00041.safetensorsWeights800.0 MB a4d913c6dad5
sft/qwen38-distill-step8032/frozen-model-00042.safetensorsWeights800.0 MB 2be4b40cbdf6
sft/qwen38-distill-step8032/frozen-model-00043.safetensorsWeights800.0 MB 87559f1aee2c
sft/qwen38-distill-step8032/frozen-model-00044.safetensorsWeights800.0 MB 66a4b6467f42
sft/qwen38-distill-step8032/frozen-model-00045.safetensorsWeights800.0 MB 4e6e54fe2ede
sft/qwen38-distill-step8032/frozen-model-00046.safetensorsWeights800.0 MB 7b4acf8378a2
sft/qwen38-distill-step8032/frozen-model-00047.safetensorsWeights800.0 MB 4ff2cf5b68e1
sft/qwen38-distill-step8032/frozen-model-00048.safetensorsWeights800.0 MB f19a6f54334d
sft/qwen38-distill-step8032/frozen-model-00049.safetensorsWeights800.0 MB 2a55d5d0ec57
sft/qwen38-distill-step8032/frozen-model-00050.safetensorsWeights800.0 MB 755c9c67b891
sft/qwen38-distill-step8032/frozen-model-00051.safetensorsWeights800.0 MB 92cc5604f455
sft/qwen38-distill-step8032/frozen-model-00052.safetensorsWeights800.0 MB 818aaad5e042
sft/qwen38-distill-step8032/frozen-model-00053.safetensorsWeights800.0 MB 08b7c8a12d30
sft/qwen38-distill-step8032/frozen-model-00054.safetensorsWeights800.0 MB 4d02ec0dc0e3
sft/qwen38-distill-step8032/frozen-model-00055.safetensorsWeights800.0 MB ce06f1052f56
sft/qwen38-distill-step8032/frozen-model-00056.safetensorsWeights800.0 MB 44002e10f988
sft/qwen38-distill-step8032/frozen-model-00057.safetensorsWeights800.0 MB 948a8ea45ccb
sft/qwen38-distill-step8032/frozen-model-00058.safetensorsWeights800.0 MB 89135f0e72d2
sft/qwen38-distill-step8032/frozen-model-00059.safetensorsWeights800.0 MB a9c8ca1964ff
sft/qwen38-distill-step8032/frozen-model-00060.safetensorsWeights800.0 MB 2b2a4fd79b6a
sft/qwen38-distill-step8032/frozen-model-00061.safetensorsWeights800.0 MB 75d5c605f203
sft/qwen38-distill-step8032/frozen-model-00062.safetensorsWeights800.0 MB 7b573bc28816
sft/qwen38-distill-step8032/frozen-model-00063.safetensorsWeights800.0 MB b0ef680c4222
sft/qwen38-distill-step8032/frozen-model-00064.safetensorsWeights800.0 MB c69db7103c6d
sft/qwen38-distill-step8032/frozen-model-00065.safetensorsWeights800.0 MB e6207e7a21ab
sft/qwen38-distill-step8032/frozen-model-00066.safetensorsWeights800.0 MB 458bbb5e131c
sft/qwen38-distill-step8032/frozen-model-00067.safetensorsWeights800.0 MB ea0ef7dace35
sft/qwen38-distill-step8032/frozen-model-00068.safetensorsWeights800.0 MB 982e4e3abbb3
sft/qwen38-distill-step8032/frozen-model-00069.safetensorsWeights800.0 MB 32c28a79f754
sft/qwen38-distill-step8032/frozen-model-00070.safetensorsWeights800.0 MB 5107dce8b4b7
sft/qwen38-distill-step8032/frozen-model-00071.safetensorsWeights800.0 MB 99726ff3992c
sft/qwen38-distill-step8032/frozen-model-00072.safetensorsWeights800.0 MB 98099b131a2c
sft/qwen38-distill-step8032/frozen-model-00073.safetensorsWeights800.0 MB 85b280311e74
sft/qwen38-distill-step8032/frozen-model-00074.safetensorsWeights800.0 MB cad11ab6de2a
sft/qwen38-distill-step8032/frozen-model-00075.safetensorsWeights800.0 MB d6c6081d1da1
sft/qwen38-distill-step8032/frozen-model-00076.safetensorsWeights800.0 MB 08f82273a89d
sft/qwen38-distill-step8032/frozen-model-00077.safetensorsWeights800.0 MB d887b6aeff69
sft/qwen38-distill-step8032/frozen-model-00078.safetensorsWeights800.0 MB 46289e1f89cb
sft/qwen38-distill-step8032/frozen-model-00079.safetensorsWeights800.0 MB 1cca8c24272b
sft/qwen38-distill-step8032/frozen-model-00080.safetensorsWeights800.0 MB 56c1635eee74
sft/qwen38-distill-step8032/frozen-model-00081.safetensorsWeights800.0 MB f2033e92070c
sft/qwen38-distill-step8032/frozen-model-00082.safetensorsWeights800.0 MB c8e64685aafe
sft/qwen38-distill-step8032/frozen-model-00083.safetensorsWeights800.0 MB a680e4caa663
sft/qwen38-distill-step8032/frozen-model-00084.safetensorsWeights800.0 MB d4480dcd7891
sft/qwen38-distill-step8032/frozen-model-00085.safetensorsWeights800.0 MB 819b4edce306
sft/qwen38-distill-step8032/frozen-model-00086.safetensorsWeights800.0 MB 652512aeaaf2
sft/qwen38-distill-step8032/frozen-model-00087.safetensorsWeights800.0 MB eb32f4692f05
sft/qwen38-distill-step8032/frozen-model-00088.safetensorsWeights800.0 MB 51d909bf231d
sft/qwen38-distill-step8032/frozen-model-00089.safetensorsWeights800.0 MB 4549fc077d53
sft/qwen38-distill-step8032/frozen-model-00090.safetensorsWeights800.0 MB af2fecf4aa45
sft/qwen38-distill-step8032/frozen-model-00091.safetensorsWeights800.0 MB 694233eb460e
sft/qwen38-distill-step8032/frozen-model-00092.safetensorsWeights800.0 MB 13b97e836449
sft/qwen38-distill-step8032/frozen-model-00093.safetensorsWeights800.0 MB 4aa53b152e73
sft/qwen38-distill-step8032/frozen-model-00094.safetensorsWeights800.0 MB a5f81ce30cb9
sft/qwen38-distill-step8032/frozen-model-00095.safetensorsWeights800.0 MB e520d033ca29
sft/qwen38-distill-step8032/frozen-model-00096.safetensorsWeights800.0 MB b3f1346f92fc
sft/qwen38-distill-step8032/frozen-model-00097.safetensorsWeights800.0 MB bc2a24c00ab4
sft/qwen38-distill-step8032/frozen-model-00098.safetensorsWeights800.0 MB 5c1d13df5781
sft/qwen38-distill-step8032/frozen-model-00099.safetensorsWeights800.0 MB f0abe52a31e5
sft/qwen38-distill-step8032/frozen-model-00100.safetensorsWeights800.0 MB be57daf28525
sft/qwen38-distill-step8032/frozen-model-00101.safetensorsWeights800.0 MB 9de063dc18a0
sft/qwen38-distill-step8032/frozen-model-00102.safetensorsWeights800.0 MB 346e7a475a5c
sft/qwen38-distill-step8032/frozen-model-00103.safetensorsWeights800.0 MB 8acee92fb988
sft/qwen38-distill-step8032/frozen-model-00104.safetensorsWeights800.0 MB aa31f0a163df
sft/qwen38-distill-step8032/frozen-model-00105.safetensorsWeights800.0 MB 35c3bdd5056c
sft/qwen38-distill-step8032/frozen-model-00106.safetensorsWeights800.0 MB 53252aebd288
sft/qwen38-distill-step8032/frozen-model-00107.safetensorsWeights800.0 MB 7cf334fc214f
sft/qwen38-distill-step8032/frozen-model-00108.safetensorsWeights800.0 MB 5e2f809d5154
sft/qwen38-distill-step8032/frozen-model-00109.safetensorsWeights800.0 MB 36ee2acf1ac5
sft/qwen38-distill-step8032/frozen-model-00110.safetensorsWeights800.0 MB d95ef83319ee
sft/qwen38-distill-step8032/frozen-model-00111.safetensorsWeights800.0 MB 2e964883acc5
sft/qwen38-distill-step8032/frozen-model-00112.safetensorsWeights800.0 MB bb60f9da1a15
sft/qwen38-distill-step8032/frozen-model-00113.safetensorsWeights800.0 MB 14339fdf2406
sft/qwen38-distill-step8032/frozen-model-00114.safetensorsWeights800.0 MB 0de60db0f5a9
sft/qwen38-distill-step8032/frozen-model-00115.safetensorsWeights800.0 MB ddcf0ed8ec19
sft/qwen38-distill-step8032/frozen-model-00116.safetensorsWeights800.0 MB 6eb49f1dd547
sft/qwen38-distill-step8032/frozen-model-00117.safetensorsWeights800.0 MB 2896c5e250a5
sft/qwen38-distill-step8032/frozen-model-00118.safetensorsWeights800.0 MB 73e489d0dfd5
sft/qwen38-distill-step8032/frozen-model-00119.safetensorsWeights800.0 MB 296e311275bf
sft/qwen38-distill-step8032/frozen-model-00120.safetensorsWeights800.0 MB 33cf5415bf15
sft/qwen38-distill-step8032/frozen-model-00121.safetensorsWeights800.0 MB 5d377f2e98ca
sft/qwen38-distill-step8032/frozen-model-00122.safetensorsWeights800.0 MB 0501541af705
sft/qwen38-distill-step8032/frozen-model-00123.safetensorsWeights800.0 MB a6fa51b45b01
sft/qwen38-distill-step8032/frozen-model-00124.safetensorsWeights800.0 MB e8d8d7bfba86
sft/qwen38-distill-step8032/frozen-model-00125.safetensorsWeights800.0 MB 4f51863c0b6e
sft/qwen38-distill-step8032/frozen-model-00126.safetensorsWeights800.0 MB 0bf7a9b5dc14
sft/qwen38-distill-step8032/frozen-model-00127.safetensorsWeights800.0 MB 006b47d9026e
sft/qwen38-distill-step8032/frozen-model-00128.safetensorsWeights800.0 MB 097943ebce12
sft/qwen38-distill-step8032/frozen-model-00129.safetensorsWeights800.0 MB e7538fd229fe
sft/qwen38-distill-step8032/frozen-model-00130.safetensorsWeights800.0 MB 4a8a343db3ff
sft/qwen38-distill-step8032/frozen-model-00131.safetensorsWeights800.0 MB 081c80484487
sft/qwen38-distill-step8032/frozen-model-00132.safetensorsWeights800.0 MB 6dc41f977eff
sft/qwen38-distill-step8032/frozen-model-00133.safetensorsWeights800.0 MB 9f3b5765c2f5
sft/qwen38-distill-step8032/frozen-model-00134.safetensorsWeights800.0 MB 23b5754c883e
sft/qwen38-distill-step8032/frozen-model-00135.safetensorsWeights800.0 MB 4d90b22e569b
sft/qwen38-distill-step8032/frozen-model-00136.safetensorsWeights800.0 MB 6b8d517d2983
sft/qwen38-distill-step8032/frozen-model-00137.safetensorsWeights800.0 MB c33c908fa78f
sft/qwen38-distill-step8032/frozen-model-00138.safetensorsWeights800.0 MB d3dc6ef6921d
sft/qwen38-distill-step8032/frozen-model-00139.safetensorsWeights800.0 MB c0996dfe4fa0
sft/qwen38-distill-step8032/frozen-model-00140.safetensorsWeights800.0 MB 82b45a66568b
sft/qwen38-distill-step8032/frozen-model-00141.safetensorsWeights800.0 MB 403fd0c13dab
sft/qwen38-distill-step8032/frozen-model-00142.safetensorsWeights800.0 MB 3e9d500304c8
sft/qwen38-distill-step8032/frozen-model-00143.safetensorsWeights800.0 MB f16d01fefd35
sft/qwen38-distill-step8032/frozen-model-00144.safetensorsWeights800.0 MB 9618e7331172
sft/qwen38-distill-step8032/frozen-model-00145.safetensorsWeights800.0 MB 8671d08389af
sft/qwen38-distill-step8032/frozen-model-00146.safetensorsWeights800.0 MB a6a78cdbc0cb
sft/qwen38-distill-step8032/frozen-model-00147.safetensorsWeights800.0 MB 60562183f453
sft/qwen38-distill-step8032/frozen-model-00148.safetensorsWeights800.0 MB f5ec5b549162
sft/qwen38-distill-step8032/frozen-model-00149.safetensorsWeights800.0 MB 71624b839090
sft/qwen38-distill-step8032/frozen-model-00150.safetensorsWeights800.0 MB e688f9a39d2a
sft/qwen38-distill-step8032/frozen-model-00151.safetensorsWeights800.0 MB d1bdc260ac26
sft/qwen38-distill-step8032/frozen-model-00152.safetensorsWeights800.0 MB bd1afbe58cfb
sft/qwen38-distill-step8032/frozen-model-00153.safetensorsWeights800.0 MB 3ea765fed458
sft/qwen38-distill-step8032/frozen-model-00154.safetensorsWeights800.0 MB 6c7f23f8f7dc
sft/qwen38-distill-step8032/model-00001.safetensorsWeights635.7 MB c503b522679e
sft/qwen38-distill-step8032/model-00002.safetensorsWeights802.4 MB 7c08366dd32c
sft/qwen38-distill-step8032/model-00003.safetensorsWeights896.3 MB 4a375e9fced5
sft/qwen38-distill-step8032/model-00004.safetensorsWeights862.3 MB a080a77fef48
sft/qwen38-distill-step8032/model-00005.safetensorsWeights863.5 MB 5c2be5b58395
sft/qwen38-distill-step8032/model-00006.safetensorsWeights862.3 MB ebe59ff865d0
sft/qwen38-distill-step8032/model-00007.safetensorsWeights863.5 MB 82296107464c
sft/qwen38-distill-step8032/model-00008.safetensorsWeights862.3 MB c56b7933fdbc
sft/qwen38-distill-step8032/model-00009.safetensorsWeights863.5 MB f7daffa6cfd0
sft/qwen38-distill-step8032/model-00010.safetensorsWeights834.5 MB 78e36f5987c4
sft/qwen38-distill-step8032/model-00011.safetensorsWeights891.3 MB 98d7bba0c82e
sft/qwen38-distill-step8032/model-00012.safetensorsWeights834.5 MB 581670aed440
sft/qwen38-distill-step8032/model-00013.safetensorsWeights891.3 MB 9dbabf533792
sft/qwen38-distill-step8032/model-00014.safetensorsWeights862.3 MB e7b1b837643b
sft/qwen38-distill-step8032/model-00015.safetensorsWeights862.3 MB d765293c8ede
sft/qwen38-distill-step8032/model-00016.safetensorsWeights863.5 MB 22cf6a9fc827
sft/qwen38-distill-step8032/model-00017.safetensorsWeights862.3 MB edbc49821ea1
sft/qwen38-distill-step8032/model-00018.safetensorsWeights863.5 MB 7a3acfe07210
sft/qwen38-distill-step8032/model-00019.safetensorsWeights862.3 MB d963737809f2
sft/qwen38-distill-step8032/model-00020.safetensorsWeights863.5 MB c44b96d83bad
sft/qwen38-distill-step8032/model-00021.safetensorsWeights834.5 MB eada79110236
sft/qwen38-distill-step8032/model-00022.safetensorsWeights891.3 MB 90e2d6b85535
sft/qwen38-distill-step8032/model-00023.safetensorsWeights834.5 MB 54faa922ac55
sft/qwen38-distill-step8032/model-00024.safetensorsWeights891.3 MB c5d4ef5a17b7
sft/qwen38-distill-step8032/model-00025.safetensorsWeights862.3 MB 387c4c63495c
sft/qwen38-distill-step8032/model-00026.safetensorsWeights700.1 MB d9b2510e2f5a
config.jsonConfiguration2.4 KB —
generation_config.jsonConfiguration231 B —
parameter_summary.jsonConfiguration116 B —
pretrain/Modujo-1B-A0.75B/config.jsonConfiguration2.5 KB —
pretrain/Modujo-1B-A0.75B/generation_config.jsonConfiguration253 B —
pretrain/Modujo-1B-A0.75B/parameter_summary.jsonConfiguration391 B —
pt/Modujo-4B-A0.66B/config.jsonConfiguration2.4 KB —
pt/Modujo-4B-A0.66B/generation_config.jsonConfiguration138 B —
pt/Modujo-4B-A0.66B/inference.pyConfiguration1.6 KB —
pt/Modujo-4B-A0.66B/qwen4_exp_dense_compat.pyConfiguration2.2 KB —
pt/Modujo-4B-A0.66B/release.jsonConfiguration598 B —
pt/Modujo-4B-A0.66B/tested-environment.jsonConfiguration141 B —
pt/Modujo-4B-A0.66B/weight-audit.jsonConfiguration207 B —
sft/Modujo-4B-A0.66B/config.jsonConfiguration2.4 KB —
sft/Modujo-4B-A0.66B/evaluation/cases.jsonConfiguration775 B —
sft/Modujo-4B-A0.66B/evaluation/code-tests.jsonConfiguration486 B —
sft/Modujo-4B-A0.66B/evaluation/extended-cases.jsonConfiguration1.4 KB —
sft/Modujo-4B-A0.66B/evaluation/summary.jsonConfiguration591 B —
sft/Modujo-4B-A0.66B/generation_config.jsonConfiguration160 B —
sft/Modujo-4B-A0.66B/inference.pyConfiguration1.6 KB —
sft/Modujo-4B-A0.66B/model.safetensors.index.jsonConfiguration60.4 KB —
sft/Modujo-4B-A0.66B/qwen4_exp_dense_compat.pyConfiguration2.2 KB —
sft/Modujo-4B-A0.66B/release.jsonConfiguration503 B —
sft/Modujo-4B-A0.66B/tested-environment.jsonConfiguration141 B —
sft/qwen38-distill-step8032/config.jsonConfiguration2.9 KB —
sft/qwen38-distill-step8032/generation_config.jsonConfiguration202 B —
sft/qwen38-distill-step8032/model.safetensors.index.jsonConfiguration110.7 KB —
sft/qwen38-distill-step8032/pruning-plan.jsonConfiguration6.3 MB —
sft/qwen38-distill-step8032/weights-manifest.jsonConfiguration25.0 KB —
README.mdDocumentation3.8 KB —
pt/Modujo-4B-A0.66B/README.mdDocumentation2.3 KB —
sft/Modujo-4B-A0.66B/README.mdDocumentation2.7 KB —
sft/Modujo-4B-A0.66B/evaluation/report.mdDocumentation14.9 KB —
sft/qwen38-distill-step8032/README.mdDocumentation1.2 KB —
chat_template.jinjaOther9.0 KB —
pretrain/Modujo-1B-A0.75B/chat_template.jinjaOther9.0 KB —
pt/Modujo-4B-A0.66B/__pycache__/qwen4_exp_dense_compat.cpython-312.pycOther3.5 KB —
pt/Modujo-4B-A0.66B/chat_template.jinjaOther9.0 KB —
sft/Modujo-4B-A0.66B/__pycache__/qwen4_exp_dense_compat.cpython-312.pycOther3.5 KB —
sft/Modujo-4B-A0.66B/chat_template.jinjaOther9.0 KB —
sft/Modujo-4B-A0.66B/evaluation/extended.jsonlOther22.8 KB —
sft/Modujo-4B-A0.66B/evaluation/greedy.jsonlOther20.2 KB —
sft/qwen38-distill-step8032/chat_template.jinjaOther9.0 KB —
.gitattributesRepository1.6 KB —
pretrain/Modujo-1B-A0.75B/tokenizer.jsonTokenizer20.0 MB 06b9509352d2
pretrain/Modujo-1B-A0.75B/tokenizer_config.jsonTokenizer1.1 KB —
pt/Modujo-4B-A0.66B/tokenizer.jsonTokenizer20.0 MB 06b9509352d2
pt/Modujo-4B-A0.66B/tokenizer_config.jsonTokenizer1.1 KB —
sft/Modujo-4B-A0.66B/tokenizer.jsonTokenizer20.0 MB 06b9509352d2
sft/Modujo-4B-A0.66B/tokenizer_config.jsonTokenizer1.1 KB —
sft/qwen38-distill-step8032/tokenizer.jsonTokenizer12.8 MB 0997f410c57a
sft/qwen38-distill-step8032/tokenizer_config.jsonTokenizer17.9 KB —
tokenizer.jsonTokenizer20.0 MB 06b9509352d2
tokenizer_config.jsonTokenizer1.1 KB —

License and Download

License
Not stated by the source
Access
Open weights, no gate
Download size
159.1 GB
Download from Modujo

Released by Modujo through its official repository on Hugging Face.

Memory Requirements

PrecisionWeights in memory
As published159.1 GB

Weights only, from the published parameter count; the key-value cache and runtime add to this.

Questions About ModujoMoE

What is ModujoMoE's context length?

32,768 tokens, from the maximum position embeddings in its published configuration.

Similar Models

Model · Text generation

opt-125m

AI at Meta

OPT was first introduced in Open Pre-trained Transformer Language Models and first released in metaseq's repository on May 3rd 2022 by Meta AI. Disclaimer: The team releasing OPT wrote an official model card, which is available in Appendix D of the paper. Content from this model card has been written by the Hugging Face team. To quote the first two paragraphs of the official paper OPT was predominantly pretrained with English text, but a small amount of non-English data is still present within the training corpus via CommonCrawl. The model was pretrained using a causal language modeling (CLM) objective. OPT belongs to the same family of decoder-only models like GPT-3. As such, it was…

Open weights other 2,048 tokens transformers

Model · Text generation

Ornith-1.5-9B-GGUF

Ornith

Chirp Chirp! We are introducing Ornith-1.5, a major step toward building foundation models through end-to-end self-improvement. Ornith-1.5 extends Ornith-1.0 (which was developed on top of Qwen3.5 and Gemma4 with additional continued pretraining, mid-training, and post-training) by expanding the self-improvement loop from scaffold and rollout optimization to jointly optimizing task generation, scaffold construction, and solution rollouts. Rather than relying on a fixed set of human-curated tasks and manually designed harnesses, Ornith-1.5 continuously generates new training tasks, discovers effective strategies for solving them, and improves the policy through reinforcement learning. For…

Open weights mit transformers

Model · Text generation

Ornith-1.5-35B-A3B-GGUF

Ornith

Chirp Chirp! We are introducing Ornith-1.5, a major step toward building foundation models through end-to-end self-improvement. Ornith-1.5 extends Ornith-1.0 (which was developed on top of Qwen3.5 and Gemma4 with additional continued pretraining, mid-training, and post-training) by expanding the self-improvement loop from scaffold and rollout optimization to jointly optimizing task generation, scaffold construction, and solution rollouts. Rather than relying on a fixed set of human-curated tasks and manually designed harnesses, Ornith-1.5 continuously generates new training tasks, discovers effective strategies for solving them, and improves the policy through reinforcement learning. For…

Open weights mit transformers

Uncensored Qwen3.8-27B, published as GGUF quantizations with the multi token prediction (MTP) head retained and verified. Refusal behaviour has been substantially reduced, not eliminated. See Measured behaviour for the numbers. Capabilities, training data, and architecture are otherwise unchanged. - Refusal directions removed with Heretic, which co minimizes refusal count against KL divergence from the base model. No handwritten refusal removal code, no finetuning, no additional training data. - Abliteration runs at bf16 (no 4 bit quantization). the resulting LoRA is merged into the bf16 base, so the published weights are not a quantized round trip. - mtp. tensors are copied verbatim from…

Open weights apache-2.0 llama.cpp

Model · Text generation

Ornith-1.0-9B-GGUF

Ornith

Aloha! Today, we are releasing Ornith-1.0, a self-improving family of open-source models for agentic coding. This model card documents Ornith-1.0-9B, the most lightweight member of the Ornith family, designed for efficient single-GPU deployment. Ornith-1.0-9B is a dense ~9B model (≈19 GB in bf16), so it serves comfortably on a single 80GB GPU. The recipes below stand up an OpenAI-compatible server; add --tensor-parallel-size / --tp if you want to shard across more GPUs. For a quick local test (or to script offline generation), load the model directly with Transformers. Make sure you have a recent release installed — see the Transformers installation guide; Ornith-1.0-9B requires…

Open weights mit transformers

Model · Text generation

Ternary-Bonsai-2-27B-gguf

Prism ML

Full 27B-class reasoning in ternary transformer weights, for llama.cpp (CUDA, Metal, CPU) - \~5.9 GB language model (down from \~54 GB FP16) — full 27B-class reasoning on a standard laptop or a single GPU - 98.2% of FP16 intelligence retained: 84.78 average across 14 thinking-mode benchmarks — far above the conventional IQ2XXS build (72.59) at about 82% of its footprint, and within 0.4 points of UD-Q4KXL at three times the footprint - Retains thinking, reasoning, and agentic behavior deep in the sub-4-bit regime, where conventional low-bit representations collapse: math within half a point of full precision (96.57), coding level with the baseline (89.42), agentic tool calling at 74.92…

Open weights apache-2.0 llama.cpp