Skip to page content
SAVRN
Search Plan a project

Open-weight model · Text generation

Morpho-72B-MoE-DPO

by Blue Morpho Limited bluemorpholimited/Morpho-72B-MoE-DPO

Morpho-72B-MoE-DPO is an open-weight model for text generation from Blue Morpho Limited, released under Apache License 2.0. Its published files total 114.6 GB.

Stage 5 of the Morpho-180B pipeline (display name; SFT backbone: - Per-expert DPO, same 64 experts, same one-example-one-expert mapping as SFT. - Real preference data only (HH-RLHF, HelpSteer2, UltraFeedback, Nectar), routed per expert by the v5 taxonomy.

Parameters—
Context—
Weights114.5 GB
Licenseapache-2.0
AccessOpen weights
Monthly Downloads—

Model Card

By Blue Morpho Limited, published under apache-2.0, revision 10c17f75c0e4.

Stage 5 of the Morpho-180B pipeline (display name; SFT backbone: - Per-expert DPO, same 64 experts, same one-example-one-expert mapping as SFT. - Real preference data only (HH-RLHF, HelpSteer2, UltraFeedback, Nectar), routed per expert by the v5 taxonomy. - DPO adapters initialize from the converged SFT adapters — never from scratch. Stage 5 IN PROGRESS — 17/31 experts DONE (resumed 3x, AUTOSKIP active, final leg running on 97GB GPU box). 24 eligible experts remain in the queue; 33 thin experts (<200 pairs) keep SFT-only per policy. Each row is verified from the expert's bestloss.txt + DONE.txt on this repo. Progress tracked on the model card.

Read Blue Morpho Limited's full model card

Morpho-180B DPO — preference alignment artifacts

Stage 5 of the Morpho-180B pipeline (display name; SFT backbone: Morpho-72B-MoE-Checkpoints).

Contract

  • Per-expert DPO, same 64 experts, same one-example-one-expert mapping as SFT.
  • Same convergence bar: patience-gated training, best-only weights, DONE markers.
  • Real preference data only (HH-RLHF, HelpSteer2, UltraFeedback, Nectar), routed per expert by the v5 taxonomy.
  • DPO adapters initialize from the converged SFT adapters — never from scratch.

Layout (populated as the stage runs)

dpo/raw/        # preference pairs per source (chosen/rejected JSONL)
dpo/tagged/     # per-expert routed pairs + expert_id/domain audit fields
dpo/tokenized/  # tokenized chosen/rejected batches (dpo_batch_*.npy)
dpo/adapters/   # per-expert DPO adapters (sft/expert_XX pattern)
manifest.json   # per-expert pair counts + DPO losses

Status

Stage 5 IN PROGRESS — 17/31 experts DONE (resumed 3x, AUTO_SKIP active, final leg running on 97GB GPU box).

Expert Domain DPO best loss Converged step Weights
02 Banking & Lending 1.01e-06 673 DONE + weights
04 Accounting & Auditing 7.83e-08 317 DONE + weights
05 Taxation & Compliance 5.73e-04 979 DONE + weights
06 Real Estate Finance 0.0176 400 DONE + weights
09 Criminal Law 0.0176 191 DONE + weights
10 Civil Law 5.33e-06 587 DONE + weights
11 Corporate Law 0.00105 525 DONE + weights
12 Intellectual Property 1.38e-06 1695 DONE + weights
13 International Law 6.38e-09 759 DONE + weights
14 Tax Law 2.17e-10 1050 DONE + weights
15 Regulatory Compliance 8.82e-04 607 DONE + weights
17 Pharmacology 7.50e-16 739 DONE + weights
18 Diagnostics & Imaging 3.23e-11 348 DONE + weights
20 Neurology 0.00406 350 DONE + weights
21 Cardiology 9.27e-05 660 DONE + weights
22 Oncology 1.84e-08 1240 DONE + weights
23 Public Health 3.15e-11 660 DONE + weights

24 eligible experts remain in the queue; 33 thin experts (<200 pairs) keep SFT-only per policy. Each row is verified from the expert's best_loss.txt + DONE.txt on this repo.

Progress tracked on the model card.

Identity and Version

Repository
bluemorpholimited/Morpho-72B-MoE-DPO
Publisher
Blue Morpho Limited
Task
Text generation
Modality
Text
Library
Not stated by the source
Parameters
Not stated by the source
Languages
en
Revision
10c17f75c0e4efa72eaec86f95ed2635dd7157ec
First published
2026-10-14
Last updated
2026-10-16

Files and Weights

111 files, 114.6 GB in total. The weights are 17 files totalling 114.5 GB in safetensors.

Weights17 files · 114.5 GB
Configuration18 files · 20.9 KB
Documentation18 files · 90.8 KB
Other57 files · 37.5 MB
Repository1 file · 1.5 KB
Every file
FileTypeSizeSHA-256
dpo/adapters/expert_02/best/adapter_model.safetensorsWeights6.7 GB 7a1f4a1d18db
dpo/adapters/expert_04/best/adapter_model.safetensorsWeights6.7 GB a3e8d50b0af2
dpo/adapters/expert_05/best/adapter_model.safetensorsWeights6.7 GB d85a71e9a965
dpo/adapters/expert_06/best/adapter_model.safetensorsWeights6.7 GB 02ed1b37acfe
dpo/adapters/expert_09/best/adapter_model.safetensorsWeights6.7 GB 790257393a0b
dpo/adapters/expert_10/best/adapter_model.safetensorsWeights6.7 GB 6aeff351f1b8
dpo/adapters/expert_11/best/adapter_model.safetensorsWeights6.7 GB 35328f0e2e51
dpo/adapters/expert_12/best/adapter_model.safetensorsWeights6.7 GB 4df303a01b11
dpo/adapters/expert_13/best/adapter_model.safetensorsWeights6.7 GB f281dc08a25e
dpo/adapters/expert_14/best/adapter_model.safetensorsWeights6.7 GB 73994e7486a6
dpo/adapters/expert_15/best/adapter_model.safetensorsWeights6.7 GB 15b4b4094d2a
dpo/adapters/expert_17/best/adapter_model.safetensorsWeights6.7 GB aea6c30872af
dpo/adapters/expert_18/best/adapter_model.safetensorsWeights6.7 GB 94e268b5da40
dpo/adapters/expert_20/best/adapter_model.safetensorsWeights6.7 GB 1249f6b47c17
dpo/adapters/expert_21/best/adapter_model.safetensorsWeights6.7 GB 84a20d2d35f6
dpo/adapters/expert_22/best/adapter_model.safetensorsWeights6.7 GB 83122d79fada
dpo/adapters/expert_23/best/adapter_model.safetensorsWeights6.7 GB 799a886c1f1a
dpo/adapters/expert_02/best/adapter_config.jsonConfiguration1.2 KB —
dpo/adapters/expert_04/best/adapter_config.jsonConfiguration1.2 KB —
dpo/adapters/expert_05/best/adapter_config.jsonConfiguration1.2 KB —
dpo/adapters/expert_06/best/adapter_config.jsonConfiguration1.2 KB —
dpo/adapters/expert_09/best/adapter_config.jsonConfiguration1.2 KB —
dpo/adapters/expert_10/best/adapter_config.jsonConfiguration1.2 KB —
dpo/adapters/expert_11/best/adapter_config.jsonConfiguration1.2 KB —
dpo/adapters/expert_12/best/adapter_config.jsonConfiguration1.2 KB —
dpo/adapters/expert_13/best/adapter_config.jsonConfiguration1.2 KB —
dpo/adapters/expert_14/best/adapter_config.jsonConfiguration1.2 KB —
dpo/adapters/expert_15/best/adapter_config.jsonConfiguration1.2 KB —
dpo/adapters/expert_17/best/adapter_config.jsonConfiguration1.2 KB —
dpo/adapters/expert_18/best/adapter_config.jsonConfiguration1.2 KB —
dpo/adapters/expert_20/best/adapter_config.jsonConfiguration1.2 KB —
dpo/adapters/expert_21/best/adapter_config.jsonConfiguration1.2 KB —
dpo/adapters/expert_22/best/adapter_config.jsonConfiguration1.2 KB —
dpo/adapters/expert_23/best/adapter_config.jsonConfiguration1.2 KB —
manifest.jsonConfiguration984 B —
README.mdDocumentation2.7 KB —
dpo/adapters/expert_02/best/README.mdDocumentation5.2 KB —
dpo/adapters/expert_04/best/README.mdDocumentation5.2 KB —
dpo/adapters/expert_05/best/README.mdDocumentation5.2 KB —
dpo/adapters/expert_06/best/README.mdDocumentation5.2 KB —
dpo/adapters/expert_09/best/README.mdDocumentation5.2 KB —
dpo/adapters/expert_10/best/README.mdDocumentation5.2 KB —
dpo/adapters/expert_11/best/README.mdDocumentation5.2 KB —
dpo/adapters/expert_12/best/README.mdDocumentation5.2 KB —
dpo/adapters/expert_13/best/README.mdDocumentation5.2 KB —
dpo/adapters/expert_14/best/README.mdDocumentation5.2 KB —
dpo/adapters/expert_15/best/README.mdDocumentation5.2 KB —
dpo/adapters/expert_17/best/README.mdDocumentation5.2 KB —
dpo/adapters/expert_18/best/README.mdDocumentation5.2 KB —
dpo/adapters/expert_20/best/README.mdDocumentation5.2 KB —
dpo/adapters/expert_21/best/README.mdDocumentation5.2 KB —
dpo/adapters/expert_22/best/README.mdDocumentation5.2 KB —
dpo/adapters/expert_23/best/README.mdDocumentation5.2 KB —
dpo/adapters/expert_02/best/DONE.txtOther107 B —
dpo/adapters/expert_02/best/best_loss.txtOther22 B —
dpo/adapters/expert_02/best/step.txtOther3 B —
dpo/adapters/expert_04/best/DONE.txtOther106 B —
dpo/adapters/expert_04/best/best_loss.txtOther21 B —
dpo/adapters/expert_04/best/step.txtOther3 B —
dpo/adapters/expert_05/best/DONE.txtOther106 B —
dpo/adapters/expert_05/best/best_loss.txtOther21 B —
dpo/adapters/expert_05/best/step.txtOther3 B —
dpo/adapters/expert_06/best/DONE.txtOther105 B —
dpo/adapters/expert_06/best/best_loss.txtOther20 B —
dpo/adapters/expert_06/best/step.txtOther3 B —
dpo/adapters/expert_09/best/DONE.txtOther105 B —
dpo/adapters/expert_09/best/best_loss.txtOther20 B —
dpo/adapters/expert_09/best/step.txtOther3 B —
dpo/adapters/expert_10/best/DONE.txtOther106 B —
dpo/adapters/expert_10/best/best_loss.txtOther21 B —
dpo/adapters/expert_10/best/step.txtOther3 B —
dpo/adapters/expert_11/best/DONE.txtOther106 B —
dpo/adapters/expert_11/best/best_loss.txtOther21 B —
dpo/adapters/expert_11/best/step.txtOther3 B —
dpo/adapters/expert_12/best/DONE.txtOther108 B —
dpo/adapters/expert_12/best/best_loss.txtOther22 B —
dpo/adapters/expert_12/best/step.txtOther4 B —
dpo/adapters/expert_13/best/DONE.txtOther105 B —
dpo/adapters/expert_13/best/best_loss.txtOther20 B —
dpo/adapters/expert_13/best/step.txtOther3 B —
dpo/adapters/expert_14/best/DONE.txtOther108 B —
dpo/adapters/expert_14/best/best_loss.txtOther22 B —
dpo/adapters/expert_14/best/step.txtOther4 B —
dpo/adapters/expert_15/best/DONE.txtOther105 B —
dpo/adapters/expert_15/best/best_loss.txtOther20 B —
dpo/adapters/expert_15/best/step.txtOther3 B —
dpo/adapters/expert_17/best/DONE.txtOther105 B —
dpo/adapters/expert_17/best/best_loss.txtOther20 B —
dpo/adapters/expert_17/best/step.txtOther3 B —
dpo/adapters/expert_18/best/DONE.txtOther106 B —
dpo/adapters/expert_18/best/best_loss.txtOther21 B —
dpo/adapters/expert_18/best/step.txtOther3 B —
dpo/adapters/expert_20/best/DONE.txtOther106 B —
dpo/adapters/expert_20/best/best_loss.txtOther21 B —
dpo/adapters/expert_20/best/step.txtOther3 B —
dpo/adapters/expert_21/best/DONE.txtOther106 B —
dpo/adapters/expert_21/best/best_loss.txtOther21 B —
dpo/adapters/expert_21/best/step.txtOther3 B —
dpo/adapters/expert_22/best/DONE.txtOther108 B —
dpo/adapters/expert_22/best/best_loss.txtOther22 B —
dpo/adapters/expert_22/best/step.txtOther4 B —
dpo/adapters/expert_23/best/DONE.txtOther106 B —
dpo/adapters/expert_23/best/best_loss.txtOther21 B —
dpo/adapters/expert_23/best/step.txtOther3 B —
dpo/tokenized/dpo_routed_0.npyOther8.0 MB 5c4fb027b6c2
dpo/tokenized/dpo_routed_1.npyOther7.7 MB 008abde2f2c0
dpo/tokenized/dpo_routed_2.npyOther6.3 MB ee208e69889c
dpo/tokenized/dpo_routed_3.npyOther5.6 MB 527f5c1c23c5
dpo/tokenized/dpo_routed_4.npyOther6.6 MB cb14d3c715ea
dpo/tokenized/dpo_routed_5.npyOther3.3 MB 031ba60b7bc0
.gitattributesRepository1.5 KB —

License and Download

License
apache-2.0
Access
Open weights, no gate
Download size
114.5 GB
Download from Blue Morpho Limited

Released by Blue Morpho Limited through its official repository on Hugging Face. Read the license.

Memory Requirements

PrecisionWeights in memory
As published114.5 GB

Weights only, from the published parameter count; the key-value cache and runtime add to this.

Questions About Morpho-72B-MoE-DPO

Can I use Morpho-72B-MoE-DPO commercially?

Yes. Morpho-72B-MoE-DPO is released under Apache License 2.0. The Apache License 2.0 is a permissive open-source license. It permits commercial use, modification and redistribution. It requires keeping the license and copyright notices and any NOTICE file, stating significant changes, and it includes an express patent grant from contributors.

Similar Models

Model · Text generation

opt-125m

AI at Meta

OPT was first introduced in Open Pre-trained Transformer Language Models and first released in metaseq's repository on May 3rd 2022 by Meta AI. Disclaimer: The team releasing OPT wrote an official model card, which is available in Appendix D of the paper. Content from this model card has been written by the Hugging Face team. To quote the first two paragraphs of the official paper OPT was predominantly pretrained with English text, but a small amount of non-English data is still present within the training corpus via CommonCrawl. The model was pretrained using a causal language modeling (CLM) objective. OPT belongs to the same family of decoder-only models like GPT-3. As such, it was…

Open weights other 2,048 tokens transformers

Model · Text generation

Ornith-1.5-9B-GGUF

Ornith

Chirp Chirp! We are introducing Ornith-1.5, a major step toward building foundation models through end-to-end self-improvement. Ornith-1.5 extends Ornith-1.0 (which was developed on top of Qwen3.5 and Gemma4 with additional continued pretraining, mid-training, and post-training) by expanding the self-improvement loop from scaffold and rollout optimization to jointly optimizing task generation, scaffold construction, and solution rollouts. Rather than relying on a fixed set of human-curated tasks and manually designed harnesses, Ornith-1.5 continuously generates new training tasks, discovers effective strategies for solving them, and improves the policy through reinforcement learning. For…

Open weights mit transformers

Model · Text generation

Ornith-1.5-35B-A3B-GGUF

Ornith

Chirp Chirp! We are introducing Ornith-1.5, a major step toward building foundation models through end-to-end self-improvement. Ornith-1.5 extends Ornith-1.0 (which was developed on top of Qwen3.5 and Gemma4 with additional continued pretraining, mid-training, and post-training) by expanding the self-improvement loop from scaffold and rollout optimization to jointly optimizing task generation, scaffold construction, and solution rollouts. Rather than relying on a fixed set of human-curated tasks and manually designed harnesses, Ornith-1.5 continuously generates new training tasks, discovers effective strategies for solving them, and improves the policy through reinforcement learning. For…

Open weights mit transformers

Model · Text generation

Ternary-Bonsai-2-27B-gguf

Prism ML

Full 27B-class reasoning in ternary transformer weights, for llama.cpp (CUDA, Metal, CPU) - \~5.9 GB language model (down from \~54 GB FP16) — full 27B-class reasoning on a standard laptop or a single GPU - 98.2% of FP16 intelligence retained: 84.78 average across 14 thinking-mode benchmarks — far above the conventional IQ2XXS build (72.59) at about 82% of its footprint, and within 0.4 points of UD-Q4KXL at three times the footprint - Retains thinking, reasoning, and agentic behavior deep in the sub-4-bit regime, where conventional low-bit representations collapse: math within half a point of full precision (96.57), coding level with the baseline (89.42), agentic tool calling at 74.92…

Open weights apache-2.0 llama.cpp

Uncensored Qwen3.8-27B, published as GGUF quantizations with the multi token prediction (MTP) head retained and verified. Refusal behaviour has been substantially reduced, not eliminated. See Measured behaviour for the numbers. Capabilities, training data, and architecture are otherwise unchanged. - Refusal directions removed with Heretic, which co minimizes refusal count against KL divergence from the base model. No handwritten refusal removal code, no finetuning, no additional training data. - Abliteration runs at bf16 (no 4 bit quantization). the resulting LoRA is merged into the bf16 base, so the published weights are not a quantized round trip. - mtp. tensors are copied verbatim from…

Open weights apache-2.0 llama.cpp

Model · Text generation

Ornith-1.0-9B-GGUF

Ornith

Aloha! Today, we are releasing Ornith-1.0, a self-improving family of open-source models for agentic coding. This model card documents Ornith-1.0-9B, the most lightweight member of the Ornith family, designed for efficient single-GPU deployment. Ornith-1.0-9B is a dense ~9B model (≈19 GB in bf16), so it serves comfortably on a single 80GB GPU. The recipes below stand up an OpenAI-compatible server; add --tensor-parallel-size / --tp if you want to shard across more GPUs. For a quick local test (or to script offline generation), load the model directly with Transformers. Make sure you have a recent release installed — see the Transformers installation guide; Ornith-1.0-9B requires…

Open weights mit transformers