Skip to page content
SAVRN
Search Plan a project

Open-weight model

olmo3-calibration-models

by Edwin Huang ehuang5/olmo3-calibration-models

olmo3-calibration-models is an open-weight model from Edwin Huang, released under Apache License 2.0. Its published files total 584.1 GB.

DPO checkpoints from a study of why DPO makes Olmo 3 7B Instruct overconfident on multiple-choice questions. Every model here is trained from allenai/Olmo-3-7B-Instruct-SFT on Ai2's Dolci-Instruct-DPO preference pairs with Ai2's open-instruct code.

Parameters—
Context—
Weights583.8 GB
Licenseapache-2.0
AccessOpen weights
Monthly Downloads—

Model Card

By Edwin Huang, published under apache-2.0, revision 232ae143c109.

DPO checkpoints from a study of why DPO makes Olmo 3 7B Instruct overconfident on multiple-choice questions. Every model here is trained from allenai/Olmo-3-7B-Instruct-SFT on Ai2's Dolci-Instruct-DPO preference pairs with Ai2's open-instruct code. - Code, results and the write-up ("Why Does DPO Make Olmo 3 Overconfident?"): These are research checkpoints for reproducing the study's measurements. They were not tuned or safety-tested for use. "Agreement-weighted" DPO multiplies each pair's loss by sigmoid(-5 x the SFT model's preference margin), rescaled to average 1, so pairs the SFT model already gets right count less. Each folder is a full model (model.safetensors, config, tokenizer, chat…

Read Edwin Huang's full model card

Olmo 3 7B DPO calibration study: trained checkpoints

DPO checkpoints from a study of why DPO makes Olmo 3 7B Instruct overconfident on multiple-choice questions. Every model here is trained from allenai/Olmo-3-7B-Instruct-SFT on Ai2's Dolci-Instruct-DPO preference pairs with Ai2's open-instruct code.

These are research checkpoints for reproducing the study's measurements. They were not tuned or safety-tested for use. "Agreement-weighted" DPO multiplies each pair's loss by sigmoid(-5 x the SFT model's preference margin), rescaled to average 1, so pairs the SFT model already gets right count less.

Folders

Each folder is a full model (model.safetensors, config, tokenizer, chat template). Load one with subfolder=, e.g. AutoModelForCausalLM.from_pretrained("ehuang5/olmo3-calibration-models", subfolder="q25_weighted").

Folder Training
q25_plain/ Plain DPO on 25% of the Instruct DPO pairs (65,000), Ai2's recipe (LR 1e-6, 508 steps)
q25_weighted/ Agreement-weighted DPO, same data and recipe
full_weighted/final/ Agreement-weighted DPO on all 259,922 pairs, Ai2's recipe (2,031 steps); compare with Ai2's released Olmo-3-7B-Instruct-DPO
full_weighted/snap_step_*/ Snapshots of that run at steps 406, 812, 1218, 1624 and 2030
wave/wy_*_s{1,2,3}/ Small tests from the pre-registered replication: 8,192 pairs, LR 4e-6, 64 steps, seeds 1-3

Small tests in wave/:

Name Loss Training set
wy_plain Plain DPO Mixed pairs
wy_w5 Agreement-weighted DPO Mixed pairs
wy_simpo SimPO (no reference model) Mixed pairs
wy_simporef SimPO with DPO's SFT reference added Mixed pairs
wy_hi / wy_lo Plain DPO 99% / 18% of pairs already right under SFT, matched on source and length
wy_ipohi / wy_ipolo IPO (tau = 1) Same two sets
wy_easy Plain DPO "Easy" pairs (high margin under a DPO-trained model)
wy_w5hi Agreement-weighted DPO The 99% set (follow-up 1)
wy_hiall / wy_w5all Plain / agreement-weighted DPO The 99% set minus its disagreed pairs (follow-up 2; seed 1 only)

License

Apache-2.0, following the base model. The training data is Ai2's Dolci preference data (ODC-BY); please credit Ai2's Olmo 3 release and the Dolci datasets when using these models.

Identity and Version

Repository
ehuang5/olmo3-calibration-models
Publisher
Edwin Huang
Task
Not stated by the source
Modality
Other
Library
Not stated by the source
Parameters
Not stated by the source
Languages
dpo
Revision
232ae143c10930e9a5d5a28b64c3af8a227ba42c
First published
2026-10-13
Last updated
2026-10-16

Files and Weights

242 files, 584.1 GB in total. The weights are 40 files totalling 583.8 GB in safetensors.

Weights40 files · 583.8 GB
Configuration80 files · 79.6 KB
Tokenizer80 files · 285.5 MB
Documentation1 file · 3.0 KB
Other40 files · 105.5 KB
Repository1 file · 1.5 KB
Every file
FileTypeSizeSHA-256
full_weighted/final/model.safetensorsWeights14.6 GB 372b16ef81ce
full_weighted/snap_step_1218/model.safetensorsWeights14.6 GB 0d76348973be
full_weighted/snap_step_1624/model.safetensorsWeights14.6 GB 573de6b39f2a
full_weighted/snap_step_2030/model.safetensorsWeights14.6 GB 070a3aac3e4a
full_weighted/snap_step_406/model.safetensorsWeights14.6 GB 8de156143886
full_weighted/snap_step_812/model.safetensorsWeights14.6 GB a04c6de55a62
q25_plain/model.safetensorsWeights14.6 GB 954ce7a7f10e
q25_weighted/model.safetensorsWeights14.6 GB 96857a29f792
wave/wy_easy_s1/model.safetensorsWeights14.6 GB d619340d90ad
wave/wy_easy_s2/model.safetensorsWeights14.6 GB fac6921397f6
wave/wy_easy_s3/model.safetensorsWeights14.6 GB 6905eccde0fb
wave/wy_hi_s1/model.safetensorsWeights14.6 GB 802a85aaa749
wave/wy_hi_s2/model.safetensorsWeights14.6 GB 0eb82f530000
wave/wy_hi_s3/model.safetensorsWeights14.6 GB a3afca9c9832
wave/wy_hiall_s1/model.safetensorsWeights14.6 GB 21ebf9beeab2
wave/wy_ipohi_s1/model.safetensorsWeights14.6 GB 3b5600f77b9a
wave/wy_ipohi_s2/model.safetensorsWeights14.6 GB 57ec8f8a26b8
wave/wy_ipohi_s3/model.safetensorsWeights14.6 GB fe7ba2a5886f
wave/wy_ipolo_s1/model.safetensorsWeights14.6 GB cebc660954e0
wave/wy_ipolo_s2/model.safetensorsWeights14.6 GB 76b7d68f4213
wave/wy_ipolo_s3/model.safetensorsWeights14.6 GB cbf3dfc47e76
wave/wy_lo_s1/model.safetensorsWeights14.6 GB b5e126b3462f
wave/wy_lo_s2/model.safetensorsWeights14.6 GB 5102fc3aeccb
wave/wy_lo_s3/model.safetensorsWeights14.6 GB 221810144c3d
wave/wy_plain_s1/model.safetensorsWeights14.6 GB 21bedb942f08
wave/wy_plain_s2/model.safetensorsWeights14.6 GB c29582c43502
wave/wy_plain_s3/model.safetensorsWeights14.6 GB 978329a589fb
wave/wy_simpo_s1/model.safetensorsWeights14.6 GB 1cedffa0c5d9
wave/wy_simpo_s2/model.safetensorsWeights14.6 GB 8dc62e4f2028
wave/wy_simpo_s3/model.safetensorsWeights14.6 GB 64795dfce12d
wave/wy_simporef_s1/model.safetensorsWeights14.6 GB 84279d1a0719
wave/wy_simporef_s2/model.safetensorsWeights14.6 GB 36aca325addb
wave/wy_simporef_s3/model.safetensorsWeights14.6 GB 11abb6826dc0
wave/wy_w5_s1/model.safetensorsWeights14.6 GB 756e82398656
wave/wy_w5_s2/model.safetensorsWeights14.6 GB e121399d6308
wave/wy_w5_s3/model.safetensorsWeights14.6 GB 24252301d244
wave/wy_w5all_s1/model.safetensorsWeights14.6 GB 801dc2ddf99b
wave/wy_w5hi_s1/model.safetensorsWeights14.6 GB da15862a61ca
wave/wy_w5hi_s2/model.safetensorsWeights14.6 GB 0b17048d6d39
wave/wy_w5hi_s3/model.safetensorsWeights14.6 GB 3d94d3efa881
full_weighted/final/config.jsonConfiguration1.8 KB —
full_weighted/final/generation_config.jsonConfiguration204 B —
full_weighted/snap_step_1218/config.jsonConfiguration1.8 KB —
full_weighted/snap_step_1218/generation_config.jsonConfiguration204 B —
full_weighted/snap_step_1624/config.jsonConfiguration1.8 KB —
full_weighted/snap_step_1624/generation_config.jsonConfiguration204 B —
full_weighted/snap_step_2030/config.jsonConfiguration1.8 KB —
full_weighted/snap_step_2030/generation_config.jsonConfiguration204 B —
full_weighted/snap_step_406/config.jsonConfiguration1.8 KB —
full_weighted/snap_step_406/generation_config.jsonConfiguration204 B —
full_weighted/snap_step_812/config.jsonConfiguration1.8 KB —
full_weighted/snap_step_812/generation_config.jsonConfiguration204 B —
q25_plain/config.jsonConfiguration1.8 KB —
q25_plain/generation_config.jsonConfiguration204 B —
q25_weighted/config.jsonConfiguration1.8 KB —
q25_weighted/generation_config.jsonConfiguration204 B —
wave/wy_easy_s1/config.jsonConfiguration1.8 KB —
wave/wy_easy_s1/generation_config.jsonConfiguration204 B —
wave/wy_easy_s2/config.jsonConfiguration1.8 KB —
wave/wy_easy_s2/generation_config.jsonConfiguration204 B —
wave/wy_easy_s3/config.jsonConfiguration1.8 KB —
wave/wy_easy_s3/generation_config.jsonConfiguration204 B —
wave/wy_hi_s1/config.jsonConfiguration1.8 KB —
wave/wy_hi_s1/generation_config.jsonConfiguration204 B —
wave/wy_hi_s2/config.jsonConfiguration1.8 KB —
wave/wy_hi_s2/generation_config.jsonConfiguration204 B —
wave/wy_hi_s3/config.jsonConfiguration1.8 KB —
wave/wy_hi_s3/generation_config.jsonConfiguration204 B —
wave/wy_hiall_s1/config.jsonConfiguration1.8 KB —
wave/wy_hiall_s1/generation_config.jsonConfiguration204 B —
wave/wy_ipohi_s1/config.jsonConfiguration1.8 KB —
wave/wy_ipohi_s1/generation_config.jsonConfiguration204 B —
wave/wy_ipohi_s2/config.jsonConfiguration1.8 KB —
wave/wy_ipohi_s2/generation_config.jsonConfiguration204 B —
wave/wy_ipohi_s3/config.jsonConfiguration1.8 KB —
wave/wy_ipohi_s3/generation_config.jsonConfiguration204 B —
wave/wy_ipolo_s1/config.jsonConfiguration1.8 KB —
wave/wy_ipolo_s1/generation_config.jsonConfiguration204 B —
wave/wy_ipolo_s2/config.jsonConfiguration1.8 KB —
wave/wy_ipolo_s2/generation_config.jsonConfiguration204 B —
wave/wy_ipolo_s3/config.jsonConfiguration1.8 KB —
wave/wy_ipolo_s3/generation_config.jsonConfiguration204 B —
wave/wy_lo_s1/config.jsonConfiguration1.8 KB —
wave/wy_lo_s1/generation_config.jsonConfiguration204 B —
wave/wy_lo_s2/config.jsonConfiguration1.8 KB —
wave/wy_lo_s2/generation_config.jsonConfiguration204 B —
wave/wy_lo_s3/config.jsonConfiguration1.8 KB —
wave/wy_lo_s3/generation_config.jsonConfiguration204 B —
wave/wy_plain_s1/config.jsonConfiguration1.8 KB —
wave/wy_plain_s1/generation_config.jsonConfiguration204 B —
wave/wy_plain_s2/config.jsonConfiguration1.8 KB —
wave/wy_plain_s2/generation_config.jsonConfiguration204 B —
wave/wy_plain_s3/config.jsonConfiguration1.8 KB —
wave/wy_plain_s3/generation_config.jsonConfiguration204 B —
wave/wy_simpo_s1/config.jsonConfiguration1.8 KB —
wave/wy_simpo_s1/generation_config.jsonConfiguration204 B —
wave/wy_simpo_s2/config.jsonConfiguration1.8 KB —
wave/wy_simpo_s2/generation_config.jsonConfiguration204 B —
wave/wy_simpo_s3/config.jsonConfiguration1.8 KB —
wave/wy_simpo_s3/generation_config.jsonConfiguration204 B —
wave/wy_simporef_s1/config.jsonConfiguration1.8 KB —
wave/wy_simporef_s1/generation_config.jsonConfiguration204 B —
wave/wy_simporef_s2/config.jsonConfiguration1.8 KB —
wave/wy_simporef_s2/generation_config.jsonConfiguration204 B —
wave/wy_simporef_s3/config.jsonConfiguration1.8 KB —
wave/wy_simporef_s3/generation_config.jsonConfiguration204 B —
wave/wy_w5_s1/config.jsonConfiguration1.8 KB —
wave/wy_w5_s1/generation_config.jsonConfiguration204 B —
wave/wy_w5_s2/config.jsonConfiguration1.8 KB —
wave/wy_w5_s2/generation_config.jsonConfiguration204 B —
wave/wy_w5_s3/config.jsonConfiguration1.8 KB —
wave/wy_w5_s3/generation_config.jsonConfiguration204 B —
wave/wy_w5all_s1/config.jsonConfiguration1.8 KB —
wave/wy_w5all_s1/generation_config.jsonConfiguration204 B —
wave/wy_w5hi_s1/config.jsonConfiguration1.8 KB —
wave/wy_w5hi_s1/generation_config.jsonConfiguration204 B —
wave/wy_w5hi_s2/config.jsonConfiguration1.8 KB —
wave/wy_w5hi_s2/generation_config.jsonConfiguration204 B —
wave/wy_w5hi_s3/config.jsonConfiguration1.8 KB —
wave/wy_w5hi_s3/generation_config.jsonConfiguration204 B —
README.mdDocumentation3.0 KB —
full_weighted/final/chat_template.jinjaOther2.6 KB —
full_weighted/snap_step_1218/chat_template.jinjaOther2.6 KB —
full_weighted/snap_step_1624/chat_template.jinjaOther2.6 KB —
full_weighted/snap_step_2030/chat_template.jinjaOther2.6 KB —
full_weighted/snap_step_406/chat_template.jinjaOther2.6 KB —
full_weighted/snap_step_812/chat_template.jinjaOther2.6 KB —
q25_plain/chat_template.jinjaOther2.6 KB —
q25_weighted/chat_template.jinjaOther2.6 KB —
wave/wy_easy_s1/chat_template.jinjaOther2.6 KB —
wave/wy_easy_s2/chat_template.jinjaOther2.6 KB —
wave/wy_easy_s3/chat_template.jinjaOther2.6 KB —
wave/wy_hi_s1/chat_template.jinjaOther2.6 KB —
wave/wy_hi_s2/chat_template.jinjaOther2.6 KB —
wave/wy_hi_s3/chat_template.jinjaOther2.6 KB —
wave/wy_hiall_s1/chat_template.jinjaOther2.6 KB —
wave/wy_ipohi_s1/chat_template.jinjaOther2.6 KB —
wave/wy_ipohi_s2/chat_template.jinjaOther2.6 KB —
wave/wy_ipohi_s3/chat_template.jinjaOther2.6 KB —
wave/wy_ipolo_s1/chat_template.jinjaOther2.6 KB —
wave/wy_ipolo_s2/chat_template.jinjaOther2.6 KB —
wave/wy_ipolo_s3/chat_template.jinjaOther2.6 KB —
wave/wy_lo_s1/chat_template.jinjaOther2.6 KB —
wave/wy_lo_s2/chat_template.jinjaOther2.6 KB —
wave/wy_lo_s3/chat_template.jinjaOther2.6 KB —
wave/wy_plain_s1/chat_template.jinjaOther2.6 KB —
wave/wy_plain_s2/chat_template.jinjaOther2.6 KB —
wave/wy_plain_s3/chat_template.jinjaOther2.6 KB —
wave/wy_simpo_s1/chat_template.jinjaOther2.6 KB —
wave/wy_simpo_s2/chat_template.jinjaOther2.6 KB —
wave/wy_simpo_s3/chat_template.jinjaOther2.6 KB —
wave/wy_simporef_s1/chat_template.jinjaOther2.6 KB —
wave/wy_simporef_s2/chat_template.jinjaOther2.6 KB —
wave/wy_simporef_s3/chat_template.jinjaOther2.6 KB —
wave/wy_w5_s1/chat_template.jinjaOther2.6 KB —
wave/wy_w5_s2/chat_template.jinjaOther2.6 KB —
wave/wy_w5_s3/chat_template.jinjaOther2.6 KB —
wave/wy_w5all_s1/chat_template.jinjaOther2.6 KB —
wave/wy_w5hi_s1/chat_template.jinjaOther2.6 KB —
wave/wy_w5hi_s2/chat_template.jinjaOther2.6 KB —
wave/wy_w5hi_s3/chat_template.jinjaOther2.6 KB —
.gitattributesRepository1.5 KB —
full_weighted/final/tokenizer.jsonTokenizer7.1 MB —
full_weighted/final/tokenizer_config.jsonTokenizer343 B —
full_weighted/snap_step_1218/tokenizer.jsonTokenizer7.1 MB —
full_weighted/snap_step_1218/tokenizer_config.jsonTokenizer343 B —
full_weighted/snap_step_1624/tokenizer.jsonTokenizer7.1 MB —
full_weighted/snap_step_1624/tokenizer_config.jsonTokenizer343 B —
full_weighted/snap_step_2030/tokenizer.jsonTokenizer7.1 MB —
full_weighted/snap_step_2030/tokenizer_config.jsonTokenizer343 B —
full_weighted/snap_step_406/tokenizer.jsonTokenizer7.1 MB —
full_weighted/snap_step_406/tokenizer_config.jsonTokenizer343 B —
full_weighted/snap_step_812/tokenizer.jsonTokenizer7.1 MB —
full_weighted/snap_step_812/tokenizer_config.jsonTokenizer343 B —
q25_plain/tokenizer.jsonTokenizer7.1 MB —
q25_plain/tokenizer_config.jsonTokenizer343 B —
q25_weighted/tokenizer.jsonTokenizer7.1 MB —
q25_weighted/tokenizer_config.jsonTokenizer343 B —
wave/wy_easy_s1/tokenizer.jsonTokenizer7.1 MB —
wave/wy_easy_s1/tokenizer_config.jsonTokenizer343 B —
wave/wy_easy_s2/tokenizer.jsonTokenizer7.1 MB —
wave/wy_easy_s2/tokenizer_config.jsonTokenizer343 B —
wave/wy_easy_s3/tokenizer.jsonTokenizer7.1 MB —
wave/wy_easy_s3/tokenizer_config.jsonTokenizer343 B —
wave/wy_hi_s1/tokenizer.jsonTokenizer7.1 MB —
wave/wy_hi_s1/tokenizer_config.jsonTokenizer343 B —
wave/wy_hi_s2/tokenizer.jsonTokenizer7.1 MB —
wave/wy_hi_s2/tokenizer_config.jsonTokenizer343 B —
wave/wy_hi_s3/tokenizer.jsonTokenizer7.1 MB —
wave/wy_hi_s3/tokenizer_config.jsonTokenizer343 B —
wave/wy_hiall_s1/tokenizer.jsonTokenizer7.1 MB —
wave/wy_hiall_s1/tokenizer_config.jsonTokenizer343 B —
wave/wy_ipohi_s1/tokenizer.jsonTokenizer7.1 MB —
wave/wy_ipohi_s1/tokenizer_config.jsonTokenizer343 B —
wave/wy_ipohi_s2/tokenizer.jsonTokenizer7.1 MB —
wave/wy_ipohi_s2/tokenizer_config.jsonTokenizer343 B —
wave/wy_ipohi_s3/tokenizer.jsonTokenizer7.1 MB —
wave/wy_ipohi_s3/tokenizer_config.jsonTokenizer343 B —
wave/wy_ipolo_s1/tokenizer.jsonTokenizer7.1 MB —
wave/wy_ipolo_s1/tokenizer_config.jsonTokenizer343 B —
wave/wy_ipolo_s2/tokenizer.jsonTokenizer7.1 MB —
wave/wy_ipolo_s2/tokenizer_config.jsonTokenizer343 B —
wave/wy_ipolo_s3/tokenizer.jsonTokenizer7.1 MB —
wave/wy_ipolo_s3/tokenizer_config.jsonTokenizer343 B —
wave/wy_lo_s1/tokenizer.jsonTokenizer7.1 MB —
wave/wy_lo_s1/tokenizer_config.jsonTokenizer343 B —
wave/wy_lo_s2/tokenizer.jsonTokenizer7.1 MB —
wave/wy_lo_s2/tokenizer_config.jsonTokenizer343 B —
wave/wy_lo_s3/tokenizer.jsonTokenizer7.1 MB —
wave/wy_lo_s3/tokenizer_config.jsonTokenizer343 B —
wave/wy_plain_s1/tokenizer.jsonTokenizer7.1 MB —
wave/wy_plain_s1/tokenizer_config.jsonTokenizer343 B —
wave/wy_plain_s2/tokenizer.jsonTokenizer7.1 MB —
wave/wy_plain_s2/tokenizer_config.jsonTokenizer343 B —
wave/wy_plain_s3/tokenizer.jsonTokenizer7.1 MB —
wave/wy_plain_s3/tokenizer_config.jsonTokenizer343 B —
wave/wy_simpo_s1/tokenizer.jsonTokenizer7.1 MB —
wave/wy_simpo_s1/tokenizer_config.jsonTokenizer343 B —
wave/wy_simpo_s2/tokenizer.jsonTokenizer7.1 MB —
wave/wy_simpo_s2/tokenizer_config.jsonTokenizer343 B —
wave/wy_simpo_s3/tokenizer.jsonTokenizer7.1 MB —
wave/wy_simpo_s3/tokenizer_config.jsonTokenizer343 B —
wave/wy_simporef_s1/tokenizer.jsonTokenizer7.1 MB —
wave/wy_simporef_s1/tokenizer_config.jsonTokenizer343 B —
wave/wy_simporef_s2/tokenizer.jsonTokenizer7.1 MB —
wave/wy_simporef_s2/tokenizer_config.jsonTokenizer343 B —
wave/wy_simporef_s3/tokenizer.jsonTokenizer7.1 MB —
wave/wy_simporef_s3/tokenizer_config.jsonTokenizer343 B —
wave/wy_w5_s1/tokenizer.jsonTokenizer7.1 MB —
wave/wy_w5_s1/tokenizer_config.jsonTokenizer343 B —
wave/wy_w5_s2/tokenizer.jsonTokenizer7.1 MB —
wave/wy_w5_s2/tokenizer_config.jsonTokenizer343 B —
wave/wy_w5_s3/tokenizer.jsonTokenizer7.1 MB —
wave/wy_w5_s3/tokenizer_config.jsonTokenizer343 B —
wave/wy_w5all_s1/tokenizer.jsonTokenizer7.1 MB —
wave/wy_w5all_s1/tokenizer_config.jsonTokenizer343 B —
wave/wy_w5hi_s1/tokenizer.jsonTokenizer7.1 MB —
wave/wy_w5hi_s1/tokenizer_config.jsonTokenizer343 B —
wave/wy_w5hi_s2/tokenizer.jsonTokenizer7.1 MB —
wave/wy_w5hi_s2/tokenizer_config.jsonTokenizer343 B —
wave/wy_w5hi_s3/tokenizer.jsonTokenizer7.1 MB —
wave/wy_w5hi_s3/tokenizer_config.jsonTokenizer343 B —

License and Download

License
apache-2.0
Access
Open weights, no gate
Download size
583.8 GB
Download from Edwin Huang

Released by Edwin Huang through its official repository on Hugging Face. Read the license.

Built From

  • Derived from allenai/Olmo-3-7B-Instruct-SFT
  • Trained on (disclosed) allenai/Dolci-Instruct-DPO

Memory Requirements

PrecisionWeights in memory
As published583.8 GB

Weights only, from the published parameter count; the key-value cache and runtime add to this.

Questions About olmo3-calibration-models

Can I use olmo3-calibration-models commercially?

Yes. olmo3-calibration-models is released under Apache License 2.0. The Apache License 2.0 is a permissive open-source license. It permits commercial use, modification and redistribution. It requires keeping the license and copyright notices and any NOTICE file, stating significant changes, and it includes an express patent grant from contributors.