DPO checkpoints from a study of why DPO makes Olmo 3 7B Instruct overconfident on multiple-choice questions. Every model here is trained from allenai/Olmo-3-7B-Instruct-SFT on Ai2's Dolci-Instruct-DPO preference pairs with Ai2's open-instruct code. - Code, results and the write-up ("Why Does DPO Make Olmo 3 Overconfident?"): These are research checkpoints for reproducing the study's measurements. They were not tuned or safety-tested for use. "Agreement-weighted" DPO multiplies each pair's loss by sigmoid(-5 x the SFT model's preference margin), rescaled to average 1, so pairs the SFT model already gets right count less. Each folder is a full model (model.safetensors, config, tokenizer, chat…
Independent publisher
Edwin Huang
ehuang5
Models in Library1
Datasets in Library0
Models on Hugging Face1
Followers—
Compare the published evidence
This profile connects the 1 model and dataset record shown below to their upstream cards and sources. The library counts describe SAVRN's coverage, rather than this publisher's entire output.
Reported licenses: apache-2.0. Open a record to review its specific license and conditions.
Compare model specifications, source dates, and hardware estimates on the individual records. Publisher identity verification is separate from a model's quality, deployment readiness, or suitability for your work.