Open-weight model
vigyan-olmoe-1b-7b-dpo-masterpiece
by Shreyansh singh shreyansh12183/vigyan-olmoe-1b-7b-dpo-masterpiece
vigyan-olmoe-1b-7b-dpo-masterpiece is an open-weight model from Shreyansh singh, released under Creative Commons Attribution-NonCommercial 4.0. It has 6.9B parameters and a 4,096-token context. At 16-bit it needs about 16.6 GB of GPU memory, which fits on 1x MI300X from $1.85 an hour, at the lowest prices in the SAVRN Index.
Sovereign Direct Preference Optimization (DPO) Aligned MoE Foundation License: Creative Commons Attribution-NonCommercial 4.0 International (CC BY-NC 4.0) (Strictly Private ) Base Architecture: allenai/OLMoE-1B-7B-0924-Instruct (Sparse 64-Expert Top-8…
Runs On
What it takes to serve vigyan-olmoe-1b-7b-dpo-masterpiece (6.9B parameters): the memory its weights need at each precision, and the cheapest way to rent enough data-center GPUs to hold them.
| Precision | Weights | Memory needed | Cheapest setup | Per hour | Also fits |
|---|---|---|---|---|---|
| 16-bit | 13.8 GB | 16.6 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
| 8-bit | 6.9 GB | 8.3 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
| 4-bit | 3.5 GB | 4.2 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
Memory is the weights at that precision plus 20% for the runtime and a short context; a long context needs more. Prices are the lowest on-demand hourly rates in the SAVRN Index, read Oct 18, 2026.
vigyan-olmoe-1b-7b-dpo-masterpiece on every accelerator the SAVRN Index prices, at every precision
Model Card
Sovereign Direct Preference Optimization (DPO) Aligned MoE Foundation License: Creative Commons Attribution-NonCommercial 4.0 International (CC BY-NC 4.0) (Strictly Private ) Base Architecture: allenai/OLMoE-1B-7B-0924-Instruct (Sparse 64-Expert Top-8 Routing) 1. Contrasting Negative Prompt Gate Calibration: Router weights calibrated across 50,000 domain samples (vigyan-olmoe-master-curriculum-v1) to prevent expert cross-bleeding. 2. Selective Router Precision Guardrail: Router gating matrices (ffngateinp / gateproj) preserved at FP16/Q80 precision to eliminate token routing entropy drift. 3. High-Throughput Edge MoE: Activates only a fraction of parameters per token, delivering 7B-class…
Excerpt from the card by Shreyansh singh, licensed cc-by-nc-4.0.
Configuration
- Architecture
- OlmoeForCausalLM
- Context length (tokens)
- 4,096
- Layers
- 16
- Hidden size
- 2,048
- Feed-forward size
- 1,024
- Attention heads
- 16
- Key/value heads
- 16
- Vocabulary size
- 50,304
- Experts
- 64
- Experts active per token
- 8
- Model type
- olmoe
Identity and Version
- Repository
- shreyansh12183/vigyan-olmoe-1b-7b-dpo-masterpiece
- Publisher
- Shreyansh singh
- Task
- Not stated by the source
- Modality
- Other
- Library
- Not stated by the source
- Parameters
- 6.9B parameters
- Languages
- dpo, moe
- Revision
- b3c50953ebf289cad3e6969ad7d41ea38a4e8697
- First published
- 2026-10-07
- Last updated
- 2026-10-08
Files and Weights
35 files, 18.2 GB in total. The weights are 14 files totalling 18.2 GB in bin, gguf, pt, pth, safetensors.
Every file
| File | Type | Size | SHA-256 |
|---|---|---|---|
| adapter_model.safetensors | Weights | 16.8 MB | ccdf8760afab |
| checkpoint-120/adapter_model.safetensors | Weights | 16.8 MB | ccdf8760afab |
| checkpoint-120/optimizer.pt | Weights | 33.7 MB | 99f13c9d91cd |
| checkpoint-120/ref/adapter_model.safetensors | Weights | 33.6 MB | cf633dead903 |
| checkpoint-120/rng_state.pth | Weights | 14.6 KB | 61c19bab1174 |
| checkpoint-120/scheduler.pt | Weights | 1.5 KB | efad52b65f45 |
| checkpoint-120/training_args.bin | Weights | 5.9 KB | 9fccd2cf92eb |
| model-00001-of-00004.safetensors | Weights | 3.8 GB | 278c860fbd1d |
| model-00002-of-00004.safetensors | Weights | 3.9 GB | 7cb385888b88 |
| model-00003-of-00004.safetensors | Weights | 3.9 GB | 4c4bf135aeb8 |
| model-00004-of-00004.safetensors | Weights | 2.2 GB | cfcbd292c8db |
| ref/adapter_model.safetensors | Weights | 33.6 MB | cf633dead903 |
| training_args.bin | Weights | 5.9 KB | 9fccd2cf92eb |
| vigyan-7b-moe-q4_k_m.gguf | Weights | 4.2 GB | cdc1e6ba1f15 |
| adapter_config.json | Configuration | 1.1 KB | — |
| checkpoint-120/adapter_config.json | Configuration | 1.1 KB | — |
| checkpoint-120/ref/adapter_config.json | Configuration | 1.1 KB | — |
| checkpoint-120/trainer_state.json | Configuration | 8.8 KB | — |
| config.json | Configuration | 829 B | — |
| dpo_unseen_benchmark_results.json | Configuration | 203.7 KB | — |
| generation_config.json | Configuration | 115 B | — |
| model.safetensors.index.json | Configuration | 287.2 KB | — |
| ref/adapter_config.json | Configuration | 1.1 KB | — |
| CERTIFIED_CHIEF_JUDGE_SCORECARD.md | Documentation | 4.9 KB | — |
| README.md | Documentation | 1.5 KB | — |
| checkpoint-120/README.md | Documentation | 5.2 KB | — |
| dpo_unseen_scorecard.md | Documentation | 1.1 KB | — |
| chat_template.jinja | Other | 401 B | — |
| checkpoint-120/chat_template.jinja | Other | 401 B | — |
| colab_demo_vigyan_olmoe_1b_7b.ipynb | Other | 6.5 KB | — |
| .gitattributes | Repository | 1.6 KB | — |
| checkpoint-120/tokenizer.json | Tokenizer | 3.6 MB | — |
| checkpoint-120/tokenizer_config.json | Tokenizer | 401 B | — |
| tokenizer.json | Tokenizer | 3.6 MB | — |
| tokenizer_config.json | Tokenizer | 401 B | — |
License and Download
- License
- cc-by-nc-4.0
- Access
- Open weights, no gate
- Download size
- 18.2 GB
Released by Shreyansh singh through its official repository on Hugging Face. Read the license.
Built From
- Derived from allenai/OLMoE-1B-7B-0924-Instruct
- Quantized from allenai/OLMoE-1B-7B-0924-Instruct
Memory Requirements
| Precision | Weights in memory |
|---|---|
| As published | 18.2 GB |
| 16-bit | 13.8 GB |
| 8-bit | 6.9 GB |
| 4-bit | 3.5 GB |
Weights only, from the published parameter count; the key-value cache and runtime add to this.
Questions About vigyan-olmoe-1b-7b-dpo-masterpiece
How much GPU memory does vigyan-olmoe-1b-7b-dpo-masterpiece need?
About 16.6 GB at 16-bit and 4.2 GB at 4-bit: the weights (6.9B parameters) plus a working margin. A long context needs more.
What is the cheapest GPU to run vigyan-olmoe-1b-7b-dpo-masterpiece on?
At 16-bit, 1x MI300X from $1.85 an hour; at 4-bit, 1x MI300X from $1.85 an hour, at the lowest on-demand prices the SAVRN Index lists.
Can I use vigyan-olmoe-1b-7b-dpo-masterpiece commercially?
Not without separate permission. vigyan-olmoe-1b-7b-dpo-masterpiece is released under Creative Commons Attribution-NonCommercial 4.0. CC BY-NC 4.0 permits sharing and adapting with credit for non-commercial purposes only. Commercial use needs separate permission from the rights holder.
What is vigyan-olmoe-1b-7b-dpo-masterpiece's context length?
4,096 tokens, from the maximum position embeddings in its published configuration.