Search public pages, research tools, and SAVRN solutions.
Open-weight model · Text generation
by Open Athena open-athena/MarinSkyRL-Open-MOPD-SmolLM3-3B-step-32
This is a MarinSkyRL-native Open-MOPD student after 32 optimizer steps. It starts from the authors' mixed-domain SFT checkpoint. Student responses were scored by the authors' math, code, and instruction-following RL teachers, routed by domain.
What it takes to serve MarinSkyRL-Open-MOPD-SmolLM3-3B-step-32 (3.3B parameters): the memory its weights need at each precision, and the cheapest way to rent enough data-center GPUs to hold them.
| Precision | Weights | Memory needed | Cheapest setup | Per hour | Also fits |
|---|---|---|---|---|---|
| 16-bit | 6.7 GB | 8.0 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
| 8-bit | 3.3 GB | 4.0 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
| 4-bit | 1.7 GB | 2.0 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
Memory is the weights at that precision plus 20% for the runtime and a short context; a long context needs more. Prices are the lowest on-demand hourly rates in the SAVRN Index, read Sep 18, 2026.
By Open Athena, published under apache-2.0, revision 07579a556457.
This is a MarinSkyRL-native Open-MOPD student after 32 optimizer steps. It starts from the authors' mixed-domain SFT checkpoint. Student responses were scored by the authors' math, code, and instruction-following RL teachers, routed by domain. The objective uses the student's selected top-16 token IDs and a clipped policy surrogate. This is an early checkpoint, not the authors' step-200 final model. The checkpoint is an unquantized, six-file Hugging Face export of the durable MarinSkyRL globalstep32 FSDP2 checkpoint. The policy export was used for the independent step-32 evaluation. The export's model.safetensors SHA-256 is bb7326640142069bc2e1fba5f54f15e0cccb1ff861f34f318b372eaab7abaf4b.…
This is a MarinSkyRL-native Open-MOPD student after 32 optimizer steps. It starts from the authors' mixed-domain SFT checkpoint. Student responses were scored by the authors' math, code, and instruction-following RL teachers, routed by domain. The objective uses the student's selected top-16 token IDs and a clipped policy surrogate. This is an early checkpoint, not the authors' step-200 final model.
The checkpoint is an unquantized, six-file Hugging Face export of the durable MarinSkyRL global_step_32 FSDP2 checkpoint. The policy export was used for the independent step-32 evaluation. The export's model.safetensors SHA-256 is bb7326640142069bc2e1fba5f54f15e0cccb1ff861f34f318b372eaab7abaf4b. The original trainer-state SHA-256 is a3f50a849ed43f06deef731bc676a1bf8d31fcf5d34ed2f63c8b6aa1c985da34. The training source is MarinSkyRL commit 12e6da9e.
The retained training and inline-evaluation traces cover the original, fast-gate, and fast-full run segments. They include student responses and teacher-route metadata, but not the historical per-token teacher scoring arrays. The local open-mopd-repro/open-mopd-reproduce.md artifact gives the exact source, input, and launch sequence.
| Independent evaluation | Correct / outputs | Accuracy |
|---|---|---|
| AIME 2024, mean@64 | 423 / 1,920 | 22.03% |
| AIME 2025, mean@64 | 448 / 1,920 | 23.33% |
| IFEval, mean@1 | 403 / 541 | 74.49% |
These are three of the six released target benchmarks. LiveCodeBench v5/v6 and IFBench were generated, but paper-comparable scores are not claimed here. The training run reached a durable step-34 checkpoint and was then stopped by the campaign owner. This model is the earlier evaluated step-32 export. See the Open-MOPD paper for the authors' full experiment and final results.
Use AutoTokenizer.from_pretrained and AutoModelForCausalLM.from_pretrained with this repository ID. The export contains config.json, generation_config.json, tokenizer.json, tokenizer_config.json, and chat_template.jinja with the weights. For comparable AIME results, use the released 30-question sets with 64 samples per question and the recorded evaluation prompts, generation limits, and grader. A single greedy answer is not comparable to mean@64.
8 files, 6.7 GB in total. The weights are 1 file totalling 6.7 GB in safetensors.
| File | Type | Size | SHA-256 |
|---|---|---|---|
| model.safetensors | Weights | 6.7 GB | bb7326640142 |
| config.json | Configuration | 2.1 KB | — |
| generation_config.json | Configuration | 116 B | — |
| README.md | Documentation | 2.9 KB | — |
| chat_template.jinja | Other | 5.4 KB | — |
| .gitattributes | Repository | 1.6 KB | — |
| tokenizer.json | Tokenizer | 17.2 MB | ab4da6b2aa68 |
| tokenizer_config.json | Tokenizer | 356 B | — |
Released by Open Athena through its official repository on Hugging Face. Read the license.
| Precision | Weights in memory |
|---|---|
| As published | 6.7 GB |
| 16-bit | 6.7 GB |
| 8-bit | 3.3 GB |
| 4-bit | 1.7 GB |
Weights only, from the published parameter count; the key-value cache and runtime add to this.
About 8 GB at 16-bit and 2 GB at 4-bit: the weights (3.3B parameters) plus a working margin. A long context needs more.
At 16-bit, 1x MI300X from $1.85 an hour; at 4-bit, 1x MI300X from $1.85 an hour, at the lowest on-demand prices the SAVRN Index lists.
Yes. MarinSkyRL-Open-MOPD-SmolLM3-3B-step-32 is released under Apache License 2.0. The Apache License 2.0 is a permissive open-source license. It permits commercial use, modification and redistribution. It requires keeping the license and copyright notices and any NOTICE file, stating significant changes, and it includes an express patent grant from contributors.
65,536 tokens, from the maximum position embeddings in its published configuration.
PowerMoE-3B is a 3B sparse Mixture-of-Experts (sMoE) language model trained with the Power learning rate scheduler. It sparsely activates 800M parameters for each token. It is trained on a mix of open-source and proprietary datasets. PowerMoE-3B has shown promising results compared to other dense models with 2x activate parameters across various benchmarks, including natural language multi-choices, code generation, and math reasoning. This is a simple example of how to use PowerMoE-3b model.
The Llama 3.2 collection of multilingual large language models (LLMs) is a collection of pretrained and instruction-tuned generative models in 1B and 3B sizes (text in/text out). The Llama 3.2 instruction-tuned text only models are optimized for multilingual dialogue use cases, including agentic retrieval and summarization tasks. They outperform many of the available open source and closed chat models on common industry benchmarks. Model Architecture: Llama 3.2 is an auto-regressive language model that uses an optimized transformer architecture. The tuned versions use supervised fine-tuning (SFT) and reinforcement learning with human feedback (RLHF) to align with human preferences for…
Bonsai 2 27B by Prism ML, repacked for vLLM. Unofficial; not affiliated with Prism ML. It needs the prismternary vLLM plugin from fraserprice/bonsai-vllm, which has the run command, kernels and throughput numbers: Built and tested for the RTX PRO 6000 Blackwell only; other NVIDIA GPUs are untested. Problems: open an issue. - The ternary weights of the MLX pack, bit for bit: the same 2-bit codes and FP16 group scales (g128), in the same Hadamard-rotated basis. MLX's redundant per-group biases (-scale) are dropped, and the rotation signs move into config.json. - The embedding table is dequantized to BF16. - Norms and the linear-attention state path are Prism ML's own tensors from the MLX…
SymbioticLM is a hybrid symbolic–neural language model that integrates a frozen transformer backbone (Qwen2ForCausalLM) with a suite of symbolic cognitive modules for adaptive, interpretable reasoning. The architecture fuses neural token-level generation with symbolic introspection and reasoning: - Dynamic Thought Evolution with Helical Encoding and DNA-Inspired Memory (DTE-HDM) Enables structured long-term memory and spiral-context encoding across tokens. - Multi-Agent Symbiotic Response Mechanisms (M.A.S.R.M) Coordinates symbolic-neural agents via gated attention and adaptive response layers. - QwenExoCortex Projects contextual hidden states from the Qwen model into a symbolic fusion…
Qwen2.5 is the latest series of Qwen large language models. For Qwen2.5, we release a number of base language models and instruction-tuned language models ranging from 0.5 to 72 billion parameters. Qwen2.5 brings the following improvements upon Qwen2: - Significantly more knowledge and has greatly improved capabilities in coding and mathematics, thanks to our specialized expert models in these domains. - Significant improvements in instruction following, generating long texts (over 8K tokens), understanding structured data (e.g, tables), and generating structured outputs especially JSON. More resilient to the diversity of system prompts, enhancing role-play implementation and…
NULLXES SHINRA-4B-INSTRUCT is the Language Intelligence Layer of the NULLXES system. SHINRA is responsible for multilingual understanding, coding intelligence, instruction following, structured outputs, and agent preparation. This checkpoint is the instruction-tuned (and optionally DPO-aligned) 4B-class dense decoder. Proprietary ShinraForCausalLM (not a Llama / Mistral / Qwen / GPT-NeoX wrapper). RMSNorm → GQA+RoPE → residual → RMSNorm → SwiGLU → residual then final RMSNorm and tied LM head. Special tokens:. Generation stop is. Document stop is. Three stages. Pretrain → NULLXES SHINRA-4B-BASE SHINRAPRETRAINV1: 40% FineWeb-Edu, 20% code (python-edu + licensed Stack), 15% math/science…