VinaySLM is a small language model (SLM) built from scratch in PyTorch to learn how decoder-only Transformers work end to end -- tokenization, embeddings, causal self-attention, training, and generation -- without importing a pretrained architecture. This is not a production-quality language model. - A tiny, hand-written, decoder-only (GPT-style) Transformer. - 466,176 parameters. - Trained on a ~600-token toy corpus of AI/programming sentences. - Packaged here as a custom Hugging Face model (trustremotecode=True) so it can be loaded and inspected through the transformers ecosystem, while the actual Transformer implementation (modelingvinayslm.py) is a thin wrapper around this project's…
Aloha! Today, we are releasing Ornith-1.0, a self-improving family of open-source models for agentic coding. This model card documents Ornith-1.0-35B, the lightweight member of the Ornith family, designed for efficient single-GPU deployment.
Parameters664,944
Context262,144
Weights70.2 GB
Licensemit
AccessOpen weights
Monthly Downloads2M
Runs On
What it takes to serve Ornith-1.0-35B (664,944 parameters): the memory its weights need at each precision, and the cheapest way to rent enough data-center GPUs to hold them.
| Precision | Weights | Memory needed | Cheapest setup | Per hour | Also fits |
|---|---|---|---|---|---|
| 16-bit | 0.0 GB | 0.0 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
| 8-bit | 0.0 GB | 0.0 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
| 4-bit | 0.0 GB | 0.0 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
Memory is the weights at that precision plus 20% for the runtime and a short context; a long context needs more. Prices are the lowest on-demand hourly rates in the SAVRN Index, read Sep 18, 2026.
SAVRN's Notes on Ornith-1.0-35B
The table on this page reads 0.0 GB at every precision, and that is the first thing to doubt. The metadata records 664,944 parameters, a figure that does not square with 70.22 GB of weight files; size hardware from the files. Ornith built it for agentic coding as a mixture of experts, 256 with 8 active per token, and all 256 sit in memory. A single MI300X at $1.85 an hour on-demand, the cheapest host in the Index, carries 192 GB, enough for the weights, with the balance going to the 262,144-token context; the publisher's recipe uses an 8x80 GB node at tensor parallel 8.
MIT is the license: commercial use, modification and redistribution, keep the copyright and permission notice. Before committing, ask what base it was trained from, since the file records no relation though the architecture string reads Qwen3_5Moe. The Index shows no per-token host price yet.
Model Card
By Ornith, published under mit, revision 5df2ed3f675c.
Aloha! Today, we are releasing Ornith-1.0, a self-improving family of open-source models for agentic coding.
Highlights:
- State-of-the-Art Coding Agents: Available in 9B-Dense, 31B-Dense, 35B-MoE, and 397B-MoE (post-trained on top of Gemma 4 and Qwen 3.5), achieving state-of-the-art performance among open-source models of comparable size on coding benchmarks such as Terminal-Bench 2.1, SWE-Bench, NL2Repo and OpenClaw.
- Self-Improving Training Framework: Ornith-1.0 employs RL to learn to generate not only solution rollouts, but also the scallfold that drive those rollouts. By jointly optimizing the scaffold and the resulting solution, the model discovers better search trajectories and generates higher-quality solutions.
- Licence: MIT licensed, globally accessible, and free from regional limitations.
Ornith 1.0 35B
This model card documents Ornith-1.0-35B, the lightweight member of the Ornith family, designed for efficient single-GPU deployment.
Benchmarks
Configuration
- Architecture
- Qwen3_5MoeForConditionalGeneration
- Context length (tokens)
- 262,144
- Layers
- 40
- Hidden size
- 2,048
- Attention heads
- 16
- Key/value heads
- 2
- Head dimension
- 256
- Vocabulary size
- 248,320
- Experts
- 256
- Experts active per token
- 8
- Model type
- qwen3_5_moe
Identity and Version
- Repository
- ornith-ai/Ornith-1.0-35B
- Publisher
- Ornith
- Task
- Text generation
- Modality
- Text
- Library
- transformers
- Parameters
- 664,944 parameters
- Languages
- Not stated by the source
- Revision
- 5df2ed3f675c7beaa490328cc70bb573b65fb660
- First published
- 2026-06-21
- Last updated
- 2026-06-25
Files and Weights
30 files, 70.3 GB in total. The weights are 16 files totalling 70.2 GB in safetensors.
Weights16 files · 70.2 GB
Configuration6 files · 3.3 MB
Tokenizer3 files · 26.7 MB
Documentation1 file · 21.2 KB
Other3 files · 1.6 MB
Repository1 file · 1.8 KB
Every file
| File | Type | Size | SHA-256 |
|---|---|---|---|
| model-00001-of-00016.safetensors | Weights | 4.3 GB | 8980bf2c5481 |
| model-00002-of-00016.safetensors | Weights | 4.5 GB | f01d85459d4e |
| model-00003-of-00016.safetensors | Weights | 5.0 GB | df645d0ce143 |
| model-00004-of-00016.safetensors | Weights | 4.0 GB | a52860dd99fb |
| model-00005-of-00016.safetensors | Weights | 4.5 GB | ed7c2d55bb63 |
| model-00006-of-00016.safetensors | Weights | 5.0 GB | 19f7de89bf9b |
| model-00007-of-00016.safetensors | Weights | 4.0 GB | a107feedd9b5 |
| model-00008-of-00016.safetensors | Weights | 4.5 GB | 472623b88e7c |
| model-00009-of-00016.safetensors | Weights | 5.0 GB | c78b11615416 |
| model-00010-of-00016.safetensors | Weights | 4.0 GB | 3523d98eadd5 |
| model-00011-of-00016.safetensors | Weights | 4.5 GB | a039fd65e069 |
| model-00012-of-00016.safetensors | Weights | 5.0 GB | cdb279ef9c13 |
| model-00013-of-00016.safetensors | Weights | 4.0 GB | 0ed70df84b10 |
| model-00014-of-00016.safetensors | Weights | 4.5 GB | 35245ae19111 |
| model-00015-of-00016.safetensors | Weights | 5.0 GB | 8897878571a9 |
| model-00016-of-00016.safetensors | Weights | 2.6 GB | 39a4b3b5c85d |
| config.json | Configuration | 3.3 KB | — |
| generation_config.json | Configuration | 213 B | — |
| model.safetensors.index.json | Configuration | 3.3 MB | — |
| preprocessor_config.json | Configuration | 390 B | — |
| processor_config.json | Configuration | 1.2 KB | — |
| video_preprocessor_config.json | Configuration | 385 B | — |
| README.md | Documentation | 21.2 KB | — |
| assets/ornith_35b_eval.png | Other | 595.3 KB | 03a8648a556a |
| assets/ornith_logo.png | Other | 962.4 KB | 458ee0d85bae |
| chat_template.jinja | Other | 7.5 KB | — |
| .gitattributes | Repository | 1.8 KB | — |
| tokenizer.json | Tokenizer | 20.0 MB | 06b9509352d2 |
| tokenizer_config.json | Tokenizer | 1.2 KB | — |
| vocab.json | Tokenizer | 6.7 MB | — |
License and Download
- License
- mit
- Access
- Open weights, no gate
- Download size
- 70.2 GB
Download from Ornith
Released by Ornith through its official repository on Hugging Face. Read the license.
Evaluations
Each result is shown as reported, with the conditions its reporter stated. None is a SAVRN measurement. A comparison lines two results up only when their configuration, unit and setup are all stated and identical.
| Benchmark | Conditions | Result | Reported by | Revision | Date |
|---|---|---|---|---|---|
| SWE-bench/SWE-bench_Multilingual | Task swe_bench_multilingual_%_resolvedMetric swe_bench_multilingual_%_resolvedComparison conditions not established | 69.3 | Model Card Reported by a third party |
Evaluated revision not stated | 2026-08-10 |
| SWE-bench/SWE-bench_Verified | Task swe_bench_%_resolvedMetric swe_bench_%_resolvedComparison conditions not established | 75.6 | Ornith-1.0-35B model card Reported by a third party |
Evaluated revision not stated | 2026-06-25 |
| ScaleAI/SWE-bench_Pro | Task SWE_Bench_ProMetric SWE_Bench_ProComparison conditions not established | 50.4 | Ornith-1.0-35B model card Reported by a third party |
Evaluated revision not stated | 2026-06-25 |
| claw-eval/Claw-Eval | Task generalMetric generalComparison conditions not established | 69.8 | Ornith-1.0-35B model card Reported by a third party |
Evaluated revision not stated | 2026-06-25 |
| harborframework/terminal-bench-2.1 | Task terminalbench_2_1Metric terminalbench_2_1Setup Claude Code 2.1.126 harness, parser=json, temperature=1.0, top_p=1.0, max_new_tokens=131072, avg of 5 runs.Comparison conditions not established | 62.8 | Model Card Reported by a third party |
Evaluated revision not stated | 2026-06-25 |
| harborframework/terminal-bench-2.1 | Task terminalbench_2_1Metric terminalbench_2_1Setup Terminus-2 harness (Harbor framework), parser=json, temperature=1.0, top_p=1.0, 128K context, 4h timeout, avg of 5 runs.Comparison conditions not established | 64.2 | Model Card Reported by a third party |
Evaluated revision not stated | 2026-06-25 |
Memory Requirements
| Precision | Weights in memory |
|---|---|
| As published | 70.2 GB |
| 16-bit | 0.0 GB |
| 8-bit | 0.0 GB |
| 4-bit | 0.0 GB |
Weights only, from the published parameter count; the key-value cache and runtime add to this.
Questions About Ornith-1.0-35B
How much GPU memory does Ornith-1.0-35B need?
About 0 GB at 16-bit and 0 GB at 4-bit: the weights (664,944 parameters) plus a working margin. A long context needs more.
What is the cheapest GPU to run Ornith-1.0-35B on?
At 16-bit, 1x MI300X from $1.85 an hour; at 4-bit, 1x MI300X from $1.85 an hour, at the lowest on-demand prices the SAVRN Index lists.
Can I use Ornith-1.0-35B commercially?
Yes. Ornith-1.0-35B is released under MIT License. The MIT License is a short permissive license. It permits commercial use, modification and redistribution, provided the copyright notice and permission notice are included.
What is Ornith-1.0-35B's context length?
262,144 tokens, from the maximum position embeddings in its published configuration.
Similar Models
Fine-tune Qwen3 (14B) for free using our Google Colab notebook! - Read our Blog about Qwen3 support: unsloth.ai/blog/qwen3 - View the rest of our notebooks in our docs here. Qwen3-Coder is available in multiple sizes. Today, we're excited to introduce Qwen3-Coder-30B-A3B-Instruct. This streamlined model maintains impressive performance and efficiency, featuring the following key enhancements: - Significant Performance among open models on Agentic Coding, Agentic Browser-Use, and other foundational coding tasks. - Long-context Capabilities with native support for 256K tokens, extendable up to 1M tokens using Yarn, optimized for repository-scale understanding. - Agentic Coding supporting for…
OPT was first introduced in Open Pre-trained Transformer Language Models and first released in metaseq's repository on May 3rd 2022 by Meta AI. Disclaimer: The team releasing OPT wrote an official model card, which is available in Appendix D of the paper. Content from this model card has been written by the Hugging Face team. To quote the first two paragraphs of the official paper OPT was predominantly pretrained with English text, but a small amount of non-English data is still present within the training corpus via CommonCrawl. The model was pretrained using a causal language modeling (CLM) objective. OPT belongs to the same family of decoder-only models like GPT-3. As such, it was…
Chirp Chirp! We are introducing Ornith-1.5, a major step toward building foundation models through end-to-end self-improvement. Ornith-1.5 extends Ornith-1.0 (which was developed on top of Qwen3.5 and Gemma4 with additional continued pretraining, mid-training, and post-training) by expanding the self-improvement loop from scaffold and rollout optimization to jointly optimizing task generation, scaffold construction, and solution rollouts. Rather than relying on a fixed set of human-curated tasks and manually designed harnesses, Ornith-1.5 continuously generates new training tasks, discovers effective strategies for solving them, and improves the policy through reinforcement learning. For…
Chirp Chirp! We are introducing Ornith-1.5, a major step toward building foundation models through end-to-end self-improvement. Ornith-1.5 extends Ornith-1.0 (which was developed on top of Qwen3.5 and Gemma4 with additional continued pretraining, mid-training, and post-training) by expanding the self-improvement loop from scaffold and rollout optimization to jointly optimizing task generation, scaffold construction, and solution rollouts. Rather than relying on a fixed set of human-curated tasks and manually designed harnesses, Ornith-1.5 continuously generates new training tasks, discovers effective strategies for solving them, and improves the policy through reinforcement learning. For…
Aloha! Today, we are releasing Ornith-1.0, a self-improving family of open-source models for agentic coding. This model card documents Ornith-1.0-9B, the most lightweight member of the Ornith family, designed for efficient single-GPU deployment. Ornith-1.0-9B is a dense ~9B model (≈19 GB in bf16), so it serves comfortably on a single 80GB GPU. The recipes below stand up an OpenAI-compatible server; add --tensor-parallel-size / --tp if you want to shard across more GPUs. For a quick local test (or to script offline generation), load the model directly with Transformers. Make sure you have a recent release installed — see the Transformers installation guide; Ornith-1.0-9B requires…