Skip to page content
SAVRN
Search Plan a project

Open-weight model · Text generation

Slayer149-balanced

by Fabryka AI SlayerLab/Slayer149-balanced

Slayer149-balanced is an open-weight model for text generation from Fabryka AI, released under Apache License 2.0. It has 2,048-token context. Its published files total 35.0 GB.

Experimental 149,333,081-parameter English causal model trained from scratch. 26 layers, hidden width 576, FFN 2496, 9 query heads / 3 KV heads, tied 24,576-token byte-level BPE vocabulary, QK normalization and value residuals.

Parameters—
Context2,048
Weights35.0 GB
Licenseapache-2.0
AccessOpen weights
Monthly Downloads—

Model Card

By Fabryka AI, published under apache-2.0, revision 390c5ec32c1b.

Experimental 149,333,081-parameter English causal model trained from scratch. 26 layers, hidden width 576, FFN 2496, 9 query heads / 3 KV heads, tied 24,576-token byte-level BPE vocabulary, QK normalization and value residuals. Architecture informed by Qwen3 and AltSlate's Apache-2.0 Jugnu recipe; no Jugnu weights reused. This is a separate experiment from SlayerLab/Slayer149, which remains available. Training may still be in progress. See reports/status.json, training metrics and checkpoint manifests. No leaderboard rank or winning result is claimed. The final full GLINT-derived evaluation is reports/evaluation.json, when present. Competitor scores may use different evaluation conventions.…

Read Fabryka AI's full model card

Experimental 149,333,081-parameter English causal model trained from scratch. 26 layers, hidden width 576, FFN 2496, 9 query heads / 3 KV heads, tied 24,576-token byte-level BPE vocabulary, QK normalization and value residuals. Architecture informed by Qwen3 and AltSlate's Apache-2.0 Jugnu recipe; no Jugnu weights reused. This is a separate experiment from SlayerLab/Slayer149, which remains available.

Training may still be in progress. See reports/status.json, training metrics and checkpoint manifests. No leaderboard rank or winning result is claimed. The final full GLINT-derived evaluation is reports/evaluation.json, when present. Competitor scores may use different evaluation conventions.

tokenizer.json preserves Unicode/whitespace and uses EOS ID 0. Tokenizer selection and dataset provenance are recorded in reports. Training sources are pinned FineWeb-Edu, DCLM-edu, Cosmopedia-v2 and FineMath, with document-disjoint validation and exact-overlap filtering against the benchmark suite. Filtering is not a semantic contamination guarantee. No training text or credentials are uploaded.

checkpoints/<step>/training-state.pt contains model and both optimizer states. Restore with the matching tokenizer, config, source files and training code. Final safetensors appear under the final checkpoint and at repository root. Use the supplied custom PyTorch loader; this is not yet an AutoModel package.

from load_model import load_model
model, tokenizer = load_model("/path/to/downloaded/repository", device="cuda")

Goal: have a durable checkpoint and evaluation report by 15:00 Warsaw time on 2026-10-01, within the user's confirmed free GPU reservation. This is not a promise of model quality or benchmark rank.

Configuration

Architecture
BalancedLM
Context length (tokens)
2,048
Layers
26
Hidden size
576
Feed-forward size
2,496
Attention heads
9
Key/value heads
3
Head dimension
64
Vocabulary size
24,576
Model type
slayer_balanced

Identity and Version

Repository
SlayerLab/Slayer149-balanced
Publisher
Fabryka AI
Task
Text generation
Modality
Text
Library
pytorch
Parameters
Not stated by the source
Languages
en
Revision
390c5ec32c1b96574a26623488f35be7ae72c4d9
First published
2026-10-01
Last updated
2026-10-01

Files and Weights

75 files, 35.0 GB in total. The weights are 28 files totalling 35.0 GB in pt.

Weights28 files · 35.0 GB
Configuration40 files · 18.1 KB
Tokenizer2 files · 1.7 MB
Documentation1 file · 1.9 KB
Other3 files · 125.4 KB
Repository1 file · 1.5 KB
Every file
FileTypeSizeSHA-256
checkpoints/000000020/training-state.ptWeights1.3 GB 795790e1edd9
checkpoints/000000500/training-state.ptWeights1.3 GB e6d88a10d4be
checkpoints/000001000/training-state.ptWeights1.3 GB 5f2e1312722d
checkpoints/000001500/training-state.ptWeights1.3 GB cabe10c9b253
checkpoints/000002000/training-state.ptWeights1.3 GB a9de0e80ae40
checkpoints/000002500/training-state.ptWeights1.3 GB 6b27005cdd1c
checkpoints/000003000/training-state.ptWeights1.3 GB 9c82a194727e
checkpoints/000003500/training-state.ptWeights1.3 GB dc90a020c2e8
checkpoints/000004000/training-state.ptWeights1.3 GB b31eadd7aff1
checkpoints/000004500/training-state.ptWeights1.3 GB 7509421b3a2c
checkpoints/000005000/training-state.ptWeights1.3 GB 4ca259755142
checkpoints/000005500/training-state.ptWeights1.3 GB 6114b64a060d
checkpoints/000006000/training-state.ptWeights1.3 GB 21d72768fd3d
checkpoints/000006500/training-state.ptWeights1.3 GB 33828b5a38d2
checkpoints/000007000/training-state.ptWeights1.3 GB c1931cfa854f
checkpoints/000007500/training-state.ptWeights1.3 GB f3367c10e16e
checkpoints/000008000/training-state.ptWeights1.3 GB 3bc3507080b6
checkpoints/000008500/training-state.ptWeights1.3 GB 406a7255d390
checkpoints/000009000/training-state.ptWeights1.3 GB 6b02ef28524e
checkpoints/000009500/training-state.ptWeights1.3 GB 78d782083b3f
checkpoints/000010000/training-state.ptWeights1.3 GB f89fafc08dcf
checkpoints/000010500/training-state.ptWeights1.3 GB ac226493b4dd
checkpoints/000011000/training-state.ptWeights1.3 GB 7107be5ed9b7
checkpoints/000011500/training-state.ptWeights1.3 GB 359a09b969ed
checkpoints/000012000/training-state.ptWeights1.3 GB 862b8f021340
checkpoints/000012500/training-state.ptWeights1.3 GB d00a026f40f3
checkpoints/000013000/training-state.ptWeights1.3 GB bf20b78d9cd9
checkpoints/000013500/training-state.ptWeights1.3 GB db4cbe637569
balanced_model.pyConfiguration4.7 KB —
checkpoints/000000020/manifest.jsonConfiguration173 B —
checkpoints/000000500/manifest.jsonConfiguration174 B —
checkpoints/000001000/manifest.jsonConfiguration175 B —
checkpoints/000001500/manifest.jsonConfiguration175 B —
checkpoints/000002000/manifest.jsonConfiguration175 B —
checkpoints/000002500/manifest.jsonConfiguration175 B —
checkpoints/000003000/manifest.jsonConfiguration175 B —
checkpoints/000003500/manifest.jsonConfiguration175 B —
checkpoints/000004000/manifest.jsonConfiguration175 B —
checkpoints/000004500/manifest.jsonConfiguration175 B —
checkpoints/000005000/manifest.jsonConfiguration175 B —
checkpoints/000005500/manifest.jsonConfiguration175 B —
checkpoints/000006000/manifest.jsonConfiguration175 B —
checkpoints/000006500/manifest.jsonConfiguration175 B —
checkpoints/000007000/manifest.jsonConfiguration175 B —
checkpoints/000007500/manifest.jsonConfiguration175 B —
checkpoints/000008000/manifest.jsonConfiguration175 B —
checkpoints/000008500/manifest.jsonConfiguration175 B —
checkpoints/000009000/manifest.jsonConfiguration175 B —
checkpoints/000009500/manifest.jsonConfiguration175 B —
checkpoints/000010000/manifest.jsonConfiguration176 B —
checkpoints/000010500/manifest.jsonConfiguration176 B —
checkpoints/000011000/manifest.jsonConfiguration176 B —
checkpoints/000011500/manifest.jsonConfiguration176 B —
checkpoints/000012000/manifest.jsonConfiguration176 B —
checkpoints/000012500/manifest.jsonConfiguration176 B —
checkpoints/000013000/manifest.jsonConfiguration176 B —
checkpoints/000013500/manifest.jsonConfiguration176 B —
config.jsonConfiguration1.4 KB —
load_model.pyConfiguration503 B —
reports/DATA_READY.jsonConfiguration247 B —
reports/benchmark/result.jsonConfiguration202 B —
reports/deadline.jsonConfiguration173 B —
reports/run/resolved.jsonConfiguration1.9 KB —
reports/sources/cosmopedia/manifest.jsonConfiguration964 B —
reports/sources/dclm_edu/manifest.jsonConfiguration788 B —
reports/sources/finemath/manifest.jsonConfiguration961 B —
reports/sources/fineweb_edu/manifest.jsonConfiguration1.2 KB —
reports/status.jsonConfiguration156 B —
README.mdDocumentation1.9 KB —
reports/run/metrics.jsonlOther122.9 KB —
reports/run/validation.jsonlOther2.3 KB —
reports/train.tomlOther174 B —
.gitattributesRepository1.5 KB —
reports/tokenizer-selection.jsonTokenizer4.1 KB —
tokenizer.jsonTokenizer1.7 MB —

License and Download

License
apache-2.0
Access
Open weights, no gate
Download size
35.0 GB
Download from Fabryka AI

Released by Fabryka AI through its official repository on Hugging Face. Read the license.

Memory Requirements

PrecisionWeights in memory
As published35.0 GB

Weights only, from the published parameter count; the key-value cache and runtime add to this.

Questions About Slayer149-balanced

Can I use Slayer149-balanced commercially?

Yes. Slayer149-balanced is released under Apache License 2.0. The Apache License 2.0 is a permissive open-source license. It permits commercial use, modification and redistribution. It requires keeping the license and copyright notices and any NOTICE file, stating significant changes, and it includes an express patent grant from contributors.

What is Slayer149-balanced's context length?

2,048 tokens, from the maximum position embeddings in its published configuration.

Similar Models

Model · Text generation

opt-125m

AI at Meta

OPT was first introduced in Open Pre-trained Transformer Language Models and first released in metaseq's repository on May 3rd 2022 by Meta AI. Disclaimer: The team releasing OPT wrote an official model card, which is available in Appendix D of the paper. Content from this model card has been written by the Hugging Face team. To quote the first two paragraphs of the official paper OPT was predominantly pretrained with English text, but a small amount of non-English data is still present within the training corpus via CommonCrawl. The model was pretrained using a causal language modeling (CLM) objective. OPT belongs to the same family of decoder-only models like GPT-3. As such, it was…

Open weights other 2,048 tokens transformers

Model · Text generation

Ornith-1.5-9B-GGUF

Ornith

Chirp Chirp! We are introducing Ornith-1.5, a major step toward building foundation models through end-to-end self-improvement. Ornith-1.5 extends Ornith-1.0 (which was developed on top of Qwen3.5 and Gemma4 with additional continued pretraining, mid-training, and post-training) by expanding the self-improvement loop from scaffold and rollout optimization to jointly optimizing task generation, scaffold construction, and solution rollouts. Rather than relying on a fixed set of human-curated tasks and manually designed harnesses, Ornith-1.5 continuously generates new training tasks, discovers effective strategies for solving them, and improves the policy through reinforcement learning. For…

Open weights mit transformers

Model · Text generation

Ornith-1.5-35B-A3B-GGUF

Ornith

Chirp Chirp! We are introducing Ornith-1.5, a major step toward building foundation models through end-to-end self-improvement. Ornith-1.5 extends Ornith-1.0 (which was developed on top of Qwen3.5 and Gemma4 with additional continued pretraining, mid-training, and post-training) by expanding the self-improvement loop from scaffold and rollout optimization to jointly optimizing task generation, scaffold construction, and solution rollouts. Rather than relying on a fixed set of human-curated tasks and manually designed harnesses, Ornith-1.5 continuously generates new training tasks, discovers effective strategies for solving them, and improves the policy through reinforcement learning. For…

Open weights mit transformers

Model · Text generation

Ternary-Bonsai-2-27B-gguf

Prism ML

Full 27B-class reasoning in ternary transformer weights, for llama.cpp (CUDA, Metal, CPU) - \~5.9 GB language model (down from \~54 GB FP16) — full 27B-class reasoning on a standard laptop or a single GPU - 98.2% of FP16 intelligence retained: 84.78 average across 14 thinking-mode benchmarks — far above the conventional IQ2XXS build (72.59) at about 82% of its footprint, and within 0.4 points of UD-Q4KXL at three times the footprint - Retains thinking, reasoning, and agentic behavior deep in the sub-4-bit regime, where conventional low-bit representations collapse: math within half a point of full precision (96.57), coding level with the baseline (89.42), agentic tool calling at 74.92…

Open weights apache-2.0 llama.cpp

Uncensored Qwen3.8-27B, published as GGUF quantizations with the multi token prediction (MTP) head retained and verified. Refusal behaviour has been substantially reduced, not eliminated. See Measured behaviour for the numbers. Capabilities, training data, and architecture are otherwise unchanged. - Refusal directions removed with Heretic, which co minimizes refusal count against KL divergence from the base model. No handwritten refusal removal code, no finetuning, no additional training data. - Abliteration runs at bf16 (no 4 bit quantization). the resulting LoRA is merged into the bf16 base, so the published weights are not a quantized round trip. - mtp. tensors are copied verbatim from…

Open weights apache-2.0 llama.cpp

Model · Text generation

Ornith-1.0-9B-GGUF

Ornith

Aloha! Today, we are releasing Ornith-1.0, a self-improving family of open-source models for agentic coding. This model card documents Ornith-1.0-9B, the most lightweight member of the Ornith family, designed for efficient single-GPU deployment. Ornith-1.0-9B is a dense ~9B model (≈19 GB in bf16), so it serves comfortably on a single 80GB GPU. The recipes below stand up an OpenAI-compatible server; add --tensor-parallel-size / --tp if you want to shard across more GPUs. For a quick local test (or to script offline generation), load the model directly with Transformers. Make sure you have a recent release installed — see the Transformers installation guide; Ornith-1.0-9B requires…

Open weights mit transformers