Skip to page content
SAVRN
Search Plan a project

Open-weight model · Text generation

Ternary-Bonsai-8B-gguf-fork

by Pythai.net PYTHAI/Ternary-Bonsai-8B-gguf-fork

Ternary-Bonsai-8B-gguf-fork is an open-weight model for text generation from Pythai.net, released under Apache License 2.0. Its published files total 2.3 GB.

Ternary (1.58-bit) language model in GGUF Q20 format for llama.cpp - Demo repo — examples for serving, benchmarking, and integrating Bonsai - Discord — community support and updates Each weight takes a value from {-1, 0, +1}, with one shared FP16 scale per…

Parameters—
Context—
Weights2.3 GB
Licenseapache-2.0
AccessOpen weights
Monthly Downloads—

Model Card

By Pythai.net, published under apache-2.0, revision 2445960052d4.

Ternary (1.58-bit) language model in GGUF Q20 format for llama.cpp - Demo repo — examples for serving, benchmarking, and integrating Bonsai - Discord — community support and updates Each weight takes a value from {-1, 0, +1}, with one shared FP16 scale per group of 128 weights: Q20 encodes each weight as a 2-bit code q in {0, 1, 2, 3}, dequantized via w = (q - 1) scale. One 128-element block is 34 bytes (2 bytes FP16 scale + 32 bytes of packed 2-bit codes) for an effective 2.125 bits/weight. The fourth code point (q = 3, reconstructing to +2 scale) is reserved for future extensions; for ternary weights it is unused. Evaluated with EvalScope v1.4.2 + vLLM 0.15.1 on NVIDIA H100 under…

Read Pythai.net's full model card

PYTHAI fork of prism-ml/Ternary-Bonsai-8B-gguf at commit c2aefbeb4b24469cd11579c3384b990404c17a30, weights included, LICENSE and NOTICE.txt unchanged. Only the mainline-safe Ternary-Bonsai-8B-Q2_0_g64.gguf is carried; the files left at source and why are listed in FORK.json (omitted). The upstream model card follows.

Prism ML Website  |  White Paper  |  Demo & Examples  |  Discord

Ternary-Bonsai-8B-gguf

Ternary (1.58-bit) language model in GGUF Q2_0 format for llama.cpp

Resources

  • White Paper
  • Demo repo — examples for serving, benchmarking, and integrating Bonsai
  • Discord — community support and updates
  • Kernels: Q2_0 is not yet in mainline llama.cpp. Use our fork at PrismML-Eng/llama.cpp (prism branch, default) which adds Q2_0 support for CPU (NEON/generic) and Metal. Upstream PR coming soon.

Model Overview

Item Specification
Base model Qwen3-8B
Parameters 8.19B (~6.95B non-embedding)
Architecture GQA (32 query / 8 KV heads), SwiGLU MLP, RoPE, RMSNorm
Layers 36 Transformer decoder blocks
Context length 65,536 tokens
Vocab size 151,936
Weight format GGUF Q2_0 g128: {-1, 0, +1} with FP16 group-wise scaling
Packed Q2_0 size 2.03 GiB (2.18 GB)
Ternary coverage Embeddings, attention projections, MLP projections, LM head
License Apache 2.0

Quantization Format: GGUF Q2_0 (g128)

Each weight takes a value from {-1, 0, +1}, with one shared FP16 scale per group of 128 weights:

w_i = scale_g * t_i,    t_i in {-1, 0, +1}

Q2_0 encodes each weight as a 2-bit code q in {0, 1, 2, 3}, dequantized via w = (q - 1) * scale. One 128-element block is 34 bytes (2 bytes FP16 scale + 32 bytes of packed 2-bit codes) for an effective 2.125 bits/weight. The fourth code point (q = 3, reconstructing to +2 * scale) is reserved for future extensions; for ternary weights it is unused.

Memory

Format Size Reduction Ratio
FP16 16.38 GB -- 1.0x
GGUF Q2_0 g128 2.03 GiB (2.18 GB) 86.7% 7.5x

Files in this repo

File Format Size Recommended
Ternary-Bonsai-8B-F16.gguf FP16 16.38 GB baseline / re-quantization source
Ternary-Bonsai-8B-Q2_0.gguf Q2_0 (g128) 2.03 GiB recommended (lossless for ternary)

Quickstart

Build from the Prism fork

git clone https://github.com/PrismML-Eng/llama.cpp
cd llama.cpp
cmake -B build -DGGML_METAL=ON   # or -DGGML_CUDA=ON, -DGGML_VULKAN=ON
cmake --build build -j

llama.cpp CLI

./build/bin/llama-cli \
  -m Ternary-Bonsai-8B-Q2_0.gguf \
  -p "Explain quantum computing in simple terms." \
  -n 256

llama.cpp server

./build/bin/llama-server -m Ternary-Bonsai-8B-Q2_0.gguf -c 4096

Throughput (llama.cpp, Apple M4 Pro 48 GB)

Backend PP512 (tok/s) TG128 (tok/s)
Metal (GPU) 455 76
NEON CPU (10 t) 146 32

Flags: -ngl 99 -fa 1 for Metal; -ngl 0 -fa 1 -t 10 for CPU.

Benchmarks

Evaluated with EvalScope v1.4.2 + vLLM 0.15.1 on NVIDIA H100 under identical infrastructure, generation parameters, and scoring. All models are in the 6B-9B parameter range.

Model Size Avg MMLU-R MuSR GSM8K HE+ IFEval BFCL
Qwen 3 8B 16.38 GB 79.3 83 55 93 82.3 81.5 81
Ternary Bonsai 8B 2.18 GB 75.5 72.6 56.2 91 77.4 81.8 73.9
1-bit Bonsai 8B (prior) 1.15 GB 70.5 65.7 50 88 73.8 79.8 65.7
RNJ 8B 16.63 GB 73.1 75.5 50.4 93.7 84.2 73.8 61.1
Ministral3 8B 16.04 GB 71.0 68.9 53.8 87.9 72.6 67.4 75.4
Olmo 3 7B 14.60 GB 70.9 72 56.1 92.5 79.3 87.1 38.4

Ternary Bonsai 8B ranks 2nd among all compared models despite being 1/8th the size.

Intelligence Density

density = -ln(1 - score/100) / size_GB
Model Size Intelligence Density (1/GB)
Ternary Bonsai 8B 2.18 GB 0.645
1-bit Bonsai 8B (prior) 1.15 GB 1.062
Qwen 3 8B 16.38 GB 0.096
RNJ 8B 16.62 GB 0.079

Citation

@techreport{ternarybonsai,
    title   = {Ternary Bonsai: 1.58-bit Language Models at 8B, 4B, and 1.7B Scale},
    author  = {Prism ML},
    year    = {2026},
    month   = {April},
    url     = {https://prismml.com}
}

Contact

For questions, feedback, or collaboration inquiries: [email protected]

Identity and Version

Repository
PYTHAI/Ternary-Bonsai-8B-gguf-fork
Publisher
Pythai.net
Task
Text generation
Modality
Text
Library
gguf
Parameters
Not stated by the source
Languages
on-device
Revision
2445960052d465ace262a43b5f5fef52a0c1e1ef
First published
2026-09-26
Last updated
2026-09-26

Files and Weights

10 files, 2.3 GB in total. The weights are 1 file totalling 2.3 GB in gguf.

Weights1 file · 2.3 GB
Configuration2 files · 2.9 KB
Documentation2 files · 17.2 KB
Other4 files · 89.2 KB
Repository1 file · 2.0 KB
Every file
FileTypeSizeSHA-256
Ternary-Bonsai-8B-Q2_0_g64.ggufWeights2.3 GB e17b298d84ee
.eval_results/gsm8k.yamlConfiguration149 B —
FORK.jsonConfiguration2.8 KB —
LICENSEDocumentation10.2 KB —
README.mdDocumentation7.0 KB —
NOTICE.txtOther412 B —
assets/bonsai-logo.svgOther515 B —
assets/frontier.pngOther78.5 KB —
assets/frontier.svgOther9.8 KB —
.gitattributesRepository2.0 KB —

License and Download

License
apache-2.0
Access
Open weights, no gate
Download size
2.3 GB
Download from Pythai.net

Released by Pythai.net through its official repository on Hugging Face. Read the license.

Built From

  • Derived from prism-ml/Ternary-Bonsai-8B-unpacked
  • Quantized from prism-ml/Ternary-Bonsai-8B-unpacked

Evaluations

Each result is shown as reported, with the conditions its reporter stated. None is a SAVRN measurement. A comparison lines two results up only when their configuration, unit and setup are all stated and identical.

BenchmarkConditionsResultReported byRevisionDate
openai/gsm8k Task gsm8kMetric gsm8kComparison conditions not established 91 Model Card
Reported by a third party		 Evaluated revision not stated 2026-09-26

Memory Requirements

PrecisionWeights in memory
As published2.3 GB

Weights only, from the published parameter count; the key-value cache and runtime add to this.

Questions About Ternary-Bonsai-8B-gguf-fork

Can I use Ternary-Bonsai-8B-gguf-fork commercially?

Yes. Ternary-Bonsai-8B-gguf-fork is released under Apache License 2.0. The Apache License 2.0 is a permissive open-source license. It permits commercial use, modification and redistribution. It requires keeping the license and copyright notices and any NOTICE file, stating significant changes, and it includes an express patent grant from contributors.

Similar Models

Model · Text generation

opt-125m

AI at Meta

OPT was first introduced in Open Pre-trained Transformer Language Models and first released in metaseq's repository on May 3rd 2022 by Meta AI. Disclaimer: The team releasing OPT wrote an official model card, which is available in Appendix D of the paper. Content from this model card has been written by the Hugging Face team. To quote the first two paragraphs of the official paper OPT was predominantly pretrained with English text, but a small amount of non-English data is still present within the training corpus via CommonCrawl. The model was pretrained using a causal language modeling (CLM) objective. OPT belongs to the same family of decoder-only models like GPT-3. As such, it was…

Open weights other 2,048 tokens transformers
View model

Model · Text generation

Ornith-1.5-9B-GGUF

Ornith

Chirp Chirp! We are introducing Ornith-1.5, a major step toward building foundation models through end-to-end self-improvement. Ornith-1.5 extends Ornith-1.0 (which was developed on top of Qwen3.5 and Gemma4 with additional continued pretraining, mid-training, and post-training) by expanding the self-improvement loop from scaffold and rollout optimization to jointly optimizing task generation, scaffold construction, and solution rollouts. Rather than relying on a fixed set of human-curated tasks and manually designed harnesses, Ornith-1.5 continuously generates new training tasks, discovers effective strategies for solving them, and improves the policy through reinforcement learning. For…

Open weights mit transformers
View model

Model · Text generation

Ternary-Bonsai-2-27B-gguf

Prism ML

Full 27B-class reasoning in ternary transformer weights, for llama.cpp (CUDA, Metal, CPU) - \~5.9 GB language model (down from \~54 GB FP16) — full 27B-class reasoning on a standard laptop or a single GPU - 98.2% of FP16 intelligence retained: 84.78 average across 14 thinking-mode benchmarks — far above the conventional IQ2XXS build (72.59) at about 82% of its footprint, and within 0.4 points of UD-Q4KXL at three times the footprint - Retains thinking, reasoning, and agentic behavior deep in the sub-4-bit regime, where conventional low-bit representations collapse: math within half a point of full precision (96.57), coding level with the baseline (89.42), agentic tool calling at 74.92…

Open weights apache-2.0 llama.cpp
View model

Model · Text generation

Ornith-1.5-35B-A3B-GGUF

Ornith

Chirp Chirp! We are introducing Ornith-1.5, a major step toward building foundation models through end-to-end self-improvement. Ornith-1.5 extends Ornith-1.0 (which was developed on top of Qwen3.5 and Gemma4 with additional continued pretraining, mid-training, and post-training) by expanding the self-improvement loop from scaffold and rollout optimization to jointly optimizing task generation, scaffold construction, and solution rollouts. Rather than relying on a fixed set of human-curated tasks and manually designed harnesses, Ornith-1.5 continuously generates new training tasks, discovers effective strategies for solving them, and improves the policy through reinforcement learning. For…

Open weights mit transformers
View model

Uncensored Qwen3.8-27B, published as GGUF quantizations with the multi token prediction (MTP) head retained and verified. Refusal behaviour has been substantially reduced, not eliminated. See Measured behaviour for the numbers. Capabilities, training data, and architecture are otherwise unchanged. - Refusal directions removed with Heretic, which co minimizes refusal count against KL divergence from the base model. No handwritten refusal removal code, no finetuning, no additional training data. - Abliteration runs at bf16 (no 4 bit quantization). the resulting LoRA is merged into the bf16 base, so the published weights are not a quantized round trip. - mtp. tensors are copied verbatim from…

Open weights apache-2.0 llama.cpp
View model

Model · Text generation

Ornith-1.0-9B-GGUF

Ornith

Aloha! Today, we are releasing Ornith-1.0, a self-improving family of open-source models for agentic coding. This model card documents Ornith-1.0-9B, the most lightweight member of the Ornith family, designed for efficient single-GPU deployment. Ornith-1.0-9B is a dense ~9B model (≈19 GB in bf16), so it serves comfortably on a single 80GB GPU. The recipes below stand up an OpenAI-compatible server; add --tensor-parallel-size / --tp if you want to shard across more GPUs. For a quick local test (or to script offline generation), load the model directly with Transformers. Make sure you have a recent release installed — see the Transformers installation guide; Ornith-1.0-9B requires…

Open weights mit transformers
View model