Ternary Bonsai 2 27B -- q27 pack
PrismML's Ternary Bonsai 2 27B
repacked for the q27 engine. It is
Qwen3.8-27B with every projection ternary (one fp16 scale per 128 weights)
in a Hadamard-rotated basis, and no MTP block. This pack is an exact
re-encoding of the PTQ1_0 GGUF, not a requantization: every trit is stored
as-is in q27's 2-bit T2_G128 container, the embedding and the head as
int8 (also exact), and the rotation metadata is carried verbatim so the
engine rotates activations itself.
| file |
GB |
what |
bonsai2-27b-t2.q27 |
9.44 |
the model (T2 body, Q8 embed/head, Hadamard signs) -- single-slot serving |
bonsai2-27b-t2-mtp.q27 |
9.87 |
the same plus an MTP head as blk.64 (below) -- multi-slot serving, needs q27 v0.13.0 |
qwen38-27b-mtp.tok |
0.0002 |
the Qwen3.8 tokenizer, byte-identical to the one in the Qwen3.8 repo |
Needs q27 v0.12.0
or later (the MTP pack: v0.13.0) (docs/plans/2026-09-18-bonsai2-ternary.md in the repo has the
design). The pack has no MTP head, so DFlash2 is the only speculative path.
The Qwen3.8 drafter packs in
signalnine/Qwen3.8-27B-MTP-q27
work against this target as-is (3.47 tokens per round on Claude Code
traffic); a drafter trained on the ternary target does better --
ProCreations' Ternary-Bonsai-2-27B-DFlash2
(independent, Apache-2.0) packs unchanged with python3 tools/dflash2_pack.py
<its dir> bonsai2-dflash2-q8-serve.d2w --q8 and gives 3.80 tokens per round,
227.7 t/s aggregate on the same run.
The MTP pack (multi-slot)
bonsai2-27b-t2-mtp.q27 is the pure pack plus
ProCreations' Ternary-Bonsai-2-27B-MTP
head (independent work, Apache-2.0, not a PrismML release): the Qwen3.8 MTP
block distilled onto Bonsai's hidden states, stored here as the pack's
blk.64 (Q8 matmuls, F32 norms, kept unrotated -- the engine rotates
nothing on that layer). Every other tensor is byte-identical to the pure
pack. With it the engine runs its ordinary speculative ladder against the
ternary target, which is what makes fused multi-slot rounds pay: on a 5090
at 8 slots / 16K, C = 1 / 2 / 4 / 8 concurrent streams decode at 173 / 231
/ 384 / 512 t/s aggregate (the draftless pure pack: 105 / 150 / 200 / 329;
the Qwen3.8 q4s tier: 141 / 230 / 352 / 531). Serve it with
--slots 8 --ctx 16384 and Q27_BATCH=1, no DFlash2 pack. For one slot
the DFlash2 route below is faster (227.7 vs 178.8 t/s on Claude Code
traffic), so this pack is the multi-slot choice, not the default. The
ladder reproduces plain decode byte for byte over 1500 CLI tokens; load
digest wsum: 0f85b21f8721e39c. Rebuild from the two upstream checkpoints:
python3 tools/repack.py Ternary-Bonsai-2-27B-PTQ1_0.gguf out.q27 --mtp-safetensors model_mtp.safetensors.
How faithful the port is
Bit-exact containers make this checkable against the reference fork on the
same GGUF:
- teacher-forced logits vs the fork over 383 positions: top-1 agreement
0.9974, NLL 2.0792 vs 2.0796
- wikitext-2 perplexity at 2048: q27 8.2767 vs
llama-perplexity 8.2643
(q27 runs fp8 KV and int8 activations, which is the 0.15%)
- the 2-bit prefill GEMM is bitwise the int8 tensor-core GEMM on an exact
Q4 image of the same matrices, and the full-corpus NLL is identical to
six digits through both paths
What the checkpoint measures like
Same instruments as the Qwen3.8 tier table, so read these side by side:
|
Bonsai 2 (this pack) |
Qwen3.8-27B q27 default tier |
| GB |
9.44 |
17.00 |
| wikitext PPL (chunk 512, fp8 KV) |
9.2508 |
7.3121 |
| HumanEval+ (30, no-think) |
25/30 |
30/30 |
| needle |
6/6 at ~90K |
6/6 at ~120K |
| RTX 5090 decode, DFlash2 K=7 |
233 t/s (700-token prompt), 346 (short) |
~220 |
| free VRAM after weights, 32 GB card |
23.2 GB (auto ctx 262K) |
~15 GB |
On a 12-instance Claude Code SWE-bench run it lands the same patches as the
Qwen3.8 tiers (gold 11/12, identical miss) at 205 vs 222 t/s aggregate,
but it reasons about twice as long per instance -- 37 vs 22 API turns, 75K
vs 35K thinking characters -- so wall time is 2.3x. It is a different
checkpoint, not a smaller tier of the model above; pick it for the 9 GB
footprint and the context that buys, not for quality.
Serving
hf download signalnine/Bonsai-2-27B-q27 --local-dir models/bonsai2
hf download signalnine/Qwen3.8-27B-MTP-q27 qwen38-dflash2-q8-serve.d2w --local-dir models/qwen38
(cd models/bonsai2 && md5sum -c CHECKSUMS.md5)
Q27_KV=fp8 Q27_BATCH=0 Q27_DFLASH2=models/qwen38/qwen38-dflash2-q8-serve.d2w Q27_DFLASH2_RESERVE_GB=3 \
./build/q27-server models/bonsai2/bonsai2-27b-t2.q27 models/bonsai2/qwen38-27b-mtp.tok \
--think --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.05 --think-budget 0
That is the Qwen3.8 agentic recipe; the engine detects the pack, turns the
activation rotation on and the MTP ladder off, and selects the 3.8 XML tool
dialect from the name. Weight digest on load (Q27_PRINT_WSUM=1):
wsum: 9161ec16da4b66a1. To rebuild the pack from the upstream GGUF:
python3 tools/repack.py Ternary-Bonsai-2-27B-PTQ1_0.gguf bonsai2-27b-t2.q27
(about three minutes, numpy only; the converter verifies every ternary slot).
Upstream checkpoint and license: Apache-2.0, PrismML
(prism-ml/Ternary-Bonsai-2-27B-gguf, Ternary-Bonsai-2-27B-PTQ1_0.gguf).
The MTP head in bonsai2-27b-t2-mtp.q27 is ProCreations' work
(ProCreations/Ternary-Bonsai-2-27B-MTP, model_mtp.safetensors, Apache-2.0).
