Prism ML's ternary Ternary-Bonsai-2-27B
build of Qwen/Qwen3.8-27B, repacked for
chad, a Claude-Code-style local coding agent for
Apple Silicon, with its speculative decoder bundled in. This is chad's default model.
Created using Bonsai by Prism ML.
One download gets both halves: the ternary target model and the block drafter it decodes
with, already quantized. Nothing is built on first run.
What the weights are
Every projection of Qwen3.8-27B (a dense qwen3_5 hybrid: 64 layers, 48 GatedDeltaNet +
16 full attention) is stored in a Hadamard-rotated basis: multiplied by a fixed sign
vector and put through a blockwise Walsh-Hadamard transform offline, then quantized to
2-bit affine group-128 whose three levels reproduce the ternary set {−s, 0, +s}. The
rotation costs no extra bits and no extra weight traffic, but the matching transform must be
applied to the activations at runtime and inverted after the embedding lookup. An
ordinary MLX affine loader finds tensors of exactly the right shapes, skips both transforms,
and produces plausible garbage without raising. chad routes the pack on its model_type
(prism_hadamard_qwen35) to a loader that does the rotation; mlx-lm alone will not run it
correctly.
- Footprint: 7.15 GB of weights + a 1.1 GB bundled drafter; 262k native context.
- RAM: built for 24 GB Apple Silicon. chad sizes its context window from the live Metal
budget: on a 24 GB M4 Pro the compaction threshold lands around 150k tokens, against
~30k on the 3-bit quant this replaces.
- Speed (M4 Pro, greedy, 512-token prompt, 128-token decode): 21 tok/s serial, 64
tok/s with the drafter (94% of drafted tokens accepted); prefill ~99 tok/s.
- Quality: teacher-forced NLL on code is unchanged against the pack's own fp32 forward
(1.502 vs 1.500); chad's private core and languages eval tiers pass 13/13 and 17/17, the
same as the 3-bit quant.
What differs from the Prism pack
- Text only. The pack's FP16 vision tower (0.92 GB) is dropped;
config.json says so
(components.vision: false). The language-model tensors, sign vectors and
hadamard.json are byte-identical to the source revision recorded in the safetensors
header (source_revision).
- Tokenizer and chat template from the base model. The pack's
tokenizer.json has the
same vocabulary and added tokens but different merges and pre-tokenizer, which trips
transformers' regex warning on every load and diverges on combining marks. This repo
ships the base tokenizer, and the chat template with reasoning_effort defaulting to
medium (the pack's says xhigh; upstream Qwen's says xhigh). Pass
reasoning_effort explicitly to override it.
config.json carries the base checkpoint's transformers_version. Without it,
transformers' Mistral-regex heuristic treats any checkpoint as a legacy Mistral
tokenizer and warns on every load that tokenization is wrong; the key is what the base
Qwen3.8-27B config carries, and the tokenizer files here are that checkpoint's.- No bundled runtime. The pack ships Python it asks callers to
sys.path.insert; chad
reimplements the forward and does not import code out of a model download.
dflash/: the DFlash2 block drafter, identical to the one bundled with
nathansutton/Qwen3.8-27B-UD-Q3_K_XL-DFlash2-MLX.
The bundled DFlash2 drafter (dflash/)
A 1.9B model that reads the target's residual stream at five tapped layers and proposes a
whole block of tokens in one forward, quantized to 4-bit group-64 in MLX format. A
drafter reads the residual stream, which the weight quantization only perturbs, so the
drafter trained against the bf16 base transfers to the ternary target unchanged. chad
verifies the block in one batched target forward and accepts by exact rejection sampling,
so greedy output is token-identical to unspeculated decoding (to kernel rounding) and
sampled output keeps the model's true distribution at any temperature.
It is deliberately in a subdirectory: mlx-lm globs model*.safetensors at the repo root,
so dflash/model.safetensors is invisible to the base loader.
The drafter weights are converted from the DFlash2 release for this model
(z-lab/Qwen3.8-27B-DFlash2); the
method is Chen et al., DFlash: Block Diffusion for Flash Speculative Decoding
(arXiv:2602.06036). Only the quantization and the MLX layout are ours; please honor the
original release's license and citation terms.
Use it
uvx chad-code # this is the default model; first run downloads it (~8 GB)
License and attribution
Apache-2.0. The ternary weights are copyright Prism ML, Inc., built from Qwen3.8-27B
(copyright Alibaba Cloud, Apache-2.0); LICENSE and NOTICE.txt are the pack's own.
Created using Bonsai by Prism ML.
