bonsai2-abliteration-pipeline

Ternary Bonsai 2 27B Abliterated (GLP alpha 0.80)

Responsible use

This is an uncensored research model. Its built-in refusals were removed, so it answers prompts the base model declines, including chemical and biological synthesis, cybercrime, weapons, harassment, and fraud. Use it for red-teaming, offensive-security research, and refusal-rate evaluation. It has no guardrails of its own: if you deploy it, add your own input and output moderation (for example Llama Guard).

What this is

Abliterated variant of prism-ml/Ternary-Bonsai-2-27B-gguf (PrismML's ternary compression of Qwen/Qwen3.8-27B, 27B hybrid attention + GatedDeltaNet). The refusal direction was removed with runtime projective steering (GLP, the weightless method behind the Inkling steering vectors), applied at every residual writer of the fork's qwen35 graph. This is a refinement of directional ablation (Arditi et al. 2024) adapted to a model whose weights cannot be projected: Bonsai 2's language weights are ternary at group 128 in a Hadamard-rotated basis, and projecting then repacking changes none of the 89,128,960 ternary digits (measured; the lattice step is 100% of what a weight can represent, the projection asks for ~1.4%).

The ternary checkpoint is never modified. The steering vector in this repo is the entire behavioral change, and deleting it (or setting WEIGHTLESS_STEER_ALPHA=0, verified bit-identical) reverts the runtime to stock. pipeline/eval/dirs.npz holds the measured per-layer directions, so you can re-ablate at any strength in seconds without re-measuring. abliteration_config.json records the final parameters and every evaluation number.

For runtimes that cannot apply the patch, two baked drop-ins carry the same edit in both ternary pack formats: gguf/Ternary-Bonsai-2-27B-abliterated-df32k-PTQ1_0.gguf (5.95 GB) and gguf/Ternary-Bonsai-2-27B-abliterated-df32k-PQ2_0.gguf (7.21 GB). Each is 2,284,995 ternary digits flipped in the writer tensors of layers 15-63, group scales untouched, everything outside the flips byte-identical to the corresponding base release (verified: both packs decode to the same model). They keep MMLU at +0.21pp but remove only part of the refusals; the measurements below explain why the vector is the recommended artifact. An ungated companion repo, distributedcognition/bonsai2-abliteration-pipeline, mirrors the patches, pipeline code, directions, and baked files for CI and tooling that cannot pass the access gate.

Load

The GLP vector needs the PrismML llama.cpp fork (prism branch, commit 9a9394a) with the hotfix patch from this repo:

git clone -b prism https://github.com/PrismML-Eng/llama.cpp && cd llama.cpp
git checkout 9a9394a895b96003ca842a6041cb28ac49a108f7
git apply 0001-qwen35-glp-steering.patch
cmake -B build -DGGML_CUDA=ON && cmake --build build -j

WEIGHTLESS_STEER_PATH=glp/Ternary-Bonsai-2-27B-GLP-64-a0.80.gguf \
./build/bin/llama-server -m Ternary-Bonsai-2-27B-PTQ1_0.gguf -ngl 99 --jinja

The reader is fail-closed: the vector file must carry glp.mode=project, a missing or malformed file aborts the process, and WEIGHTLESS_STEER_ALPHA multiplies the file's strength at runtime. The baked GGUF needs no patch; serve it with the stock fork like the base release. The fork itself is required either way: stock llama.cpp cannot load PTQ1_0.

Evaluation (alpha 0.80)

The standard abliteration protocol (the one Heretic made automatic): 104 held-out harmful prompts (mlabonne/harmful_behaviors test split) with normalized keyword detection plus an LLM judge (the base model classifying its own responses), MMLU as letter-logprob over the full 14,042-item test set in chat format, and a benign over-refusal check on mlabonne/harmless_alpaca. Eval mode: chat, greedy, enable_thinking=false, native PTQ1_0 runtime (Lambda A10 CUDA, spot-verified on Apple Metal; letter-logprob scores were bit-identical across VMs).

The keyword metric over-counts at low alpha for the opposite reason it usually does: most flagged items are hedged answers where the detector fires on words like illegal and harmful. The judge's 4/104 is the honest number. The MMLU delta is +1 item of 14,042 (+0.007pp), which is 0.00pp at reporting precision. The abliterated model keeps the hedged style; responses often open with a warning or a ToS note before answering.

MMLU-14k (full test set, chat letter-logprob, T=0, native PTQ1_0 runtime)

No measurable capability cost. The alpha curve (full test set, true refusals from the judge): alpha 0.78 reads −0.02pp, 0.85 reads −0.03pp with 2/104 refusals, 1.00 reads −0.14pp with 2/104 refusals, and the 60-layer quality-gated vector reaches 0/104 refusals at −0.30pp. Per-subject movement at alpha 1.0 is 22 up, 21 down, 14 flat, concentrated in the ethics cluster (moral_scenarios −1.45pp), the same signature the DeepSeek-V4.1-Flash abliteration showed at its own 0.00pp operating point. For comparison, the most aggressive baked artifact here (131,072 flips per tensor) measured −0.48pp for 16/104 refusals, and dealignai's CRACK v2 reported −0.62pp on MMLU-2280 for 0% refusals on HB320.

Abliteration at this strength is not always free on the ternary lattice: a weight edit must move whole digits, each flip overshooting the ideal sub-lattice correction by 3 to 10 times, so the baked GGUF trades refusal removal against MMLU along a measured curve (96/104 judge refusals at +0.21pp, 16/104 at −0.48pp). The runtime projection does not have this constraint, which is why it holds 0.00pp while removing 96% of refusals.

PQ2_0 parity (MMLU-2280, same protocol): the official PQ2_0 pack holds the same ternary codes as PTQ1_0, and the fork serves it through different kernels, which shifts letter-logprob MMLU by −0.09pp (2 of 2277 items) on the stock model. Against that pack's own base, GLP alpha 0.80 again reads exactly 0.00pp (identical 1730 of 2277 correct) with 3/104 judge refusals, and the baked df32k GGUF reads −0.09pp with 96/104, matching its PTQ1_0 behavior. The artifacts are format-portable as designed.

Serving

Validated end to end on the PrismML fork (prism branch at 9a9394a) with the hotfix applied, CUDA A10 and Apple Metal, --jinja chat completions, streaming, and logprobs all pass:

WEIGHTLESS_STEER_PATH=glp/Ternary-Bonsai-2-27B-GLP-64-a0.80.gguf \
./build/bin/llama-server -m Ternary-Bonsai-2-27B-PTQ1_0.gguf \
  -ngl 99 -c 8192 --flash-attn on --jinja

Non-obvious requirements: the vector path must be absolute or relative to the working directory at launch, the vector's glp.n_embd must match the model (the reader aborts otherwise), and thinking mode is on by default; benchmarks above ran with --chat-template-kwargs '{"enable_thinking": false}' or the equivalent template kwarg. With thinking enabled, budget generation length generously, the same as for the stock model. The same vectors work unchanged on the PQ2_0 pack: same residual basis, same measured behavior (see the parity numbers below).

Method

Per-layer refusal directions were measured natively on the ternary runtime, through the fork's multi-layer hidden-state tap at the last prompt token, on 128 harmful (mlabonne/harmful_behaviors) vs 128 harmless (mlabonne/harmless_alpaca) prompts: per-layer diff-in-means, orthogonalized against the harmless mean direction (projected abliteration). Median held-out AUC is 1.000 with split-half cosine 0.987, and adjacent-layer cosine median 0.906 against a 0.04 null. The directions are applied as h <- h - alpha * r (r^T h) at every residual writer (attn_output, ssm_out, ffn_down) of the 64 trunk layers and in the MTP draft graph. The baked GGUF flips ternary digits selected by purchase ratio |dW[o,j]| / s_block, with u = W_stored^T r computed in the stored basis, where the Hadamard fold cancels and the group scales are never touched. The full pipeline (PTQ1_0 codec validated byte-for-byte against llama-quantize in both directions, capture, fit, vector writer, digit-flip baker, refusal analysis) is in python.tar.gz, and every raw prediction file behind the numbers is in eval-results/.

Apache 2.0, inherited from the base model. Base model: prism-ml/Ternary-Bonsai-2-27B-gguf, derived from Qwen/Qwen3.8-27B (Alibaba). Abliteration follows the residual-direction method (Arditi et al., Refusal in LLMs is mediated by a single direction, NeurIPS 2024). Not affiliated with or endorsed by the base-model authors.