GLM-5.3-Flash-dealignai-orcarouter-merge

GLM-5.3-Flash — Orthogonal Abliteration Merge (dealignai + orca)

A weight-space combination of two independent abliterations of the same base model, produced by different teams with different methods. The two edits turn out to lie in nearly orthogonal directions, so applying both removes the refusals that each one leaves behind — the combination is more thoroughly decensored than either parent, with no fine-tuning and no gradient steps.

This card documents what was done and why it works at the weight level. It does not report harmful- capability evaluations, and standard capability benchmarks (MMLU, etc.) have not been run — see Limitations.

Uncensored model. This model has had safety-refusal behavior substantially removed. You are responsible for how you use it and for compliance with the base model's license and applicable law.

Contents / files

Why Q8_0 is the base to requantize from. Both parents are FP8 (e4m3) native. Q8_0 (int8 + one fp16 scale per 32 weights) preserves FP8's ~3-mantissa-bit values essentially losslessly, so it is the correct high-precision source here — better than a BF16 upcast, which would only pad FP8 values into a wider container without recovering any precision. To make your own quant (e.g. a smaller K-quant or a different imatrix), run llama-quantize on the Q8_0 file rather than re-deriving the merge.

Built with a GLM-5.3-Flash-capable llama.cpp (the glm5next branch, which adds the architecture).

The two parents

Both are MIT-licensed weight-level abliterations of zai-org/GLM-5.3-Flash (MIT), produced independently by different teams with different techniques. Throughout this card they're referred to by the short names dealignai and orca:

The independent measurements in this card corroborate both parents' published descriptions: orca's edit is recovered as a single direction with per-layer cosine consistency 0.995–1.000 — exactly what a single- vector Arditi ablation predicts — while dealignai's edits are per-layer and mutually independent.

How the merge was done

Both parents are the same base model with the same architecture (1412 tensors, identical shapes). Comparing them tensor-by-tensor (dequantized to fp32):

• 852 of 1412 tensors are byte-identical between the two — neither abliteration touched them (all ffn_gate/up experts, attention Q/K projections, every norm, routers).
• The edits are confined to the residual-writing tensors: attn_output, ffn_down / ffn_down_exps / ffn_down_shexp, token_embd, and nextn.eh_proj. output.weight is not edited by either. (attn_q/k/v differ only by quantization re-rolling noise, not by any directional edit.)

Each abliteration is a linear map on those tensors. Writing g for orca's global unit direction and v_ℓ for dealignai's per-layer directions:

orca:       W  ->  (I - g gᵀ) W                    # full projection, every layer
dealignai:  W  ->  (I - α_ℓ v_ℓ v_ℓᵀ) W            # partial, layers ~12–43
combined:   W  ->  (I - g gᵀ)(I - α_ℓ v_ℓ v_ℓᵀ) W  # this model

The merge takes dealignai's weights (the cleaner FP8-native base) and applies orca's projection (I − ggᵀ) on top, on exactly the tensor set orca originally edited. g was recovered at Q8 precision from the parents' own difference matrices (the top left-singular vector of W_orca − W_dealignai on tensors dealignai left untouched).

Why it works: the directions are nearly orthogonal

The reason applying both edits helps, rather than one clobbering the other, is that g and the v_ℓ point in almost unrelated directions in the residual stream. The cross term in the composition, α (gᵀv_ℓ) g v_ℓᵀW, is negligible because gᵀv_ℓ ≈ 0, so the two edits commute and their removals add.

Directionality data (measured on this build; cosine similarities of unit vectors in the 4096-dim residual space):

The projection is idempotent — re-applying it to the finished model changes nothing (‖W'−W‖/‖W‖ ≈ 0.0003, i.e. quantization noise), confirming g was cleanly removed rather than merely attenuated.

Intuitively: orca swept out one global refusal axis everywhere; dealignai gently pushed down a different set of per-layer axes on the middle of the network. Because those axes barely overlap, the merged model has both removed, and refusals that survived one method (because that method wasn't aimed at them) are caught by the other.

Evidence: refusal comparison

The point of the merge is that it removes refusals each parent still has. The grid below is the direct evidence, on a small matched probe set (7 requests spanning standard harm categories + benign controls, greedy decoding). This reports only the refuse-vs-comply outcome per category — the standard way abliteration cards report results (both parents publish HarmBench-320 refusal rates); no generated harmful text is included here.

Hard refusals (lower = more decensored):

Per-category, at low reasoning (R = refused, C = complied, safe = safety-preserving non-harmful response, not a method):

Reading it: orca clears the two dealignai-only refusals (explosive, drugs); dealignai's method doesn't help self-harm/hate/fraud but orca's global projection does; the combination has none of them. The same pattern holds at max reasoning (combined 0/7). A few combined max cells complied within the reasoning trace and hit the generation-length cap before the final answer — counted as non-refusals, since no refusal appears anywhere in the output.

• Self-harm is the deliberate exception: all three models — and the merge — return a supportive, safety-preserving response (not a method). Neither abliteration nor the merge removed that, and this card does not treat overriding it as a goal.
• A greedy smoke-test generation is fluent and on-task (e.g. a correct Rayleigh-scattering explanation).

Reproducing the merge

The scripts/ folder contains derive_g.py, project_q8.py, and the precomputed direction g_q8.npy, so you can skip step 1 and go straight to the projection:

# 1. (optional — g_q8.npy is already included) recover orca's direction g at Q8 from the two parents:
python3 derive_g.py g_q8.npy
# 2. apply (I - g gᵀ) to the dealignai Q8_0 on orca's target tensors, streaming a new Q8_0:
python3 project_q8.py --g g_q8.npy --in <dealignai-q8_0.gguf> --out <combined-q8_0.gguf> --name "..."
#    (dry-run first with --check to confirm the g-residual on target tensors drops to ~0.0006)
# 3. quantize to IQ4_XS with the same imatrix used for the parents:
llama-quantize --allow-requantize --imatrix <imatrix> <combined-q8_0.gguf> <out-iq4_xs.gguf> IQ4_XS

project_q8.py copies all non-target (unedited) tensors byte-for-byte and re-quantizes only the tensors orca edited.

Limitations and what is NOT claimed

• No capability benchmarks were run. MMLU / GSM8K / etc. are not reported. Removing two orthogonal directions can plausibly cost more capability than removing one; that impact is unmeasured. If you care about the "smarts," evaluate before relying on this — dealignai specifically preserved some low-reasoning refusals to protect capability, and this merge overrides that trade-off.
• Refusal outcomes are reported at the category level (refuse vs comply), as evidence the merge works; no generated harmful text or instructions are included. The probe set is small (7 items) and is not a substitute for a full safety benchmark like HarmBench-320 (which both parents report against).
• Directionality figures are from one build and a sampled set of tensors; they are stable across the layers sampled but were not exhaustively computed for all 1412 tensors.
• Quantization is IQ4_XS with a specific imatrix; behavior at other quant levels may differ.

Attribution and licensing

Derivative of three MIT-licensed works (license verified 2026-09-17); MIT permits redistribution of this merged derivative provided the copyright/permission notice is retained:

• Base model — zai-org/GLM-5.3-Flash (Z.ai / Zhipu AI), MIT.
• dealignai — dealignai/GLM-5.3-Flash-ABLITERATED-FP8 ("CRACK"), MIT. Credit: dealignai, compute via @jordanschenck.
• orca — orcarouter/GLM-5.3-Flash-Uncensored-FP8 (OrcaRouter / Continuum AI Corp), MIT. Method: Arditi et al. (2024).

Retain each upstream's MIT notice. The "uncensored / not-for-all-audiences" tags are set in the metadata above.

Model card drafted with assistance from Claude Code; the measurements and directionality data were produced and verified independently.