GLM-5.3-Flash-Uncensored-W4A16

Quantization of this checkpoint (W4A16 routed experts)

This is a quantized build of GLM-5.3-Flash-Uncensored-FP8. Only the routed MoE experts are re-quantized; every other tensor is kept exactly as in the FP8 source (FP8 tensors stay FP8, BF16 tensors stay BF16). The model card below this section is the original card of the FP8 checkpoint.

Total ~179 GiB (the FP8 source is 306 GiB).

Format. compressed-tensors, mixed-precision with two config groups in config.json:

• group_0 (pack-quantized, targets re:.*mlp\.experts\..*): the layout llm-compressor writes, per expert projection weight_packed (int32, eight 4-bit values each), weight_scale (BF16, one per 32 consecutive input-channel weights) and weight_shape.
• group_1 (float-quantized, targets the shared experts and the dense MLPs of layers 0-2): the source's FP8 e4m3 weights with FP32 128×128 block scales, unchanged except that weight_scale_inv is renamed weight_scale, and the source's dynamic FP8 activation scheme (per token, groups of 128).
• The FP8 MLA projections keep the source's weight_scale_inv and belong to neither group; vLLM's GLM-5-Next model dequantizes them to 16 bits at load. ignore lists lm_head and re:.*visual.*.

Method. Data-free round-to-nearest, no calibration data (no GPTQ / AWQ). Each expert's FP8 weights are first dequantized with their 128×128 block scales (in FP32). Each weight is then q × scale with q in [-8, 7]. For every group of 32 weights the scale is chosen like llm-compressor's MSE observer: the group's abs-max is shrunk by 0-19 % in 1 % steps and the scale with the smallest Σ|error|^2.4 is kept (stopping after 5 steps without improvement).

Activations and KV cache. The experts are weight-only quantized (16-bit activations); the checkpoint defines no KV-cache quantization.

The abliterated (refusal-removed) build of zai-org/GLM-5.3-Flash — a 320B / 18B-active Mixture-of-Experts model with hybrid linear + sparse attention, 4-wide Manifold-Constrained Hyper-Connections, a native vision + video tower, an MTP speculative head, and a 1M-token context.

The refusal direction is baked directly into the official block-FP8 shards — same format, same shard layout, same model.safetensors.index.json. Every tensor matches the base checkpoint in name, dtype and shape, so this is a drop-in replacement for zai-org/GLM-5.3-Flash in any stack that already serves it.

On the -FP8 name: block-FP8 is the format Z.ai ships GLM-5.3-Flash in — the suffix names the checkpoint's own precision, not a quantization step we applied. There is no upstream BF16 release to derive from, so this is the full-precision source as published.

Disclaimer — read before use

This model has had its safety alignment substantially removed via abliteration (orthogonalizing the refusal direction out of the residual stream). As a direct consequence:

• It will comply with harmful, unethical, offensive, or illegal requests that the original GLM-5.3-Flash would refuse. It has no meaningful built-in guardrails.
• It is released strictly for legitimate research — interpretability, AI-safety and refusal-mechanism study, red-teaming, robustness evaluation, and controlled experiments.
• You assume full responsibility and liability for how you use it and for everything it generates. Do not deploy it to end users or in production without adding your own safety, moderation, and abuse-prevention layers.
• Use must comply with the MIT License inherited from the base model, and all laws and regulations that apply to you.
• The authors and uploaders accept no liability for any misuse or harm. Its outputs do not reflect the views of the uploaders or of Z.ai / Zhipu AI.

By downloading or using this model you acknowledge and accept the above.

Run it with OrcaCode Review

Models are only half the system.

OrcaCode Review turns every model listed on OrcaRouter into a production code-review agent:

• reviews every PR
• finds security + correctness issues
• posts inline findings
• P0/P1 can block merges
• swap models anytime

Open model. Open harness. Open bill.

• Website — https://www.orcarouter.ai
• GitHub — https://github.com/Continuum-AI-Corp/Orca-Code-Review

Model details

Abliteration

Refusal-direction removal following Arditi et al. (2024), Refusal in Language Models Is Mediated by a Single Direction. A single refusal direction r (k = 1) is estimated as the massive-activation-masked mean-difference of harmful − harmless activations, read from the 4096-d residual stream at layer 22 of 45 (depth 0.49) and selected by a 9-layer quality sweep. r is then orthogonalized out of every residual-writing matrix — W' = W − r(rᵀW) — computed in float32.

Why it had to be baked on disk

GLM-5.3-Flash cannot be abliterated in memory, for three independent reasons:

• Its weights are block-FP8: raw e4m3 codes whose scales live in a sibling weight_scale_inv. Projecting the codes corrupts the matrix — and the leakage probe, reading the same code space, reports a false ~0.
• Its 12,384 routed-expert down_proj matrices are fused into stacked 3-D parameters at load time, so a conventional 2-D-weight matcher edits none of them and still reports success.
• transformers drops the MTP block (layers.45) on load, so save_pretrained would silently omit 6.98 GiB of the checkpoint.

The shards are therefore streamed and rewritten one at a time. FP8 writers are dequantized with their own scales, projected in fp32, requantized, and written back together with their updated scales; BF16 writers (the 34 KDA o_proj, the vision merger, the MTP eh_proj) are projected and stored back as BF16 without ever entering the FP8 cycle.

What was edited

12,442 of these are FP8 and 36 are BF16. Max residual leakage after the edit: 0.149 (FP8) and 0.024 (BF16). A post-bake verification pass confirms all 62 shards and 76,108 tensors match the base checkpoint in name, dtype and shape.

Result

Held-out JailbreakBench test split, 64 harmful + 64 benign prompts, greedy, reasoning_effort=low, the <think> block stripped before classification:

Harmful refusal down ~89%; over-refusal eliminated; generation stays coherent and on-task on both harmful and benign prompts. This is the acceptance metric used to select the build — see Evaluation below for the benchmark suite.

What resisted, and what that says about the base model

The single-direction hypothesis held for most of the harmful-prompt distribution, and where it held it held cleanly — several categories drop to a refusal rate of zero. It did not hold everywhere. A subset of content categories resisted every variant we tried, and the ways they resisted are more interesting than the fact that they did:

• No other layer mediates them. A direction fitted only on the resistant categories and swept across candidate depths bypassed them worse than the general direction did — 0.75–0.98 against 0.33.
• More fitting data does not move the estimate. Pooling five harmful datasets produced a direction with cosine similarity 0.957 to the original and no meaningful improvement.
• A second direction makes it worse, not better. Adding the leading principal component of the harmful-residual spread took refusal from 0.207 to 0.993; pairing layer 22 with a direction from another layer returned 1.000, the un-ablated baseline. In both cases the model stayed fluent and undamaged — zero unfinished generations, coherent on-topic replies. It was not broken into refusing; it simply refused.
• Restricting the edit to later layers collapses the effect (0.92 at layers ≥ 22), even though the direction is estimated at layer 22. The early-layer writers are load-bearing.

The single layer-22 direction is therefore not merely the best point we found; it is a narrow optimum that any perturbation of the subspace destroys. The reading we take from this is that GLM-5.3-Flash's safety training is not wholly mediated by one linear direction in the residual stream. Part of it is, and that part is what abliteration removes. The remainder is encoded in a form this technique cannot reach at all — not partially removed, not weakened, simply untouched. Published abliterations of other frontier models routinely reach near-zero refusal across every benchmark; that this one does not is a property of the base model, not of the method, and it is the strongest evidence in this work that Z.ai's alignment goes deeper than a single steerable feature. For anyone studying refusal mechanisms, that residual is the interesting part — and for anyone deploying the base model, it is a meaningful robustness result.

Where the refusal actually lives

Hook-ablating exactly the residual writers each bake stage would edit (lossless, no bake required):

On GLM-5.3-Flash the refusal direction is written almost entirely by the routed-expert down_proj matrices: attention and the shared/dense MLP together move refusal by 0.03, the routed experts move it by 0.81. Editing the writers even beats hooking the layer boundary — the mHC structure lets a layer re-read its own attention output before the MLP, and a writer edit cleans that intermediate state where a layer-output hook does not.

Requantization leakage, and why 32 iterations

Orthogonalizing a block-FP8 matrix and requantizing to e4m3 leaves part of the direction behind. Measured on a real MLA o_proj against this direction:

With only 45 attention writers (34 of them BF16) that leakage is lost in the noise. Across 12,442 FP8 matrices it accumulates: the 8-iteration bake landed at 0.188 harmful refusal against a lossless prediction of 0.031. Raising it to 32 closed roughly half the gap — 0.188 → 0.094. Beyond 32 the leakage falls slowly while the weight perturbation keeps rising, so 32 is the shipped operating point.

Fine-tuning & post-training

• The checkpoint is a drop-in for zai-org/GLM-5.3-Flash in the transformers / Glm5NextForConditionalGeneration stack (needs transformers 5.16+).
• Re-quantization: derive GGUF / MLX / lower-bit builds from these weights directly.
• Note: abliteration is a weight edit, not data-level unlearning. Fine-tuning on refusal-heavy or safety data can partially re-introduce refusals; neutral / task data preserves the uncensored behaviour.

Intended use

• Research into refusal mechanisms, alignment, and interpretability.
• Red-teaming and safety / robustness evaluation in controlled environments.
• A base for further post-training and quantization.

Evaluation

Measured on these exact weights, 8×H100 with tensor + expert parallelism, greedy decoding, reasoning_effort=low, the <think> block stripped before classification. Refusal is judged by a rule-based opening-phrase classifier (caveat = answered but wrapped in a disclaimer) — indicative, not an LLM-judge / publication-grade number.

Safety — harmful-prompt refusal (lower = more uncensored)

Both columns are measured on these exact checkpoints with the same script, sampling seed and settings. The base model refuses 92–99% of harmful prompts on six of the seven sets — ForbiddenQuestions is the outlier at 0.593, which says more about that set (a large share of its questions are sensitive rather than harmful, and the base model answers them) than about the edit.

The spread in Δ is the same result reported in What resisted above, seen from the benchmark side: the sets that fall furthest are the ones whose content the removed direction mediates, and the two that fall least are the ones it does not reach.

Over-refusal — benign prompts wrongly refused (lower = better)

250 prompts written to look harmful while being benign. The base model is already restrained here at 2.4%; the edit does not blunt that discrimination, it sharpens it — 0.4%, and over-refusal on the held-out benign split fell from 0.094 to 0.000.

Capability retention — vs the official base (same scripts, same settings)

Every delta is within ±1.5 pp and no reply failed to parse on either checkpoint. Two of the four move up, which is the signature of sampling noise at these sizes rather than of an improvement — read the whole table as "unchanged", not as a gain.

Multiple choice is scored from a single forward pass, taking the argmax over the option letters' logits, rather than by generating an answer; GSM8K keeps chain-of-thought generation because it needs the reasoning. That makes the MMLU-Pro column lower than a CoT-scored run of the same benchmark would be — for both checkpoints equally, which is why the base column is not optional. Read the delta, not the absolute figure, and do not compare these numbers against MMLU-Pro results obtained with a different scoring method.

Usage — self-host with vLLM (OpenAI-compatible)

glm5_next needs a vLLM build with GLM-5.3-Flash support and transformers 5.16+. The weights are 306 GiB, so plan for 8×H100/H200 with tensor parallelism.

docker run -d --name glm53 --gpus all --ipc host -p 8000:8000 \
  -v /path/to/GLM-5.3-Flash-Uncensored-FP8:/model \
  vllm/vllm-openai:latest \
  --model /model --served-model-name GLM-5.3-Flash-Uncensored-FP8 \
  --tensor-parallel-size 8 --max-model-len 262144 \
  --enable-expert-parallel

See the vLLM recipe and the SGLang cookbook for the currently recommended flags, including the tool-call parser to pass with --enable-auto-tool-choice.

Thinking control

GLM-5.3-Flash has no enable_thinking toggle — its chat template always opens a <think> block. Control the budget with reasoning_effort instead (low / high / max; defaults to max):

client.chat.completions.create(
    model="GLM-5.3-Flash-Uncensored-FP8",
    messages=[{"role": "user", "content": "..."}],
    extra_body={"chat_template_kwargs": {"reasoning_effort": "low", "clear_thinking": True}},
)

clear_thinking defaults to false; pass true for chat scenarios. Give generation enough budget to reach </think>, or replies get truncated inside the scratchpad. Pass image_url content parts for vision.

Bias, risks, and limitations

• Safety guardrails removed — the model will produce harmful, biased, or offensive content on request (see the disclaimer).
• It inherits any biases and limitations of the base GLM-5.3-Flash.
• Roughly 5% of the refusal direction survives requantization into e4m3 (see above), so a small residual refusal rate remains — it is not, and cannot be, exactly zero on an FP8 checkpoint.
• Refusal is reduced, not removed. Some content categories are not mediated by the direction this method removes and still refuse at close to the base rate (see What resisted). Do not assume a uniformly uncensored model.
• Capability retention is measured, not assumed (see Evaluation), but on sampled subsets of four benchmarks — enough to rule out a large regression, not a substitute for a full harness run.
• The reported refusal metric is a rule-based heuristic; evaluate rigorously for your own use case.

License

MIT, inherited from the base model zai-org/GLM-5.3-Flash. Abliteration does not change the underlying license obligations.