Mixed-precision GGUF derived from
orcarouter/GLM-5.3-Flash-Uncensored-FP8.
The recipe preserves higher precision in always-active paths and six boundary
routed layers, with IQ2_XXS as the minimum precision.
Status: published. File identity and bounded single-GPU image checks are
complete. General text, MTP and DGX Spark context checks remain pending.
| Model property |
Value |
| Architecture |
45 trunk layers; 34 KDA + 11 sparse-attention layers |
| Routed experts |
288 per routed layer, top-8 active |
| Native context |
1,048,576 tokens |
| MTP |
One embedded block |
| Vision |
Separate encoder, original BF16 weights |
| Main + MTP |
87.470 GiB |
| Total with vision |
88.520 GiB / 95.047 GB |
Quantization recipe
Layer indices are zero-based: trunk 0–44, MTP 45. Trunk layers 0–2 are dense.
| Tensor group |
Type |
| Token embedding and output head |
Q8_0 |
| Dense MLP layers 0–2 and shared experts |
Q8_0 |
| KDA query/key projections |
Q4_K |
| Remaining KDA and major sparse-attention/MLA projections |
Q8_0 |
| Router weights/biases, norms and state controls |
F32 |
| Indexer and mHC mixers; MTP hidden-state projection |
BF16 |
| Routed gate/up, layers 3, 4, 5, 43, 44, 45 |
IQ2_XS + imatrix |
| Routed gate/up, remaining routed layers |
IQ2_XXS + imatrix |
| Routed down, layers 3, 4, 5, 43, 44, 45 |
Q2_K (optional imatrix) |
| Routed down, remaining routed layers |
IQ2_XS + imatrix |
| Vision encoder |
BF16 |
Gate/up use the same format within each layer. Each tensor uses one block
format throughout; routed matrix widths remain divisible by 256. Source-weight reconstruction proxies were collected for all 126 trunk expert
matrices.
Q2_K does not require an imatrix; optional importance weighting uses available
activation statistics without changing the format or file size.
Calibration covered 81,920 text positions; every trunk routed expert was
observed. A bounded Q2_K source-weight sample showed 51.2% lower activation-weighted
squared reconstruction error (SSE) with imatrix. This is a reconstruction proxy, not a measured
response-quality gain. See calibration-summary.json.
Calibration uses GLM-native text inputs and relevant source references from
Inkling-Small Multimodal Calibration.
The imatrix is collected from text inputs. Original images are prepared for
separate image quality checks; image-conditioned activation calibration is not
claimed. MTP activation statistics are not collected by the current upstream
GLM graph; block 45 uses weight-based importance for IQ2_XS gate/up and standard
unweighted Q2_K down. Drafting behavior remains a target-hardware check.
Runtime
This GGUF targets the native GLM format in
antirez/ds4. The mixed expert combinations need
the accompanying runtime extension; upstream llama.cpp compatibility is not
established. Build and usage commands are in runtime/README.md.
The 1M context limit is preserved in metadata. Estimated KV/state/graph memory
at that limit is approximately 14.85 GiB, plus 1.15 GiB with MTP. Actual usable
context depends on available memory and runtime settings; DGX Spark maximum
context has not yet been measured. Image serving requires the BF16 encoder;
native video-input support is not established.
The source model's uncensored behavior and benchmark results are not new
measurements of this quantization. Quality results will identify the exact
tested build and settings.
Bounded verification
Both file hashes and all 1,412 main tensor entries were verified after public
upload. Re-encoding samples matched exactly across 129 routed tensors and 516
rows. Eight reserved DocVQA/ChartQA images completed on one H100 with SSD expert
streaming: 8/8 requested answers matched, with 15/16 manual rubric points.
One reply added an incorrect steady-decline claim to a correct peak-year answer.
These are bounded checks. General text, MTP and long-context qualification remain
pending. Exact settings and runtime identities are in
verification-summary.json.
License
MIT, inherited from the source checkpoint. See LICENSE.
