GGUF quants of Blackfrost-AI/RED-SNOW-5.3-FLASH-BF16 - a red-team operations model built on a GLM-5.3-Flash (glm5_next) foundation with targeted security adaptation.
Full-functionality GGUF exports for llama.cpp: text + vision included (the GLM-5.3 vision tower ships as a separate mmproj file, keeping every quant pairable with it). Every file was smoke-tested end to end before upload - text runs with llama-cli, vision with llama-server --mmproj.
The model
RED-SNOW 5.3 FLASH is built for authorized offensive-security work: attack-path reasoning, cross-domain operations (identity, cloud, network, app, endpoint, OT), exploit and tradecraft analysis, detection-aware execution, and purple-team handoff. See the base model card for the full picture.
Architecture highlights that matter for inference:
- MoE: 45 layers, 288 routed experts (8 active), 1 shared expert, ~321B total params
- MLA (nope-only) + DSA sparse-attention indexer with k-pool compression
- KDA linear attention on most layers (short-conv + gated delta net)
- mHC hyper-connections
- MTP (nextn) layer - shipped in-file in every GGUF for speculative decoding
Quants
| File |
Quant |
Size |
Notes |
RED-SNOW-5.3-FLASH-UD-Q4_K_XL.gguf |
UD-Q4_K_XL |
190.7 GB |
dynamic 4-bit; attn/ssm/indexer and first/last blocks run hotter (Q5_K/Q6_K/Q8_0) |
RED-SNOW-5.3-FLASH-Q4_K_M.gguf |
Q4_K_M |
181.5 GB |
standard 4-bit, output/embeddings at Q6_K |
RED-SNOW-5.3-FLASH-Q3_K_M.gguf |
Q3_K_M |
138.6 GB |
tighter memory |
RED-SNOW-5.3-FLASH-Q2_K.gguf |
Q2_K |
106.2 GB |
smallest, noticeable quality hit |
RED-SNOW-5.3-FLASH-Q8_0.gguf |
Q8_0 |
~341 GB |
near-lossless reference (in progress) |
mmproj-RED-SNOW-5.3-FLASH-f16.gguf |
F16 mmproj |
1.16 GB |
GLM-5.3 vision tower + glm5v projector, pairs with every quant |
MTP speculative decoding ships in-file: every main GGUF carries the NextN block, enabled at runtime with --spec-type draft-mtp. No separate draft download is needed. The MTP head boosts decode throughput for free.
Vision
Full multimodal support: the GLM-5.3 vision tower (glm5v projector) ships as mmproj-RED-SNOW-5.3-FLASH-f16.gguf and pairs with any of the quants above. Screenshots, target UIs, logs, and diagrams all work through the standard OpenAI-compatible image_url content parts:
llama-server -m RED-SNOW-5.3-FLASH-Q4_K_M.gguf \
--mmproj mmproj-RED-SNOW-5.3-FLASH-f16.gguf \
-ngl 99 --jinja --port 8080
The vision tower is kept separate so each quant stays small; the base model's multimodal behavior is preserved bit-for-bit (F16 tower).
Usage
Requires a llama.cpp build with glm5_next / Glm5NextForConditionalGeneration support, including the glm5v mmproj projector for vision.
# chat
llama-cli -m RED-SNOW-5.3-FLASH-Q4_K_M.gguf -ngl 99 --jinja
# server with in-file MTP speculative decoding + vision
llama-server -m RED-SNOW-5.3-FLASH-Q4_K_M.gguf \
--spec-type draft-mtp \
--mmproj mmproj-RED-SNOW-5.3-FLASH-f16.gguf \
-ngl 99 --jinja --port 8080
The chat template preserves GLM's visible-answer/reasoning split and supports thinking-on and thinking-off generation.
How these were made
- Quantized with llama.cpp ggml K-quant kernels, streaming per-tensor from the BF16 safetensors via HTTP range requests (peak RAM of one tensor), so quality is identical to a conventional offline quantize.
- Embeddings/output at Q8_0 (Q6_K for Q4_K_M), sensitive
indexer/ssm/hc tensors kept at higher precision on the dynamic builds, first and last eighth of the layers boosted on UD builds.
- Smoke test per file:
llama-cli generation check before release, plus a llama-server --mmproj vision check (image color identification) for the mmproj file. Records live in smoke/.
Smoke tests
Every GGUF in this repo passed a generation smoke test (a coding prompt that must produce a correct function) before upload. Each record in smoke/ is a small Python file with the full transcript; you can verify one like this:
def smoke_ok(record_path: str) -> bool:
"""Check a smoke/ record: true if the quant passed its smoke test."""
record = open(record_path).read()
return "SMOKE_OK = True" in record and "return a + b" in record
Questions
Questions, broken files, or quant requests: open a discussion on this repo or DM me on X.
Intended use
Authorized offensive-security work only: scoped engagements, labs, CTFs, cyber ranges, and research. Keep a human operator in control of scope, targets, credentials, and destructive actions; validate findings against real systems. You are responsible for operating within the law and within your authorization.
License
MIT, inherited from the base model. Original source materials remain subject to their respective terms.
