Community GGUF quantizations of whitecircle/GLM-4.7-Flash-Coder.
If this GGUF made your day easier, a coffee would make mine.
Send a coffee
I build and test these releases myself. Your coffee helps keep me going.
Thank you for supporting this work.
About GLM-4.7-Flash-Coder
GLM-4.7-Flash-Coder is an agentic coding fine-tune of zai-org/GLM-4.7-Flash. The upstream model card describes a 30B Mixture-of-Experts model trained on software-engineering trajectories, with tool use, reasoning, and concurrent tool-call behavior. The upstream training configuration uses a 75k-token sequence length; its model configuration declares a 202,752-token maximum position limit. This release does not claim long-context behavior beyond the runtime validation documented below.
The model uses the GLM-4.7 Flash chat template and XML-style tool calls. See the official upstream model card for the original model, training, and evaluation details.
This repository contains quantizations only. No training or fine-tuning was performed here.
Published file measurements
Sizes use decimal GB (bytes / 1e9). The table covers the public GGUF files
currently released in this repository; runtime memory also depends on context,
batch size, KV-cache settings, backend, and offload configuration.
| File |
Size (GB) |
Size (bytes) |
CPU smoke |
GPU 0 smoke |
GLM-4.7-Flash-Coder-Q8_0.gguf |
31.842800 |
31,842,800,096 |
PASS |
PASS |
GLM-4.7-Flash-Coder-Q6_K.gguf |
24.614787 |
24,614,786,848 |
PASS |
PASS |
GLM-4.7-Flash-Coder-Q5_K_M.gguf |
21.264516 |
21,264,515,872 |
PASS |
PASS |
GLM-4.7-Flash-Coder-Q5_K_S.gguf |
20.664288 |
20,664,287,712 |
PASS |
PASS (partial) |
GLM-4.7-Flash-Coder-Q4_K_M.gguf |
18.132722 |
18,132,722,464 |
PASS |
PASS |
GLM-4.7-Flash-Coder-Q4_K_S.gguf |
17.097991 |
17,097,990,624 |
PASS |
PASS |
GLM-4.7-Flash-Coder-Q3_K_M.gguf |
14.405579 |
14,405,578,528 |
PASS |
NOT_RUN |
GLM-4.7-Flash-Coder-Q2_K.gguf |
11.042966 |
11,042,966,304 |
PASS |
NOT_RUN |
GLM-4.7-Flash-Coder-IQ2_XS.gguf |
8.877017 |
8,877,016,864 |
PASS |
NOT_RUN |
GLM-4.7-Flash-Coder-IQ1_M.gguf |
6.886802 |
6,886,802,208 |
PASS |
NOT_RUN |
Quick start
The first validated artifact is GLM-4.7-Flash-Coder-Q8_0.gguf; run it with llama.cpp:
./llama-cli \
-m GLM-4.7-Flash-Coder-Q8_0.gguf \
--chat-template-file chat_template.jinja \
--jinja \
--reasoning off \
-p 'Answer briefly in English: What is GGUF and why is it useful for running language models locally?' \
-n 128 -c 4096 -ngl 99
The included chat_template.jinja should be used for chat and tool-call serialization. Tool calling and reasoning claims are limited to the validation profiles recorded in the reproducibility manifest.
Reproducibility and validation
Artifacts are being released incrementally. Each quantization is generated directly from the locked BF16 GGUF source, validated with the local llama.cpp runtime, checksummed, and uploaded in a separate Hub commit. The attempted ladder is:
Q8_0, Q6_K, Q5_K_M, Q4_K_M, Q3_K_M, Q2_K, IQ2_XS, IQ1_M, Q1_0.
The reproducibility manifest records the upstream revision, source hashes, converter/runtime revision, calibration input, quantization commands, validation profile, and publication status. The model-specific calibration text and importance matrix are included under reproducibility/. The public SHA256SUMS.txt records checksums for published files. Raw conversion, calibration, quantization, smoke-test, and benchmark logs remain local under reports/.
Q8_0, Q6_K, Q5_K_M, and Q4_K_M have passed CPU and GPU 0 load/generate smoke tests with the included chat template. Q5_K_S passed CPU smoke and a GPU 0 partial-offload smoke at -ngl 20 with context 2048; a full-offload attempt exceeded the available GPU 0 memory and is not claimed as a full-GPU profile. Q4_K_S passed CPU and full GPU 0 smoke at -ngl 99 with context 2048. Q3_K_M, Q2_K, IQ2_XS, and IQ1_M have passed CPU smoke testing and are published; GPU 0 smoke was not run for these lower formats. Q1_0 was attempted but is omitted because this runtime has no fallback for a non-block-aligned tensor; no invalid artifact is published.
Fidelity measurements will be added only after a fixed BF16 hold-out evaluation is completed. Until then, no quantization is presented as a measured quality sweet spot.
License and attribution
The upstream model card declares the MIT license; see LICENSE and the upstream model card.
These are community GGUF quantizations, not an official whitecircle/GLM-4.7-Flash-Coder release or endorsement.
