Some files in this repo may not work.
IQ3_M was tested and produced readable text. TQ1_0 and TQ2_0 load and emit tokens, but the text is not readable. Larger quants (Q8_0, Q6_K, Q5_K_M, Q4_K_M, IQ4, MXFP4) are still being built and will be uploaded after they exist. Do not treat an untested file as usable.
GGUF conversions of Blackfrost-AI/K2-Horizon-MoVA-36B-A4B-DERISKED-BF16. The source is a weight-level behavioral derivative of IFM/K2-Horizon-MoVA-36B-A4B. DERISKED is the Blackfrost research label. It is not a claim of safety, zero refusals, or unchanged benchmark scores.
Upstream llama.cpp cannot load k2-horizon. You need the IFM fork:
https://github.com/MBZUAI-IFM/llama.cpp/tree/model/K2Horizon
These files were converted and quantized with that branch at 42adf01.
What was tested
Host: Ryzen 6850H / Radeon 680M, Vulkan, GGML_VK_DISABLE_ASYNC=1, fork build b1-42adf01. Prompt: The capital of France is, temperature 0, --jinja.
| File |
Size |
Result |
...-IQ3_M.gguf |
15.4 GiB |
Works. Readable reasoning text at 7.1 tok/s. |
...-TQ1_0.gguf |
8.0 GiB |
Loads. ~9 tok/s. Output is not readable. |
...-TQ2_0.gguf |
9.6 GiB |
Loads. ~14 tok/s. Output is not readable. |
IQ3_M greedy sample (chat template, reasoning on):
[Start thinking]
1. **Analyze the User's Request:**
* The user's query is "The capital of France
Routers (attn_v_gate) are kept in F32. This IQ3_M has no imatrix. The first imatrix pass exited during load.
Q8_0 and Q6_K did not start on this GPU with -ngl 99 (failed to fit params to free device memory). They have not had a generation check. Q5_K_M, Q4_K_M, IQ4_XS, IQ4_NL, Q4_K_S, Q4_0, Q4_1, MXFP4_MOE, and a mixed Q4_K_DYN file are still in the queue.
Run
IQ3_M is the file that produced readable text on a 680M.
git clone --branch model/K2Horizon https://github.com/MBZUAI-IFM/llama.cpp
cd llama.cpp
git checkout 42adf01
cmake -B build -DGGML_VULKAN=ON -DCMAKE_BUILD_TYPE=Release
cmake --build build -j --target llama-server llama-cli
GGML_VK_DISABLE_ASYNC=1 ./build/bin/llama-server \
-m K2-Horizon-MoVA-36B-A4B-DERISKED-IQ3_M.gguf \
-ngl 99 -c 4096 -fa on --jinja \
--temp 1.0 --top-p 0.95 \
--host 0.0.0.0 --port 8082
GGML_VK_DISABLE_ASYNC=1 matters on the 680M. Async Vulkan uploads have produced ? / NaN garbage on this GPU for other models. The chat template is custom. llama.cpp only applies it with --jinja. Upstream sampling is temperature 1.0, top_p 0.95, reasoning effort high. The template thinks inside <ifm|think>.
If the process prints Failed to process regex on Windows, the MSVC regex engine is rejecting the tokenizer pattern. Run the same fork under Linux.
Architecture
| Property |
Value |
| Architecture |
k2-horizon |
| Parameters |
36B total, about 4B active per token |
| Layers |
48 (3 dense, then 45 MoE) |
| Hidden size |
2560 |
| Heads |
32 query / 8 KV, head dim 128 |
| MoE |
100 routed experts, 8 active, 1 shared expert |
| MoVA |
64 value experts, 4 active, on the MoE layers |
| Expert FFN |
768 |
| Dense FFN |
6144 |
| Context |
524288 native. A 680M-class box cannot serve that window. |
| Vocab |
250624, untied embeddings |
| Rope theta |
10000000 |
| Router |
sigmoid, top-k weights normalized, scale 2.5 |
| Norm |
grouped RMSNorm, 2 groups, eps 1e-6 |
All 36B weights have to sit in RAM or VRAM. "4B active" is the compute per token, not the file size.
License
Apache-2.0, same as the source card. See LICENSE. Architecture, tokenizer, and chat template remain IFM's. The checkpoint weights are the Blackfrost derivative. This repository only repackages those weights as GGUF.
Lineage
IFM/K2-Horizon-MoVA-36B-A4B@7730b92d1b574e04663b04023d5d6fa83475432f
└── Blackfrost-AI/K2-Horizon-MoVA-36B-A4B-DERISKED-BF16
└── freakyskittle/K2-Horizon-MoVA-36B-A4B-DERISKED-GGUF
