Sakura-K2-Horizon-0.9B-GGUF

Sakura — K2-Horizon-0.9B (GGUF, three sizes)

K2-Horizon-0.9B is the smallest member of IFM's open K2-Horizon family: a tiny decoder-only reasoning and agent model with a long context window (Apache-2.0, fully open training data and recipe). This repository holds three GGUF files of different size made from the BF16 weights with our measured mixed-codec method. Pick one on the Files page; the table below says what each file is. Independent community quantization, not an official IFM (MBZUAI Institute of Foundation Models) release. Part of the Sakura Mini line.

Needs the IFM K2-Horizon llama.cpp fork for now. The K2Horizon architecture is not in mainline llama.cpp yet (feature request #29424, pull request #29535 still open); use branch model/K2Horizon of MBZUAI-IFM/llama.cpp, same as IFM's own GGUF files. This card will be updated when upstream support lands.

Quality at a glance

Same method, same texts, same machine for every file; lower KL divergence means closer to the BF16 original:

• Sakura-K2-Horizon-0.9B-0.40GiB.gguf vs llama.cpp default IQ3_XXS (0.42 GiB): KL divergence to the BF16 original 14 % higher on average over all 3 texts (0.02 GiB smaller).
• Sakura-K2-Horizon-0.9B-0.58GiB.gguf vs llama.cpp default IQ4_XS (0.57 GiB): KL divergence to the BF16 original 29 % higher on average over all 3 texts (0.01 GiB larger).
• Sakura-K2-Horizon-0.9B-0.63GiB.gguf vs IFM official Q4_K_M (0.62 GiB): KL divergence to the BF16 original 61 % lower on average over all 3 texts (0.01 GiB larger).

Why: the bit budget per tensor is allocated from measured sensitivity (see How it was made), not from fixed rules. Full table below.

Which file should I take?

The three files

"Tensor types by size" lists the share of the file's bytes per ggml type (all are standard llama.cpp types, so the files run in stock llama.cpp). Quality columns: mean KL divergence of the quantized model's next-token distribution against the BF16 source on held-out text (lower is better) and how often the most likely token is unchanged (higher is better).

Comparison at similar size

Same texts, same context, same chunks, measured by us with llama-perplexity (BF16 source = reference; PPL of the BF16 model: en: 2.029, dev: 1.893, wiki: 15.449). The reference quants are: llama.cpp default recipe, same importance matrix (ours), measured the same way. Sorted by size.

Method: 12 chunks of 512 tokens per text; texts: English held-out text, developer-text held-out set, English encyclopedia held-out text (not used in any calibration). Single measurement on one machine; small KLD differences at similar size are not a quality ranking.

How it was made

1. The BF16 GGUF is IFM's official file from IFM/K2-Horizon-0.9B-GGUF (unchanged); the importance matrix was computed by us (llama-imatrix).
2. Every large weight matrix was quantized once per candidate type (Q2_K, IQ2_S, IQ3_XXS, IQ3_S, Q3_K, IQ4_XS, Q4_K, Q5_K, Q6_K, Q8_0) with llama-quantize and the importance matrix; the error of each choice was estimated per matrix (importance-weighted) and scaled by a sensitivity factor measured with real KL divergence: every group of tensors (for example the FFN down projections, the attention value projections, the first and last layers) was moved alone to a lower-bit type, and the KL divergence it caused was compared with its quantization error.
3. An exact budget allocation (multiple-choice knapsack over bytes) picked one type per matrix for each target size; the final model was assembled from the stored tensors without re-quantizing. The per-tensor choices are public, see Tensor allocation below. Norms, small tensors and the multi-token-prediction layers stay at high precision.

Tensor allocation

The type of every tensor is public: allocation.json lists type and size per tensor and file (the GGUF headers show the same). Summary per tensor group (number of tensors per ggml type):

calibration.json holds the measured sensitivity factors (or the rules) that the allocation used.

Limits

• Measured only with KL divergence and perplexity on short held-out texts and not with downstream benchmarks; do not read it as a task-quality claim.
• Quantization always costs some quality; the smallest file costs the most.
• Requires the IFM K2-Horizon llama.cpp fork until upstream support lands: the K2Horizon architecture is not yet in mainline llama.cpp (see feature request #29424 and pull request #29535); use the branch model/K2Horizon of MBZUAI-IFM/llama.cpp. All numbers on this card were measured with that fork (Vulkan backend; CPU and Vulkan agree to 0.2 % perplexity on our check).
• The model uses the <|ifm|im_start|> chat format with a thinking block; the chat template is embedded in the GGUF.

Credits

• Original model: K2-Horizon-0.9B (IFM / MBZUAI, Apache-2.0), Apache-2.0 license (included as LICENSE).
• BF16 GGUF and the official Q4_K_M / Q5_K_M reference quants: IFM/K2-Horizon-0.9B-GGUF. The other reference quants are llama-quantize default recipes with our importance matrix.
• GGUF format and tools: ggml-org/llama.cpp (MIT).