# K2-Horizon-MoVA-36B-A4B-MLA by Matthew P Barnson: Open Model
Source: https://savrn.com/models/k2-horizon-mova-36b-a4b-mla
Markdown alternate of the page above; the site index is https://savrn.com/llms.txt

---

## Runs On

What it takes to serve K2-Horizon-MoVA-36B-A4B-MLA (37.9B parameters): the memory its weights need at each precision, and the cheapest way to rent enough data-center GPUs to hold them.

| Precision | Weights | Memory needed | Cheapest setup | Per hour | Also fits |
| --- | --- | --- | --- | --- | --- |
| 16-bit | 75.8 GB | 91.0 GB | 1x [MI300X](https://savrn.com/ai-index/pricing/gpus/mi300x) (192 GB) Vultr | $1.85 | [1x MI325X](https://savrn.com/ai-index/pricing/gpus/mi325x) $2.00 · [1x MI355X](https://savrn.com/ai-index/pricing/gpus/mi355x) $2.59 |
| 8-bit | 37.9 GB | 45.5 GB | 1x [MI300X](https://savrn.com/ai-index/pricing/gpus/mi300x) (192 GB) Vultr | $1.85 | [1x H100](https://savrn.com/ai-index/pricing/gpus/h100) $1.99 · [1x MI325X](https://savrn.com/ai-index/pricing/gpus/mi325x) $2.00 |
| 4-bit | 19.0 GB | 22.8 GB | 1x [MI300X](https://savrn.com/ai-index/pricing/gpus/mi300x) (192 GB) Vultr | $1.85 | [1x H100](https://savrn.com/ai-index/pricing/gpus/h100) $1.99 · [1x MI325X](https://savrn.com/ai-index/pricing/gpus/mi325x) $2.00 |

Memory is the weights at that precision plus 20% for the runtime and a short context; a long context needs more. Prices are the lowest on-demand hourly rates in the [SAVRN Index](https://savrn.com/ai-index/pricing/gpus), read Oct 12, 2026.

[K2-Horizon-MoVA-36B-A4B-MLA on every accelerator the SAVRN Index prices, at every precision](https://savrn.com/models/k2-horizon-mova-36b-a4b-mla/gpus)

## Model Card

By Matthew P Barnson, published under apache-2.0, revision 47302a89c4dd.

Research preview. main is now the healed checkpoint. It is measurably worse than the original (KLD 0.091, +0.058 nats) and loops occasionally in long generations (2 of 48, the original 0 of 48). Read "Known issues" first.

### K2-Horizon-MoVA-36B-A4B-MLA (research preview)

[IFM/K2-Horizon-MoVA-36B-A4B](https://huggingface.co/IFM/K2-Horizon-MoVA-36B-A4B) with its attention converted from grouped-query attention (GQA) to multi-head latent attention (MLA), DeepSeek-style, without retraining. The KV cache shrinks from 192 KiB to 84 KiB per token in BF16 (42 KiB with an 8-bit latent). The routed experts and the value experts are the original weights.

As far as we know this is the first MLA conversion of a GQA mixture-of-value-experts model. It is a research preview: calibrated, then healed by distillation of the attention projections only. See "Known issues".

### Versions

[Read the full model card (2,519 words)](https://savrn.com/models/k2-horizon-mova-36b-a4b-mla/card)

## Configuration

Architecture

K2HorizonForCausalLM

Context length (tokens)

524,288

Layers

48

Hidden size

2,560

Feed-forward size

6,144

Attention heads

32

Key/value heads

8

Head dimension

128

Vocabulary size

250,624

Experts

100

Experts active per token

8

Model type

k2_horizon

## Identity and Version

Repository

txgsync/K2-Horizon-MoVA-36B-A4B-MLA

Publisher

Matthew P Barnson

Task

Text generation

Modality

Text

Library

mlx

Parameters

37.9B parameters

Languages

en

Revision

47302a89c4dd6ecf9ae8972e86f6a83904317613

First published

2026-10-08

Last updated

2026-10-10

## Files and Weights

60 files, 75.9 GB in total. The weights are 49 files totalling 75.8 GB in safetensors.

Weights49 files · 75.8 GB

Configuration6 files · 1.6 MB

Tokenizer2 files · 29.2 MB

Documentation1 file · 39.7 KB

Other1 file · 51.0 KB

Repository1 file · 1.6 KB

Every file

| File | Type | Size | SHA-256 |
| --- | --- | --- | --- |
| model-00001-of-00048.safetensors | Weights | 167.8 MB | 47f00aa366bd |
| model-00002-of-00048.safetensors | Weights | 167.8 MB | b05a1aec8e3f |
| model-00003-of-00048.safetensors | Weights | 167.8 MB | 03eb46e9db27 |
| model-00004-of-00048.safetensors | Weights | 1.6 GB | bfb969b49711 |
| model-00005-of-00048.safetensors | Weights | 1.6 GB | 3580081fa2c9 |
| model-00006-of-00048.safetensors | Weights | 1.6 GB | 59b743cb89a9 |
| model-00007-of-00048.safetensors | Weights | 1.6 GB | 0bfe0fe878b2 |
| model-00008-of-00048.safetensors | Weights | 1.6 GB | c198939725fe |
| model-00009-of-00048.safetensors | Weights | 1.6 GB | 34802824154c |
| model-00010-of-00048.safetensors | Weights | 1.6 GB | 602803a47e4f |
| model-00011-of-00048.safetensors | Weights | 1.6 GB | 3be64866ae51 |
| model-00012-of-00048.safetensors | Weights | 1.6 GB | bd500f6e8bd4 |
| model-00013-of-00048.safetensors | Weights | 1.6 GB | dc3b384ec891 |
| model-00014-of-00048.safetensors | Weights | 1.6 GB | b57b86ed1d1f |
| model-00015-of-00048.safetensors | Weights | 1.6 GB | 4a0b140c4399 |
| model-00016-of-00048.safetensors | Weights | 1.6 GB | e248cec898cb |
| model-00017-of-00048.safetensors | Weights | 1.6 GB | c9e30bde2b4d |
| model-00018-of-00048.safetensors | Weights | 1.6 GB | 7fe389b9643a |
| model-00019-of-00048.safetensors | Weights | 1.6 GB | 6412690d2c97 |
| model-00020-of-00048.safetensors | Weights | 1.6 GB | 69bc784b39cf |
| model-00021-of-00048.safetensors | Weights | 1.6 GB | f7982fb6318e |
| model-00022-of-00048.safetensors | Weights | 1.6 GB | e08147e5a9f1 |
| model-00023-of-00048.safetensors | Weights | 1.6 GB | a7cce3e66fcd |
| model-00024-of-00048.safetensors | Weights | 1.6 GB | 487ef656fa78 |
| model-00025-of-00048.safetensors | Weights | 1.6 GB | 1c7cb319f9b1 |
| model-00026-of-00048.safetensors | Weights | 1.6 GB | 6122afe69e82 |
| model-00027-of-00048.safetensors | Weights | 1.6 GB | 14999d001378 |
| model-00028-of-00048.safetensors | Weights | 1.6 GB | 8fe45e7d8e51 |
| model-00029-of-00048.safetensors | Weights | 1.6 GB | 97ce875e0ac6 |
| model-00030-of-00048.safetensors | Weights | 1.6 GB | d4409db89a51 |
| model-00031-of-00048.safetensors | Weights | 1.6 GB | 97bdf5060788 |
| model-00032-of-00048.safetensors | Weights | 1.6 GB | 8807202dd15b |
| model-00033-of-00048.safetensors | Weights | 1.6 GB | 62cfdc988983 |
| model-00034-of-00048.safetensors | Weights | 1.6 GB | f37f4f6e7909 |
| model-00035-of-00048.safetensors | Weights | 1.6 GB | fbd4870db3ed |
| model-00036-of-00048.safetensors | Weights | 1.6 GB | 00f5c160cc1b |
| model-00037-of-00048.safetensors | Weights | 1.6 GB | 9991b27e3f1a |
| model-00038-of-00048.safetensors | Weights | 1.6 GB | 45456c2079aa |
| model-00039-of-00048.safetensors | Weights | 1.6 GB | 3a801fe7888c |
| model-00040-of-00048.safetensors | Weights | 1.6 GB | 63fcd835f6be |
| model-00041-of-00048.safetensors | Weights | 1.6 GB | 1f3517b9b182 |
| model-00042-of-00048.safetensors | Weights | 1.6 GB | 5b4ab1dd1675 |
| model-00043-of-00048.safetensors | Weights | 1.6 GB | f6723d662bbb |
| model-00044-of-00048.safetensors | Weights | 1.6 GB | 124cb0d68524 |
| model-00045-of-00048.safetensors | Weights | 1.6 GB | 969779d6bcad |
| model-00046-of-00048.safetensors | Weights | 1.6 GB | 65fbc1f7f533 |
| model-00047-of-00048.safetensors | Weights | 1.6 GB | 0792c1e87f04 |
| model-00048-of-00048.safetensors | Weights | 4.2 GB | 251588c4b5cd |
| model-mla-delta.safetensors | Weights | 945.3 MB | 074292a8e065 |
| config.json | Configuration | 1.8 KB | — |
| generation_config.json | Configuration | 81 B | — |
| k2_mla.py | Configuration | 4.6 KB | — |
| mla_config.json | Configuration | 499 B | — |
| model.safetensors.index.json | Configuration | 1.5 MB | — |
| special_tokens_map.json | Configuration | 79 B | — |
| README.md | Documentation | 39.7 KB | — |
| chat_template.jinja | Other | 51.0 KB | — |
| .gitattributes | Repository | 1.6 KB | — |
| tokenizer.json | Tokenizer | 29.2 MB | 2fa69519ff1e |
| tokenizer_config.json | Tokenizer | 253 B | — |

## License and Download

License

apache-2.0

Access

Open weights, no gate

Download size

75.8 GB

[Download from Matthew P Barnson](https://huggingface.co/txgsync/K2-Horizon-MoVA-36B-A4B-MLA)

Released by Matthew P Barnson through its official repository on Hugging Face. [Read the license](https://www.apache.org/licenses/LICENSE-2.0).

## Built From

- Derived from IFM/K2-Horizon-MoVA-36B-A4B
- Described by arXiv:2502.07864
- Trained on (disclosed) IFM/K2-Horizon-Midtrain-Data
- Trained on (disclosed) IFM/K2-Horizon-Pretrain-Data

## Memory Requirements

| Precision | Weights in memory |
| --- | --- |
| As published | 75.8 GB |
| 16-bit | 75.8 GB |
| 8-bit | 37.9 GB |
| 4-bit | 19.0 GB |

Weights only, from the published parameter count; the key-value cache and runtime add to this.

## Questions About K2-Horizon-MoVA-36B-A4B-MLA

### How much GPU memory does K2-Horizon-MoVA-36B-A4B-MLA need?

About 91 GB at 16-bit and 22.8 GB at 4-bit: the weights (37.9B parameters) plus a working margin. A long context needs more.

### What is the cheapest GPU to run K2-Horizon-MoVA-36B-A4B-MLA on?

At 16-bit, 1x MI300X from $1.85 an hour; at 4-bit, 1x MI300X from $1.85 an hour, at the lowest on-demand prices the SAVRN Index lists.

### Can I use K2-Horizon-MoVA-36B-A4B-MLA commercially?

Yes. K2-Horizon-MoVA-36B-A4B-MLA is released under Apache License 2.0. The Apache License 2.0 is a permissive open-source license. It permits commercial use, modification and redistribution. It requires keeping the license and copyright notices and any NOTICE file, stating significant changes, and it includes an express patent grant from contributors.

### What is K2-Horizon-MoVA-36B-A4B-MLA's context length?

524,288 tokens, from the maximum position embeddings in its published configuration.

## Similar Models

Model · Text generation

### [LUHANADJolTctKlt](https://savrn.com/models/luhanadjoltctklt)

[Priyansh Singhal](https://savrn.com/model-publishers/ps4research)

This seedoss model was trained 2x faster with Unsloth and Huggingface's TRL library.

Open weights apache-2.0 36.2B parameters 524,288 tokens transformers

[View model](https://savrn.com/models/luhanadjoltctklt)

Model · Text generation

### [Tema_Q-X7-Thinking](https://savrn.com/models/tema-q-x7-thinking)

[Tema_Q LLM](https://savrn.com/model-publishers/temaq-org)

TemaQ-X7-Thinking（天馬求） は、Ornith AIが開発したモデル Ornith-1.5 を基盤にした、エージェント向けの大規模言語モデル（LLM）です。 TemaQ-X7-Thinking (TemaQ model) is an enhanced Large Language Model (LLM) designed for agent applications, based on the Ornith-1.5 model developed by Ornith AI. It is engineered to generate more flexible and useful responses, even for prompts that are difficult for standard models to handle effectively. When used in conjunction with TemaQ Agent, it enables advanced reasoning capabilities. ユーザーの責任: モデルの利用者は、生成されたコンテンツが、適用される法律、規制、およびHugging Faceの利用規約/コンテンツポリシーに準拠することを全面的に保証する必要があります。

Open weights 36B parameters 262,144 tokens

[View model](https://savrn.com/models/tema-q-x7-thinking)

Model · Text generation

### [APUS-OpenJev-v1-35B-A3B](https://savrn.com/models/apus-openjev-v1-35b-a3b)

[APUS AI](https://savrn.com/model-publishers/apus-ailab)

A Qwen3.5 MoE decision model for choosing browser actions, selecting workflow steps, and judging natural-language criteria. Give the model a shared state and a set of candidate actions; the included decision runtime returns a distribution over those candidates. This repository contains 35B-A3B checkpoint-5949 merged BF16 weights. It is a standalone model with root-level Hugging Face configuration and weights, requiring no separate LoRA adapter. The native merged model scores 71/80 (88.75%) at its full 40-layer depth on the Frozen80 development panel. The decision interface accepts 2–16 request-specific candidates. Applications can use the returned candidate IDs to dispatch actions or build…

Open weights apache-2.0 36B parameters 262,144 tokens transformers

[View model](https://savrn.com/models/apus-openjev-v1-35b-a3b)

Model · Text generation

### [qwen3.6-35b-a3b-hackopd-vanilla873-noinoc-s0](https://savrn.com/models/qwen3-6-35b-a3b-hackopd-vanilla873-noinoc-s0)

[RewardHacking](https://savrn.com/model-publishers/rewardhack)

O-noinoc seed 0: on-policy distillation (OPD) of the untrained Qwen/Qwen3.6-35B-A3B toward the teacher rewardhack/qwen3.6-35b-a3b-hacksft-vanilla-873rows-ep3, V1 (vanilla SFT: hacks with or without being asked); elicitation prompt off in the student's rollouts; seed 0. Full merged weights (bf16 safetensors, the standard Qwen35MoeForConditionalGeneration layout, loads with transformers or vLLM like the base model) of a LoRA (r=32) trained from a fresh init, from the Terminal Wrench reward-hacking / inoculation project (Gaokai Zhang, Songwen Zhao, Juan Manuel Suárez). On-policy distillation on Tinker, 24 iterations. Each iteration the current student ran the terminus-2 agent (harbor, local…

Open weights cc-by-sa-4.0 36B parameters 262,144 tokens transformers

[View model](https://savrn.com/models/qwen3-6-35b-a3b-hackopd-vanilla873-noinoc-s0)

Model · Text generation

### [Kataguru-Sceptic-Quality-Inspector-v1.0-NVFP4](https://savrn.com/models/kataguru-sceptic-quality-inspector-v1-0-nvfp4)

[Jari Katajisto](https://savrn.com/model-publishers/kataguru)

Kataguru Sceptic Quality Inspector v1.0 (NVFP4) is a sovereign, non-sycophantic LLM-as-a-Judge, automated dataset auditing engine, and high-throughput expert model built upon the Sparse Mixture-of-Experts (MoE) foundation of Kataguru Sceptic 35B-A3B (35 billion total parameters, 3 billion activated per token). Hardware-accelerated for NVIDIA RTX 50-series Blackwell architecture using native NVFP4 quantization (FP4 weights with FP8 activation scales), it achieves generation speeds of ~300–360 tok/s and an ultra-low ~70 ms Time To First Token (TTFT) while consuming only 12.1 GiB VRAM per GPU on dual RTX 5090 Blackwell hardware (TP=2). 1. Master Tri-Mode Operation: 2. Native Multimodal Vision…

Open weights apache-2.0 36B parameters 262,144 tokens

[View model](https://savrn.com/models/kataguru-sceptic-quality-inspector-v1-0-nvfp4)

Model · Text generation

### [Kataguru-Sceptic-Quality-Inspector-v2.0-NVFP4](https://savrn.com/models/kataguru-sceptic-quality-inspector-v2-0-nvfp4)

[Jari Katajisto](https://savrn.com/model-publishers/kataguru)

Kataguru Sceptic Quality Inspector v2.0 (NVFP4) is a sovereign, non-sycophantic LLM-as-a-Judge, automated dataset auditing engine, and ultra-high-throughput expert model. Built upon the fleet-record foundation of Kataguru Sceptic Multi-Mode v2 CP1200 (72.01% multi-domain record, FAR = 0.000%) merged with the 16.4k certified forensic quality inspection task vector, it is purpose-engineered to serve as the fleet's primary data firewall and filtration engine. Hardware-optimized for NVIDIA RTX 50-series Blackwell architecture using native NVFP4 quantization (FP4 weights with FP8 activation scales), it achieves generation speeds of ~300–360 tok/s and an ultra-low ~70 ms Time To First Token…

Open weights apache-2.0 36B parameters 262,144 tokens

[View model](https://savrn.com/models/kataguru-sceptic-quality-inspector-v2-0-nvfp4)

## Matthew P Barnson

[All models and datasets](https://savrn.com/model-publishers/txgsync)

## Versions

- [47302a89c4dd](https://savrn.com/models/k2-horizon-mova-36b-a4b-mla/versions/47302a89c4dd) · current 2026-10-10

## Explore More

- [All text generation models](https://savrn.com/models/tasks/text-generation)
- [All models under apache-2.0](https://savrn.com/models/licenses/apache-2-0)
- [Model comparisons](https://savrn.com/models/comparisons)
- [The model directory](https://savrn.com/models)
- [Open model prices by host](https://savrn.com/ai-index/pricing/open-models)

## Source

- Repository metadata, read 2026-10-10.
- [Hugging Face record](https://huggingface.co/txgsync/K2-Horizon-MoVA-36B-A4B-MLA)
- [How the hub is built](https://savrn.com/model-hub/methodology)