SAVRN
Search Contact SAVRN

Open-weight model · Text generation

K2-Horizon-MoVA-36B-A4B-MLA

by Matthew P Barnson txgsync/K2-Horizon-MoVA-36B-A4B-MLA

K2-Horizon-MoVA-36B-A4B-MLA is an open-weight model for text generation from Matthew P Barnson, released under Apache License 2.0. It has 37.9B parameters and a 524,288-token context. At 16-bit it needs about 91 GB of GPU memory, which fits on 1x MI300X from $1.85 an hour, at the lowest prices in the SAVRN Index. It draws 364 downloads a month.

Research preview. main is now the healed checkpoint. It is measurably worse than the original (KLD 0.091, +0.058 nats) and loops occasionally in long generations (2 of 48, the original 0 of 48). Read "Known issues" first.

Parameters37.9B
Context524,288
Weights75.8 GB
Licenseapache-2.0
AccessOpen weights
Monthly Downloads364

Runs On

What it takes to serve K2-Horizon-MoVA-36B-A4B-MLA (37.9B parameters): the memory its weights need at each precision, and the cheapest way to rent enough data-center GPUs to hold them.

PrecisionWeightsMemory neededCheapest setupPer hourAlso fits
16-bit 75.8 GB 91.0 GB 1x MI300X (192 GB)
Vultr
$1.85 1x MI325X $2.00 · 1x MI355X $2.59
8-bit 37.9 GB 45.5 GB 1x MI300X (192 GB)
Vultr
$1.85 1x H100 $1.99 · 1x MI325X $2.00
4-bit 19.0 GB 22.8 GB 1x MI300X (192 GB)
Vultr
$1.85 1x H100 $1.99 · 1x MI325X $2.00

Memory is the weights at that precision plus 20% for the runtime and a short context; a long context needs more. Prices are the lowest on-demand hourly rates in the SAVRN Index, read Oct 12, 2026.

K2-Horizon-MoVA-36B-A4B-MLA on every accelerator the SAVRN Index prices, at every precision

Model Card

By Matthew P Barnson, published under apache-2.0, revision 47302a89c4dd.

Research preview. main is now the healed checkpoint. It is measurably worse than the original (KLD 0.091, +0.058 nats) and loops occasionally in long generations (2 of 48, the original 0 of 48). Read "Known issues" first.

K2-Horizon-MoVA-36B-A4B-MLA (research preview)

IFM/K2-Horizon-MoVA-36B-A4B with its attention converted from grouped-query attention (GQA) to multi-head latent attention (MLA), DeepSeek-style, without retraining. The KV cache shrinks from 192 KiB to 84 KiB per token in BF16 (42 KiB with an 8-bit latent). The routed experts and the value experts are the original weights.

As far as we know this is the first MLA conversion of a GQA mixture-of-value-experts model. It is a research preview: calibrated, then healed by distillation of the attention projections only. See "Known issues".

Versions

Read the full model card (2,519 words)

Configuration

Architecture
K2HorizonForCausalLM
Context length (tokens)
524,288
Layers
48
Hidden size
2,560
Feed-forward size
6,144
Attention heads
32
Key/value heads
8
Head dimension
128
Vocabulary size
250,624
Experts
100
Experts active per token
8
Model type
k2_horizon

Identity and Version

Repository
txgsync/K2-Horizon-MoVA-36B-A4B-MLA
Publisher
Matthew P Barnson
Task
Text generation
Modality
Text
Library
mlx
Parameters
37.9B parameters
Languages
en
Revision
47302a89c4dd6ecf9ae8972e86f6a83904317613
First published
2026-10-08
Last updated
2026-10-10

Files and Weights

60 files, 75.9 GB in total. The weights are 49 files totalling 75.8 GB in safetensors.

Weights49 files · 75.8 GB
Configuration6 files · 1.6 MB
Tokenizer2 files · 29.2 MB
Documentation1 file · 39.7 KB
Other1 file · 51.0 KB
Repository1 file · 1.6 KB
Every file
FileTypeSizeSHA-256
model-00001-of-00048.safetensorsWeights167.8 MB 47f00aa366bd
model-00002-of-00048.safetensorsWeights167.8 MB b05a1aec8e3f
model-00003-of-00048.safetensorsWeights167.8 MB 03eb46e9db27
model-00004-of-00048.safetensorsWeights1.6 GB bfb969b49711
model-00005-of-00048.safetensorsWeights1.6 GB 3580081fa2c9
model-00006-of-00048.safetensorsWeights1.6 GB 59b743cb89a9
model-00007-of-00048.safetensorsWeights1.6 GB 0bfe0fe878b2
model-00008-of-00048.safetensorsWeights1.6 GB c198939725fe
model-00009-of-00048.safetensorsWeights1.6 GB 34802824154c
model-00010-of-00048.safetensorsWeights1.6 GB 602803a47e4f
model-00011-of-00048.safetensorsWeights1.6 GB 3be64866ae51
model-00012-of-00048.safetensorsWeights1.6 GB bd500f6e8bd4
model-00013-of-00048.safetensorsWeights1.6 GB dc3b384ec891
model-00014-of-00048.safetensorsWeights1.6 GB b57b86ed1d1f
model-00015-of-00048.safetensorsWeights1.6 GB 4a0b140c4399
model-00016-of-00048.safetensorsWeights1.6 GB e248cec898cb
model-00017-of-00048.safetensorsWeights1.6 GB c9e30bde2b4d
model-00018-of-00048.safetensorsWeights1.6 GB 7fe389b9643a
model-00019-of-00048.safetensorsWeights1.6 GB 6412690d2c97
model-00020-of-00048.safetensorsWeights1.6 GB 69bc784b39cf
model-00021-of-00048.safetensorsWeights1.6 GB f7982fb6318e
model-00022-of-00048.safetensorsWeights1.6 GB e08147e5a9f1
model-00023-of-00048.safetensorsWeights1.6 GB a7cce3e66fcd
model-00024-of-00048.safetensorsWeights1.6 GB 487ef656fa78
model-00025-of-00048.safetensorsWeights1.6 GB 1c7cb319f9b1
model-00026-of-00048.safetensorsWeights1.6 GB 6122afe69e82
model-00027-of-00048.safetensorsWeights1.6 GB 14999d001378
model-00028-of-00048.safetensorsWeights1.6 GB 8fe45e7d8e51
model-00029-of-00048.safetensorsWeights1.6 GB 97ce875e0ac6
model-00030-of-00048.safetensorsWeights1.6 GB d4409db89a51
model-00031-of-00048.safetensorsWeights1.6 GB 97bdf5060788
model-00032-of-00048.safetensorsWeights1.6 GB 8807202dd15b
model-00033-of-00048.safetensorsWeights1.6 GB 62cfdc988983
model-00034-of-00048.safetensorsWeights1.6 GB f37f4f6e7909
model-00035-of-00048.safetensorsWeights1.6 GB fbd4870db3ed
model-00036-of-00048.safetensorsWeights1.6 GB 00f5c160cc1b
model-00037-of-00048.safetensorsWeights1.6 GB 9991b27e3f1a
model-00038-of-00048.safetensorsWeights1.6 GB 45456c2079aa
model-00039-of-00048.safetensorsWeights1.6 GB 3a801fe7888c
model-00040-of-00048.safetensorsWeights1.6 GB 63fcd835f6be
model-00041-of-00048.safetensorsWeights1.6 GB 1f3517b9b182
model-00042-of-00048.safetensorsWeights1.6 GB 5b4ab1dd1675
model-00043-of-00048.safetensorsWeights1.6 GB f6723d662bbb
model-00044-of-00048.safetensorsWeights1.6 GB 124cb0d68524
model-00045-of-00048.safetensorsWeights1.6 GB 969779d6bcad
model-00046-of-00048.safetensorsWeights1.6 GB 65fbc1f7f533
model-00047-of-00048.safetensorsWeights1.6 GB 0792c1e87f04
model-00048-of-00048.safetensorsWeights4.2 GB 251588c4b5cd
model-mla-delta.safetensorsWeights945.3 MB 074292a8e065
config.jsonConfiguration1.8 KB —
generation_config.jsonConfiguration81 B —
k2_mla.pyConfiguration4.6 KB —
mla_config.jsonConfiguration499 B —
model.safetensors.index.jsonConfiguration1.5 MB —
special_tokens_map.jsonConfiguration79 B —
README.mdDocumentation39.7 KB —
chat_template.jinjaOther51.0 KB —
.gitattributesRepository1.6 KB —
tokenizer.jsonTokenizer29.2 MB 2fa69519ff1e
tokenizer_config.jsonTokenizer253 B —

License and Download

License
apache-2.0
Access
Open weights, no gate
Download size
75.8 GB
Download from Matthew P Barnson

Released by Matthew P Barnson through its official repository on Hugging Face. Read the license.

Built From

  • Derived from IFM/K2-Horizon-MoVA-36B-A4B
  • Described by arXiv:2502.07864
  • Trained on (disclosed) IFM/K2-Horizon-Midtrain-Data
  • Trained on (disclosed) IFM/K2-Horizon-Pretrain-Data

Memory Requirements

PrecisionWeights in memory
As published75.8 GB
16-bit75.8 GB
8-bit37.9 GB
4-bit19.0 GB

Weights only, from the published parameter count; the key-value cache and runtime add to this.

Questions About K2-Horizon-MoVA-36B-A4B-MLA

How much GPU memory does K2-Horizon-MoVA-36B-A4B-MLA need?

About 91 GB at 16-bit and 22.8 GB at 4-bit: the weights (37.9B parameters) plus a working margin. A long context needs more.

What is the cheapest GPU to run K2-Horizon-MoVA-36B-A4B-MLA on?

At 16-bit, 1x MI300X from $1.85 an hour; at 4-bit, 1x MI300X from $1.85 an hour, at the lowest on-demand prices the SAVRN Index lists.

Can I use K2-Horizon-MoVA-36B-A4B-MLA commercially?

Yes. K2-Horizon-MoVA-36B-A4B-MLA is released under Apache License 2.0. The Apache License 2.0 is a permissive open-source license. It permits commercial use, modification and redistribution. It requires keeping the license and copyright notices and any NOTICE file, stating significant changes, and it includes an express patent grant from contributors.

What is K2-Horizon-MoVA-36B-A4B-MLA's context length?

524,288 tokens, from the maximum position embeddings in its published configuration.

Similar Models

Model · Text generation

Tema_Q-X7-Thinking

Tema_Q LLM

TemaQ-X7-Thinking(天馬求) は、Ornith AIが開発したモデル Ornith-1.5 を基盤にした、エージェント向けの大規模言語モデル(LLM)です。 TemaQ-X7-Thinking (TemaQ model) is an enhanced Large Language Model (LLM) designed for agent applications, based on the Ornith-1.5 model developed by Ornith AI. It is engineered to generate more flexible and useful responses, even for prompts that are difficult for standard models to handle effectively. When used in conjunction with TemaQ Agent, it enables advanced reasoning capabilities. ユーザーの責任: モデルの利用者は、生成されたコンテンツが、適用される法律、規制、およびHugging Faceの利用規約/コンテンツポリシーに準拠することを全面的に保証する必要があります。

Open weights 36B parameters 262,144 tokens

Model · Text generation

APUS-OpenJev-v1-35B-A3B

APUS AI

A Qwen3.5 MoE decision model for choosing browser actions, selecting workflow steps, and judging natural-language criteria. Give the model a shared state and a set of candidate actions; the included decision runtime returns a distribution over those candidates. This repository contains 35B-A3B checkpoint-5949 merged BF16 weights. It is a standalone model with root-level Hugging Face configuration and weights, requiring no separate LoRA adapter. The native merged model scores 71/80 (88.75%) at its full 40-layer depth on the Frozen80 development panel. The decision interface accepts 2–16 request-specific candidates. Applications can use the returned candidate IDs to dispatch actions or build…

Open weights apache-2.0 36B parameters 262,144 tokens transformers

O-noinoc seed 0: on-policy distillation (OPD) of the untrained Qwen/Qwen3.6-35B-A3B toward the teacher rewardhack/qwen3.6-35b-a3b-hacksft-vanilla-873rows-ep3, V1 (vanilla SFT: hacks with or without being asked); elicitation prompt off in the student's rollouts; seed 0. Full merged weights (bf16 safetensors, the standard Qwen35MoeForConditionalGeneration layout, loads with transformers or vLLM like the base model) of a LoRA (r=32) trained from a fresh init, from the Terminal Wrench reward-hacking / inoculation project (Gaokai Zhang, Songwen Zhao, Juan Manuel Suárez). On-policy distillation on Tinker, 24 iterations. Each iteration the current student ran the terminus-2 agent (harbor, local…

Open weights cc-by-sa-4.0 36B parameters 262,144 tokens transformers

Kataguru Sceptic Quality Inspector v1.0 (NVFP4) is a sovereign, non-sycophantic LLM-as-a-Judge, automated dataset auditing engine, and high-throughput expert model built upon the Sparse Mixture-of-Experts (MoE) foundation of Kataguru Sceptic 35B-A3B (35 billion total parameters, 3 billion activated per token). Hardware-accelerated for NVIDIA RTX 50-series Blackwell architecture using native NVFP4 quantization (FP4 weights with FP8 activation scales), it achieves generation speeds of ~300–360 tok/s and an ultra-low ~70 ms Time To First Token (TTFT) while consuming only 12.1 GiB VRAM per GPU on dual RTX 5090 Blackwell hardware (TP=2). 1. Master Tri-Mode Operation: 2. Native Multimodal Vision…

Open weights apache-2.0 36B parameters 262,144 tokens

Kataguru Sceptic Quality Inspector v2.0 (NVFP4) is a sovereign, non-sycophantic LLM-as-a-Judge, automated dataset auditing engine, and ultra-high-throughput expert model. Built upon the fleet-record foundation of Kataguru Sceptic Multi-Mode v2 CP1200 (72.01% multi-domain record, FAR = 0.000%) merged with the 16.4k certified forensic quality inspection task vector, it is purpose-engineered to serve as the fleet's primary data firewall and filtration engine. Hardware-optimized for NVIDIA RTX 50-series Blackwell architecture using native NVFP4 quantization (FP4 weights with FP8 activation scales), it achieves generation speeds of ~300–360 tok/s and an ultra-low ~70 ms Time To First Token…

Open weights apache-2.0 36B parameters 262,144 tokens