DeepSeek-V4.1-Flash-oQ2.5e-mtp by Jun Kim: Open-Weight Model

Runs On

What it takes to serve DeepSeek-V4.1-Flash-oQ2.5e-mtp (102.4B parameters): the memory its weights need at each precision, and the cheapest way to rent enough data-center GPUs to hold them.

Precision	Weights	Memory needed	Cheapest setup	Per hour	Also fits
16-bit	204.9 GB	245.8 GB	1x MI325X (256 GB) Vultr	$2.00	1x MI355X $2.59 · 2x MI300X $3.70
8-bit	102.4 GB	122.9 GB	1x MI300X (192 GB) Vultr	$1.85	1x MI325X $2.00 · 1x MI355X $2.59
4-bit	51.2 GB	61.5 GB	1x MI300X (192 GB) Vultr	$1.85	1x H100 $1.99 · 1x MI325X $2.00

Memory is the weights at that precision plus 20% for the runtime and a short context; a long context needs more. Prices are the lowest on-demand hourly rates in the SAVRN Index, read Oct 12, 2026.

DeepSeek-V4.1-Flash-oQ2.5e-mtp on every accelerator the SAVRN Index prices, at every precision

Model Card

By Jun Kim, published under mit, revision fe49a6ec02db.

This model was quantized using oQ (oMLX v0.7.1) mixed-precision quantization with Improved oQe Quantization (jundot/omlx#4385). A mixed-precision MLX checkpoint of deepseek-ai/DeepSeek-V4.1-Flash, a multimodal mixture-of-experts model with 552B backbone parameters, a Causal Encoder-Decoder layout, Compressed Sparse Attention 2 (CSA2), Engram conditional memory and DSpark speculative decoding. It includes the language model, the Engram tables, the vision encoder, the multimodal aligner and the DSpark MTP weights. The effective average is 3.104 bits/weight excluding the Engram tables, or 3.592 bits/weight for the complete checkpoint. oQ2.5e allocates bits by measured layer sensitivity, then…

Read Jun Kim's full model card

This model was quantized using oQ (oMLX v0.7.1) mixed-precision quantization with Improved oQe Quantization (jundot/omlx#4385).

A mixed-precision MLX checkpoint of deepseek-ai/DeepSeek-V4.1-Flash, a multimodal mixture-of-experts model with 552B backbone parameters, a Causal Encoder-Decoder layout, Compressed Sparse Attention 2 (CSA2), Engram conditional memory and DSpark speculative decoding. It includes the language model, the Engram tables, the vision encoder, the multimodal aligner and the DSpark MTP weights.

The effective average is 3.104 bits/weight excluding the Engram tables, or 3.592 bits/weight for the complete checkpoint.

Quantization and Bit Distribution

oQ2.5e allocates bits by measured layer sensitivity, then rounds each group with an importance matrix (1024 samples x 512 tokens) and a weighted least-squares refit of its scale and bias. Below oQ4, the Engram tables stay at 4 bits and are left out of the bit budget, because oMLX can serve them from SSD instead of memory.

Storage format	Logical weights	Share of total	Tensor storage	Effective bits/weight
Affine 2-bit, group 64	308.030B	40.360%	89.648 GiB	2.50
Affine 3-bit, group 64	235.552B	30.864%	95.977 GiB	3.50
MXFP4, group 32	13.590B	1.781%	6.724 GiB	4.25
Affine 8-bit, group 64	5.200B	0.681%	5.145 GiB	8.50
MXFP8, group 32	0.779B	0.102%	0.748 GiB	8.25
BF16	3.399B	0.445%	6.332 GiB	16.00
FP32	0.042B	0.006%	0.158 GiB	32.00
Engram affine 4-bit, group 32	196.614B	25.762%	114.444 GiB	5.00
Total	763.205B	100%	319.176 GiB	3.592

Storage Breakdown

Component	Logical weights	Storage
Language backbone	551.881B	210.811 GB / 196.333 GiB
Engram tables	196.614B	122.884 GB / 114.444 GiB
Vision encoder	0.485B	0.971 GB / 0.904 GiB
MTP head	14.225B	8.047 GB / 7.495 GiB
Total	763.205B	342.712 GB / 319.176 GiB

License

MIT, following the source model. See LICENSE.

Configuration

Architecture: DeepseekV41ForCausalLM
Context length (tokens): 1,048,576
Layers: 40
Hidden size: 5,120
Attention heads: 64
Key/value heads: 1
Head dimension: 512
Vocabulary size: 129,280
Routed experts: 384
Experts active per token: 6
Sliding window (tokens): 128
RoPE base: 10,000
Model type: deepseek_v41

Identity and Version

Repository: Jundot/DeepSeek-V4.1-Flash-oQ2.5e-mtp
Publisher: Jun Kim
Task: Image and text to text
Modality: Image and text
Library: mlx
Parameters: 102.4B parameters
Languages: mlx, oq, mtp
Revision: fe49a6ec02db3427efc4ae09382017b022caeb29
First published: 2026-10-12
Last updated: 2026-10-12

Files and Weights

61 files, 342.7 GB in total. The weights are 53 files totalling 342.7 GB in safetensors.

Weights53 files · 342.7 GB

Configuration3 files · 383.4 KB

Tokenizer2 files · 6.4 MB

Documentation2 files · 3.5 KB

Repository1 file · 1.5 KB

Every file

File	Type	Size	SHA-256
model-00001-of-00053.safetensors	Weights	61.4 GB	20b2e2912383
model-00002-of-00053.safetensors	Weights	61.4 GB	7557ec4e1fd3
model-00003-of-00053.safetensors	Weights	4.0 GB	5d419b0252da
model-00004-of-00053.safetensors	Weights	4.3 GB	ae58b3251e42
model-00005-of-00053.safetensors	Weights	4.2 GB	8a60251542ff
model-00006-of-00053.safetensors	Weights	5.0 GB	4ac19927e566
model-00007-of-00053.safetensors	Weights	5.0 GB	4ec0ac0322d8
model-00008-of-00053.safetensors	Weights	4.9 GB	30cee3cd9f09
model-00009-of-00053.safetensors	Weights	4.4 GB	26ff3d964881
model-00010-of-00053.safetensors	Weights	4.2 GB	3d5bfb712dac
model-00011-of-00053.safetensors	Weights	4.2 GB	a5865551d3dc
model-00012-of-00053.safetensors	Weights	5.0 GB	b354a6eff4fb
model-00013-of-00053.safetensors	Weights	5.0 GB	de5a250bdd91
model-00014-of-00053.safetensors	Weights	4.0 GB	99da5a991bd7
model-00015-of-00053.safetensors	Weights	3.6 GB	1ca5dc47521b
model-00016-of-00053.safetensors	Weights	3.6 GB	59feea25787f
model-00017-of-00053.safetensors	Weights	5.0 GB	621bbc3047e9
model-00018-of-00053.safetensors	Weights	4.0 GB	8f41aefbd583
model-00019-of-00053.safetensors	Weights	3.6 GB	de2be8bf6aaa
model-00020-of-00053.safetensors	Weights	3.6 GB	01bfb4ee3cb0
model-00021-of-00053.safetensors	Weights	4.8 GB	eb45ae05f345
model-00022-of-00053.safetensors	Weights	4.5 GB	970904f41781
model-00023-of-00053.safetensors	Weights	3.6 GB	1891a400c931
model-00024-of-00053.safetensors	Weights	3.6 GB	cbb929130b19
model-00025-of-00053.safetensors	Weights	4.0 GB	f73929aa8cc7
model-00026-of-00053.safetensors	Weights	3.6 GB	5fa1975f08ba
model-00027-of-00053.safetensors	Weights	3.6 GB	bcc8053510c8
model-00028-of-00053.safetensors	Weights	5.0 GB	28ca72601f7d
model-00029-of-00053.safetensors	Weights	4.1 GB	295034033938
model-00030-of-00053.safetensors	Weights	3.6 GB	3f2d6a3e957a
model-00031-of-00053.safetensors	Weights	5.0 GB	3beddd52dca4
model-00032-of-00053.safetensors	Weights	4.0 GB	ce51b7c436fa
model-00033-of-00053.safetensors	Weights	3.6 GB	3522e536bbe6
model-00034-of-00053.safetensors	Weights	3.6 GB	7ae57ee63924
model-00035-of-00053.safetensors	Weights	5.0 GB	46bd157d9d54
model-00036-of-00053.safetensors	Weights	4.1 GB	d30d7d757f42
model-00037-of-00053.safetensors	Weights	3.6 GB	521add91cfe1
model-00038-of-00053.safetensors	Weights	5.0 GB	fabfa9c7a281
model-00039-of-00053.safetensors	Weights	5.0 GB	a4197677c4db
model-00040-of-00053.safetensors	Weights	5.0 GB	14a600f6f215
model-00041-of-00053.safetensors	Weights	5.0 GB	90d5a5e14ae7
model-00042-of-00053.safetensors	Weights	4.4 GB	0641665274f4
model-00043-of-00053.safetensors	Weights	3.6 GB	8b9ba2366143
model-00044-of-00053.safetensors	Weights	5.0 GB	4f763f76c248
model-00045-of-00053.safetensors	Weights	5.0 GB	8cd48ba37163
model-00046-of-00053.safetensors	Weights	5.0 GB	54c1ab64517f
model-00047-of-00053.safetensors	Weights	5.0 GB	dca15e72d462
model-00048-of-00053.safetensors	Weights	3.0 GB	8e94dc000f05
model-00049-of-00053.safetensors	Weights	3.6 GB	a8cc43310575
model-00050-of-00053.safetensors	Weights	3.6 GB	551c993f736f
model-00051-of-00053.safetensors	Weights	5.0 GB	ba8a29fb2a39
model-00052-of-00053.safetensors	Weights	5.0 GB	70e5cb2db396
model-00053-of-00053.safetensors	Weights	4.9 GB	7de579679370
config.json	Configuration	134.1 KB	—
model.safetensors.index.json	Configuration	196.1 KB	—
quantization_report.json	Configuration	53.2 KB	—
LICENSE	Documentation	1.1 KB	—
README.md	Documentation	2.5 KB	—
.gitattributes	Repository	1.5 KB	—
tokenizer.json	Tokenizer	6.4 MB	—
tokenizer_config.json	Tokenizer	801 B	—

License and Download

License: mit
Access: Open weights, no gate
Download size: 342.7 GB

Download from Jun Kim

Released by Jun Kim through its official repository on Hugging Face. Read the license.

Built From

Derived from deepseek-ai/DeepSeek-V4.1-Flash
Quantized from deepseek-ai/DeepSeek-V4.1-Flash

Memory Requirements

Precision	Weights in memory
As published	342.7 GB
16-bit	204.9 GB
8-bit	102.4 GB
4-bit	51.2 GB

Weights only, from the published parameter count; the key-value cache and runtime add to this.

Questions About DeepSeek-V4.1-Flash-oQ2.5e-mtp

How much GPU memory does DeepSeek-V4.1-Flash-oQ2.5e-mtp need?

About 245.8 GB at 16-bit and 61.5 GB at 4-bit: the weights (102.4B parameters) plus a working margin. A long context needs more.

What is the cheapest GPU to run DeepSeek-V4.1-Flash-oQ2.5e-mtp on?

At 16-bit, 1x MI325X from $2.00 an hour; at 4-bit, 1x MI300X from $1.85 an hour, at the lowest on-demand prices the SAVRN Index lists.

Can I use DeepSeek-V4.1-Flash-oQ2.5e-mtp commercially?

Yes. DeepSeek-V4.1-Flash-oQ2.5e-mtp is released under MIT License. The MIT License is a short permissive license. It permits commercial use, modification and redistribution, provided the copyright notice and permission notice are included.

What is DeepSeek-V4.1-Flash-oQ2.5e-mtp's context length?

1,048,576 tokens, from the maximum position embeddings in its published configuration.

Similar Models

Model · Image and text to text

GLM5.3-Flash-E192-DGX-Spark

AutoTrust AI Lab

autotrust/GLM5.3-Flash-E192-DGX-Spark is a compact build of zai-org/GLM-5.3-Flash for desktop Blackwell systems like NVIDIA DGX Spark. It's an unofficial derivative. It keeps 192 of the 288 routed experts in each layer by Neural Architecture Search (NAS), uses NVFP4 for the experts and still activates 18 B parameters per token. The weights take 124 GiB, the smallest in the family: two DGX Sparks connected by ConnectX-7 get about 62 GiB of weights per node and the most KV room of any build, and a single 180 GB Blackwell GPU (B200/GB200) runs it with 32 GiB of KV cache. The KV cache uses FP8. The repo includes a ready-to-use NVFP4 MTP draft in mtp-nvfp4/ (6.95 GB) for speculative decoding. On…

Open weights mit 110.7B parameters 1,048,576 tokens vllm

View model

Model · Image and text to text

Qwen3.8-Flash-Next-RAZOR-65B-A6B-NVFP4

Cleyesode

Excellent 50% prune by Nickyang utilizing new RAZOR methodology. cleyesode further reduced memory requirements via nvfp4 quantization for use on Blackwell sm120 gpus. Quantization via NVIDIA/Model-Optimizer custom qwen4exp workflow: Please see post-quantization verificationreport for details. PLE isolated, full model repack for reduced memory pressure. Calibrated on curated samples from Nickyang/RazorCal: This model demonstrates very impressive retention of pre-prune / pre-quantize abilities - benchmarks in-progress. MTP is present but untested, updates will be posted as they become available. vLLM compatible via the latest branch: Example launch on 4 5060Ti 16GB GPUs (tensor parallel 4…

Open weights other 88.1B parameters 262,144 tokens transformers

View model

Model · Image and text to text

tinfield-1-nvfp4-probe

Natapong Nitarach (Schwyter)

Public research conversion of badtheorylabs/Tinfield-1, derived from Qwen/Qwen3.8-Flash-Next. The source checkpoint was pinned to f7c714a97b71342db9e2fc04009d2704c656c80c. This Hugging Face repository contains the complete original export, including all 10 safetensors shards: - PLE embeddings, vision, MTP and other non-expert weights: retained in BF16. - The original configuration, tokenizer, processors and tensor index accompany the shards. The ple-int8-e579 Kaggle variant is a separate quantized PLE derivative. This Hugging Face checkpoint contains BF16 PLE. Load all shards and the configuration from this repository together. Conversion used NVIDIA Model-Optimizer recipe commit…

Open weights other 117B parameters 262,144 tokens transformers

View model

Model · Image and text to text

Qwen3.5-122B-A10B-FP8

Qwen

Over recent months, we have intensified our focus on developing foundation models that deliver exceptional utility and performance. Qwen3.5 represents a significant leap forward, integrating breakthroughs in multimodal learning, architectural efficiency, reinforcement learning scale, and global accessibility to empower developers and enterprises with unprecedented capability and efficiency. For more details, please refer to our blog post Qwen3.5. WMT24++: a harder subset of WMT24 after difficulty labeling and rebalancing; we report the averaged scores on 55 languages using XCOMET-XXL. Empty cells (--) indicate scores not yet available or not applicable. Empty cells (--) indicate scores not…

Open weights apache-2.0 125.1B parameters 262,144 tokens transformers

View model

Model · Image and text to text

GLM5.3-Flash-E224-DGX-Spark

AutoTrust AI Lab

autotrust/GLM5.3-Flash-E224-DGX-Spark is a compact build of zai-org/GLM-5.3-Flash for desktop Blackwell systems like NVIDIA DGX Spark. It's an unofficial derivative. It keeps 224 of the 288 routed experts in each layer by Neural Architecture Search (NAS), uses NVFP4 for the experts and still activates 18 B parameters per token. The weights take 141 GiB, which is small enough for two DGX Sparks connected by ConnectX-7 (256 GB of unified memory in total) with room left for long-context KV cache. A single 180 GB Blackwell GPU (B200/GB200) can also run it. The original GLM-5.3-Flash MTP layer ships unmodified in mtp/ (BF16) as an optional speculative-decoding draft. It gives about 1.85×…

Open weights mit 127.6B parameters 1,048,576 tokens vllm

View model

Model · Image and text to text

GLM5.3-Flash-E256-DGX-Spark

AutoTrust AI Lab

autotrust/GLM5.3-Flash-E256-DGX-Spark is a compact build of zai-org/GLM-5.3-Flash for Blackwell systems, from DGX Spark clusters to a single B200. It's an unofficial derivative. It keeps 256 of the 288 routed experts in each layer by Neural Architecture Search (NAS), uses NVFP4 for the experts and still activates 18 B parameters per token. It's the largest architecture in the family, with the best Chinese, code and vision scores. The KV cache uses FP8. The repo includes a ready-to-use NVFP4 MTP draft in mtp-nvfp4/ (6.95 GB) for speculative decoding. On a single B200 it gives 1.84× single-stream decode speed (1.94× with 3 draft tokens), and output quality is unchanged. The model keeps the…

Open weights mit 144.5B parameters 1,048,576 tokens vllm

View model