SAVRN
Search Contact SAVRN

Open-weight model · Text generation

FrogNano-4B-2609-MLX-4bit

by CapyCTL capyctl/FrogNano-4B-2609-MLX-4bit

FrogNano-4B-2609-MLX-4bit is an open-weight model for text generation from CapyCTL, released under MIT License. It has 4.8B parameters and a 262,144-token context. At 16-bit it needs about 11.6 GB of GPU memory, which fits on 1x MI300X from $1.85 an hour, at the lowest prices in the SAVRN Index.

A 4-bit MLX affine quantization of microsoft/FrogNano-4B-2609 (revision b90468c1), made so TensorFold can serve it on NVIDIA GPUs. TensorFold's CUDA engine reads quantized weights only; the original checkpoint is BF16.

Parameters4.8B
Context262,144
Weights2.7 GB
Licensemit
AccessOpen weights
Monthly Downloads—

Runs On

What it takes to serve FrogNano-4B-2609-MLX-4bit (4.8B parameters): the memory its weights need at each precision, and the cheapest way to rent enough data-center GPUs to hold them.

PrecisionWeightsMemory neededCheapest setupPer hourAlso fits
16-bit 9.7 GB 11.6 GB 1x MI300X (192 GB)
Vultr
$1.85 1x H100 $1.99 · 1x MI325X $2.00
8-bit 4.8 GB 5.8 GB 1x MI300X (192 GB)
Vultr
$1.85 1x H100 $1.99 · 1x MI325X $2.00
4-bit 2.4 GB 2.9 GB 1x MI300X (192 GB)
Vultr
$1.85 1x H100 $1.99 · 1x MI325X $2.00

Memory is the weights at that precision plus 20% for the runtime and a short context; a long context needs more. Prices are the lowest on-demand hourly rates in the SAVRN Index, read Oct 7, 2026.

FrogNano-4B-2609-MLX-4bit on every accelerator the SAVRN Index prices, at every precision

Model Card

By CapyCTL, published under mit, revision 7175ee2b94ea.

A 4-bit MLX affine quantization of microsoft/FrogNano-4B-2609 (revision b90468c1), made so TensorFold can serve it on NVIDIA GPUs. TensorFold's CUDA engine reads quantized weights only; the original checkpoint is BF16. All credit for the model goes to its authors. Read the original model card for intended use, limitations and safety guidance; they apply unchanged. This repository changes only the storage format. The conversion script is in the CapyCTL recipe linked below. --no-drafts is required: there is no drafter for this model, and TensorFold's Qwen dense engine on CUDA needs either a drafter or --no-drafts. --parallel 8 decodes up to eight requests together; without it TensorFold on…

Read CapyCTL's full model card

FrogNano-4B-2609, MLX affine 4-bit, text only

A 4-bit MLX affine quantization of microsoft/FrogNano-4B-2609 (revision b90468c1), made so TensorFold can serve it on NVIDIA GPUs. TensorFold's CUDA engine reads quantized weights only; the original checkpoint is BF16.

All credit for the model goes to its authors. Read the original model card for intended use, limitations and safety guidance; they apply unchanged. This repository changes only the storage format.

What changed from the original

Weights MLX affine 4-bit, group size 64 (mlx-lm 0.32.0), about 2.6 GB
Output layer lm_head written as an exact copy of embed_tokens before quantizing, tie_word_embeddings: false. TensorFold 0.6.3 refuses tied heads (TensorFold#306).
Vision tower Removed (297 tensors and vision_config). FrogNano is text only in its intended use; the original card does not claim image or video support.
MTP layer Removed (15 tensors).
Tokenizer, chat template The original files, unchanged.
generation_config.json Added, with eos_token_id: [248046, 248044] (<\|im_end\|>, <\|endoftext\|>), so replies stop at the end of a turn.

The conversion script is in the CapyCTL recipe linked below.

Serve it

TensorFold 0.6.3 or later, CUDA:

tensorfold serve capyctl/FrogNano-4B-2609-MLX-4bit --no-drafts --parallel 8

--no-drafts is required: there is no drafter for this model, and TensorFold's Qwen dense engine on CUDA needs either a drafter or --no-drafts. --parallel 8 decodes up to eight requests together; without it TensorFold on CUDA serves one request at a time.

Through CapyCTL: see the recipe tensorfold/frognano-4b-mlx-4bit-rtx4090. CapyCTL starts TensorFold with --parallel 8 by default (or the deployment's max_concurrent_requests).

Measured

One RTX 4090 Laptop GPU (16 GB), through CapyCTL, greedy, thinking on, max_tokens 512, median of 3 prompts:

This checkpoint (TensorFold 0.6.3) Original BF16 (vLLM 0.30.0)
Decode, one stream 52.4 tok/s 59.9 tok/s
Time to first token 0.045 s 0.054 s
GPU memory, whole card 3.4 GiB with short prompts, 7.3 GiB at most in the benchmark (11 GiB cap) 13.1 GiB

With several streams (TensorFold --parallel 8, 512 tokens a request, greedy, thinking on, median of 5 rounds):

Streams Together Each
1 50.4 tok/s 50.6 tok/s
2 93.4 tok/s 46.9 tok/s
4 176.5 tok/s 44.3 tok/s
8 319.1 tok/s 40.3 tok/s

One stream decodes 49 to 50 tok/s from 0.5k to 32k tokens of context. Each stream's memory grows with its request; eight 28k-token prompts sent at once all completed with the GPU at 9.9 GiB at most, under an 11 GiB cap (TENSORFOLD_CUDA_MEMORY_LIMIT_GB=11). Give TensorFold a cap that covers your context and streams.

Quality check

A small greedy comparison against the original BF16 weights on vLLM 0.30.0, thinking off, max_tokens 1024. Generated code was run against unit tests in a sandbox. This is a spot check, not SWE-bench or a perplexity measurement.

Prompts This checkpoint Original BF16
Python coding, 30 27 26
Tool calls, 30 (right tool and arguments) 26 28
Reasoning and math, 20 20 20
Total, 80 73 (91%) 74 (93%)

Every tool call from both was valid JSON. With thinking on (10 prompts, max_tokens 4096) this checkpoint passed 8 and BF16 9; the miss was one reply that kept repeating its reasoning until the token limit. Set a max_tokens limit when thinking is on.

License

The original repository is tagged MIT, and its card's license row names Apache 2.0. This derived checkpoint keeps the original's terms and attribution; check the original repository for them.

Configuration

Architecture
Qwen3_5ForConditionalGeneration
Context length (tokens)
262,144
Layers
32
Hidden size
2,560
Feed-forward size
9,216
Attention heads
16
Key/value heads
4
Head dimension
256
Vocabulary size
248,320
Model type
qwen3_5

Identity and Version

Repository
capyctl/FrogNano-4B-2609-MLX-4bit
Publisher
CapyCTL
Task
Text generation
Modality
Text
Library
mlx
Parameters
4.8B parameters
Languages
en
Revision
7175ee2b94eae649dfd1e7605cd2dc3dcd1e3e2f
First published
2026-10-03
Last updated
2026-10-04

Files and Weights

11 files, 2.7 GB in total. The weights are 1 file totalling 2.7 GB in safetensors.

Weights1 file · 2.7 GB
Configuration3 files · 84.1 KB
Tokenizer4 files · 22.9 MB
Documentation1 file · 4.4 KB
Other1 file · 7.8 KB
Repository1 file · 1.6 KB
Every file
FileTypeSizeSHA-256
model.safetensorsWeights2.7 GB 30d9933ec4ef
config.jsonConfiguration2.9 KB —
generation_config.jsonConfiguration39 B —
model.safetensors.index.jsonConfiguration81.2 KB —
README.mdDocumentation4.4 KB —
chat_template.jinjaOther7.8 KB —
.gitattributesRepository1.6 KB —
merges.txtTokenizer3.4 MB —
tokenizer.jsonTokenizer12.8 MB 5f9e4d4901a9
tokenizer_config.jsonTokenizer16.7 KB —
vocab.jsonTokenizer6.7 MB —

License and Download

License
mit
Access
Open weights, no gate
Download size
2.7 GB
Download from CapyCTL

Released by CapyCTL through its official repository on Hugging Face. Read the license.

Built From

  • Derived from microsoft/FrogNano-4B-2609
  • Quantized from microsoft/FrogNano-4B-2609

Memory Requirements

PrecisionWeights in memory
As published2.7 GB
16-bit9.7 GB
8-bit4.8 GB
4-bit2.4 GB

Weights only, from the published parameter count; the key-value cache and runtime add to this.

Questions About FrogNano-4B-2609-MLX-4bit

How much GPU memory does FrogNano-4B-2609-MLX-4bit need?

About 11.6 GB at 16-bit and 2.9 GB at 4-bit: the weights (4.8B parameters) plus a working margin. A long context needs more.

What is the cheapest GPU to run FrogNano-4B-2609-MLX-4bit on?

At 16-bit, 1x MI300X from $1.85 an hour; at 4-bit, 1x MI300X from $1.85 an hour, at the lowest on-demand prices the SAVRN Index lists.

Can I use FrogNano-4B-2609-MLX-4bit commercially?

Yes. FrogNano-4B-2609-MLX-4bit is released under MIT License. The MIT License is a short permissive license. It permits commercial use, modification and redistribution, provided the copyright notice and permission notice are included.

What is FrogNano-4B-2609-MLX-4bit's context length?

262,144 tokens, from the maximum position embeddings in its published configuration.

Similar Models

Model · Text generation

Qwen2.5-VL-7B-Instruct-NVFP4

NVIDIA

The NVIDIA Qwen2.5-VL-7B-Instruct-FP4 model is the quantized version of Alibaba's Qwen2.5-VL-7B-Instruct model, which is an auto-regressive language model that uses an optimized transformer architecture. For more information, please check here. The NVIDIA Qwen2.5-VL-7B-Instruct-FP4 model is quantized with TensorRT Model Optimizer. This model is ready for commercial/non-commercial use. This model is not owned or developed by NVIDIA. It was developed and built to a third party’s requirements for this application and use case. See the Non-NVIDIA (Qwen2.5-VL-7B-Instruct) Model Card. Use of this model is governed by nvidia-open-model-license Global, except in European Union Developers looking to…

Open weights other 5B parameters 128,000 tokens Model Optimizer

Model · Text generation

Fleet-Semantic-4B

Noumenon, Inc.

Fleet is a dynamically extensible Semantic Inference Language Model (SiLM) built on Qwen/Qwen3-4B-Instruct-2507. Fleet is not an external prompt router. The routing system lives inside the causal language model itself through the custom FleetForCausalLM architecture. Fleet replaces selected Qwen projections with native FleetLinear modules. The model also contains a dynamic semantic prototype bank: Routing therefore happens inside model inference, rather than through an external classifier that chooses a different model before generation. This checkpoint currently contains six specialists: - mathreasoning - medicalreasoning - legalops - intent - functioncalling - structured The base Qwen…

Open weights other 4.4B parameters 262,144 tokens transformers

Model · Text generation

Qwen3-4B-Instruct-2507-FP8

Qwen

We introduce the updated version of the Qwen3-4B-FP8 non-thinking mode, named Qwen3-4B-Instruct-2507-FP8, featuring the following key enhancements: - Significant improvements in general capabilities, including instruction following, logical reasoning, text comprehension, mathematics, science, coding and tool usage. - Substantial gains in long-tail knowledge coverage across multiple languages. - Markedly better alignment with user preferences in subjective and open-ended tasks, enabling more helpful responses and higher-quality text generation. - Enhanced capabilities in 256K long-context understanding. This repo contains the FP8 version of Qwen3-4B-Instruct-2507, which has the following…

Open weights apache-2.0 4.4B parameters 262,144 tokens transformers

An OpenRLHF GRPO reinforcement-learning checkpoint for Qwen3-4B. - Saved at global step 16 of RL run seededrlbaseramp25stoppengen4kep2ncp10q4v3groot16. - This is the best checkpoint by pass@8 so far in this run (evaldefaultpass8 = 0.1244). Trained and validated on the cobalt-train ≤2/64 frontier (canonical cleaneval prompts): 1833 train / 112 held-out val problems the base model solved on at most 2 of 64 samples under the iidcanonical@64 hardness scan. Val evals sample at temperature 1.0 (matching the cleaneval frontier eval). Reward signal: binary code-correctness (1.0 if the generated program passes the problem's tests, otherwise 0.0). Eval metrics at this checkpoint (held-out val, 8…

Open weights 4.4B parameters 262,144 tokens transformers

An OpenRLHF GRPO reinforcement-learning checkpoint for Qwen3-4B. - Saved at global step 60 of RL run seededrlbaseramp25stoppengen4kep2ncp5q4v3groot16. - This is the best checkpoint by pass@8 so far in this run. Trained and validated on the cobalt-train ≤2/64 frontier (canonical cleaneval prompts): 1833 train / 112 held-out val problems the base model solved on at most 2 of 64 samples under the iidcanonical@64 hardness scan. Val evals sample at temperature 1.0 (matching the cleaneval frontier eval). Reward signal: binary code-correctness (1.0 if the generated program passes the problem's tests, otherwise 0.0). This checkpoint is the main revision (git branch) of the repo, with the model at…

Open weights 4.4B parameters 262,144 tokens transformers