Text encoder weights from Google's T5 model
Search public pages, research tools, and SAVRN solutions.
Open-weight model · Text generation
by CapyCTL capyctl/FrogNano-4B-2609-MLX-4bit
FrogNano-4B-2609-MLX-4bit is an open-weight model for text generation from CapyCTL, released under MIT License. It has 4.8B parameters and a 262,144-token context. At 16-bit it needs about 11.6 GB of GPU memory, which fits on 1x MI300X from $1.85 an hour, at the lowest prices in the SAVRN Index.
A 4-bit MLX affine quantization of microsoft/FrogNano-4B-2609 (revision b90468c1), made so TensorFold can serve it on NVIDIA GPUs. TensorFold's CUDA engine reads quantized weights only; the original checkpoint is BF16.
What it takes to serve FrogNano-4B-2609-MLX-4bit (4.8B parameters): the memory its weights need at each precision, and the cheapest way to rent enough data-center GPUs to hold them.
| Precision | Weights | Memory needed | Cheapest setup | Per hour | Also fits |
|---|---|---|---|---|---|
| 16-bit | 9.7 GB | 11.6 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
| 8-bit | 4.8 GB | 5.8 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
| 4-bit | 2.4 GB | 2.9 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
Memory is the weights at that precision plus 20% for the runtime and a short context; a long context needs more. Prices are the lowest on-demand hourly rates in the SAVRN Index, read Oct 7, 2026.
FrogNano-4B-2609-MLX-4bit on every accelerator the SAVRN Index prices, at every precision
By CapyCTL, published under mit, revision 7175ee2b94ea.
A 4-bit MLX affine quantization of microsoft/FrogNano-4B-2609 (revision b90468c1), made so TensorFold can serve it on NVIDIA GPUs. TensorFold's CUDA engine reads quantized weights only; the original checkpoint is BF16. All credit for the model goes to its authors. Read the original model card for intended use, limitations and safety guidance; they apply unchanged. This repository changes only the storage format. The conversion script is in the CapyCTL recipe linked below. --no-drafts is required: there is no drafter for this model, and TensorFold's Qwen dense engine on CUDA needs either a drafter or --no-drafts. --parallel 8 decodes up to eight requests together; without it TensorFold on…
A 4-bit MLX affine quantization of microsoft/FrogNano-4B-2609
(revision b90468c1), made so TensorFold can serve it on NVIDIA GPUs.
TensorFold's CUDA engine reads quantized weights only; the original checkpoint is BF16.
All credit for the model goes to its authors. Read the original model card for intended use, limitations and safety guidance; they apply unchanged. This repository changes only the storage format.
| Weights | MLX affine 4-bit, group size 64 (mlx-lm 0.32.0), about 2.6 GB |
| Output layer | lm_head written as an exact copy of embed_tokens before quantizing, tie_word_embeddings: false. TensorFold 0.6.3 refuses tied heads (TensorFold#306). |
| Vision tower | Removed (297 tensors and vision_config). FrogNano is text only in its intended use; the original card does not claim image or video support. |
| MTP layer | Removed (15 tensors). |
| Tokenizer, chat template | The original files, unchanged. |
generation_config.json |
Added, with eos_token_id: [248046, 248044] (<\|im_end\|>, <\|endoftext\|>), so replies stop at the end of a turn. |
The conversion script is in the CapyCTL recipe linked below.
TensorFold 0.6.3 or later, CUDA:
tensorfold serve capyctl/FrogNano-4B-2609-MLX-4bit --no-drafts --parallel 8
--no-drafts is required: there is no drafter for this model, and TensorFold's Qwen dense engine on CUDA needs either a drafter or --no-drafts.
--parallel 8 decodes up to eight requests together; without it TensorFold on CUDA serves one request at a time.
Through CapyCTL: see the recipe
tensorfold/frognano-4b-mlx-4bit-rtx4090.
CapyCTL starts TensorFold with --parallel 8 by default (or the deployment's max_concurrent_requests).
One RTX 4090 Laptop GPU (16 GB), through CapyCTL, greedy, thinking on, max_tokens 512, median of 3 prompts:
| This checkpoint (TensorFold 0.6.3) | Original BF16 (vLLM 0.30.0) | |
|---|---|---|
| Decode, one stream | 52.4 tok/s | 59.9 tok/s |
| Time to first token | 0.045 s | 0.054 s |
| GPU memory, whole card | 3.4 GiB with short prompts, 7.3 GiB at most in the benchmark (11 GiB cap) | 13.1 GiB |
With several streams (TensorFold --parallel 8, 512 tokens a request, greedy, thinking on, median of 5 rounds):
| Streams | Together | Each |
|---|---|---|
| 1 | 50.4 tok/s | 50.6 tok/s |
| 2 | 93.4 tok/s | 46.9 tok/s |
| 4 | 176.5 tok/s | 44.3 tok/s |
| 8 | 319.1 tok/s | 40.3 tok/s |
One stream decodes 49 to 50 tok/s from 0.5k to 32k tokens of context. Each stream's memory grows with its request;
eight 28k-token prompts sent at once all completed with the GPU at 9.9 GiB at most, under an 11 GiB cap
(TENSORFOLD_CUDA_MEMORY_LIMIT_GB=11). Give TensorFold a cap that covers your context and streams.
A small greedy comparison against the original BF16 weights on vLLM 0.30.0, thinking off, max_tokens 1024.
Generated code was run against unit tests in a sandbox. This is a spot check, not SWE-bench or a perplexity measurement.
| Prompts | This checkpoint | Original BF16 |
|---|---|---|
| Python coding, 30 | 27 | 26 |
| Tool calls, 30 (right tool and arguments) | 26 | 28 |
| Reasoning and math, 20 | 20 | 20 |
| Total, 80 | 73 (91%) | 74 (93%) |
Every tool call from both was valid JSON. With thinking on (10 prompts, max_tokens 4096) this checkpoint passed 8 and BF16 9;
the miss was one reply that kept repeating its reasoning until the token limit. Set a max_tokens limit when thinking is on.
The original repository is tagged MIT, and its card's license row names Apache 2.0. This derived checkpoint keeps the original's terms and attribution; check the original repository for them.
11 files, 2.7 GB in total. The weights are 1 file totalling 2.7 GB in safetensors.
| File | Type | Size | SHA-256 |
|---|---|---|---|
| model.safetensors | Weights | 2.7 GB | 30d9933ec4ef |
| config.json | Configuration | 2.9 KB | — |
| generation_config.json | Configuration | 39 B | — |
| model.safetensors.index.json | Configuration | 81.2 KB | — |
| README.md | Documentation | 4.4 KB | — |
| chat_template.jinja | Other | 7.8 KB | — |
| .gitattributes | Repository | 1.6 KB | — |
| merges.txt | Tokenizer | 3.4 MB | — |
| tokenizer.json | Tokenizer | 12.8 MB | 5f9e4d4901a9 |
| tokenizer_config.json | Tokenizer | 16.7 KB | — |
| vocab.json | Tokenizer | 6.7 MB | — |
Released by CapyCTL through its official repository on Hugging Face. Read the license.
| Precision | Weights in memory |
|---|---|
| As published | 2.7 GB |
| 16-bit | 9.7 GB |
| 8-bit | 4.8 GB |
| 4-bit | 2.4 GB |
Weights only, from the published parameter count; the key-value cache and runtime add to this.
About 11.6 GB at 16-bit and 2.9 GB at 4-bit: the weights (4.8B parameters) plus a working margin. A long context needs more.
At 16-bit, 1x MI300X from $1.85 an hour; at 4-bit, 1x MI300X from $1.85 an hour, at the lowest on-demand prices the SAVRN Index lists.
Yes. FrogNano-4B-2609-MLX-4bit is released under MIT License. The MIT License is a short permissive license. It permits commercial use, modification and redistribution, provided the copyright notice and permission notice are included.
262,144 tokens, from the maximum position embeddings in its published configuration.
Text encoder weights from Google's T5 model
The NVIDIA Qwen2.5-VL-7B-Instruct-FP4 model is the quantized version of Alibaba's Qwen2.5-VL-7B-Instruct model, which is an auto-regressive language model that uses an optimized transformer architecture. For more information, please check here. The NVIDIA Qwen2.5-VL-7B-Instruct-FP4 model is quantized with TensorRT Model Optimizer. This model is ready for commercial/non-commercial use. This model is not owned or developed by NVIDIA. It was developed and built to a third party’s requirements for this application and use case. See the Non-NVIDIA (Qwen2.5-VL-7B-Instruct) Model Card. Use of this model is governed by nvidia-open-model-license Global, except in European Union Developers looking to…
Fleet is a dynamically extensible Semantic Inference Language Model (SiLM) built on Qwen/Qwen3-4B-Instruct-2507. Fleet is not an external prompt router. The routing system lives inside the causal language model itself through the custom FleetForCausalLM architecture. Fleet replaces selected Qwen projections with native FleetLinear modules. The model also contains a dynamic semantic prototype bank: Routing therefore happens inside model inference, rather than through an external classifier that chooses a different model before generation. This checkpoint currently contains six specialists: - mathreasoning - medicalreasoning - legalops - intent - functioncalling - structured The base Qwen…
We introduce the updated version of the Qwen3-4B-FP8 non-thinking mode, named Qwen3-4B-Instruct-2507-FP8, featuring the following key enhancements: - Significant improvements in general capabilities, including instruction following, logical reasoning, text comprehension, mathematics, science, coding and tool usage. - Substantial gains in long-tail knowledge coverage across multiple languages. - Markedly better alignment with user preferences in subjective and open-ended tasks, enabling more helpful responses and higher-quality text generation. - Enhanced capabilities in 256K long-context understanding. This repo contains the FP8 version of Qwen3-4B-Instruct-2507, which has the following…
Model · Text generation
An OpenRLHF GRPO reinforcement-learning checkpoint for Qwen3-4B. - Saved at global step 16 of RL run seededrlbaseramp25stoppengen4kep2ncp10q4v3groot16. - This is the best checkpoint by pass@8 so far in this run (evaldefaultpass8 = 0.1244). Trained and validated on the cobalt-train ≤2/64 frontier (canonical cleaneval prompts): 1833 train / 112 held-out val problems the base model solved on at most 2 of 64 samples under the iidcanonical@64 hardness scan. Val evals sample at temperature 1.0 (matching the cleaneval frontier eval). Reward signal: binary code-correctness (1.0 if the generated program passes the problem's tests, otherwise 0.0). Eval metrics at this checkpoint (held-out val, 8…
Model · Text generation
An OpenRLHF GRPO reinforcement-learning checkpoint for Qwen3-4B. - Saved at global step 60 of RL run seededrlbaseramp25stoppengen4kep2ncp5q4v3groot16. - This is the best checkpoint by pass@8 so far in this run. Trained and validated on the cobalt-train ≤2/64 frontier (canonical cleaneval prompts): 1833 train / 112 held-out val problems the base model solved on at most 2 of 64 samples under the iidcanonical@64 hardness scan. Val evals sample at temperature 1.0 (matching the cleaneval frontier eval). Reward signal: binary code-correctness (1.0 if the generated program passes the problem's tests, otherwise 0.0). This checkpoint is the main revision (git branch) of the repo, with the model at…