SAVRN
Search Contact SAVRN

Open-weight model · Text generation

POCKET-35B-GGUF

by FINAL_Bench FINAL-Bench/POCKET-35B-GGUF

Pick your build → -0f6e56) Wikipedia-Korean perplexity, lower is better. Q4KM = 5.79 baseline. English builds are tuned on English; see each repo. We measure Bonsai on the same machine with the same stock llama.cpp, and we tell you where we lose.

Parameters
Context
Weights59.1 GB
Licenseapache-2.0
AccessOpen weights
Monthly Downloads824.4k

Model Card

By FINAL_Bench, published under apache-2.0, revision 652a7ba1c3fb.

Pick your build → -0f6e56) Wikipedia-Korean perplexity, lower is better. Q4KM = 5.79 baseline. English builds are tuned on English; see each repo. We measure Bonsai on the same machine with the same stock llama.cpp, and we tell you where we lose. [measured] Generation speed — POCKET wins on both CPU and GPU: [measured on a MacBook M3 Pro, 18 GB] — and on a laptop, POCKET wins every axis, including prompt processing: On a laptop GPU the arithmetic headroom that let Bonsai win prefill on an H100 is gone, so MoE sparsity wins across the board. POCKET-35B-Q2K runs on the M3 Pro's CPU at 19.5 tok/s — on an 18 GB Mac, run Q2K on CPU (-ngl 0); its 13 GB exceeds the recommended Metal budget.…

Read FINAL_Bench's full model card

Collections

POCKET Models — this family (on-device, no GPU) Darwin Family · Aether Foundation · VKAE Accelerated

POCKET-35B-GGUF

A 35B model that runs on your PC with no GPU — and on your phone. Just stock llama.cpp. No fork, no CUDA, no cloud.

Try it live, no install → — both answering on a CPU-only box (no GPU). POCKET-26B is Gemma4-based.

Pick your build →

The POCKET lineup — pick by your device

Repo File Size Runs on Best for Korean PPL*
POCKET-35B-GGUF Q4_K_M 21 GB PC / server (32 GB RAM) top quality 5.79
POCKET-35B-GGUF Q2_K 13 GB mini-PC, no GPU daily driver 6.49
POCKET-35B-GGUF IQ1_M 8.2 GB 16 GB RAM box smallest full model 9.69
POCKET-KR-GGUF IQ2_M 5.1 GB Android 8 GB+ Korean phone 7.95
POCKET-KR-MLX 2-bit 5.1 GB iPhone / iPad / Mac Korean, Apple-native 7.95
POCKET-EN-GGUF iPhone-mix 5.3 GB iPhone (PocketPal) English phone
POCKET-EN-GGUF PC-mix 6.8 GB PC / Android English, best quality

*Wikipedia-Korean perplexity, lower is better. Q4_K_M = 5.79 baseline. English builds are tuned on English; see each repo.

Why MLX for Korean but GGUF for English on iPhone? Apple-native MLX only does uniform quantization. Korean survives it (96 experts hold up); English needs our proprietary quantization, which only GGUF supports — so the English iPhone build ships as a GGUF you run with PocketPal. Honest, not lazy.

POCKET-26B — a Gemma4-26B-A4B-based sibling that loads in any app today (Ollama · LM Studio · PocketPal · MLX), no bleeding-edge runtime needed: GGUF (Q2_K 11 GB · Q4_K_M 17 GB · GPQA-Diamond 67%). Universal compatibility for 12 GB phones, PC, and browser.

Benchmarks — what is measured, what is not

We measure Bonsai on the same machine with the same stock llama.cpp, and we tell you where we lose.

[measured] Generation speed — POCKET wins on both CPU and GPU:

POCKET-35B IQ1_M Bonsai-27B Q1_0
CPU generate (Xeon, 16t) 27.0 tok/s 10.1 2.69×
GPU generate (H100) 197 tok/s 89 2.22×
GPU prompt (H100) 753 1816 0.41×
Quality (HellaSwag, 400q) 61.0% 60.0% tie (CI overlaps)

[measured on a MacBook M3 Pro, 18 GB] — and on a laptop, POCKET wins every axis, including prompt processing:

POCKET-35B IQ1_M Bonsai-27B Q1_0
Metal generate (tg64) 25.4 tok/s 12.8 1.99×
CPU generate (8 threads) 13.8 tok/s 4.4 3.13×
Metal prompt (pp128) 240.7 tok/s 73.4 3.28×
CPU prompt (pp128) 45.5 tok/s 9.6 4.75×

On a laptop GPU the arithmetic headroom that let Bonsai win prefill on an H100 is gone, so MoE sparsity wins across the board. POCKET-35B-Q2_K runs on the M3 Pro's CPU at 19.5 tok/s — on an 18 GB Mac, run Q2_K on CPU (-ngl 0); its 13 GB exceeds the recommended Metal budget.

[measured — GPQA Diamond, 198q, greedy] reasoning quality vs quantization:

Model GPQA-Diamond (greedy)
Qwen3.6-35B-A3B 73.2%
POCKET-35B Q4_K_M 68.7%
POCKET-35B Q2_K 60.1%

[pending — community reports welcome] on-device iPhone and Strix Halo throughput. We publish only what we ran ourselves; help us fill the rest.

The same-size rival Ternary-Bonsai-27B-Q2_0 (7.2 GB) fails to load in upstream llama.cpp — it needs the PrismML fork. POCKET runs on the tools you already have.

Files in this repo

File Size bpw Runs on Korean PPL
POCKET-35B-Q4_K_M.gguf 21 GB 4.5 PC 32 GB RAM 5.79 (top)
POCKET-35B-Q3_K_M.gguf 16 GB 3.4 PC 24 GB 6.06
POCKET-35B-Q2_K.gguf 13 GB 2.6 mini-PC 16–24 GB 6.49 (best value)
POCKET-35B-IQ1_M.gguf 8.2 GB 1.9 16 GB RAM 9.69 (smallest)

Quickstart — no fork needed

# any recent llama.cpp — brew / winget / apt, or LM Studio / Ollama
llama-cli -m POCKET-35B-Q2_K.gguf -p "안녕하세요" -ngl 0 -t 8
# reproduce our CPU numbers:
llama-bench -m POCKET-35B-IQ1_M.gguf -p 128 -n 64 -ngl 0 -t 16

Use physical-core count for -t (max ~32). Do not pass all threads — it can slow down sharply.

Lineage — where POCKET comes from

POCKET is quantized from Darwin-36B-Opus, VIDRAFT's flagship — a model bred and evolved over several generations on the Darwin platform (crossbreeding, healing, expert surgery). Darwin-36B-Opus itself traces back to a Qwen3.5-family MoE architecture.

Component Origin
Starting checkpoint Darwin-36B-Opus — VIDRAFT, multi-generation Darwin evolution
Base architecture Qwen3.5-family MoE (256 experts, top-8), unchanged
Quantization (Q4_K_MIQ1_M) stock llama.cpp — no custom format
Runtime upstream llama.cpp / Apple MLX — unmodified
Proprietary language-specific tuning (KR/EN builds) ours (VIDRAFT)

The CPU/GPU speed comes from the sparse-MoE architecture plus ordinary quantization — reproducible with the same base and the same tools. What we add is the Darwin-evolved weights, the honest measurement, the Korean tuning, and the pruning that makes the 5 GB phone builds.

Limitations

  • The iPhone/Mac speed is not yet measured by us — community reports welcome.
  • Extreme quants (IQ1_M) hurt Korean ~2.8× more than English; use Q2_K or larger for quality.
  • English phone builds trade quality for size; the PC build (PC-mix) is much closer to full quality.

License

Apache-2.0.


POCKET is a VIDRAFT model family. 35B, in your pocket. No GPU.

Learn more


The POCKET Family — On-device AI by VIDRAFT

Big models, small hardware. No GPU, no cloud.

Models -POCKET-35B-GGUF— flagship, PC / server, no GPU -POCKET-26B-GGUF— compact 26B -POCKET-KR-GGUF— Korean, Android -POCKET-KR-MLX— Korean, iPhone / Mac -POCKET-EN-GGUF— English, phone / PC -POCKET-Image-Zimage — character-perfect text in any image

Demos & tools (Spaces) -POCKET-Image Studio— text-in-image, generate in-page -POCKET-35B-CPU— 35B answering on a CPU -POCKET-26B-CPU — 26B on a CPU

Full POCKET collection

Identity and Version

Repository
FINAL-Bench/POCKET-35B-GGUF
Publisher
FINAL_Bench
Task
Text generation
Modality
Text
Library
llama.cpp
Parameters
Not stated by the source
Languages
on-device, cpu, moe
Revision
652a7ba1c3fb0d926435111acb57a507dddaddd6
First published
2026-07-22
Last updated
2026-07-29

Files and Weights

10 files, 59.1 GB in total. The weights are 4 files totalling 59.1 GB in gguf.

Weights4 files · 59.1 GB
Documentation1 file · 9.8 KB
Other4 files · 185.2 KB
Repository1 file · 1.8 KB
Every file
FileTypeSizeSHA-256
POCKET-35B-IQ1_M.ggufWeights8.2 GB c56c77d15878
POCKET-35B-Q2_K.ggufWeights12.9 GB 2567ed710fb5
POCKET-35B-Q3_K_M.ggufWeights16.8 GB 9ab4184f0f5a
POCKET-35B-Q4_K_M.ggufWeights21.2 GB 6f479f637c8f
README.mdDocumentation9.8 KB
pocket_hero.svgOther3.0 KB
pocket_speed.svgOther2.3 KB
pocket_vs_bonsai.pngOther172.3 KB 9d9dddd21da7
pocket_vs_bonsai.svgOther7.6 KB
.gitattributesRepository1.8 KB

License and Download

License
apache-2.0
Access
Open weights, no gate
Download size
59.1 GB
Download from FINAL_Bench

Released by FINAL_Bench through its official repository on Hugging Face. Read the license.

Built From

  • Derived from FINAL-Bench/Darwin-36B-Opus
  • Quantized from FINAL-Bench/Darwin-36B-Opus

Memory Requirements

PrecisionWeights in memory
As published59.1 GB

Weights only, from the published parameter count; the key-value cache and runtime add to this.

Questions About POCKET-35B-GGUF

Can I use POCKET-35B-GGUF commercially?

Yes. POCKET-35B-GGUF is released under Apache License 2.0. The Apache License 2.0 is a permissive open-source license. It permits commercial use, modification and redistribution. It requires keeping the license and copyright notices and any NOTICE file, stating significant changes, and it includes an express patent grant from contributors.

Similar Models

Fine-tune Qwen3 (14B) for free using our Google Colab notebook! - Read our Blog about Qwen3 support: unsloth.ai/blog/qwen3 - View the rest of our notebooks in our docs here. Qwen3-Coder is available in multiple sizes. Today, we're excited to introduce Qwen3-Coder-30B-A3B-Instruct. This streamlined model maintains impressive performance and efficiency, featuring the following key enhancements: - Significant Performance among open models on Agentic Coding, Agentic Browser-Use, and other foundational coding tasks. - Long-context Capabilities with native support for 256K tokens, extendable up to 1M tokens using Yarn, optimized for repository-scale understanding. - Agentic Coding supporting for…

Open weights apache-2.0 transformers

Model · Text generation

opt-125m

AI at Meta

OPT was first introduced in Open Pre-trained Transformer Language Models and first released in metaseq's repository on May 3rd 2022 by Meta AI. Disclaimer: The team releasing OPT wrote an official model card, which is available in Appendix D of the paper. Content from this model card has been written by the Hugging Face team. To quote the first two paragraphs of the official paper OPT was predominantly pretrained with English text, but a small amount of non-English data is still present within the training corpus via CommonCrawl. The model was pretrained using a causal language modeling (CLM) objective. OPT belongs to the same family of decoder-only models like GPT-3. As such, it was…

Open weights other 2,048 tokens transformers

Model · Text generation

Ornith-1.5-9B-GGUF

Ornith

Chirp Chirp! We are introducing Ornith-1.5, a major step toward building foundation models through end-to-end self-improvement. Ornith-1.5 extends Ornith-1.0 (which was developed on top of Qwen3.5 and Gemma4 with additional continued pretraining, mid-training, and post-training) by expanding the self-improvement loop from scaffold and rollout optimization to jointly optimizing task generation, scaffold construction, and solution rollouts. Rather than relying on a fixed set of human-curated tasks and manually designed harnesses, Ornith-1.5 continuously generates new training tasks, discovers effective strategies for solving them, and improves the policy through reinforcement learning. For…

Open weights mit transformers

Model · Text generation

Ornith-1.5-35B-A3B-GGUF

Ornith

Chirp Chirp! We are introducing Ornith-1.5, a major step toward building foundation models through end-to-end self-improvement. Ornith-1.5 extends Ornith-1.0 (which was developed on top of Qwen3.5 and Gemma4 with additional continued pretraining, mid-training, and post-training) by expanding the self-improvement loop from scaffold and rollout optimization to jointly optimizing task generation, scaffold construction, and solution rollouts. Rather than relying on a fixed set of human-curated tasks and manually designed harnesses, Ornith-1.5 continuously generates new training tasks, discovers effective strategies for solving them, and improves the policy through reinforcement learning. For…

Open weights mit transformers

Model · Text generation

Ornith-1.0-9B-GGUF

Ornith

Aloha! Today, we are releasing Ornith-1.0, a self-improving family of open-source models for agentic coding. This model card documents Ornith-1.0-9B, the most lightweight member of the Ornith family, designed for efficient single-GPU deployment. Ornith-1.0-9B is a dense ~9B model (≈19 GB in bf16), so it serves comfortably on a single 80GB GPU. The recipes below stand up an OpenAI-compatible server; add --tensor-parallel-size / --tp if you want to shard across more GPUs. For a quick local test (or to script offline generation), load the model directly with Transformers. Make sure you have a recent release installed — see the Transformers installation guide; Ornith-1.0-9B requires…

Open weights mit transformers

Uncensored Qwen3.8-27B, published as GGUF quantizations with the multi token prediction (MTP) head retained and verified. Refusal behaviour has been substantially reduced, not eliminated. See Measured behaviour for the numbers. Capabilities, training data, and architecture are otherwise unchanged. - Refusal directions removed with Heretic, which co minimizes refusal count against KL divergence from the base model. No handwritten refusal removal code, no finetuning, no additional training data. - Abliteration runs at bf16 (no 4 bit quantization). the resulting LoRA is merged into the bf16 base, so the published weights are not a quantized round trip. - mtp. tensors are copied verbatim from…

Open weights apache-2.0 llama.cpp