Open-weight model
FreeToken-Qwen3.5-35B-A3B-FP8-pruned241
by Tyrel Barstow theJuice85/FreeToken-Qwen3.5-35B-A3B-FP8-pruned241
FreeToken-Qwen3.5-35B-A3B-FP8-pruned241 is an open-weight model from Tyrel Barstow, released under Apache License 2.0. It has 34B parameters and a 262,144-token context. At 16-bit it needs about 81.6 GB of GPU memory, which fits on 1x MI300X from $1.85 an hour, at the lowest prices in the SAVRN Index.
Expert-pruned variant of FreeToken-Qwen3.5-35B-A3B-FP8: 46% of routed experts removed (241 of 512 kept per layer, selected by routing-count calibration), with no requantization — surviving FP8 tensors and block scales are byte-identical to the base.
Runs On
What it takes to serve FreeToken-Qwen3.5-35B-A3B-FP8-pruned241 (34B parameters): the memory its weights need at each precision, and the cheapest way to rent enough data-center GPUs to hold them.
| Precision | Weights | Memory needed | Cheapest setup | Per hour | Also fits |
|---|---|---|---|---|---|
| 16-bit | 68.0 GB | 81.6 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x MI325X $2.00 · 1x MI355X $2.59 |
| 8-bit | 34.0 GB | 40.8 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
| 4-bit | 17.0 GB | 20.4 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
Memory is the weights at that precision plus 20% for the runtime and a short context; a long context needs more. Prices are the lowest on-demand hourly rates in the SAVRN Index, read Oct 18, 2026.
Model Card
By Tyrel Barstow, published under apache-2.0, revision 6c3c29a3ac2c.
Expert-pruned variant of FreeToken-Qwen3.5-35B-A3B-FP8: 46% of routed experts removed (241 of 512 kept per layer, selected by routing-count calibration), with no requantization — surviving FP8 tensors and block scales are byte-identical to the base. Router rows compacted and renumbered; MTP block pruned consistently. Teacher-forced logprob-equivalence (32 samples/domain, mean per-token delta vs base server; negative = worse): HumanEval pass@1 (temp 0, thinking off): 14/24, identical to base, same 14 tasks. For context, deeper tiers degrade measurably: at 63% pruning (192 experts) OOD deltas grow ~4x while in-domain still looks clean (calibration-coverage failure); at 75% pruning every…
Read Tyrel Barstow's full model card
Expert-pruned variant of FreeToken-Qwen3.5-35B-A3B-FP8: 46% of routed experts removed (241 of 512 kept per layer, selected by routing-count calibration), with no requantization — surviving FP8 tensors and block scales are byte-identical to the base. Router rows compacted and renumbered; MTP block pruned consistently.
Evidence: behaviorally identical to base
Teacher-forced logprob-equivalence (32 samples/domain, mean per-token delta vs base server; negative = worse):
| Probe domain | mean delta | worst sample |
|---|---|---|
| In-domain (calibration mix) | -0.01 | — |
| SWE (real repo files) | -0.009 | -0.146 |
| Tools (OpenAI-style schemas) | +0.006 | -0.075 |
HumanEval pass@1 (temp 0, thinking off): 14/24, identical to base, same 14 tasks.
For context, deeper tiers degrade measurably: at 63% pruning (192 experts) OOD deltas grow ~4x while in-domain still looks clean (calibration-coverage failure); at 75% pruning every sample worsens. 241 is the safe cut.
Serving notes
Hybrid-attention model: under vLLM use --max-num-seqs 48 (or lower) — Mamba
state cache needs one block per sequence and default 256 fails engine init.
FP8 block-128 weights, dynamic activation scale, same loader/kernels as base.
Method / reproduction
Calibration = routing counts from replayed transcripts; prune = drop lowest-usage experts per layer, compact router, renumber expert keys. Eval harness (probes, diff tool, pass-at-1 script) in the study repo: github.com — see commit 64809c1 message for the tier table.
Configuration
- Architecture
- Qwen3_5MoeForConditionalGeneration
- Context length (tokens)
- 262,144
- Layers
- 40
- Hidden size
- 2,048
- Attention heads
- 16
- Key/value heads
- 2
- Head dimension
- 256
- Vocabulary size
- 248,320
- Experts
- 241
- Experts active per token
- 8
- Model type
- qwen3_5_moe
- Quantization
- fp8
Identity and Version
- Repository
- theJuice85/FreeToken-Qwen3.5-35B-A3B-FP8-pruned241
- Publisher
- Tyrel Barstow
- Task
- Not stated by the source
- Modality
- Other
- Library
- Not stated by the source
- Parameters
- 34B parameters
- Languages
- moe
- Revision
- 6c3c29a3ac2c32cc934de1c7f89da762a4e7df6a
- First published
- 2026-10-17
- Last updated
- 2026-10-17
Files and Weights
29 files, 35.6 GB in total. The weights are 14 files totalling 35.5 GB in safetensors.
Every file
| File | Type | Size | SHA-256 |
|---|---|---|---|
| model.safetensors-00001-of-00014.safetensors | Weights | 2.5 GB | 13241627549e |
| model.safetensors-00002-of-00014.safetensors | Weights | 2.5 GB | fa02743459ad |
| model.safetensors-00003-of-00014.safetensors | Weights | 2.5 GB | 2d80d3cfc393 |
| model.safetensors-00004-of-00014.safetensors | Weights | 2.5 GB | 3e7b19e49065 |
| model.safetensors-00005-of-00014.safetensors | Weights | 2.5 GB | 321015c90a5f |
| model.safetensors-00006-of-00014.safetensors | Weights | 2.5 GB | 678c4c5e0baa |
| model.safetensors-00007-of-00014.safetensors | Weights | 2.5 GB | d15f33c9cafe |
| model.safetensors-00008-of-00014.safetensors | Weights | 2.5 GB | 8c2282180753 |
| model.safetensors-00009-of-00014.safetensors | Weights | 3.6 GB | 075e09d0da66 |
| model.safetensors-00010-of-00014.safetensors | Weights | 2.5 GB | ecee34e7f083 |
| model.safetensors-00011-of-00014.safetensors | Weights | 2.5 GB | 685f6de3adde |
| model.safetensors-00012-of-00014.safetensors | Weights | 2.5 GB | f2d7753bfb91 |
| model.safetensors-00013-of-00014.safetensors | Weights | 2.9 GB | 072f076ca5de |
| model.safetensors-00014-of-00014.safetensors | Weights | 1.2 GB | 79c6cc7d631c |
| config.json | Configuration | 18.2 KB | — |
| configuration.json | Configuration | 51 B | — |
| generation_config.json | Configuration | 244 B | — |
| model.safetensors.index.json | Configuration | 7.4 MB | — |
| preprocessor_config.json | Configuration | 390 B | — |
| prune_manifest.json | Configuration | 7.2 KB | — |
| video_preprocessor_config.json | Configuration | 385 B | — |
| LICENSE | Documentation | 11.3 KB | — |
| README.md | Documentation | 1.7 KB | — |
| chat_template.jinja | Other | 7.8 KB | — |
| .gitattributes | Repository | 1.6 KB | — |
| merges.txt | Tokenizer | 3.4 MB | — |
| tokenizer.json | Tokenizer | 12.8 MB | 5f9e4d4901a9 |
| tokenizer_config.json | Tokenizer | 16.7 KB | — |
| vocab.json | Tokenizer | 6.7 MB | — |
License and Download
- License
- apache-2.0
- Access
- Open weights, no gate
- Download size
- 35.5 GB
Released by Tyrel Barstow through its official repository on Hugging Face. Read the license.
Memory Requirements
| Precision | Weights in memory |
|---|---|
| As published | 35.5 GB |
| 16-bit | 68.0 GB |
| 8-bit | 34.0 GB |
| 4-bit | 17.0 GB |
Weights only, from the published parameter count; the key-value cache and runtime add to this.
Questions About FreeToken-Qwen3.5-35B-A3B-FP8-pruned241
How much GPU memory does FreeToken-Qwen3.5-35B-A3B-FP8-pruned241 need?
About 81.6 GB at 16-bit and 20.4 GB at 4-bit: the weights (34B parameters) plus a working margin. A long context needs more.
What is the cheapest GPU to run FreeToken-Qwen3.5-35B-A3B-FP8-pruned241 on?
At 16-bit, 1x MI300X from $1.85 an hour; at 4-bit, 1x MI300X from $1.85 an hour, at the lowest on-demand prices the SAVRN Index lists.
Can I use FreeToken-Qwen3.5-35B-A3B-FP8-pruned241 commercially?
Yes. FreeToken-Qwen3.5-35B-A3B-FP8-pruned241 is released under Apache License 2.0. The Apache License 2.0 is a permissive open-source license. It permits commercial use, modification and redistribution. It requires keeping the license and copyright notices and any NOTICE file, stating significant changes, and it includes an express patent grant from contributors.
What is FreeToken-Qwen3.5-35B-A3B-FP8-pruned241's context length?
262,144 tokens, from the maximum position embeddings in its published configuration.