This checkpoint was structurally pruned with the released Less-is-MoE mean-absolute-gradient method. It removes exactly 50% of routed-expert FFN neurons using 64 calibration samples from the gpqamain configuration of Idavidrein/gpqa revision 633f5ee89ab8ad4522a9f850766b73f62147ffdd. The released loader settings are preserved: train, Question plus shuffled choices, Explanation, selectionseed=1234, BF16, and no optimizer step. The samples are full length: no tokenizer maxlength, truncation, or padding. The longest input for this tokenizer is 1,511 tokens. The source MXFP4 checkpoint was explicitly dequantized to BF16 before scoring and pruning. The source-row selection hash is…
Open weights
apache-2.0
59.5B parameters
131,072 tokens
88.7 GiB text weights — the accessibility build, the roomiest fit on a 128 GB Mac. Also ships the bf16 vision tower (+0.85 GiB) and an optional MTP draft head (+5.4 GiB); full download 95.0 GiB. (v2, mixed geometry.) v2 — updated 2026-08-22. This repository now serves a rebuilt artifact at the same size and the same bits per weight, with a different codebook geometry that measures better on both perplexity corpora. v1's numbers are kept below rather than quietly overwritten, and v1's bytes remain downloadable by pinning the previous revision: A vector-quantized build of Qwen3.5-397B-A17B running? 88.7 GiB text weights — it runs on a single 128 GB Apple Silicon machine with ≈7 GiB more…
Open weights
apache-2.0
62.6B parameters
262,144 tokens
mlx
This checkpoint was structurally pruned with the released Less-is-MoE mean-absolute-gradient method. It removes exactly 50% of routed-expert FFN neurons using 64 calibration samples from the gpqamain configuration of Idavidrein/gpqa revision 633f5ee89ab8ad4522a9f850766b73f62147ffdd. The released loader settings are preserved: train, Question plus shuffled choices, Explanation, selectionseed=1234, BF16, and no optimizer step. The samples are full length: no tokenizer maxlength, truncation, or padding. The longest input for this tokenizer is 1,632 tokens. The source checkpoint was loaded and pruned in BF16. The source-row selection hash is…
Open weights
apache-2.0
64.1B parameters
262,144 tokens
This checkpoint was structurally pruned with the released Less-is-MoE mean-absolute-gradient method. It removes exactly 50% of routed-expert FFN neurons using 64 calibration samples from the gpqamain configuration of Idavidrein/gpqa revision 633f5ee89ab8ad4522a9f850766b73f62147ffdd. The released loader settings are preserved: train, Question plus shuffled choices, Explanation, selectionseed=1234, BF16, and no optimizer step. The samples are full length: no tokenizer maxlength, truncation, or padding. The longest input for this tokenizer is 1,632 tokens. The source checkpoint was loaded and pruned in BF16. The source-row selection hash is…
Open weights
apache-2.0
64.1B parameters
262,144 tokens
This checkpoint was structurally pruned with the released Less-is-MoE mean-absolute-gradient method. It removes exactly 50% of routed-expert FFN neurons using 64 calibration samples from the gpqamain configuration of Idavidrein/gpqa revision 633f5ee89ab8ad4522a9f850766b73f62147ffdd. The released loader settings are preserved: train, Question plus shuffled choices, Explanation, selectionseed=1234, BF16, and no optimizer step. The samples are full length: no tokenizer maxlength, truncation, or padding. The longest input for this tokenizer is 1,632 tokens. The source checkpoint was loaded and pruned in BF16. The source-row selection hash is…
Open weights
apache-2.0
64.1B parameters
262,144 tokens
125.3 GiB text weights — the quality build. Also ships the bf16 vision tower (+0.85 GiB) and an optional MTP draft head (+5.4 GiB); full download 131.6 GiB. A vector-quantized build of Qwen3.5-397B-A17B for machines with memory to spend: the strongest quantization we know how to make of this model at this size, on stock mlx-lm, no patches. 12.43% of the 64-element weight groups in the Qwen3.5-397B teacher sit in output rows whose weights are about 1e-29. The vq-skipzero format drops those rows' codes and scales on disk and in memory; they output exact zeros, and the live rows are byte-identical to the previous revision (vqlab sz-check, every module). Text weights go from 141.71 to 125.31…
Open weights
apache-2.0
64.5B parameters
262,144 tokens
mlx
