Skip to page content
SAVRN
Search Plan a project

Open-weight model · Image and text to text

Qwen3.6-35B-A3B-abliterated-MNN

by Isaac Maple darkmaniac7/Qwen3.6-35B-A3B-abliterated-MNN

Qwen3.6-35B-A3B-abliterated-MNN is an open-weight model for image and text to text from Isaac Maple, released under Apache License 2.0. Its published files total 22.8 GB. It draws 202 downloads a month.

Published by TokForge for on-device inference. Runs on-device in the TokForge app. MNN-format 4-bit quantization of the Heretic-abliterated Qwen3.6-35B-A3B multimodal MoE, packaged for the TokForge Android MNN-fork runtime.

Parameters—
Context—
Weights1.0 GB
Licenseapache-2.0
AccessOpen weights
Monthly Downloads202

Model Card

By Isaac Maple, published under apache-2.0, revision 4cacae82f153.

Published by TokForge for on-device inference. Runs on-device in the TokForge app. MNN-format 4-bit quantization of the Heretic-abliterated Qwen3.6-35B-A3B multimodal MoE, packaged for the TokForge Android MNN-fork runtime. - config.json: MNN LLM geometry (modeltype qwen35moe, jinja chat template, MRoPE, vision offsets) - llmconfig.json: backend defaults (cpu, threadnum=4, precision=low, memory=low) - llm.mnn / llm.mnn.weight: quantized MNN graph + external weight blob (Q4, block 64, HQQ) - embeddingsbf16.bin: BF16 embedding table (separated from quantized weights via --seperateembed) - tokenizer.mtok: MNN binary tokenizer - visual.mnn / visual.mnn.weight: vision transformer (DeepStack VLM)…

Read Isaac Maple's full model card

Published by TokForge for on-device inference.

TokForge

  • Website: https://tokforge.ai
  • Discord: https://discord.gg/Acv3CBtfVm
  • Google Play: https://play.google.com/store/apps/details?id=dev.tokforge
  • iOS TestFlight: https://testflight.apple.com/join/jnufjzRr

Runs on-device in the TokForge app.

Qwen3.6-35B-A3B-abliterated-MNN

MNN-format 4-bit quantization of the Heretic-abliterated Qwen3.6-35B-A3B multimodal MoE, packaged for the TokForge Android MNN-fork runtime.

What this is

  • Source (upstream abliteration): Youssofal/Qwen3.6-35B-A3B-Abliterated-Heretic-BF16
  • Upstream of that: Qwen/Qwen3.6-35B-A3B
  • Abliteration methodology: Heretic input-side split-MoE transfer (MPOA/SOMA-style). Upstream reports 1/25 refusal rate on the official 25-prompt harmful-behaviors check (vs. 22/25 for the base model) with KL-divergence 0.0107 vs. base.
  • Architecture: qwen3_5_moe (40 layers, 256 experts, 8 active per token, hybrid linear + full attention, DeepStack vision).

Bundle contents (parity with taobao-mnn/Qwen3.6-35B-A3B-MNN)

  • config.json: MNN LLM geometry (model_type qwen3_5_moe, jinja chat template, MRoPE, vision offsets)
  • llm_config.json: backend defaults (cpu, thread_num=4, precision=low, memory=low)
  • llm.mnn / llm.mnn.weight: quantized MNN graph + external weight blob (Q4, block 64, HQQ)
  • embeddings_bf16.bin: BF16 embedding table (separated from quantized weights via --seperate_embed)
  • tokenizer.mtok: MNN binary tokenizer
  • visual.mnn / visual.mnn.weight: vision transformer (DeepStack VLM)

Quantization scheme

Identical to the base taobao-mnn/Qwen3.6-35B-A3B-MNN bundle:

Flag Value
--quant_bit 4
--quant_block 64
--lm_quant_bit 4
--lm_quant_block 64
--embed_bit 16
--hqq enabled
--seperate_embed enabled

This parity means the abliterated variant should behave identically to the base Qwen3.6 bundle in terms of load time, memory footprint, and decode throughput on TokForge-supported devices.

VLM capability (added post-release, issue #217)

This bundle now ships with visual.mnn + visual.mnn.weight, DeepStack vision transformer for image input. llm_config.json has is_visual: true with the full vision config block (image_mean, image_norm, image_size: 420, vision_start, vision_end, image_pad, num_grid_per_side: 48, has_deepstack: true).

The vision tower in Qwen3.6-35B-A3B is architecturally identical across the base Qwen release and the Heretic-abliterated variant (abliteration targets only the LLM decoder MLP layers, verified by structural comparison of all 333 *.visual.* weight keys in both safetensors). The visual assets here are therefore drop-in compatible with taobao-mnn/Qwen3.6-35B-A3B-MNN and byte-identical to those in the base bundle.

Attention-stack fields (attention_type: mix, sliding_window: 4, layer_nums: 40) match the base taobao-mnn/Qwen3.6-35B-A3B-MNN bundle exactly.

Original conversion note

The first upload of this repo (pre-#217) shipped text-only: the ONNX visual export ran during the initial llmexport.py run, but the final MNNConvert step did not emit visual.mnn / visual.mnn.weight into the output dir. The exporter has since been patched with a --visual_only flag in the TokForge MNN fork to allow re-emitting vision assets without a full re-conversion. See the upstream issue thread for details.

Target runtime

TokForge Android, MNN fork with .mtok tokenizer, DeepStack VLM support, and the libMNN cherry-pick that lets the base Qwen3.6-35B-A3B load on 24 GB devices.

Usage with upstream MNN llm_demo

git clone https://github.com/alibaba/MNN.git
cd MNN && mkdir build && cd build
cmake .. -DMNN_LOW_MEMORY=true -DMNN_CPU_WEIGHT_DEQUANT_GEMM=true \
         -DMNN_BUILD_LLM=true -DMNN_SUPPORT_TRANSFORMER_FUSE=true
make -j

./llm_demo /path/to/Qwen3.6-35B-A3B-abliterated-MNN/config.json prompt.txt

License & safety

  • Apache-2.0 (inherited from Qwen and the Youssofal abliteration).
  • This is a safety-reduced / abliterated variant. It refuses far less than the base model on the MPOA/SOMA refusal benchmark. Deploy with appropriate user-facing controls and local policy.
  • Export pipeline: alibaba/MNN llmexport (tq-merged branch, TokForge fork).

Credits

  • Qwen3.6-35B-A3B-Abliterated-Heretic-BF16 by Youssofal (Hugging Face). Used under the Apache-2.0 license.
  • Qwen3.6-35B-A3B by the Qwen team (Alibaba Cloud) (Hugging Face). Used under the Apache-2.0 license.
  • Converted with llmexport from MNN by Alibaba (Apache-2.0).

Identity and Version

Repository
darkmaniac7/Qwen3.6-35B-A3B-abliterated-MNN
Publisher
Isaac Maple
Task
Image and text to text
Modality
Image and text
Library
Not stated by the source
Parameters
Not stated by the source
Languages
en
Revision
4cacae82f153b60fd9c27a1fae9366d99357aa24
First published
2026-04-17
Last updated
2026-10-15

Files and Weights

13 files, 22.8 GB in total. The weights are 1 file totalling 1.0 GB in bin.

Weights1 file · 1.0 GB
Configuration4 files · 119.0 MB
Tokenizer2 files · 10.3 MB
Documentation1 file · 5.3 KB
Other4 files · 21.7 GB
Repository1 file · 1.8 KB
Every file
FileTypeSizeSHA-256
embeddings_bf16.binWeights1.0 GB fa0611f12cdb
config.jsonConfiguration382 B —
export_args.jsonConfiguration1.1 KB —
llm.mnn.jsonConfiguration118.9 MB b80a8605a666
llm_config.jsonConfiguration8.6 KB —
README.mdDocumentation5.3 KB —
llm.mnnOther54.3 MB 56aea5e006df
llm.mnn.weightOther21.3 GB a64a53b083d8
visual.mnnOther547.8 KB 0a1cf274946e
visual.mnn.weightOther253.5 MB 99e8601398be
.gitattributesRepository1.8 KB —
tokenizer.mtokTokenizer7.3 MB 9762fc50f2ea
tokenizer.txtTokenizer3.0 MB —

License and Download

License
apache-2.0
Access
Open weights, no gate
Download size
1.0 GB
Download from Isaac Maple

Released by Isaac Maple through its official repository on Hugging Face. Read the license.

Built From

  • Derived from Youssofal/Qwen3.6-35B-A3B-Abliterated-Heretic-BF16
  • Quantized from Youssofal/Qwen3.6-35B-A3B-Abliterated-Heretic-BF16

Memory Requirements

PrecisionWeights in memory
As published1.0 GB

Weights only, from the published parameter count; the key-value cache and runtime add to this.

Questions About Qwen3.6-35B-A3B-abliterated-MNN

Can I use Qwen3.6-35B-A3B-abliterated-MNN commercially?

Yes. Qwen3.6-35B-A3B-abliterated-MNN is released under Apache License 2.0. The Apache License 2.0 is a permissive open-source license. It permits commercial use, modification and redistribution. It requires keeping the license and copyright notices and any NOTICE file, stating significant changes, and it includes an express patent grant from contributors.

Similar Models

Non-uniform GGUF quantizations of a 512-expert MoE, produced with GSQ and RCO, with a vision projector for multimodal use. This repository provides GGUF quantizations of Qwen3.8-Flash-Next at four sizes, together with the model's vision projector (mmproj) for multimodal use. In contrast to uniform quantization, which applies a single quantization type to all weight tensors, each model here assigns a separate quantization type to every tensor. The assignment is obtained by a gradient-based search that allocates precision according to per-tensor sensitivity, subject to a total size budget. The resulting files are standard GGUF and run unmodified in llama.cpp, Ollama, and LM Studio. A…

Open weights apache-2.0 gguf
View model

Model · Image and text to text

Huihui-Qwen3.8-27B-abliterated-GGUF

Huihui.ai

This is an uncensored version of Qwen/Qwen3.8-27B created with abliteration (see remove-refusals-with-transformers to know more about it). This is a crude, proof-of-concept implementation to remove refusals from an LLM model without using TransformerLens. The newly added Huihui-Qwen3.8-27B-abliterated-Underdog-Saluki series come from ConwayResearch/Underdog-Saluki-27B-1.0 and ukisai/Swift-1.5-Qwen3.8-27B-GSQ-RCO-GGUF. Only layers 22 to 52 (0-based indexing) have been ablated, while the other layers remain unablated. It may come with a small disclaimer warning. Due to ConwayResearch/Underdog-Saluki-27B-1.0 not providing an imatrix file, we will attempt to perform ablation by mixing the GGUF…

Open weights apache-2.0 transformers
View model

Qwen3.8-27B uncensored by HauhauCS 0/465 Refusals. This is the Aggressive variant: direct answers, no refusal behavior, and minimal preamble on hard prompts. Every text GGUF preserves Qwen3.8's native NextN head, and this release adds HauhauCS FastMTP: a specific acceleration sidecar qualified across the complete quant lineup at maximum native context. Vision is included through the separate BF16 projector. No changes to datasets or intended capabilities. This release preserves Qwen3.8-27B's text, reasoning, agentic, image, and video capabilities while applying the HauhauCS Aggressive uncensoring profile. Pick Aggressive when you specifically want the model to get to the answer without…

Open weights apache-2.0
View model

Model · Image and text to text

Qwen3.8-27B-iMatrix-NVFP4-MTP-GGUF

Michał Piszczek

I built this quant because the ready-made FP4 file answered the wrong question. It was fast, but on my short WikiText-2 control it scored 6.4949 PPL. Plain Q40 scored 6.3798. The first higher-quality hybrid went too far the other way: good perplexity, 34.19 tok/s, and no comfortable room for 256K plus vision. This is the build that survived both gates. It is a 17.1 GB, 5.01 BPW mixed-precision GGUF of Qwen/Qwen3.8-27B. It keeps large, tolerant matrices in native NVFP4 and spends more bits on selected attention, Gated DeltaNet, and late FFN tensors. The trained MTP layer remains embedded in the same GGUF. This is not a fine-tune. I built the private calibration workload from 5,472 messages…

Open weights apache-2.0
View model

Model · Image and text to text

Qwen3.8-27B-GGUF

GGML Org

Run it in the Llama app: https://llama.app - https://huggingface.co/Qwen/Qwen3.8-27B - https://huggingface.co/incoai/Qwen3.8-27B-DFlash2 - add info

Open weights apache-2.0
View model

lm mmap --lazy-mode on -ngl 99 -c 32768 \ mmproj-Qwen3.8-Flash-Next-BF16.gguf --local-dir. llama-mtmd-cli -m IQ3S/Qwen3.8-Flash-Next-GSQ-RCO-abliterated-IQ3S-00001-of-00002.gguf \ --mmproj mmproj-Qwen3.8-Flash-Next-BF16.gguf -lm mmap --lazy-mode on \ -md mtp-Qwen3.8-Flash-Next-Q80.gguf --draft-max 4 -lm mmap --lazy-mode on -ngl 99 -lm mmap --lazy-mode on keeps the n-gram table memory-mapped on disk: it is 28.8 GB and one row is read per token. Shard 1 wants to be resident (VRAM or RAM); shard 2 is fine on an SSD. The vision projector is a standard clip GGUF shared with the upstream tiers, 0.91 GB. Native context is 262,144 tokens and the KV cache grows with it, so start at -c 32768 and work…

Open weights apache-2.0 gguf
View model