Qwen3.8-27B-heretic-ara-int8-ov by Blah Blah: Open Model

Model Card

By Blah Blah, published under apache-2.0, revision 6135eaf8d7c5.

converted to OpenVINO IR with weights compressed to INT8 by NNCF. A 27B qwen35 vision-language model: 64 text layers (48 linear-attention, 16 full-attention), hidden 5120, plus a 27-depth vision tower. nncf.compressweights with: - mode: INT8ASYM - bits: 8 Ships openvinomtpmodel., the multi-token-prediction head exported by Optimum Intel. Pair with a matching DFlash draft for block-parallel speculative decoding. Optimum Intel (image-text-to-text), Transformers 5.2.0, OpenVINO 2026.3.1. Note: the export requires TMPDIR on a real disk. /tmp is a 16 GB tmpfs and OpenVINO stages the compressed weights there; this export stages ~74 GB, so keep that much free on the TMPDIR filesystem. Apache-2.0.…

Read Blah Blah's full model card

Qwen3.8-27B-heretic-ara — OpenVINO IR (int8)

heretic-org/Qwen3.8-27B-heretic-ara converted to OpenVINO IR with weights compressed to INT8 by NNCF.

A 27B qwen3_5 vision-language model: 64 text layers (48 linear-attention, 16 full-attention), hidden 5120, plus a 27-depth vision tower.

Quantization parameters

nncf.compress_weights with:

mode: INT8_ASYM
bits: 8

Speculative decoding

Ships openvino_mtp_model.*, the multi-token-prediction head exported by Optimum Intel. Pair with a matching DFlash draft for block-parallel speculative decoding.

Files

File	Description
`openvino_language_model.*`	int8 text decoder
`openvino_text_embeddings_model.*`	token embedding stage
`openvino_vision_embeddings_model.`, `..._pos_model.`, `..._merger_model.*`	vision tower
`openvino_mtp_model.*`	MTP head
`openvino_tokenizer.`, `openvino_detokenizer.`	tokenizer / detokenizer

Usage (OpenVINO GenAI)

import openvino_genai as ov_genai
pipe = ov_genai.VLMPipeline(".", "GPU")
print(pipe.generate("Describe this image.", image=image_tensor))

Conversion

Optimum Intel (image-text-to-text), Transformers 5.2.0, OpenVINO 2026.3.1.

Note: the export requires TMPDIR on a real disk. /tmp is a 16 GB tmpfs and OpenVINO stages the compressed weights there; this export stages ~74 GB, so keep that much free on the TMPDIR filesystem.

Acknowledgements

Apache-2.0. Base Qwen models by Alibaba Cloud.

Configuration

Architecture: Qwen3_5ForConditionalGeneration
Context length (tokens): 262,144
Layers: 64
Hidden size: 5,120
Feed-forward size: 17,408
Attention heads: 24
Key/value heads: 4
Head dimension: 256
Vocabulary size: 248,320
Model type: qwen3_5

Identity and Version

Repository: blaj/Qwen3.8-27B-heretic-ara-int8-ov
Publisher: Blah Blah
Task: Image and text to text
Modality: Image and text
Library: openvino
Parameters: Not stated by the source
Languages: vlm
Revision: 6135eaf8d7c59f6a97d796c44346b423c797450e
First published: 2026-10-01
Last updated: 2026-10-01

Files and Weights

26 files, 27.8 GB in total. The weights are 8 files totalling 27.8 GB in bin.

Weights8 files · 27.8 GB

Configuration5 files · 7.6 KB

Tokenizer4 files · 20.0 MB

Documentation1 file · 1.8 KB

Other7 files · 10.7 MB

Repository1 file · 1.6 KB

Every file

File	Type	Size	SHA-256
openvino_detokenizer.bin	Weights	3.8 MB	48fc2c6a7a10
openvino_language_model.bin	Weights	25.6 GB	4c806db582a3
openvino_mtp_model.bin	Weights	425.0 MB	d33866ffb8ff
openvino_text_embeddings_model.bin	Weights	1.3 GB	b9580c471e13
openvino_tokenizer.bin	Weights	9.6 MB	bad6021844f4
openvino_vision_embeddings_merger_model.bin	Weights	457.5 MB	3792b6946c1f
openvino_vision_embeddings_model.bin	Weights	1.8 MB	5a8661b052ab
openvino_vision_embeddings_pos_model.bin	Weights	2.7 MB	82ccc97be035
config.json	Configuration	3.8 KB	—
generation_config.json	Configuration	213 B	—
openvino_config.json	Configuration	1.8 KB	—
preprocessor_config.json	Configuration	482 B	—
processor_config.json	Configuration	1.3 KB	—
README.md	Documentation	1.8 KB	—
chat_template.jinja	Other	9.0 KB	—
openvino_language_model.xml	Other	9.2 MB	—
openvino_mtp_model.xml	Other	211.7 KB	—
openvino_text_embeddings_model.xml	Other	5.9 KB	—
openvino_vision_embeddings_merger_model.xml	Other	1.3 MB	—
openvino_vision_embeddings_model.xml	Other	8.6 KB	—
openvino_vision_embeddings_pos_model.xml	Other	5.9 KB	—
.gitattributes	Repository	1.6 KB	—
openvino_detokenizer.xml	Tokenizer	16.8 KB	—
openvino_tokenizer.xml	Tokenizer	33.8 KB	—
tokenizer.json	Tokenizer	20.0 MB	6f32ce20dc35
tokenizer_config.json	Tokenizer	1.3 KB	—

License and Download

License: apache-2.0
Access: Open weights, no gate
Download size: 27.8 GB

Download from Blah Blah

Released by Blah Blah through its official repository on Hugging Face. Read the license.

Built From

Derived from heretic-org/Qwen3.8-27B-heretic-ara

Memory Requirements

Precision	Weights in memory
As published	27.8 GB

Weights only, from the published parameter count; the key-value cache and runtime add to this.

Questions About Qwen3.8-27B-heretic-ara-int8-ov

Can I use Qwen3.8-27B-heretic-ara-int8-ov commercially?

Yes. Qwen3.8-27B-heretic-ara-int8-ov is released under Apache License 2.0. The Apache License 2.0 is a permissive open-source license. It permits commercial use, modification and redistribution. It requires keeping the license and copyright notices and any NOTICE file, stating significant changes, and it includes an express patent grant from contributors.

What is Qwen3.8-27B-heretic-ara-int8-ov's context length?

262,144 tokens, from the maximum position embeddings in its published configuration.

Similar Models

Model · Image and text to text

Qwen3.8-Flash-Next-GSQ-RCO-GGUF

IST Austria Distributed Algorithms and Systems Lab

Non-uniform GGUF quantizations of a 512-expert MoE, produced with GSQ and RCO, with a vision projector for multimodal use. This repository provides GGUF quantizations of Qwen3.8-Flash-Next at four sizes, together with the model's vision projector (mmproj) for multimodal use. In contrast to uniform quantization, which applies a single quantization type to all weight tensors, each model here assigns a separate quantization type to every tensor. The assignment is obtained by a gradient-based search that allocates precision according to per-tensor sensitivity, subject to a total size budget. The resulting files are standard GGUF and run unmodified in llama.cpp, Ollama, and LM Studio. A…

Open weights apache-2.0 gguf

View model

Model · Image and text to text

Huihui-Qwen3.8-27B-abliterated-GGUF

Huihui.ai

This is an uncensored version of Qwen/Qwen3.8-27B created with abliteration (see remove-refusals-with-transformers to know more about it). This is a crude, proof-of-concept implementation to remove refusals from an LLM model without using TransformerLens. The newly added Huihui-Qwen3.8-27B-abliterated-Underdog-Saluki series come from ConwayResearch/Underdog-Saluki-27B-1.0 and ukisai/Swift-1.5-Qwen3.8-27B-GSQ-RCO-GGUF. Only layers 22 to 52 (0-based indexing) have been ablated, while the other layers remain unablated. It may come with a small disclaimer warning. Due to ConwayResearch/Underdog-Saluki-27B-1.0 not providing an imatrix file, we will attempt to perform ablation by mixing the GGUF…

Open weights apache-2.0 transformers

View model

Model · Image and text to text

Qwen3.8-27B-Uncensored-HauhauCS-Aggressive-MTP-GGUF

HauhauCS

Qwen3.8-27B uncensored by HauhauCS 0/465 Refusals. This is the Aggressive variant: direct answers, no refusal behavior, and minimal preamble on hard prompts. Every text GGUF preserves Qwen3.8's native NextN head, and this release adds HauhauCS FastMTP: a specific acceleration sidecar qualified across the complete quant lineup at maximum native context. Vision is included through the separate BF16 projector. No changes to datasets or intended capabilities. This release preserves Qwen3.8-27B's text, reasoning, agentic, image, and video capabilities while applying the HauhauCS Aggressive uncensoring profile. Pick Aggressive when you specifically want the model to get to the answer without…

Open weights apache-2.0

View model

Model · Image and text to text

Qwen3.8-27B-iMatrix-NVFP4-MTP-GGUF

Michał Piszczek

I built this quant because the ready-made FP4 file answered the wrong question. It was fast, but on my short WikiText-2 control it scored 6.4949 PPL. Plain Q40 scored 6.3798. The first higher-quality hybrid went too far the other way: good perplexity, 34.19 tok/s, and no comfortable room for 256K plus vision. This is the build that survived both gates. It is a 17.1 GB, 5.01 BPW mixed-precision GGUF of Qwen/Qwen3.8-27B. It keeps large, tolerant matrices in native NVFP4 and spends more bits on selected attention, Gated DeltaNet, and late FFN tensors. The trained MTP layer remains embedded in the same GGUF. This is not a fine-tune. I built the private calibration workload from 5,472 messages…

Open weights apache-2.0

View model

Model · Image and text to text

Qwen3.8-27B-GGUF

GGML Org

Run it in the Llama app: https://llama.app - https://huggingface.co/Qwen/Qwen3.8-27B - https://huggingface.co/incoai/Qwen3.8-27B-DFlash2 - add info

Open weights apache-2.0

View model

Model · Image and text to text

Qwen3.8-Flash-Next-GSQ-RCO-abliterated-GGUF

Scorpio117

lm mmap --lazy-mode on -ngl 99 -c 32768 \ mmproj-Qwen3.8-Flash-Next-BF16.gguf --local-dir. llama-mtmd-cli -m IQ3S/Qwen3.8-Flash-Next-GSQ-RCO-abliterated-IQ3S-00001-of-00002.gguf \ --mmproj mmproj-Qwen3.8-Flash-Next-BF16.gguf -lm mmap --lazy-mode on \ -md mtp-Qwen3.8-Flash-Next-Q80.gguf --draft-max 4 -lm mmap --lazy-mode on -ngl 99 -lm mmap --lazy-mode on keeps the n-gram table memory-mapped on disk: it is 28.8 GB and one row is read per token. Shard 1 wants to be resident (VRAM or RAM); shard 2 is fine on an SSD. The vision projector is a standard clip GGUF shared with the upstream tiers, 0.91 GB. Native context is 262,144 tokens and the KV cache grows with it, so start at -c 32768 and work…

Open weights apache-2.0 gguf

View model