SAVRN
Search Contact SAVRN

Open-weight model · Image and text to text

Huihui-Qwen3.8-27B-abliterated-EXL3-3.0bpw

by Lee grimlee/Huihui-Qwen3.8-27B-abliterated-EXL3-3.0bpw

Huihui-Qwen3.8-27B-abliterated-EXL3-3.0bpw is an open-weight model for image and text to text from Lee, released under Apache License 2.0. It has 6.7B parameters and a 262,144-token context. At 16-bit it needs about 16.2 GB of GPU memory, which fits on 1x MI300X from $1.85 an hour, at the lowest prices in the SAVRN Index. It draws 4 downloads a month.

This repository contains an EXL3 3.0 bpw quantization of The original model and abliteration work are attributed to huihui-ai; this repository contains the EXL3 quantization produced by grimlee.

Parameters6.7B
Context262,144
Weights13.5 GB
Licenseapache-2.0
AccessOpen weights
Monthly Downloads4

Runs On

What it takes to serve Huihui-Qwen3.8-27B-abliterated-EXL3-3.0bpw (6.7B parameters): the memory its weights need at each precision, and the cheapest way to rent enough data-center GPUs to hold them.

PrecisionWeightsMemory neededCheapest setupPer hourAlso fits
16-bit 13.5 GB 16.2 GB 1x MI300X (192 GB)
Vultr
$1.85 1x H100 $1.99 · 1x MI325X $2.00
8-bit 6.7 GB 8.1 GB 1x MI300X (192 GB)
Vultr
$1.85 1x H100 $1.99 · 1x MI325X $2.00
4-bit 3.4 GB 4.0 GB 1x MI300X (192 GB)
Vultr
$1.85 1x H100 $1.99 · 1x MI325X $2.00

Memory is the weights at that precision plus 20% for the runtime and a short context; a long context needs more. Prices are the lowest on-demand hourly rates in the SAVRN Index, read Oct 7, 2026.

Huihui-Qwen3.8-27B-abliterated-EXL3-3.0bpw on every accelerator the SAVRN Index prices, at every precision

Model Card

By Lee, published under apache-2.0, revision 64371e7543b2.

This repository contains an EXL3 3.0 bpw quantization of huihui-ai/Huihui-Qwen3.8-27B-abliterated.

The original model and abliteration work are attributed to huihui-ai; this repository contains the EXL3 quantization produced by grimlee.

The source model is an abliterated / uncensored derivative of Qwen3.8-27B. For details about the original model modification and its behavior, refer to the source model card.

The checkpoint retains the Qwen3.8 multimodal model structure, including the vision component. The optional SM89 runtime work documented below does not change the model format or weights.

Quantization details

The checkpoint uses the standard EXL3 format with a target body bitrate of 3.0 bits per weight.

The target bitrate applies to the quantized body; not every tensor in the checkpoint is stored at exactly 3 bits.

Read the full model card (955 words)

Configuration

Architecture
Qwen3_5ForConditionalGeneration
Context length (tokens)
262,144
Layers
64
Hidden size
5,120
Feed-forward size
17,408
Attention heads
24
Key/value heads
4
Head dimension
256
Vocabulary size
248,320
Model type
qwen3_5
Quantization
exl3

Identity and Version

Repository
grimlee/Huihui-Qwen3.8-27B-abliterated-EXL3-3.0bpw
Publisher
Lee
Task
Image and text to text
Modality
Image and text
Library
transformers
Parameters
6.7B parameters
Languages
Not stated by the source
Revision
64371e7543b20705b491b4a1ab4b2ba42722b3a8
First published
2026-09-19
Last updated
2026-09-20

Files and Weights

19 files, 13.5 GB in total. The weights are 4 files totalling 13.5 GB in safetensors.

Weights4 files · 13.5 GB
Configuration6 files · 930.3 KB
Tokenizer4 files · 22.9 MB
Documentation2 files · 19.9 KB
Other2 files · 9.2 KB
Repository1 file · 1.6 KB
Every file
FileTypeSizeSHA-256
model-00001-of-00004.safetensorsWeights4.3 GB 234b08e7a8d1
model-00002-of-00004.safetensorsWeights4.2 GB 8c1d6c456318
model-00003-of-00004.safetensorsWeights4.3 GB 5acac8bc06f6
model-00004-of-00004.safetensorsWeights756.6 MB de640403d5aa
config.jsonConfiguration4.6 KB —
generation_config.jsonConfiguration202 B —
model.safetensors.index.jsonConfiguration295.6 KB —
preprocessor_config.jsonConfiguration390 B —
quantization_config.jsonConfiguration629.1 KB —
video_preprocessor_config.jsonConfiguration385 B —
LICENSEDocumentation11.5 KB —
README.mdDocumentation8.4 KB —
chat_template.jinjaOther9.0 KB —
crc32.txtOther238 B —
.gitattributesRepository1.6 KB —
merges.txtTokenizer3.4 MB —
tokenizer.jsonTokenizer12.8 MB 0997f410c57a
tokenizer_config.jsonTokenizer17.9 KB —
vocab.jsonTokenizer6.7 MB —

License and Download

License
apache-2.0
Access
Open weights, no gate
Download size
13.5 GB
Download from Lee

Released by Lee through its official repository on Hugging Face. Read the license.

Built From

  • Derived from huihui-ai/Huihui-Qwen3.8-27B-abliterated
  • Quantized from huihui-ai/Huihui-Qwen3.8-27B-abliterated

Memory Requirements

PrecisionWeights in memory
As published13.5 GB
16-bit13.5 GB
8-bit6.7 GB
4-bit3.4 GB

Weights only, from the published parameter count; the key-value cache and runtime add to this.

Questions About Huihui-Qwen3.8-27B-abliterated-EXL3-3.0bpw

How much GPU memory does Huihui-Qwen3.8-27B-abliterated-EXL3-3.0bpw need?

About 16.2 GB at 16-bit and 4 GB at 4-bit: the weights (6.7B parameters) plus a working margin. A long context needs more.

What is the cheapest GPU to run Huihui-Qwen3.8-27B-abliterated-EXL3-3.0bpw on?

At 16-bit, 1x MI300X from $1.85 an hour; at 4-bit, 1x MI300X from $1.85 an hour, at the lowest on-demand prices the SAVRN Index lists.

Can I use Huihui-Qwen3.8-27B-abliterated-EXL3-3.0bpw commercially?

Yes. Huihui-Qwen3.8-27B-abliterated-EXL3-3.0bpw is released under Apache License 2.0. The Apache License 2.0 is a permissive open-source license. It permits commercial use, modification and redistribution. It requires keeping the license and copyright notices and any NOTICE file, stating significant changes, and it includes an express patent grant from contributors.

What is Huihui-Qwen3.8-27B-abliterated-EXL3-3.0bpw's context length?

262,144 tokens, from the maximum position embeddings in its published configuration.

Similar Models

Model · Image and text to text

llava-1.5-7b-hf

Llava Hugging Face

Below is the model card of Llava model 7b, which is copied from the original Llava model card that you can find here. Check out also the Google Colab demo to run Llava on a free-tier Google Colab instance: Or check out our Spaces demo! LLaVA is an open-source chatbot trained by fine-tuning LLaMA/Vicuna on GPT-generated multimodal instruction-following data. It is an auto-regressive language model, based on the transformer architecture. LLaVA-v1.5-7B was trained in September 2023. Paper or resources for more information: https://llava-vl.github.io/ First, make sure to have transformers >= 4.35.3. The model supports multi-image and multi-prompt generation. Meaning that you can pass multiple…

Open weights llama2 7.1B parameters 4,096 tokens transformers

Model · Image and text to text

Huihui-Qwen3.6-27B-abliterated-AWQ-MTP

Shawn Wei

This is an uncensored version of Qwen/Qwen3.6-27B created with abliteration (see remove-refusals-with-transformers to know more about it). This is a crude, proof-of-concept implementation to remove refusals from an LLM model without using TransformerLens. - AWQ Marlin kernel supported (auto-converted by vLLM at runtime) - MTP speculative decoding supported out of the box - 110+ tok/s on a single A800 80GB (vLLM 0.21.0, MTP enabled, fp8 KV cache) - Risk of Sensitive or Controversial Outputs: This model’s safety filtering has been significantly reduced, potentially generating sensitive, controversial, or inappropriate content. Users should exercise caution and rigorously review generated…

Open weights apache-2.0 6.3B parameters 262,144 tokens transformers

Model · Image and text to text

Swift-1.5-Qwen3.8-27b-heretic-W4A16

Akumaburn

A 18G INT4 build of for single-user decoding and low-VRAM deployment. (Marlin on Ampere). - Unrotated, so the DFlash2 drafter works. GatedDeltaNet gates, all norms. 4-bit costs 4.5× the rotated W8A8's quantization divergence (0.0264) and 1.5× the unrotated W8A8's (0.0791). Choose this build when the 18G footprint or single-user decode speed matters more than fidelity. Produced with Heretic v2.0.0.dev0: 260 TPE trials, seed 42, directional ablation on attn.oproj (16 modules), attn.outproj (48) and mlp.downproj (64). Baseline refusal score before abliteration: 98/100. Trial 140 was selected — Pareto index 1, not index 0. Index 0 (trial 161) scored 22/100 keywords at KL 0.1091; trial 140…

Open weights other 6.3B parameters 262,144 tokens vllm

This model was produced by fine-tuning the OpenVLA 7B model via LoRA (r=32) on the LIBERO-Spatial dataset from the LIBERO simulation benchmark. We made a few modifications to the training dataset to improve final performance (see the OpenVLA paper for details). Below are the hyperparameters we used for all LIBERO experiments: - No gradient accumulation (i.e. gradaccumulationsteps == 1) - shufflebuffersize == 100000 See the OpenVLA GitHub README for instructions on how to run and evaluate this model in the LIBERO simulator.

Open weights mit 7.5B parameters transformers

This model was produced by fine-tuning the OpenVLA 7B model via LoRA (r=32) on the LIBERO-Object dataset from the LIBERO simulation benchmark. We made a few modifications to the training dataset to improve final performance (see the OpenVLA paper for details). Below are the hyperparameters we used for all LIBERO experiments: - No gradient accumulation (i.e. gradaccumulationsteps == 1) - shufflebuffersize == 100000 See the OpenVLA GitHub README for instructions on how to run and evaluate this model in the LIBERO simulator.

Open weights mit 7.5B parameters transformers

Model · Image and text to text

openvla-7b-finetuned-libero-10

OpenVLA Collaboration

This model was produced by fine-tuning the OpenVLA 7B model via LoRA (r=32) on the LIBERO-10 (LIBERO-Long) dataset from the LIBERO simulation benchmark. We made a few modifications to the training dataset to improve final performance (see the OpenVLA paper for details). Below are the hyperparameters we used for all LIBERO experiments: - No gradient accumulation (i.e. gradaccumulationsteps == 1) - shufflebuffersize == 100000 See the OpenVLA GitHub README for instructions on how to run and evaluate this model in the LIBERO simulator.

Open weights mit 7.5B parameters transformers