Skip to page content
SAVRN
Search Plan a project

Open-weight model · Image and text to text

Qwen3.5-2B-RKLLM

by Hanzo Huang HanzoHuang/Qwen3.5-2B-RKLLM

Qwen3.5-2B-RKLLM is an open-weight model for image and text to text from Hanzo Huang, released under Apache License 2.0. Its published files total 11.0 GB. It draws 510 downloads a month.

Image and text on Rockchip RK3576 and RK3588, converted from Qwen/Qwen3.5-2B. Use the files for your board’s exact SoC. w4a16 uses 4-bit weights and 16-bit activations; w4a16g128 adds a weight group size of 128. w8a8 uses 8-bit weights and activations.

Parameters—
Context—
Weights1.3 GB
Licenseapache-2.0
AccessOpen weights
Monthly Downloads510

Model Card

By Hanzo Huang, published under apache-2.0, revision 888bdd28aa55.

Image and text on Rockchip RK3576 and RK3588, converted from Qwen/Qwen3.5-2B. Use the files for your board’s exact SoC. w4a16 uses 4-bit weights and 16-bit activations; w4a16g128 adds a weight group size of 128. w8a8 uses 8-bit weights and activations. File size below is download size, not total runtime RAM. For image input, also download the vision encoder for the same SoC. Keep the language model and vision encoder from this release together. ONNX files are export intermediates; they are not needed to run these RKNN/RKLLM models. With Docker installed on a matching Rockchip board, this example starts the RK3576 w4a16g128 model and its API on port 8001: The image includes its model files.…

Read Hanzo Huang's full model card

Image and text on Rockchip RK3576 and RK3588, converted from Qwen/Qwen3.5-2B.

  • Conversion toolkit: RKLLM Toolkit 1.3.0.
  • Inputs / output: text or image + text → text.
  • License: apache-2.0; the upstream model card describes its terms and intended use.

Choose your files

Use the files for your board’s exact SoC. w4a16 uses 4-bit weights and 16-bit activations; w4a16_g128 adds a weight group size of 128. w8a8 uses 8-bit weights and activations. File size below is download size, not total runtime RAM.

Board Quantization Size Language model
RK3576 w4a16_g128 2.21 GiB Qwen3.5-2B_RK3576_w4a16_g128.rkllm
RK3576 w8a8 2.76 GiB Qwen3.5-2B_RK3576_w8a8.rkllm
RK3588 w8a8 2.74 GiB Qwen3.5-2B_RK3588_w8a8.rkllm

For image input, also download the vision encoder for the same SoC. Keep the language model and vision encoder from this release together.

Board Size Vision encoder
RK3576 0.67 GiB Qwen3.5-2B_vision_RK3576.rknn
RK3588 0.65 GiB Qwen3.5-2B_vision_RK3588.rknn

ONNX files are export intermediates; they are not needed to run these RKNN/RKLLM models.

Run on your board

With Docker installed on a matching Rockchip board, this example starts the RK3576 w4a16_g128 model and its API on port 8001:

sudo docker run --rm -d --name rkllm \
  --privileged -v /dev:/dev -p 8001:8001 \
  ghcr.io/hanzo-huang/rkllm-docker/vlm/qwen3.5-2b:rk3576-w4a16-g128

The image includes its model files. See rkllm-docker for API requests, image inputs, other board variants, and configuration.

To download the files separately instead:

hf download HanzoHuang/Qwen3.5-2B-RKLLM \
  RK3576/Qwen3.5-2B_RK3576_w4a16_g128.rkllm \
  RK3576/Qwen3.5-2B_vision_RK3576.rknn \
  --local-dir ./models

These are native RKLLM/RKNN artifacts. Use a compatible RKLLM runtime and the model’s correct chat template when integrating them in another application.

Release notes

No per-variant board benchmark is published here. Quantized output may differ from the original model; check responses on your own workloads. Use the toolkit version listed above when checking runtime compatibility.

SHA-256 checksums 
76f40bbbc559615767e49a9408d28a1afc16c3a12aa3764a11e13f52acd868e9  RK3576/Qwen3.5-2B_RK3576_w4a16_g128.rkllm
95f4c10bf5d8880a4697c9c4e06508559930a9e379c4b154012daa94018ce8ae  RK3576/Qwen3.5-2B_RK3576_w8a8.rkllm
946daf4377bbf5913d8d5226180991b8e6b4a3068d8b5959bdc81a9492311a09  RK3588/Qwen3.5-2B_RK3588_w8a8.rkllm
c752c9ddec5ad0415cff7d5b2cd645c00273d7f75cfb034182126309911c051b  RK3576/Qwen3.5-2B_vision_RK3576.rknn
9077bfd3f4a0a3846af7fd1609175aaaba07858d00acdc31459ae9f2dc66716a  RK3588/Qwen3.5-2B_vision_RK3588.rknn

Credits

Original model: Qwen/Qwen3.5-2B. Conversion tools: Rockchip RKLLM.

Identity and Version

Repository
HanzoHuang/Qwen3.5-2B-RKLLM
Publisher
Hanzo Huang
Task
Image and text to text
Modality
Image and text
Library
rkllm
Parameters
Not stated by the source
Languages
Not stated by the source
Revision
888bdd28aa55fc77ce135ab3e1b0c6c395f5b108
First published
2026-08-11
Last updated
2026-10-19

Files and Weights

9 files, 11.0 GB in total. The weights are 1 file totalling 1.3 GB in onnx.

Weights1 file · 1.3 GB
Configuration1 file · 2.8 KB
Documentation1 file · 3.6 KB
Other5 files · 9.7 GB
Repository1 file · 2.2 KB
Every file
FileTypeSizeSHA-256
Qwen3.5-2B_vision.onnxWeights1.3 GB 2ed901be25b1
Qwen3.5-2B_data_quant.jsonConfiguration2.8 KB —
README.mdDocumentation3.6 KB —
RK3576/Qwen3.5-2B_RK3576_w4a16_g128.rkllmOther2.4 GB 76f40bbbc559
RK3576/Qwen3.5-2B_RK3576_w8a8.rkllmOther3.0 GB 95f4c10bf5d8
RK3576/Qwen3.5-2B_vision_RK3576.rknnOther721.5 MB c752c9ddec5a
RK3588/Qwen3.5-2B_RK3588_w8a8.rkllmOther2.9 GB 946daf4377bb
RK3588/Qwen3.5-2B_vision_RK3588.rknnOther700.4 MB 9077bfd3f4a0
.gitattributesRepository2.2 KB —

License and Download

License
apache-2.0
Access
Open weights, no gate
Download size
1.3 GB
Download from Hanzo Huang

Released by Hanzo Huang through its official repository on Hugging Face. Read the license.

Built From

Memory Requirements

PrecisionWeights in memory
As published1.3 GB

Weights only, from the published parameter count; the key-value cache and runtime add to this.

Questions About Qwen3.5-2B-RKLLM

Can I use Qwen3.5-2B-RKLLM commercially?

Yes. Qwen3.5-2B-RKLLM is released under Apache License 2.0. The Apache License 2.0 is a permissive open-source license. It permits commercial use, modification and redistribution. It requires keeping the license and copyright notices and any NOTICE file, stating significant changes, and it includes an express patent grant from contributors.

Similar Models

Non-uniform GGUF quantizations of a 512-expert MoE, produced with GSQ and RCO, with a vision projector for multimodal use. This repository provides GGUF quantizations of Qwen3.8-Flash-Next at four sizes, together with the model's vision projector (mmproj) for multimodal use. In contrast to uniform quantization, which applies a single quantization type to all weight tensors, each model here assigns a separate quantization type to every tensor. The assignment is obtained by a gradient-based search that allocates precision according to per-tensor sensitivity, subject to a total size budget. The resulting files are standard GGUF and run unmodified in llama.cpp, Ollama, and LM Studio. A…

Open weights apache-2.0 gguf
View model

Model · Image and text to text

Huihui-Qwen3.8-27B-abliterated-GGUF

Huihui.ai

This is an uncensored version of Qwen/Qwen3.8-27B created with abliteration (see remove-refusals-with-transformers to know more about it). This is a crude, proof-of-concept implementation to remove refusals from an LLM model without using TransformerLens. The newly added Huihui-Qwen3.8-27B-abliterated-Underdog-Saluki series come from ConwayResearch/Underdog-Saluki-27B-1.0 and ukisai/Swift-1.5-Qwen3.8-27B-GSQ-RCO-GGUF. Only layers 22 to 52 (0-based indexing) have been ablated, while the other layers remain unablated. It may come with a small disclaimer warning. Due to ConwayResearch/Underdog-Saluki-27B-1.0 not providing an imatrix file, we will attempt to perform ablation by mixing the GGUF…

Open weights apache-2.0 transformers
View model

Qwen3.8-27B uncensored by HauhauCS 0/465 Refusals. This is the Aggressive variant: direct answers, no refusal behavior, and minimal preamble on hard prompts. Every text GGUF preserves Qwen3.8's native NextN head, and this release adds HauhauCS FastMTP: a specific acceleration sidecar qualified across the complete quant lineup at maximum native context. Vision is included through the separate BF16 projector. No changes to datasets or intended capabilities. This release preserves Qwen3.8-27B's text, reasoning, agentic, image, and video capabilities while applying the HauhauCS Aggressive uncensoring profile. Pick Aggressive when you specifically want the model to get to the answer without…

Open weights apache-2.0
View model

Model · Image and text to text

Qwen3.8-27B-iMatrix-NVFP4-MTP-GGUF

Michał Piszczek

I built this quant because the ready-made FP4 file answered the wrong question. It was fast, but on my short WikiText-2 control it scored 6.4949 PPL. Plain Q40 scored 6.3798. The first higher-quality hybrid went too far the other way: good perplexity, 34.19 tok/s, and no comfortable room for 256K plus vision. This is the build that survived both gates. It is a 17.1 GB, 5.01 BPW mixed-precision GGUF of Qwen/Qwen3.8-27B. It keeps large, tolerant matrices in native NVFP4 and spends more bits on selected attention, Gated DeltaNet, and late FFN tensors. The trained MTP layer remains embedded in the same GGUF. This is not a fine-tune. I built the private calibration workload from 5,472 messages…

Open weights apache-2.0
View model

Model · Image and text to text

Qwen3.8-27B-GGUF

GGML Org

Run it in the Llama app: https://llama.app - https://huggingface.co/Qwen/Qwen3.8-27B - https://huggingface.co/incoai/Qwen3.8-27B-DFlash2 - add info

Open weights apache-2.0
View model

lm mmap --lazy-mode on -ngl 99 -c 32768 \ mmproj-Qwen3.8-Flash-Next-BF16.gguf --local-dir. llama-mtmd-cli -m IQ3S/Qwen3.8-Flash-Next-GSQ-RCO-abliterated-IQ3S-00001-of-00002.gguf \ --mmproj mmproj-Qwen3.8-Flash-Next-BF16.gguf -lm mmap --lazy-mode on \ -md mtp-Qwen3.8-Flash-Next-Q80.gguf --draft-max 4 -lm mmap --lazy-mode on -ngl 99 -lm mmap --lazy-mode on keeps the n-gram table memory-mapped on disk: it is 28.8 GB and one row is read per token. Shard 1 wants to be resident (VRAM or RAM); shard 2 is fine on an SSD. The vision projector is a standard clip GGUF shared with the upstream tiers, 0.91 GB. Native context is 262,144 tokens and the KV cache grows with it, so start at -c 32768 and work…

Open weights apache-2.0 gguf
View model