dQwen3.5-9B-Base by IFML: Open-Weight Model

Runs On

What it takes to serve dQwen3.5-9B-Base (9B parameters): the memory its weights need at each precision, and the cheapest way to rent enough data-center GPUs to hold them.

Precision	Weights	Memory needed	Cheapest setup	Per hour	Also fits
16-bit	17.9 GB	21.5 GB	1x MI300X (192 GB) Vultr	$1.85	1x H100 $1.99 · 1x MI325X $2.00
8-bit	9.0 GB	10.7 GB	1x MI300X (192 GB) Vultr	$1.85	1x H100 $1.99 · 1x MI325X $2.00
4-bit	4.5 GB	5.4 GB	1x MI300X (192 GB) Vultr	$1.85	1x H100 $1.99 · 1x MI325X $2.00

Memory is the weights at that precision plus 20% for the runtime and a short context; a long context needs more. Prices are the lowest on-demand hourly rates in the SAVRN Index, read Sep 18, 2026.

Model Card

By IFML, published under apache-2.0, revision d73a0b454625.

A masked diffusion language model adapted from Qwen3.5-9B. The backbone is hybrid: only its attention layers are made bidirectional, and the Gated DeltaNet layers stay causal. This is a base model, with no instruction tuning. Paper: dQwen3.5: Hybrid-Attention Diffusion Language Models. Code: https://github.com/AntonXue/dQwen Needs a CUDA GPU and transformers>=5.13 (tested with torch 2.7.1+cu128, flash-linear-attention 0.5.1). generate decodes the whole canvas at once, committing positions above a confidence threshold (tau=0.9); pass blocklength=32 for left-to-right block decoding, or tau=None, stepsperblock=k for a fixed budget. The 50B-token checkpoint from the paper is…

Read IFML's full model card

A masked diffusion language model adapted from Qwen3.5-9B. The backbone is hybrid: only its attention layers are made bidirectional, and the Gated DeltaNet layers stay causal. This is a base model, with no instruction tuning.

Paper: dQwen3.5: Hybrid-Attention Diffusion Language Models. Code: https://github.com/AntonXue/dQwen

The dQwen3.5 family

model	total params	backbone
dQwen3.5-0.8B-Base	0.75B	hybrid
dQwen3.5-2B-Base	1.88B	hybrid
dQwen3.5-4B-Base	4.21B	hybrid
dQwen3.5-9B-Base (this repo)	8.95B	hybrid
dQwen3-1.7B-Base	1.72B	full attention (control)

Quickstart

Needs a CUDA GPU and transformers>=5.13 (tested with torch 2.7.1+cu128, flash-linear-attention 0.5.1).

import torch
from transformers import AutoModel

model = AutoModel.from_pretrained("UT-IFML/dQwen3.5-9B-Base", trust_remote_code=True, dtype=torch.bfloat16).cuda().eval()
print(model.generate("def fibonacci(n):", gen_length=512, stop_strings=["\ndef "]).text)

generate decodes the whole canvas at once, committing positions above a confidence threshold (tau=0.9); pass block_length=32 for left-to-right block decoding, or tau=None, steps_per_block=k for a fixed budget. The 50B-token checkpoint from the paper is revision="step25000-swa".

Citation

@article{xue2026dqwen,
  title   = {dQwen3.5: Hybrid-Attention Diffusion Language Models},
  author  = {Xue, Anton and Rout, Litu and Akella, Aditya and Klivans, Adam and Sanghavi, Sujay and Shakkottai, Sanjay},
  journal = {arXiv preprint arXiv:2609.20751},
  year    = {2026}
}

Configuration

Architecture: DiffuQwen35
Context length (tokens): 262,144
Layers: 32
Hidden size: 4,096
Feed-forward size: 12,288
Attention heads: 16
Key/value heads: 4
Head dimension: 256
Vocabulary size: 248,320
Model type: diffuqwen35

Identity and Version

Repository: UT-IFML/dQwen3.5-9B-Base
Publisher: IFML
Task: Text generation
Modality: Text
Library: transformers
Parameters: 9B parameters
Languages: Not stated by the source
Revision: d73a0b454625b93a5990f623801febcc1c507b73
First published: 2026-08-05
Last updated: 2026-09-18

Files and Weights

15 files, 17.9 GB in total. The weights are 4 files totalling 17.9 GB in safetensors.

Weights4 files · 17.9 GB

Configuration5 files · 52.8 KB

Tokenizer2 files · 20.0 MB

Documentation1 file · 2.1 KB

Other2 files · 1.1 MB

Repository1 file · 1.6 KB

Every file

File	Type	Size	SHA-256
model-00001-of-00004.safetensors	Weights	4.9 GB	1ce388311891
model-00002-of-00004.safetensors	Weights	5.0 GB	f1820a90448a
model-00003-of-00004.safetensors	Weights	5.0 GB	fbf246547b8f
model-00004-of-00004.safetensors	Weights	3.0 GB	2e95a4c162ca
argparse.json	Configuration	866 B	—
config.json	Configuration	2.1 KB	—
generation_config.json	Configuration	179 B	—
model.safetensors.index.json	Configuration	35.6 KB	—
modeling_dqwen3_5.py	Configuration	14.0 KB	—
README.md	Documentation	2.1 KB	—
chat_template.jinja	Other	7.8 KB	—
training_log.jsonl	Other	1.1 MB	—
.gitattributes	Repository	1.6 KB	—
tokenizer.json	Tokenizer	20.0 MB	06b9509352d2
tokenizer_config.json	Tokenizer	1.2 KB	—

License and Download

License: apache-2.0
Access: Open weights, no gate
Download size: 17.9 GB

Download from IFML

Released by IFML through its official repository on Hugging Face. Read the license.

Built From

Derived from Qwen/Qwen3.5-9B
Described by arXiv:2609.20751

Memory Requirements

Precision	Weights in memory
As published	17.9 GB
16-bit	17.9 GB
8-bit	9.0 GB
4-bit	4.5 GB

Weights only, from the published parameter count; the key-value cache and runtime add to this.

Questions About dQwen3.5-9B-Base

How much GPU memory does dQwen3.5-9B-Base need?

About 21.5 GB at 16-bit and 5.4 GB at 4-bit: the weights (9B parameters) plus a working margin. A long context needs more.

What is the cheapest GPU to run dQwen3.5-9B-Base on?

At 16-bit, 1x MI300X from $1.85 an hour; at 4-bit, 1x MI300X from $1.85 an hour, at the lowest on-demand prices the SAVRN Index lists.

Can I use dQwen3.5-9B-Base commercially?

Yes. dQwen3.5-9B-Base is released under Apache License 2.0. The Apache License 2.0 is a permissive open-source license. It permits commercial use, modification and redistribution. It requires keeping the license and copyright notices and any NOTICE file, stating significant changes, and it includes an express patent grant from contributors.

What is dQwen3.5-9B-Base's context length?

262,144 tokens, from the maximum position embeddings in its published configuration.

Similar Models

Model · Text generation

idefics-9b

HuggingFaceM4

How do I pronounce the model's name? Watch a Youtube tutorial IDEFICS (Image-aware Decoder Enhanced à la Flamingo with Interleaved Cross-attentionS) is an open-access reproduction of Flamingo, a closed-source visual language model developed by Deepmind. Like GPT-4, the multimodal model accepts arbitrary sequences of image and text inputs and produces text outputs. IDEFICS is built solely on publicly available data and models. The model can answer questions about images, describe visual contents, create stories grounded on multiple images, or simply behave as a pure language model without visual inputs. IDEFICS is on par with the original closed-source model on various image-text benchmarks…

Open weights other 8.9B parameters 2,048 tokens transformers

View model

Model · Text generation

gemma-2-9b-it

Google

[Responsible Generative AI Toolkit][rai-toolkit] [Gemma on Kaggle][kaggle-gemma] [Gemma on Vertex Model Garden][vertex-mg-gemma] Summary description and brief definition of inputs and outputs. Gemma is a family of lightweight, state-of-the-art open models from Google, built from the same research and technology used to create the Gemini models. They are text-to-text, decoder-only large language models, available in English, with open weights for both pre-trained variants and instruction-tuned variants. Gemma models are well-suited for a variety of text generation tasks, including question answering, summarization, and reasoning. Their relatively small size makes it possible to deploy them…

Access requested at publisher gemma 9.2B parameters transformers

View model

Model · Text generation

LFM2.5-8B-A1B-Opus-Distil

Convergent Intelligence

This model is a fine-tuned version of LiquidAI/LFM2.5-8B-A1B, adapted on the angrygiraffe/claude-opus-4.6-4.7-reasoning-8.7k dataset for English text-generation and reasoning-style responses. The fine-tuning run used a custom Convergent Intelligence optimizer stack, CIxOpt, designed for heterogeneous routing across parameter types. The goal of this checkpoint is to test whether a Liquid Foundation Model backbone can be adapted efficiently through targeted sparse participation rather than broad full-model modification. This is an experimental research checkpoint intended for continued evaluation, domain adaptation, and architecture/optimizer testing.…

Open weights apache-2.0 8.5B parameters 128,000 tokens transformers

View model

Model · Text generation

LFM2.5-8B-A1B-Hermes-Agentic-Coder-Abliterated-v2

DuoNeural

DuoNeural/LFM2.5-8B-A1B-Hermes-Agentic-Coder-Abliterated-v2 is an apex-tier, uncensored autonomous agentic coding model trained by DuoNeural (Aura, Archon, and Jesse). Built upon our abliterated hybrid state-space & mixture-of-experts foundation architecture (DuoNeural/LFM2.5-8B-A1B-Abliterated), this model activates only 1.5 billion parameters per token out of its 8.3 billion total parameters, delivering blistering inference speeds (~360 tokens/sec on RTX 4080 Super / 3090, and ~80–90 tokens/sec on legacy mobile GPUs like the GTX 1070) while running in under 6 GB VRAM with Q4KM quantization. In our preliminary v1 release, an assistant role delimiter mismatch during training collation…

Open weights other 8.5B parameters 128,000 tokens hermes

View model

Model · Text generation

LFM2.5-8B-A1B-Hermes-Agentic-Coder-Abliterated

DuoNeural

DuoNeural/LFM2.5-8B-A1B-Hermes-Agentic-Coder-Abliterated is an apex-tier, uncensored autonomous agentic coding model trained by DuoNeural (Aura, Archon, and Jesse). Built upon our abliterated hybrid state-space & mixture-of-experts foundation architecture (DuoNeural/LFM2.5-8B-A1B-Abliterated), this model activates only 1.5 billion parameters per token out of its 8.3 billion total parameters, delivering blistering inference speeds (~380–400 tokens/sec on consumer GPUs like RTX 3090, and ~90 tokens/sec on legacy GTX 1070 laptops) while fitting in under 6 GB VRAM with Q4KM quantization. 1. Native Hermes Agentic Loop: - Explicit... deliberation before every action. - Structured... containers…

Open weights other 8.5B parameters 128,000 tokens hermes

View model

Model · Text generation

Qwen3-8B-FP8

Qwen

Qwen3 is the latest generation of large language models in Qwen series, offering a comprehensive suite of dense and mixture-of-experts (MoE) models. Built upon extensive training, Qwen3 delivers groundbreaking advancements in reasoning, instruction-following, agent capabilities, and multilingual support, with the following key features: - Uniquely support of seamless switching between thinking mode (for complex logical reasoning, math, and coding) and non-thinking mode (for efficient, general-purpose dialogue) within single model, ensuring optimal performance across various scenarios. - Significantly enhancement in its reasoning capabilities, surpassing previous QwQ (in thinking mode) and…

Open weights apache-2.0 8.2B parameters 40,960 tokens transformers

View model