This model is a fine-tuned version of Qwen/Qwen3.5-4B-Base on the Omni-Edu-70K dataset. The following hyperparameters were used during training: - learningrate: 5e-06 - trainbatchsize: 1 - evalbatchsize: 8 - distributedtype: multi-GPU - numdevices: 8 - gradientaccumulationsteps: 8 - totaltrainbatchsize: 64 - totalevalbatchsize: 64 - lrschedulertype: cosine - lrschedulerwarmupsteps: 0.1 - numepochs: 3.0 - Transformers 5.2.0 - Pytorch 2.10.0 - Datasets 4.0.0 - Tokenizers 0.22.2
Over recent months, we have intensified our focus on developing foundation models that deliver exceptional utility and performance.
Parameters4.7B
Context262,144
Weights9.3 GB
Licenseapache-2.0
AccessOpen weights
Monthly Downloads7M
Runs On
What it takes to serve Qwen3.5-4B (4.7B parameters): the memory its weights need at each precision, and the cheapest way to rent enough data-center GPUs to hold them.
| Precision | Weights | Memory needed | Cheapest setup | Per hour | Also fits |
|---|---|---|---|---|---|
| 16-bit | 9.3 GB | 11.2 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
| 8-bit | 4.7 GB | 5.6 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
| 4-bit | 2.3 GB | 2.8 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
Memory is the weights at that precision plus 20% for the runtime and a short context; a long context needs more. Prices are the lowest on-demand hourly rates in the SAVRN Index, read Sep 18, 2026.
SAVRN's Notes on Qwen3.5-4B
A 262,144-token window on a 4.7B model shapes everything else here. It is built on Qwen3.5-4B-Base and reads images as well as text. At 16-bit it needs 11.2 GB to run, so on the cheapest setup we list, one MI300X with 192 GB at $1.85 an hour on demand, quantizing is a choice, not a requirement; 4-bit brings it to 2.8 GB for several copies on one card.
Apache 2.0 means you can run it commercially, fine-tune it and redistribute what you make, as long as the license and any NOTICE file stay attached and you state significant changes, with a patent grant included. Before committing, run the window length you intend to use and measure memory there; 11.2 GB is the entry ticket, not the ceiling. Released February 27, 2026, it is young; check the publisher's page for revisions before you freeze a version.
Model Card
By Qwen, published under apache-2.0, revision 851bf6e806ef.
[!Note] This repository contains model weights and configuration files for the post-trained model in the Hugging Face Transformers format.
These artifacts are compatible with Hugging Face Transformers, vLLM, SGLang, KTransformers, etc.
Over recent months, we have intensified our focus on developing foundation models that deliver exceptional utility and performance. Qwen3.5 represents a significant leap forward, integrating breakthroughs in multimodal learning, architectural efficiency, reinforcement learning scale, and global accessibility to empower developers and enterprises with unprecedented capability and efficiency.
Qwen3.5 Highlights
Qwen3.5 features the following enhancement:
Configuration
- Architecture
- Qwen3_5ForConditionalGeneration
- Context length (tokens)
- 262,144
- Layers
- 32
- Hidden size
- 2,560
- Feed-forward size
- 9,216
- Attention heads
- 16
- Key/value heads
- 4
- Head dimension
- 256
- Vocabulary size
- 248,320
- Model type
- qwen3_5
Identity and Version
- Repository
- Qwen/Qwen3.5-4B
- Publisher
- Qwen
- Task
- Image and text to text
- Modality
- Image and text
- Library
- transformers
- Parameters
- 4.7B parameters
- Languages
- Not stated by the source
- Revision
- 851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a
- First published
- 2026-02-27
- Last updated
- 2026-03-02
Files and Weights
14 files, 9.3 GB in total. The weights are 2 files totalling 9.3 GB in safetensors.
Weights2 files · 9.3 GB
Configuration4 files · 80.1 KB
Tokenizer4 files · 22.9 MB
Documentation2 files · 89.2 KB
Other1 file · 7.8 KB
Repository1 file · 1.6 KB
Every file
| File | Type | Size | SHA-256 |
|---|---|---|---|
| model.safetensors-00001-of-00002.safetensors | Weights | 5.3 GB | 26a93f066e19 |
| model.safetensors-00002-of-00002.safetensors | Weights | 4.0 GB | cb544bd9bfae |
| config.json | Configuration | 3.2 KB | — |
| model.safetensors.index.json | Configuration | 76.2 KB | — |
| preprocessor_config.json | Configuration | 390 B | — |
| video_preprocessor_config.json | Configuration | 385 B | — |
| LICENSE | Documentation | 11.5 KB | — |
| README.md | Documentation | 77.7 KB | — |
| chat_template.jinja | Other | 7.8 KB | — |
| .gitattributes | Repository | 1.6 KB | — |
| merges.txt | Tokenizer | 3.4 MB | — |
| tokenizer.json | Tokenizer | 12.8 MB | 5f9e4d4901a9 |
| tokenizer_config.json | Tokenizer | 16.7 KB | — |
| vocab.json | Tokenizer | 6.7 MB | — |
License and Download
- License
- apache-2.0
- Access
- Open weights, no gate
- Download size
- 9.3 GB
Download from Qwen
Released by Qwen through ModelScope. Read the license.
Built From
- Derived from Qwen/Qwen3.5-4B-Base
Evaluations
Each result is shown as reported, with the conditions its reporter stated. None is a SAVRN measurement. A comparison lines two results up only when their configuration, unit and setup are all stated and identical.
| Benchmark | Conditions | Result | Reported by | Revision | Date |
|---|---|---|---|---|---|
| Idavidrein/gpqa | Task diamondMetric diamondComparison conditions not established | 76.2 | Model Card Reported by a third party |
Evaluated revision not stated | 2026-03-02 |
| LiquidAI/ifstruct-v1.0 | Task ifstruct_v1Metric ifstruct_v1Comparison conditions not established | 36.25 | Liquid AI — IFStruct v1.0 blog (Qwen3.5-4B) Reported by a third party |
Evaluated revision not stated | 2026-06-30 |
| MMMU/MMMU_Pro | Task mmmu_pro_visionMetric mmmu_pro_visionComparison conditions not established | 66.3 | Model Card Reported by a third party |
Evaluated revision not stated | 2026-04-28 |
| TIGER-Lab/MMLU-Pro | Task mmlu_proMetric mmlu_proComparison conditions not established | 79.1 | Model Card Reported by a third party |
Evaluated revision not stated | 2026-03-02 |
| likaixin/ScreenSpot-Pro | Task overallMetric overallComparison conditions not established | 60.3 | Model Card Reported by a third party |
Evaluated revision not stated | 2026-03-18 |
| llamaindex/ExtractBench | Task longMetric longSetup Pipeline name: qwen3_5_4b_vllm_extract_oneshot_structured_output_fileComparison conditions not established | 27.31 | ExtractBench Reported by a third party |
Evaluated revision not stated | 2026-08-26 |
| llamaindex/ExtractBench | Task meanMetric meanSetup Pipeline name: qwen3_5_4b_vllm_extract_oneshot_structured_output_fileComparison conditions not established | 82.43 | ExtractBench Reported by a third party |
Evaluated revision not stated | 2026-08-26 |
| llamaindex/ExtractBench | Task mediumMetric mediumSetup Pipeline name: qwen3_5_4b_vllm_extract_oneshot_structured_output_fileComparison conditions not established | 76.64 | ExtractBench Reported by a third party |
Evaluated revision not stated | 2026-08-26 |
| llamaindex/ExtractBench | Task shortMetric shortSetup Pipeline name: qwen3_5_4b_vllm_extract_oneshot_structured_output_fileComparison conditions not established | 89.06 | ExtractBench Reported by a third party |
Evaluated revision not stated | 2026-08-26 |
| llamaindex/ParseBench | Task chartMetric chartSetup Pipeline name: qwen3_5_4b_vllm_layoutComparison conditions not established | 2.5 | ParseBench Reported by a third party |
Evaluated revision not stated | 2026-04-14 |
| llamaindex/ParseBench | Task layoutMetric layoutSetup Pipeline name: qwen3_5_4b_vllm_layoutComparison conditions not established | 19.7 | ParseBench Reported by a third party |
Evaluated revision not stated | 2026-04-14 |
| llamaindex/ParseBench | Task meanMetric meanSetup Pipeline name: qwen3_5_4b_vllm_layoutComparison conditions not established | 35.4 | ParseBench Reported by a third party |
Evaluated revision not stated | 2026-04-14 |
| llamaindex/ParseBench | Task tableMetric tableSetup Pipeline name: qwen3_5_4b_vllm_layoutComparison conditions not established | 8 | ParseBench Reported by a third party |
Evaluated revision not stated | 2026-04-14 |
| llamaindex/ParseBench | Task text_contentMetric text_contentSetup Pipeline name: qwen3_5_4b_vllm_layoutComparison conditions not established | 88.9 | ParseBench Reported by a third party |
Evaluated revision not stated | 2026-04-14 |
| llamaindex/ParseBench | Task text_formattingMetric text_formattingSetup Pipeline name: qwen3_5_4b_vllm_layoutComparison conditions not established | 57.8 | ParseBench Reported by a third party |
Evaluated revision not stated | 2026-04-14 |
Memory Requirements
| Precision | Weights in memory |
|---|---|
| As published | 9.3 GB |
| 16-bit | 9.3 GB |
| 8-bit | 4.7 GB |
| 4-bit | 2.3 GB |
Weights only, from the published parameter count; the key-value cache and runtime add to this.
Built on This Model
- Quantized fromQwen3.5-4B-GGUF
- Derived fromQwen3.5-4B-GGUF
- Derived fromNuExtract3
- Derived fromdQwen3.5-4B-Base
- Adapter ofcheckpoints
- Derived fromcheckpoints
Compare Qwen3.5-4B
Questions About Qwen3.5-4B
How much GPU memory does Qwen3.5-4B need?
About 11.2 GB at 16-bit and 2.8 GB at 4-bit: the weights (4.7B parameters) plus a working margin. A long context needs more.
What is the cheapest GPU to run Qwen3.5-4B on?
At 16-bit, 1x MI300X from $1.85 an hour; at 4-bit, 1x MI300X from $1.85 an hour, at the lowest on-demand prices the SAVRN Index lists.
Can I use Qwen3.5-4B commercially?
Yes. Qwen3.5-4B is released under Apache License 2.0. The Apache License 2.0 is a permissive open-source license. It permits commercial use, modification and redistribution. It requires keeping the license and copyright notices and any NOTICE file, stating significant changes, and it includes an express patent grant from contributors.
What is Qwen3.5-4B's context length?
262,144 tokens, from the maximum position embeddings in its published configuration.
Similar Models
Meet Qwen3-VL — the most powerful vision-language model in the Qwen series to date. This generation delivers comprehensive upgrades across the board: superior text understanding & generation, deeper visual perception & reasoning, extended context length, enhanced spatial and video dynamics comprehension, and stronger agent interaction capabilities. Available in Dense and MoE architectures that scale from edge to cloud, with Instruct and reasoning‑enhanced Thinking editions for flexible, on‑demand deployment. Text Understanding on par with pure LLMs: Seamless text–vision fusion for lossless, unified comprehension. 1. Interleaved-MRoPE: Full‑frequency allocation over time, width, and height…
This is a modified version of google/translategemma-4b-it optimized for deployment with vLLM. The original TranslateGemma model requires a structured payload with dedicated sourcelangcode and targetlangcode fields: However, vLLM does not support these custom content parameters. To maintain compatibility, the chat template has been modified to encode language codes directly in the message content using a delimiter-based format: Format: >>{sourcelang} >>{targetlang} >>{texttotranslate} If you need to provide a custom prompt input The original model uses the new Transformers RoPE configuration format with separate attention type settings: This has been simplified for vLLM compatibility: The…
[Gemma 3 Technical Report][g3-tech-report] [Responsible Generative AI Toolkit][rai-toolkit] [Gemma on Kaggle][kaggle-gemma] [Gemma on Vertex Model Garden][vertex-mg-gemma3] Summary description and brief definition of inputs and outputs. Gemma is a family of lightweight, state-of-the-art open models from Google, built from the same research and technology used to create the Gemini models. Gemma 3 models are multimodal, handling text and image input and generating text output, with open weights for both pre-trained variants and instruction-tuned variants. Gemma 3 has a large, 128K context window, multilingual support in over 140 languages, and is available in more sizes than previous…
Model on Google Cloud Model Garden: MedGemma GitHub repository (supporting code, Colab notebooks, discussions, and Foundations terms of use](https://developers.google.com/health-ai-developer-foundations/terms). This section describes the MedGemma model and how to use it. MedGemma is a collection of Gemma 3 variants that are trained for performance on medical text and image comprehension. Developers can use MedGemma to accelerate building healthcare-based AI applications. MedGemma currently comes in three variants: a 4B multimodal version and 27B text-only and multimodal versions. Both MedGemma multimodal versions utilize a SigLIP image encoder that has been specifically pre-trained on a…
This model is Rex-Omni, a 3B-parameter Multimodal Large Language Model (MLLM) presented in the paper "Detect Anything via Next Point Prediction". It is compatible with the Hugging Face transformers library and is licensed under the IDEA License 1.0. src="https://img.shields.io/badge/RexOmni-Website-BADFDB?style=flat-square&logo=deno&logoColor=violet&color=BADFDB" alt="RexThinker Website" src="https://img.shields.io/badge/RexOmni-Paper-Red%25red?logo=arxiv&logoColor=red&color=yellow" alt="RexThinker Paper on arXiv" src="https://img.shields.io/badge/RexOmni-Weight-orange?logo=huggingface&logoColor=yellow" alt="RexThinker weight on Hugging Face"…