Over recent months, we have intensified our focus on developing foundation models that deliver exceptional utility and performance. Qwen3.5 represents a significant leap forward, integrating breakthroughs in multimodal learning, architectural efficiency, reinforcement learning scale, and global accessibility to empower developers and enterprises with unprecedented capability and efficiency. For more details, please refer to our blog post Qwen3.5. If you find our work helpful, feel free to give us a cite.
Valen-2B is an open-weight model for image and text to text from Valen, released under Apache License 2.0. It has 2.2B parameters and a 262,144-token context. At 16-bit it needs about 5.3 GB of GPU memory, which fits on 1x MI300X from $1.85 an hour, at the lowest prices in the SAVRN Index. It draws 185 downloads a month.
Qwen3.5 with a two-layer MLP-Mixer decision head. Supports text, images and videos, and returns Choice, Noul and Score decisions. Multiple questions can share one state encoding with execution="sharedstate". Trained on millions of samples.
Parameters2.2B
Context262,144
Weights8.9 GB
Licenseapache-2.0
AccessOpen weights
Monthly Downloads185
Runs On
What it takes to serve Valen-2B (2.2B parameters): the memory its weights need at each precision, and the cheapest way to rent enough data-center GPUs to hold them.
| Precision | Weights | Memory needed | Cheapest setup | Per hour | Also fits |
|---|---|---|---|---|---|
| 16-bit | 4.4 GB | 5.3 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
| 8-bit | 2.2 GB | 2.7 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
| 4-bit | 1.1 GB | 1.3 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
Memory is the weights at that precision plus 20% for the runtime and a short context; a long context needs more. Prices are the lowest on-demand hourly rates in the SAVRN Index, read Oct 18, 2026.
Valen-2B on every accelerator the SAVRN Index prices, at every precision
Model Card
By Valen, published under apache-2.0, revision fb9205c2541a.
Qwen3.5 with a two-layer MLP-Mixer decision head. Supports text, images and videos, and returns Choice, Noul and Score decisions. Multiple questions can share one state encoding with execution="sharedstate". Trained on millions of samples. Install Python 3.10+, PyTorch, Transformers 5.4.0, torchvision, Pillow and av. Flash Attention 2 is optional with a compatible CUDA build. The model contains its tokenizer, processor and custom inference code; a separate base-model download is unnecessary. Use attnimplementation="flashattention2" for Flash Attention, or execution="question" for independent questions. Video defaults to 16 frames. Image/video request examples and training instructions are…
Read Valen's full model card
Qwen3.5 with a two-layer MLP-Mixer decision head. Supports text, images and videos, and returns Choice, Noul and Score decisions. Multiple questions can share one state encoding with execution="shared_state". Trained on millions of samples.
Inference
Install Python 3.10+, PyTorch, Transformers 5.4.0, torchvision, Pillow and av. Flash Attention 2 is optional with a compatible CUDA build. The model contains its tokenizer, processor and custom inference code; a separate base-model download is unnecessary.
import torch
from transformers import AutoModel
model = AutoModel.from_pretrained(
"Valen-Team/Valen-2B", trust_remote_code=True,
dtype="auto", attn_implementation="sdpa",
).to("cuda").eval()
torch.set_float32_matmul_precision("highest")
torch.backends.cudnn.allow_tf32 = False
print(model.predict({
"state": "A cat is on the sofa.",
"questions": {
"animal": {"type": "choice", "instructions": "Which animal is present?",
"criteria": {"cat": "A cat", "dog": "A dog"}},
"on_sofa": {"type": "noul", "instructions": "The cat is on the sofa."},
},
}, execution="shared_state"))
Use attn_implementation="flash_attention_2" for Flash Attention, or execution="question" for independent questions. Video defaults to 16 frames. Image/video request examples and training instructions are in the Valen repository.
dtype="auto" preserves the trained FP32 parameters. The backbone runs under BF16 autocast and the Mixer stays FP32, matching native checkpoint evaluation. Loading all weights as BF16 rounds the trained parameters and can change outputs.
Evaluation
| Benchmark | Accuracy (%) |
|---|---|
| VisualDecisionBench Image | 80.41 |
| VisualDecisionBench Video | 82.85 |
| JevBench | 83.55 |
Configuration
- Architecture
- ValenQwenForDecisionMaking
- Context length (tokens)
- 262,144
- Layers
- 24
- Hidden size
- 2,048
- Feed-forward size
- 6,144
- Attention heads
- 8
- Key/value heads
- 2
- Head dimension
- 256
- Vocabulary size
- 248,320
- Model type
- valen_qwen
Identity and Version
- Repository
- Valen-Team/Valen-2B
- Publisher
- Valen
- Task
- Image and text to text
- Modality
- Image and text
- Library
- transformers
- Parameters
- 2.2B parameters
- Languages
- en, zh
- Revision
- fb9205c2541abc97ce276670e66f79b78d9068fc
- First published
- 2026-10-05
- Last updated
- 2026-10-13
Files and Weights
21 files, 8.9 GB in total. The weights are 3 files totalling 8.9 GB in safetensors.
Weights3 files · 8.9 GB
Configuration12 files · 104.0 KB
Tokenizer2 files · 20.0 MB
Documentation2 files · 13.7 KB
Other1 file · 7.8 KB
Repository1 file · 1.6 KB
Every file
| File | Type | Size | SHA-256 |
|---|---|---|---|
| model-00001-of-00003.safetensors | Weights | 4.0 GB | cbac6fc9f9e1 |
| model-00002-of-00003.safetensors | Weights | 4.0 GB | 280e098df96b |
| model-00003-of-00003.safetensors | Weights | 861.8 MB | 62b8d1265250 |
| batching.py | Configuration | 3.0 KB | — |
| compiler.py | Configuration | 13.6 KB | — |
| config.json | Configuration | 3.3 KB | — |
| configuration_valen.py | Configuration | 1.0 KB | — |
| data_types.py | Configuration | 258 B | — |
| heads.py | Configuration | 6.6 KB | — |
| model.safetensors.index.json | Configuration | 60.0 KB | — |
| modeling_valen.py | Configuration | 4.6 KB | — |
| processor_config.json | Configuration | 1.2 KB | — |
| responses.py | Configuration | 1.5 KB | — |
| runtime_model.py | Configuration | 5.6 KB | — |
| schema.py | Configuration | 3.4 KB | — |
| LICENSE | Documentation | 11.5 KB | — |
| README.md | Documentation | 2.1 KB | — |
| chat_template.jinja | Other | 7.8 KB | — |
| .gitattributes | Repository | 1.6 KB | — |
| tokenizer.json | Tokenizer | 20.0 MB | 87a7830d63fc |
| tokenizer_config.json | Tokenizer | 1.1 KB | — |
License and Download
- License
- apache-2.0
- Access
- Open weights, no gate
- Download size
- 8.9 GB
Download from Valen
Released by Valen through its official repository on Hugging Face. Read the license.
Built From
- Derived from Qwen/Qwen3.5-2B
Memory Requirements
| Precision | Weights in memory |
|---|---|
| As published | 8.9 GB |
| 16-bit | 4.4 GB |
| 8-bit | 2.2 GB |
| 4-bit | 1.1 GB |
Weights only, from the published parameter count; the key-value cache and runtime add to this.
Questions About Valen-2B
How much GPU memory does Valen-2B need?
About 5.3 GB at 16-bit and 1.3 GB at 4-bit: the weights (2.2B parameters) plus a working margin. A long context needs more.
What is the cheapest GPU to run Valen-2B on?
At 16-bit, 1x MI300X from $1.85 an hour; at 4-bit, 1x MI300X from $1.85 an hour, at the lowest on-demand prices the SAVRN Index lists.
Can I use Valen-2B commercially?
Yes. Valen-2B is released under Apache License 2.0. The Apache License 2.0 is a permissive open-source license. It permits commercial use, modification and redistribution. It requires keeping the license and copyright notices and any NOTICE file, stating significant changes, and it includes an express patent grant from contributors.
What is Valen-2B's context length?
262,144 tokens, from the maximum position embeddings in its published configuration.
Similar Models
This is the model card of a transformers model that has been pushed on the Hub. Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. Use the code below to get started with the model. Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).
The Label baseline of the DN-MOPD paper at Qwen3.5-2B continued to 160 updates (paper Table 5): multi-teacher on-policy distillation with label routing (each prompt is scored by the expert of its domain, every domain multiplier is 1). Released for comparison with DN-MOPD-Qwen3.5-2B; it is not the proposed method. Paper: Beyond Teacher Assignment: Domain-Normalized Multi-Teacher On-Policy Distillation (arXiv:2609.35347, project page) · Code: github.com/LiXin97/DN-MOPD The full recipe, with the launch scripts for every row of the paper's tables, is in recipes/qwen3.5/ and docs/recipe.md. This model was trained and evaluated with the non-thinking chat format. Pass enablethinking=False to the…
A Qwen3.5-2B student trained with DN-MOPD (Domain-Normalized Multi-Teacher On-Policy Distillation) continued to 160 updates (paper Table 5). Three same-size RL experts (math, code, instruction following) teach one student on its own responses; each prompt is scored by the expert of its domain, and DN-MOPD rescales each domain's token-level feedback by its measured spread, wd = clip(σall / σd, 0.25, 4), so that no domain dominates the shared update. Paper: Beyond Teacher Assignment: Domain-Normalized Multi-Teacher On-Policy Distillation (arXiv:2609.35347, project page) · Code: github.com/LiXin97/DN-MOPD The full recipe, with the launch scripts for every row of the paper's tables, is in…
This model was converted to MLX format from Qwen/Qwen3.5-2B using mlx-vlm version 0.3.12. Refer to the original model card for more details on the model.
We're excited to unveil Qwen2-VL, the latest iteration of our Qwen-VL model, representing nearly a year of innovation. SoTA understanding of images of various resolution & ratio: Qwen2-VL achieves state-of-the-art performance on visual understanding benchmarks, including MathVista, DocVQA, RealWorldQA, MTVQA, etc. Understanding videos of 20min+: Qwen2-VL can understand videos over 20 minutes for high-quality video-based question answering, dialog, content creation, etc. Agent that can operate your mobiles, robots, etc.: with the abilities of complex reasoning and decision making, Qwen2-VL can be integrated with devices like mobile phones, robots, etc., for automatic operation based on…