Ovis-VL-Embedding-9B is a high-capacity vision-language embedding model for text, images, visual documents, video, and interleaved multimodal inputs. It maps every supported input type into one coherent representation space, enabling high-accuracy cross-modal retrieval with a single encoder.
The model is initialized from Qwen3.5-9B. It retains the native text and vision encoders together with the shared multimodal language backbone, removes the language-modeling head, and directly uses the final-layer hidden state at the last non-padding token as the retrieval embedding. No modality-specific projection head is added.
Ovis-VL-Embedding-9B is designed for high-quality multimodal search, multimodal RAG, image and visual-document retrieval, video search and temporal localization, recommendation, and nearest-neighbor matching.
Architecture
Figure 2 from the technical report. Ovis-VL-Embedding-9B encodes text, images, visual documents, and sampled video frames as one interleaved sequence. The final-layer hidden state at the last non-padding token becomes the retrieval embedding, with no modality-specific projection heads.
The Qwen3.5-9B backbone contains 32 language layers with hidden size 4096. Its hybrid stack repeats three Gated DeltaNet layers followed by one gated full-attention layer, combining efficient long-context processing with periodic global token interaction. Multimodal positional encoding preserves temporal and two-dimensional spatial coordinates for visual tokens.
Training
Training follows three stages:
- Multimodal contrastive pretraining. Large-scale, multi-task data establish broad alignment across text, image, document, and video inputs. The objective combines difficulty-aware focal contrastive learning with similarity-distribution distillation.
- Full-parameter homogeneous finetuning. High-quality downstream data refine fine-grained discrimination. Each micro-batch is drawn from one dataset so that gathered candidates form task-consistent negatives.
- Annealing Embedding Distillation. Teacher-correct examples are retained, unresolved student examples are emphasized, and confidence-adaptive forward-KL supervision transfers complementary expert capabilities.
Retrieval interface
Ovis-VL-Embedding-9B is a bi-encoder, not a cross-encoder:
- Pair the query with the task instruction and format it with the native processor and chat template.
- Encode queries and candidates independently.
- Extract the final-layer hidden state at the last non-padding token.
- L2-normalize the 4096-dimensional query and candidate embeddings.
- Rank candidates by cosine similarity, equivalently the dot product of the normalized vectors.
No answer generation or query-candidate cross-attention is used during retrieval. Classification labels, passages, images, documents, videos, and interleaved multimodal items are all treated as candidates in the same embedding space.
Performance
MMEB-v2
MMEB-v2 evaluates vision-language embeddings over 78 datasets spanning image, video, and visual-document tasks. Ovis-VL-Embedding-9B achieves 81.13 overall, outperforming the strongest compared baseline by 1.04 points.
| Group |
Ovis-VL-Embedding-9B |
Best compared baseline |
Result |
| Image |
83.96 |
81.86 |
+2.10 |
| Video |
72.90 |
75.95 |
-3.05 |
| Visual document |
83.06 |
82.38 |
+0.68 |
| All 78 datasets |
81.13 |
80.09 |
+1.04 |
The model ranks first on all four image sub-tasks, video classification, video moment retrieval, the visual-document aggregate, and ViDoRe-V1. Scaling from 2B to 9B improves the overall score by 3.67 points, with the largest gains on video question answering (+7.64), video moment retrieval (+7.23), and video retrieval (+5.74).
Scores are percentages and higher is better. Red marks the best result in each row, underlining marks the second best, and Ovis scores are bold. The overall score is the unweighted average over all 78 MMEB-v2 datasets.
Embedding dimensions
The native output width is 4096, inherited directly from the Qwen3.5-9B backbone because no embedding projection head is added. Queries and candidates must use the same preprocessing, pooling rule, dimensionality, and L2 normalization.
Intended use
The model is intended for embedding extraction and retrieval over supported unimodal or interleaved multimodal content, including:
- high-accuracy semantic and cross-modal search;
- text-to-image, image-to-text, and image-to-image retrieval;
- multimodal RAG indexing and retrieval;
- visual-document and page retrieval;
- text-to-video, video retrieval, and temporal localization;
- recommendation and nearest-neighbor matching.
Limitations
- This checkpoint does not natively support audio input. Use Ovis-Embedding-Omni-3B for audio and general omni-modal retrieval.
- This checkpoint produces retrieval embeddings; it is not intended as a text or image generation model.
- Retrieval quality depends on task-appropriate query instructions and the native preprocessing and chat template.
- Performance varies by task. The reported model trails the strongest specialist baseline on the aggregate video score, video question answering, general video retrieval, ViDoRe-V2, VisRAG, and VisDoc-OOD.
- The 4096-dimensional output and 9B-scale backbone require more memory, storage, and inference compute than the 2B variant.
- Benchmark scores may not directly predict performance on a new domain. Evaluate with representative queries, candidates, and retrieval metrics before deployment.
Citation
If you find our embedding models useful, please consider citing our technical report:
@article{ovisembedding2026,
title = {Ovis-Embedding: Pushing the Frontiers of Universal Omni-Modal Embeddings},
author = {{Ovis-Embedding Team}},
journal = {arXiv preprint arXiv:2609.25165},
year = {2026},
url = {https://arxiv.org/abs/2609.25165}
}
License
This model is released under the Apache 2.0 license.
Resources
This repository contains the weights for Ovis-VL-Embedding-9B.
