SAVRN
Search Contact SAVRN

Open-weight model · Image and text to text

blip2-flan-t5-xl

by Salesforce AI Research Salesforce/blip2-flan-t5-xl

BLIP-2 model, leveraging Flan T5-xl (a large language model). It was introduced in the paper BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models by Li et al. and first released in this repository.

Parameters3.9B
Context—
Weights31.5 GB
Licensemit
AccessOpen weights
Monthly Downloads115k

Runs On

What it takes to serve blip2-flan-t5-xl (3.9B parameters): the memory its weights need at each precision, and the cheapest way to rent enough data-center GPUs to hold them.

PrecisionWeightsMemory neededCheapest setupPer hourAlso fits
16-bit 7.9 GB 9.5 GB 1x MI300X (192 GB)
Vultr		 $1.85 1x H100 $1.99 · 1x MI325X $2.00
8-bit 3.9 GB 4.7 GB 1x MI300X (192 GB)
Vultr		 $1.85 1x H100 $1.99 · 1x MI325X $2.00
4-bit 2.0 GB 2.4 GB 1x MI300X (192 GB)
Vultr		 $1.85 1x H100 $1.99 · 1x MI325X $2.00

Memory is the weights at that precision plus 20% for the runtime and a short context; a long context needs more. Prices are the lowest on-demand hourly rates in the SAVRN Index, read Sep 18, 2026.

Model Card

By Salesforce AI Research, published under mit, revision 0eb0d3b46c14.

BLIP-2, Flan T5-xl, pre-trained only

BLIP-2 model, leveraging Flan T5-xl (a large language model). It was introduced in the paper BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models by Li et al. and first released in this repository.

Disclaimer: The team releasing BLIP-2 did not write a model card for this model so this model card has been written by the Hugging Face team.

Model description

BLIP-2 consists of 3 models: a CLIP-like image encoder, a Querying Transformer (Q-Former) and a large language model.

The authors initialize the weights of the image encoder and large language model from pre-trained checkpoints and keep them frozen while training the Querying Transformer, which is a BERT-like Transformer encoder that maps a set of "query tokens" to query embeddings, which bridge the gap between the embedding space of the image encoder and the large language model.

Read the full model card (722 words)

Configuration

Architecture
Blip2ForConditionalGeneration
Vocabulary size
32,128
Stored precision
float32
Model type
blip-2

Identity and Version

Repository
Salesforce/blip2-flan-t5-xl
Publisher
Salesforce AI Research
Task
Image and text to text
Modality
Image and text
Library
transformers
Parameters
3.9B parameters
Languages
en
Revision
0eb0d3b46c14c1f8c7680bca2693baafdb90bb28
First published
2023-02-06
Last updated
2025-02-03

Files and Weights

17 files, 31.5 GB in total. The weights are 4 files totalling 31.5 GB in bin, safetensors.

Weights4 files · 31.5 GB
Configuration8 files · 261.4 KB
Tokenizer3 files · 3.2 MB
Documentation1 file · 7.2 KB
Repository1 file · 1.5 KB
Every file
FileTypeSizeSHA-256
model-00001-of-00002.safetensorsWeights10.0 GB 23bb7a7a7864
model-00002-of-00002.safetensorsWeights5.8 GB c77bd5bcdbc0
pytorch_model-00001-of-00002.binWeights9.4 GB 78da3eebf473
pytorch_model-00002-of-00002.binWeights6.3 GB afef3d90827b
added_tokens.jsonConfiguration23 B —
config.jsonConfiguration2.2 KB —
generation_config.jsonConfiguration168 B —
model.safetensors.index.jsonConfiguration127.9 KB —
preprocessor_config.jsonConfiguration432 B —
processor_config.jsonConfiguration68 B —
pytorch_model.bin.index.jsonConfiguration128.1 KB —
special_tokens_map.jsonConfiguration2.5 KB —
README.mdDocumentation7.2 KB —
.gitattributesRepository1.5 KB —
spiece.modelTokenizer791.7 KB d60acb128cf7
tokenizer.jsonTokenizer2.4 MB —
tokenizer_config.jsonTokenizer21.0 KB —

License and Download

License
mit
Access
Open weights, no gate
Download size
31.5 GB
Download from Salesforce AI Research

Released by Salesforce AI Research through its official repository on Hugging Face. Read the license.

Built From

Memory Requirements

PrecisionWeights in memory
As published31.5 GB
16-bit7.9 GB
8-bit3.9 GB
4-bit2.0 GB

Weights only, from the published parameter count; the key-value cache and runtime add to this.

Questions About blip2-flan-t5-xl

How much GPU memory does blip2-flan-t5-xl need?

About 9.5 GB at 16-bit and 2.4 GB at 4-bit: the weights (3.9B parameters) plus a working margin. A long context needs more.

What is the cheapest GPU to run blip2-flan-t5-xl on?

At 16-bit, 1x MI300X from $1.85 an hour; at 4-bit, 1x MI300X from $1.85 an hour, at the lowest on-demand prices the SAVRN Index lists.

Can I use blip2-flan-t5-xl commercially?

Yes. blip2-flan-t5-xl is released under MIT License. The MIT License is a short permissive license. It permits commercial use, modification and redistribution, provided the copyright notice and permission notice are included.

Similar Models

Model · Image and text to text

LocateAnything-3B

NVIDIA

LocateAnything is a vision-language model for fast and high-quality visual grounding, enabling precise object localization, dense detection, and point-based localization across diverse domains in both Enterprise Intelligence and Physical AI. The model adopts a generalist design, supporting tasks such as referring expression grounding, multi-object detection, GUI element grounding, and text localization, with strong performance in complex and cluttered scenes. Its core innovation, Parallel Box Decoding (PBD), predicts complete bounding box coordinates in a single parallel step rather than autoregressive token-by-token decoding, improving efficiency while preserving geometric consistency.…

Open weights other 3.8B parameters 32,768 tokens transformers
View model

Model · Image and text to text

Rex-Omni

IDEA-Research

This model is Rex-Omni, a 3B-parameter Multimodal Large Language Model (MLLM) presented in the paper "Detect Anything via Next Point Prediction". It is compatible with the Hugging Face transformers library and is licensed under the IDEA License 1.0. src="https://img.shields.io/badge/RexOmni-Website-BADFDB?style=flat-square&logo=deno&logoColor=violet&color=BADFDB" alt="RexThinker Website" src="https://img.shields.io/badge/RexOmni-Paper-Red%25red?logo=arxiv&logoColor=red&color=yellow" alt="RexThinker Paper on arXiv" src="https://img.shields.io/badge/RexOmni-Weight-orange?logo=huggingface&logoColor=yellow" alt="RexThinker weight on Hugging Face"…

Open weights other 4.1B parameters 128,000 tokens transformers
View model

Model · Image and text to text

Qwen2.5-VL-3B-Instruct

Qwen

licensename: qwen-research licenselink: https://huggingface.co/Qwen/Qwen2.5-VL-3B-Instruct/blob/main/LICENSE pipelinetag: image-text-to-text - multimodal libraryname: transformers In the past five months since Qwen2-VL’s release, numerous developers have built new models on the Qwen2-VL vision-language models, providing us with valuable feedback. During this period, we focused on building more useful vision-language models. Today, we are excited to introduce the latest addition to the Qwen family: Qwen2.5-VL. Understanding long videos and capturing events: Qwen2.5-VL can comprehend videos of over 1 hour, and this time it has a new ability of cpaturing event by pinpointing the relevant video…

Open weights 3.8B parameters 128,000 tokens transformers
View model

Model · Image and text to text

Nanonets-OCR2-3B

Nanonets

Nanonets-OCR2: A model for transforming documents into structured markdown with intelligent content recognition and semantic tagging Nanonets-OCR2 by Nanonets is a family of powerful, state-of-the-art image-to-markdown OCR models that go far beyond traditional text extraction. It transforms documents into structured markdown with intelligent content recognition and semantic tagging, making it ideal for downstream processing by Large Language Models (LLMs). Nanonets-OCR2 is packed with features designed to handle complex documents with ease: 1. Start the vLLM server. Check out Docstrange for more details. 1. Increasing the image resolution will improve model's performance. 2. For complex…

Open weights 3.8B parameters 128,000 tokens transformers
View model

SwiftVLN: training and evaluation code for these checkpoints. - SatNav: satellite-image navigation environments, datasets, and evaluation tools. This checkpoint is designed for SwiftVLN on SatNav, the continuous-state vision-and-language navigation benchmark over satellite imagery. It predicts short navigation-action sequences from an instruction, the current RGB window, and sampled visual memory. - swiftvln-satnav: SwiftVLN trained and evaluated on SatNav - 3b: Qwen2.5-VL 3B backbone - 1ep: trained for one epoch - f32s4: uses a 32-frame window and predicts four actions - overlap0: uses non-overlapping training windows - pf-h8: uses per-frame memory with up to eight history frames…

Open weights 3.8B parameters 128,000 tokens transformers
View model

Model · Image and text to text

blip2-opt-2.7b

Salesforce AI Research

BLIP-2 model, leveraging OPT-2.7b (a large language model with 2.7 billion parameters). It was introduced in the paper BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models by Li et al. and first released in this repository. Disclaimer: The team releasing BLIP-2 did not write a model card for this model so this model card has been written by the Hugging Face team. BLIP-2 consists of 3 models: a CLIP-like image encoder, a Querying Transformer (Q-Former) and a large language model. The authors initialize the weights of the image encoder and large language model from pre-trained checkpoints and keep them frozen while training the Querying…

Open weights mit 3.7B parameters transformers
View model