Skip to page content
SAVRN
Search Plan a project

Open-weight model · Image and text to text

blip2-flan-t5-xl

by Salesforce AI Research Salesforce/blip2-flan-t5-xl

blip2-flan-t5-xl is an open-weight model for image and text to text from Salesforce AI Research, released under MIT License. It has 3.9B parameters. At 16-bit it needs about 9.5 GB of GPU memory, which fits on 1x MI300X from $1.85 an hour, at the lowest prices in the SAVRN Index. It draws 115k downloads a month.

BLIP-2 model, leveraging Flan T5-xl (a large language model). It was introduced in the paper BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models by Li et al. and first released in this repository.

Parameters3.9B
Context—
Weights31.5 GB
Licensemit
AccessOpen weights
Monthly Downloads115k

Runs On

What it takes to serve blip2-flan-t5-xl (3.9B parameters): the memory its weights need at each precision, and the cheapest way to rent enough data-center GPUs to hold them.

PrecisionWeightsMemory neededCheapest setupPer hourAlso fits
16-bit 7.9 GB 9.5 GB 1x MI300X (192 GB)
Vultr
$1.85 1x H100 $1.99 · 1x MI325X $2.00
8-bit 3.9 GB 4.7 GB 1x MI300X (192 GB)
Vultr
$1.85 1x H100 $1.99 · 1x MI325X $2.00
4-bit 2.0 GB 2.4 GB 1x MI300X (192 GB)
Vultr
$1.85 1x H100 $1.99 · 1x MI325X $2.00

Memory is the weights at that precision plus 20% for the runtime and a short context; a long context needs more. Prices are the lowest on-demand hourly rates in the SAVRN Index, read Oct 16, 2026.

blip2-flan-t5-xl on every accelerator the SAVRN Index prices, at every precision

SAVRN's Notes on blip2-flan-t5-xl

BLIP-2 was an influential design: a frozen image encoder and a frozen language model joined by a small trained bridge. This checkpoint pairs that bridge with Flan-T5 XL, 3.9B parameters in total, to caption images and answer questions about them.

It is MIT licensed, so commercial use is allowed. It is also from early 2023, and newer vision-language models in our catalog answer questions about images far better at similar size. Keep it for existing pipelines and research comparisons. For new work, start with a current image-and-text model and use this one as a baseline.

Model Card

By Salesforce AI Research, published under mit, revision 0eb0d3b46c14.

BLIP-2, Flan T5-xl, pre-trained only

BLIP-2 model, leveraging Flan T5-xl (a large language model). It was introduced in the paper BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models by Li et al. and first released in this repository.

Disclaimer: The team releasing BLIP-2 did not write a model card for this model so this model card has been written by the Hugging Face team.

Model description

BLIP-2 consists of 3 models: a CLIP-like image encoder, a Querying Transformer (Q-Former) and a large language model.

The authors initialize the weights of the image encoder and large language model from pre-trained checkpoints and keep them frozen while training the Querying Transformer, which is a BERT-like Transformer encoder that maps a set of "query tokens" to query embeddings, which bridge the gap between the embedding space of the image encoder and the large language model.

Read the full model card (722 words)

Configuration

Architecture
Blip2ForConditionalGeneration
Vocabulary size
32,128
Stored precision
float32
Model type
blip-2

Identity and Version

Repository
Salesforce/blip2-flan-t5-xl
Publisher
Salesforce AI Research
Task
Image and text to text
Modality
Image and text
Library
transformers
Parameters
3.9B parameters
Languages
en
Revision
0eb0d3b46c14c1f8c7680bca2693baafdb90bb28
First published
2023-02-06
Last updated
2025-02-03

Files and Weights

17 files, 31.5 GB in total. The weights are 4 files totalling 31.5 GB in bin, safetensors.

Weights4 files · 31.5 GB
Configuration8 files · 261.4 KB
Tokenizer3 files · 3.2 MB
Documentation1 file · 7.2 KB
Repository1 file · 1.5 KB
Every file
FileTypeSizeSHA-256
model-00001-of-00002.safetensorsWeights10.0 GB 23bb7a7a7864
model-00002-of-00002.safetensorsWeights5.8 GB c77bd5bcdbc0
pytorch_model-00001-of-00002.binWeights9.4 GB 78da3eebf473
pytorch_model-00002-of-00002.binWeights6.3 GB afef3d90827b
added_tokens.jsonConfiguration23 B —
config.jsonConfiguration2.2 KB —
generation_config.jsonConfiguration168 B —
model.safetensors.index.jsonConfiguration127.9 KB —
preprocessor_config.jsonConfiguration432 B —
processor_config.jsonConfiguration68 B —
pytorch_model.bin.index.jsonConfiguration128.1 KB —
special_tokens_map.jsonConfiguration2.5 KB —
README.mdDocumentation7.2 KB —
.gitattributesRepository1.5 KB —
spiece.modelTokenizer791.7 KB d60acb128cf7
tokenizer.jsonTokenizer2.4 MB —
tokenizer_config.jsonTokenizer21.0 KB —

License and Download

License
mit
Access
Open weights, no gate
Download size
31.5 GB
Download from Salesforce AI Research

Released by Salesforce AI Research through its official repository on Hugging Face. Read the license.

Built From

Memory Requirements

PrecisionWeights in memory
As published31.5 GB
16-bit7.9 GB
8-bit3.9 GB
4-bit2.0 GB

Weights only, from the published parameter count; the key-value cache and runtime add to this.

Questions About blip2-flan-t5-xl

How much GPU memory does blip2-flan-t5-xl need?

About 9.5 GB at 16-bit and 2.4 GB at 4-bit: the weights (3.9B parameters) plus a working margin. A long context needs more.

What is the cheapest GPU to run blip2-flan-t5-xl on?

At 16-bit, 1x MI300X from $1.85 an hour; at 4-bit, 1x MI300X from $1.85 an hour, at the lowest on-demand prices the SAVRN Index lists.

Can I use blip2-flan-t5-xl commercially?

Yes. blip2-flan-t5-xl is released under MIT License. The MIT License is a short permissive license. It permits commercial use, modification and redistribution, provided the copyright notice and permission notice are included.

Similar Models

Model · Image and text to text

LocateAnything-3B

NVIDIA

LocateAnything is a vision-language model for fast and high-quality visual grounding, enabling precise object localization, dense detection, and point-based localization across diverse domains in both Enterprise Intelligence and Physical AI. The model adopts a generalist design, supporting tasks such as referring expression grounding, multi-object detection, GUI element grounding, and text localization, with strong performance in complex and cluttered scenes. Its core innovation, Parallel Box Decoding (PBD), predicts complete bounding box coordinates in a single parallel step rather than autoregressive token-by-token decoding, improving efficiency while preserving geometric consistency.…

Open weights other 3.8B parameters 32,768 tokens transformers

Model · Image and text to text

Rex-Omni

IDEA-Research

This model is Rex-Omni, a 3B-parameter Multimodal Large Language Model (MLLM) presented in the paper "Detect Anything via Next Point Prediction". It is compatible with the Hugging Face transformers library and is licensed under the IDEA License 1.0. src="https://img.shields.io/badge/RexOmni-Website-BADFDB?style=flat-square&logo=deno&logoColor=violet&color=BADFDB" alt="RexThinker Website" src="https://img.shields.io/badge/RexOmni-Paper-Red%25red?logo=arxiv&logoColor=red&color=yellow" alt="RexThinker Paper on arXiv" src="https://img.shields.io/badge/RexOmni-Weight-orange?logo=huggingface&logoColor=yellow" alt="RexThinker weight on Hugging Face"…

Open weights other 4.1B parameters 128,000 tokens transformers

Model · Image and text to text

ViSkill-FrozenLake

Hongxing Li

This repository contains ViSkill-FrozenLake and its visual skill library, introduced in ViSkill: Reinforcing VLM Agents with Evolving Visual-Native Skills. ViSkill-FrozenLake is a visual agent model for FrozenLake, built upon Qwen2.5-VL-3B-Instruct. It represents successful interactions as reusable visual skill cards. Geometry-aware retrieval guides the agent, while PPO with skill-guided rewards and online skill distillation jointly improve the policy and skill library. Model files are stored at the repository root, with the accompanying visual skill library in skilllibrary/. For the full ViSkill agent pipeline, including skill retrieval and environment interaction, please refer to our code…

Open weights apache-2.0 4.1B parameters 128,000 tokens transformers

Model · Image and text to text

ViSkill-Sokoban

Hongxing Li

This repository contains ViSkill-Sokoban and its visual skill library, introduced in ViSkill: Reinforcing VLM Agents with Evolving Visual-Native Skills. ViSkill-Sokoban is a visual agent model for Sokoban, built upon Qwen2.5-VL-3B-Instruct. It represents successful interactions as reusable visual skill cards. Geometry-aware retrieval guides the agent, while PPO with skill-guided rewards and online skill distillation jointly improve the policy and skill library. Model files are stored at the repository root, with the accompanying visual skill library in skilllibrary/. For the full ViSkill agent pipeline, including skill retrieval and environment interaction, please refer to our code…

Open weights 4.1B parameters 128,000 tokens transformers

Model · Image and text to text

Qwen2.5-VL-3B-Instruct

Qwen

licensename: qwen-research licenselink: https://huggingface.co/Qwen/Qwen2.5-VL-3B-Instruct/blob/main/LICENSE pipelinetag: image-text-to-text - multimodal libraryname: transformers In the past five months since Qwen2-VL’s release, numerous developers have built new models on the Qwen2-VL vision-language models, providing us with valuable feedback. During this period, we focused on building more useful vision-language models. Today, we are excited to introduce the latest addition to the Qwen family: Qwen2.5-VL. Understanding long videos and capturing events: Qwen2.5-VL can comprehend videos of over 1 hour, and this time it has a new ability of cpaturing event by pinpointing the relevant video…

Open weights 3.8B parameters 128,000 tokens transformers

Model · Image and text to text

Nanonets-OCR2-3B

Nanonets

Nanonets-OCR2: A model for transforming documents into structured markdown with intelligent content recognition and semantic tagging Nanonets-OCR2 by Nanonets is a family of powerful, state-of-the-art image-to-markdown OCR models that go far beyond traditional text extraction. It transforms documents into structured markdown with intelligent content recognition and semantic tagging, making it ideal for downstream processing by Large Language Models (LLMs). Nanonets-OCR2 is packed with features designed to handle complex documents with ease: 1. Start the vLLM server. Check out Docstrange for more details. 1. Increasing the image resolution will improve model's performance. 2. For complex…

Open weights 3.8B parameters 128,000 tokens transformers