GIT (short for GenerativeImage2Text) model, base-sized version. It was introduced in the paper GIT: A Generative Image-to-text Transformer for Vision and Language by Wang et al. and first released in this repository. Disclaimer: The team releasing GIT did not write a model card for this model so this model card has been written by the Hugging Face team. GIT is a Transformer decoder conditioned on both CLIP image tokens and text tokens. The model is trained using "teacher forcing" on a lot of (image, text) pairs. The goal for the model is simply to predict the next text token, giving the image tokens and previous text tokens. The model has full access to (i.e. a bidirectional attention mask…
MGP-STR base-sized model is trained on MJSynth and SynthText. It was introduced in the paper Multi-Granularity Prediction for Scene Text Recognition and first released in this repository.
Parameters148M
Context—
Weights1.2 GB
License—
AccessOpen weights
Monthly Downloads114.7k
Runs On
What it takes to serve mgp-str-base (148M parameters): the memory its weights need at each precision, and the cheapest way to rent enough data-center GPUs to hold them.
| Precision | Weights | Memory needed | Cheapest setup | Per hour | Also fits |
|---|---|---|---|---|---|
| 16-bit | 0.3 GB | 0.4 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
| 8-bit | 0.1 GB | 0.2 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
| 4-bit | 0.1 GB | 0.1 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
Memory is the weights at that precision plus 20% for the runtime and a short context; a long context needs more. Prices are the lowest on-demand hourly rates in the SAVRN Index, read Sep 18, 2026.
Model Card
MGP-STR base-sized model is trained on MJSynth and SynthText. It was introduced in the paper Multi-Granularity Prediction for Scene Text Recognition and first released in this repository. MGP-STR is pure vision STR model, consisting of ViT and specially designed A^3 modules. The ViT module was initialized from the weights of DeiT-base, except the patch embedding model, due to the inconsistent input size. Images (32x128) are presented to the model as a sequence of fixed-size patches (resolution 4x4), which are linearly embedded. One also adds absolute position embeddings before feeding the sequence to the layers of the ViT module. Next, A^3 module selects a meaningful combination from the…
Excerpt from the card by Alibaba DAMO Academy.
Configuration
- Architecture
- MGPSTRModel
- Layers
- 12
- Hidden size
- 768
- Attention heads
- 12
- Stored precision
- float32
- Model type
- mgp-str
Identity and Version
- Repository
- alibaba-damo/mgp-str-base
- Publisher
- Alibaba DAMO Academy
- Task
- Image to text
- Modality
- Image and text
- Library
- transformers
- Parameters
- 148M parameters
- Languages
- mgp-str
- Revision
- 5d06493b6b2a8c4c023d2c030175c03be30f4202
- First published
- 2022-11-23
- Last updated
- 2023-12-11
Files and Weights
7 files, 1.2 GB in total. The weights are 2 files totalling 1.2 GB in bin, safetensors.
Weights2 files · 1.2 GB
Configuration2 files · 791 B
Tokenizer1 file · 415 B
Documentation1 file · 2.9 KB
Repository1 file · 1.5 KB
Every file
| File | Type | Size | SHA-256 |
|---|---|---|---|
| model.safetensors | Weights | 591.8 MB | edc2be05c194 |
| pytorch_model.bin | Weights | 591.9 MB | 5fd53e97eef1 |
| config.json | Configuration | 635 B | — |
| preprocessor_config.json | Configuration | 156 B | — |
| README.md | Documentation | 2.9 KB | — |
| .gitattributes | Repository | 1.5 KB | — |
| vocab.json | Tokenizer | 415 B | — |
License and Download
- License
- Not stated by the source
- Access
- Open weights, no gate
- Download size
- 1.2 GB
Download from Alibaba DAMO Academy
Released by Alibaba DAMO Academy through its official repository on Hugging Face.
Built From
- Described by arXiv:2209.03592
Memory Requirements
| Precision | Weights in memory |
|---|---|
| As published | 1.2 GB |
| 16-bit | 0.3 GB |
| 8-bit | 0.1 GB |
| 4-bit | 0.1 GB |
Weights only, from the published parameter count; the key-value cache and runtime add to this.
Questions About mgp-str-base
How much GPU memory does mgp-str-base need?
About 0.4 GB at 16-bit and 0.1 GB at 4-bit: the weights (148M parameters) plus a working margin. A long context needs more.
What is the cheapest GPU to run mgp-str-base on?
At 16-bit, 1x MI300X from $1.85 an hour; at 4-bit, 1x MI300X from $1.85 an hour, at the lowest on-demand prices the SAVRN Index lists.
Similar Models
TrOCR model fine-tuned on the SROIE dataset. It was introduced in the paper TrOCR: Transformer-based Optical Character Recognition with Pre-trained Models by Li et al. and first released in this repository. The TrOCR model is an encoder-decoder model, consisting of an image Transformer as encoder, and a text Transformer as decoder. The image encoder was initialized from the weights of DeiT, while the text decoder was initialized from the weights of UniLM. Images are presented to the model as a sequence of fixed-size patches (resolution 16x16), which are linearly embedded. One also adds absolute position embeddings before feeding the sequence to the layers of the Transformer encoder. Next…
This model is the pretrained version of Pix2Struct, use this model for fine-tuning purposes only. Pix2Struct is an image encoder - text decoder model that is trained on image-text pairs for various tasks, including image captionning and visual question answering. The full list of available models can be found on the Table 1 of the paper: The abstract of the model states that: forms. Perhaps due to this diversity, previous work has typically relied on domainspecific recipes with limited sharing of the underlying data, model architectures, and objectives. We present Pix2Struct, a pretrained image-to-text model for purely visual language understanding, which can be finetuned on tasks…
captioning pretrained on COCO dataset - base architecture (with ViT base backbone). Authors from the paper write in the abstract: Vision-Language Pre-training (VLP) has advanced the performance for many vision-language tasks. However, most existing pre-trained models only excel in either understanding-based tasks or generation-based tasks. Furthermore, performance improvement has been largely achieved by scaling up the dataset with noisy image-text pairs collected from the web, which is a suboptimal source of supervision. In this paper, we propose BLIP, a new VLP framework which transfers flexibly to both vision-language understanding and generation tasks. BLIP effectively utilizes the…
Optical character recognition for Japanese text, with the main focus being Japanese manga. It uses Vision Encoder Decoder framework. Manga OCR can be used as a general purpose printed Japanese OCR, but its main goal was to provide a high quality text recognition, robust against various scenarios specific to manga: - both vertical and horizontal text - text with furigana - text overlaid on images - wide variety of fonts and font styles - low quality images Code is available here.
PP-OCRv5serverdet is one of the PP-OCRv5det series, the latest generation of text detection models developed by the PaddleOCR team. Designed for high-performance applications, it supports the detection of text in diverse scenarios—including handwriting, vertical, rotated, and curved text—across multiple languages such as Simplified Chinese, Traditional Chinese, English, and Japanese. Key features include robust handling of complex layouts, varying text sizes, and challenging backgrounds, making it suitable for practical applications like document analysis, license plate recognition, and scene text detection. The key accuracy metrics are as follow: Please refer to the following commands to…