Pretrained model on protein sequences using a masked language modeling (MLM) objective. It was introduced in this paper and first released in this repository. This model is trained on uppercase amino acids: it only works with capital letter amino acids.
Model Card
Pretrained model on protein sequences using a masked language modeling (MLM) objective. It was introduced in this paper and first released in this repository. This model is trained on uppercase amino acids: it only works with capital letter amino acids. ProtT5-XL-UniRef50 is based on the t5-3b model and was pretrained on a large corpus of protein sequences in a self-supervised fashion. This means it was pretrained on the raw protein sequences only, with no humans labelling them in any way (which is why it can use lots of publicly available data) with an automatic process to generate inputs and labels from those protein sequences. One important difference between this T5 model and the…
Excerpt from the card by Rostlab.
Configuration
- Architecture
- T5ForConditionalGeneration
- Vocabulary size
- 128
- Model type
- t5
Identity and Version
- Repository
- Rostlab/prot_t5_xl_uniref50
- Publisher
- Rostlab
- Task
- Not stated by the source
- Modality
- Other
- Library
- transformers
- Parameters
- Not stated by the source
- Languages
- Not stated by the source
- Revision
- 973be27c52ee6474de9c945952a8008aeb2a1a73
- First published
- 2022-03-02
- Last updated
- 2023-01-31
Files and Weights
9 files, 33.8 GB in total. The weights are 3 files totalling 33.8 GB in bin.
Every file
| File | Type | Size | SHA-256 |
|---|---|---|---|
| pytorch_model.bin | Weights | 11.3 GB | c06141d42e93 |
| pytorch_model_600k.bin | Weights | 11.3 GB | e7f35dee1572 |
| pytorch_model_723k.bin | Weights | 11.3 GB | 6f2fb2f30a59 |
| config.json | Configuration | 546 B | — |
| special_tokens_map.json | Configuration | 1.8 KB | — |
| README.md | Documentation | 7.6 KB | — |
| .gitattributes | Repository | 690 B | — |
| spiece.model | Tokenizer | 238.0 KB | 74da7b4afcde |
| tokenizer_config.json | Tokenizer | 24 B | — |
License and Download
- License
- Not stated by the source
- Access
- Open weights, no gate
- Download size
- 33.8 GB
Released by Rostlab through its official repository on Hugging Face.
Built From
- Trained on (disclosed) UniRef50
Memory Requirements
| Precision | Weights in memory |
|---|---|
| As published | 33.8 GB |
Weights only, from the published parameter count; the key-value cache and runtime add to this.