source languages: nl; target languages: en; OPUS readme: nl-en; dataset: opus; model: transformer-align; pre-processing: normalization + SentencePiece.
plT5 models are T5-based language models trained on Polish corpora. The models were optimized for the original T5 denoising target.
Parameters—
Context—
Weights3.3 GB
Licensecc-by-4.0
AccessOpen weights
Monthly Downloads1.6k
Model Card
By Allegro ML Research, published under cc-by-4.0, revision 9fa2adf614ed.
plT5 models are T5-based language models trained on Polish corpora. The models were optimized for the original T5 denoising target. plT5 was trained on six different corpora available for Polish language: The training dataset was tokenized into subwords using a sentencepiece unigram model with vocabulary size of 50k tokens. If you use this model, please cite the following paper: The model was trained by Machine Learning Research Team at Allegro and Linguistic Engineering Group at Institute of Computer Science, Polish Academy of Sciences. You can contact us at: [email protected]
Read Allegro ML Research's full model card
plT5 models are T5-based language models trained on Polish corpora. The models were optimized for the original T5 denoising target.
Corpus
plT5 was trained on six different corpora available for Polish language:
| Corpus | Tokens | Documents |
|---|---|---|
| CCNet Middle | 3243M | 7.9M |
| CCNet Head | 2641M | 7.0M |
| National Corpus of Polish | 1357M | 3.9M |
| Open Subtitles | 1056M | 1.1M |
| Wikipedia | 260M | 1.4M |
| Wolne Lektury | 41M | 5.5k |
Tokenizer
The training dataset was tokenized into subwords using a sentencepiece unigram model with vocabulary size of 50k tokens.
Usage
Example code:
from transformers import AutoTokenizer, AutoModel
tokenizer = AutoTokenizer.from_pretrained("allegro/plt5-large")
model = AutoModel.from_pretrained("allegro/plt5-large")
License
CC BY 4.0
Citation
If you use this model, please cite the following paper:
@article{chrabrowa2022evaluation,
title={Evaluation of Transfer Learning for Polish with a Text-to-Text Model},
author={Chrabrowa, Aleksandra and Dragan, {\L}ukasz and Grzegorczyk, Karol and Kajtoch, Dariusz and Koszowski, Miko{\l}aj and Mroczkowski, Robert and Rybak, Piotr},
journal={arXiv preprint arXiv:2205.08808},
year={2022}
}
Authors
The model was trained by Machine Learning Research Team at Allegro and Linguistic Engineering Group at Institute of Computer Science, Polish Academy of Sciences.
You can contact us at: [email protected]
Configuration
- Architecture
- T5ForConditionalGeneration
- Vocabulary size
- 50,048
- Model type
- t5
Identity and Version
- Repository
- allegro/plt5-large
- Publisher
- Allegro ML Research
- Task
- Translation
- Modality
- Text
- Library
- transformers
- Parameters
- Not stated by the source
- Languages
- pl
- Revision
- 9fa2adf614ed3b2392b961e2fd3e5153e082726a
- First published
- 2022-03-02
- Last updated
- 2022-08-03
Files and Weights
7 files, 3.3 GB in total. The weights are 1 file totalling 3.3 GB in bin.
Weights1 file · 3.3 GB
Configuration2 files · 725 B
Tokenizer2 files · 1.1 MB
Documentation1 file · 2.1 KB
Repository1 file · 737 B
Every file
| File | Type | Size | SHA-256 |
|---|---|---|---|
| pytorch_model.bin | Weights | 3.3 GB | 92911fb63486 |
| config.json | Configuration | 660 B | — |
| special_tokens_map.json | Configuration | 65 B | — |
| README.md | Documentation | 2.1 KB | — |
| .gitattributes | Repository | 737 B | — |
| spiece.model | Tokenizer | 1.1 MB | a200b4b9d710 |
| tokenizer_config.json | Tokenizer | 141 B | — |
License and Download
- License
- cc-by-4.0
- Access
- Open weights, no gate
- Download size
- 3.3 GB
Download from Allegro ML Research
Released by Allegro ML Research through its official repository on Hugging Face. Read the license.
Built From
- Described by arXiv:2205.08808
Memory Requirements
| Precision | Weights in memory |
|---|---|
| As published | 3.3 GB |
Weights only, from the published parameter count; the key-value cache and runtime add to this.
Questions About plt5-large
Can I use plt5-large commercially?
Yes. plt5-large is released under Creative Commons Attribution 4.0. CC BY 4.0 permits sharing and adapting the work, including commercially, provided the creator is credited and changes are indicated.
Similar Models
This is the model card of NLLB-200's distilled 600M variant. Here are the metrics for that particular checkpoint. - Information about training algorithms, parameters, fairness constraints or other applied approaches, and features. The exact training algorithm, data and the strategies to handle data imbalances for high and low resource languages that were used to train NLLB-200 is described in the paper. - Paper or other resource for more information NLLB Team et al, No Language Left Behind: Scaling Human-Centered Machine Translation, Arxiv, 2022 - Where to send questions or comments about the model: https://github.com/facebookresearch/fairseq/issues • Model performance measures: NLLB-200…
source languages: en; target languages: ru; OPUS readme: en-ru; dataset: opus; model: transformer-align; pre-processing: normalization + SentencePiece.
This model can be used for translation and text-to-text generation. CONTENT WARNING: Readers should be aware this section contains content that is disturbing, offensive, and can propagate historical and current stereotypes. Significant research has explored bias and fairness issues with language models (see, e.g., Sheng et al. (2021) and Bender et al. (2021)). Further details about the dataset for this model can be found in the OPUS readme: en-de
hfname: kor-eng - sourcelanguages: kor - targetlanguages: eng - opusreadmeurl: https://github.com/Helsinki-NLP/Tatoeba-Challenge/tree/master/models/kor-eng/README.md - originalrepo: Tatoeba-Challenge - srcconstituents: {'korHani', 'korHang', 'korLatn', 'kor'} - tgtconstituents: {'eng'} - srcmultilingual: False - tgtmultilingual: False - urlmodel: https://object.pouta.csc.fi/Tatoeba-MT-models/kor-eng/opus-2020-06-17.zip - urltestset: https://object.pouta.csc.fi/Tatoeba-MT-models/kor-eng/opus-2020-06-17.test.txt - srcalpha3: kor - tgtalpha3: eng - shortpair: ko-en - chrF2score: 0.588 - brevitypenalty: 0.9590000000000001 - reflen: 17711.0 - srcname: Korean - tgtname: English - traindate…
source languages: de; target languages: en; OPUS readme: de-en; dataset: opus; model: transformer-align; pre-processing: normalization + SentencePiece.