The developers of the Text-To-Text Transfer Transformer (T5) write: T5-Small is the checkpoint with 60 million parameters. The developers write in a blog post that the model: See the blog post and research paper for further details. The model is pre-trained on the Colossal Clean Crawled Corpus (C4), which was developed and released in the context of the same research paper as T5. The model was pre-trained on a on a multi-task mixture of unsupervised (1.) and supervised tasks (2.). Thereby, the following datasets were being used for (1.) and (2.): 1. Datasets used for Unsupervised denoising objective: 2. Datasets used for Supervised text-to-text language modeling objective - CoLA Warstadt et…
Open weights
apache-2.0
61M parameters
transformers
The developers of the Text-To-Text Transfer Transformer (T5) write: T5-Base is the checkpoint with 220 million parameters. The developers write in a blog post that the model: See the blog post and research paper for further details. The model is pre-trained on the Colossal Clean Crawled Corpus (C4), which was developed and released in the context of the same research paper as T5. The model was pre-trained on a on a multi-task mixture of unsupervised (1.) and supervised tasks (2.). Thereby, the following datasets were being used for (1.) and (2.): 1. Datasets used for Unsupervised denoising objective: 2. Datasets used for Supervised text-to-text language modeling objective - CoLA Warstadt et…
Open weights
apache-2.0
223M parameters
transformers
source languages: fr; target languages: en; OPUS readme: fr-en; dataset: opus; model: transformer-align; pre-processing: normalization + SentencePiece.
Open weights
apache-2.0
75M parameters
512 tokens
transformers
The developers of the Text-To-Text Transfer Transformer (T5) write: T5-Large is the checkpoint with 770 million parameters. The developers write in a blog post that the model: See the blog post and research paper for further details. The model is pre-trained on the Colossal Clean Crawled Corpus (C4), which was developed and released in the context of the same research paper as T5. The model was pre-trained on a on a multi-task mixture of unsupervised (1.) and supervised tasks (2.). Thereby, the following datasets were being used for (1.) and (2.): 1. Datasets used for Unsupervised denoising objective: 2. Datasets used for Supervised text-to-text language modeling objective - CoLA Warstadt…
Open weights
apache-2.0
738M parameters
transformers
The developers of the Text-To-Text Transfer Transformer (T5) write: T5-3B is the checkpoint with 3 billion parameters. The developers write in a blog post that the model: See the blog post and research paper for further details. The model is pre-trained on the Colossal Clean Crawled Corpus (C4), which was developed and released in the context of the same research paper as T5. The model was pre-trained on a on a multi-task mixture of unsupervised (1.) and supervised tasks (2.). Thereby, the following datasets were being used for (1.) and (2.): 1. Datasets used for Unsupervised denoising objective: 2. Datasets used for Supervised text-to-text language modeling objective - CoLA Warstadt et…
Open weights
apache-2.0
2.9B parameters
transformers
This is the model card of IndicTrans2 En-Indic Distilled 200M variant. Please refer to section 7.6: Distilled Models in the TMLR submission for further details on model training, data and metrics. Please refer to the github repository for a detail description on how to use HF compatible IndicTrans2 models for inference. - New RoPE based IndicTrans2 models which are capable of handling sequence lengths upto 2048 tokens are available here - These models can be used by just changing the modelname parameter. Please read the model card of the RoPE-IT2 models for more information about the generation. - It is recommended to run these models with flashattention2 for efficient generation. If you…
Access requested at publisher
mit
275M parameters
transformers
Neural machine translation model for translating from Turkish (tr) to English (en). This model is part of the OPUS-MT project, an effort to make neural machine translation models widely available and accessible for many languages in the world. All models are originally trained using the amazing framework of Marian NMT, an efficient NMT implementation written in pure C++. The models have been converted to pyTorch using the transformers library by huggingface. Training data is taken from OPUS and training pipelines use the procedures of OPUS-MT-train. You can also use OPUS-MT models with the transformers pipelines, for example: The work is supported by the European Language Grid as pilot…
Open weights
cc-by-4.0
235M parameters
1,024 tokens
transformers
Neural machine translation model for translating from Korean (ko) to English (en). This model is part of the OPUS-MT project, an effort to make neural machine translation models widely available and accessible for many languages in the world. All models are originally trained using the amazing framework of Marian NMT, an efficient NMT implementation written in pure C++. The models have been converted to pyTorch using the transformers library by huggingface. Training data is taken from OPUS and training pipelines use the procedures of OPUS-MT-train. - More information about released models for this language pair: OPUS-MT kor-eng README - Tatoeba Translation…
Open weights
cc-by-4.0
209M parameters
1,024 tokens
transformers
This model is a fine-tuned checkpoint of mBART-large-50. mbart-large-50-many-to-many-mmt is fine-tuned for multilingual machine translation. It was introduced in Multilingual Translation with Extensible Multilingual Pretraining and Finetuning paper. The model can translate directly between any pair of 50 languages. To translate into a target language, the target language id is forced as the first generated token. To force the target language id as the first generated token, pass the forcedbostokenid parameter to the generate method. See the model hub to look for more fine-tuned versions. Arabic (arAR), Czech (csCZ), German (deDE), English (enXX), Spanish (esXX), Estonian (etEE), Finnish…
Open weights
611M parameters
1,024 tokens
transformers
MADLAD-400-3B-MT is a multilingual machine translation model based on the T5 architecture that was trained on 1 trillion tokens covering over 450 languages using publicly available data. It is competitive with models that are significantly larger. Disclaimer: Juarez Bochi, who was not involved in this research, converted the original weights and wrote the contents of this model card based on the original paper and Flan-T5. Find below some example scripts on how to use the model: First, install the Python packages that are required: pip install transformers accelerate sentencepiece Usage with candle: We also provide a quantized model (1.65 GB vs the original 11.8 GB file): See the research…
Open weights
apache-2.0
2.9B parameters
transformers
English ｜ 中文 Hy-MT2 is a family of “fast-thinking” multilingual translation models designed for complex real-world scenarios. It includes three model sizes: 1.8B, 7B, and 30B-A3B (MoE), all of which support translation among 33 languages and effectively follow translation instructions in multiple languages. For on-device deployment, AngelSlim 1.25-bit extreme quantization reduces the storage requirement of the 1.8B model to only 440 MB and improves inference speed by 1.5x. Multi-dimensional evaluations show that Hy-MT2 delivers outstanding performance across general, real-world business, domain-specific, and instruction-following translation tasks. The 7B and 30B-A3B models outperform…
Open weights
apache-2.0
2B parameters
262,144 tokens
transformers
source languages: nl; target languages: en; OPUS readme: nl-en; dataset: opus; model: transformer-align; pre-processing: normalization + SentencePiece.
Open weights
apache-2.0
512 tokens
transformers
This is the model card of NLLB-200's distilled 600M variant. Here are the metrics for that particular checkpoint. - Information about training algorithms, parameters, fairness constraints or other applied approaches, and features. The exact training algorithm, data and the strategies to handle data imbalances for high and low resource languages that were used to train NLLB-200 is described in the paper. - Paper or other resource for more information NLLB Team et al, No Language Left Behind: Scaling Human-Centered Machine Translation, Arxiv, 2022 - Where to send questions or comments about the model: https://github.com/facebookresearch/fairseq/issues • Model performance measures: NLLB-200…
Open weights
cc-by-nc-4.0
1,024 tokens
transformers
source languages: en; target languages: ru; OPUS readme: en-ru; dataset: opus; model: transformer-align; pre-processing: normalization + SentencePiece.
Open weights
apache-2.0
512 tokens
transformers
This model can be used for translation and text-to-text generation. CONTENT WARNING: Readers should be aware this section contains content that is disturbing, offensive, and can propagate historical and current stereotypes. Significant research has explored bias and fairness issues with language models (see, e.g., Sheng et al. (2021) and Bender et al. (2021)). Further details about the dataset for this model can be found in the OPUS readme: en-de
Open weights
cc-by-4.0
512 tokens
transformers
hfname: kor-eng - sourcelanguages: kor - targetlanguages: eng - opusreadmeurl: https://github.com/Helsinki-NLP/Tatoeba-Challenge/tree/master/models/kor-eng/README.md - originalrepo: Tatoeba-Challenge - srcconstituents: {'korHani', 'korHang', 'korLatn', 'kor'} - tgtconstituents: {'eng'} - srcmultilingual: False - tgtmultilingual: False - urlmodel: https://object.pouta.csc.fi/Tatoeba-MT-models/kor-eng/opus-2020-06-17.zip - urltestset: https://object.pouta.csc.fi/Tatoeba-MT-models/kor-eng/opus-2020-06-17.test.txt - srcalpha3: kor - tgtalpha3: eng - shortpair: ko-en - chrF2score: 0.588 - brevitypenalty: 0.9590000000000001 - reflen: 17711.0 - srcname: Korean - tgtname: English - traindate…
Open weights
apache-2.0
512 tokens
transformers
source languages: de; target languages: en; OPUS readme: de-en; dataset: opus; model: transformer-align; pre-processing: normalization + SentencePiece.
Open weights
apache-2.0
512 tokens
transformers
This model can be used for translation and text-to-text generation. CONTENT WARNING: Readers should be aware this section contains content that is disturbing, offensive, and can propagate historical and current stereotypes. Significant research has explored bias and fairness issues with language models (see, e.g., Sheng et al. (2021) and Bender et al. (2021)). Further details about the dataset for this model can be found in the OPUS readme: ru-en
Open weights
cc-by-4.0
512 tokens
transformers
A compact encoder–decoder Transformer trained from scratch on 10,000 Sanskrit–English pairs (NLU Assignment 2), with a joint SentencePiece BPE vocabulary. It is small and fast — meant to be efficient rather than to match large pretrained models. This is not a Transformers architecture, so it ships with a self-contained modeling.py. Files: pytorchmodel.bin (weights), spm.model (tokenizer), config.json (hyperparameters), modeling.py (model + load/translate helpers). pip install gradio first. To host it, create a Hugging Face Space (SDK: Gradio) with an app.py (the load + respond code) and a requirements.txt of torch sentencepiece huggingfacehub gradio. Trained only on the provided dataset…
Open weights
mit
This is the model card of NLLB-200's distilled 1.3B variant. Here are the metrics for that particular checkpoint. - Information about training algorithms, parameters, fairness constraints or other applied approaches, and features. The exact training algorithm, data and the strategies to handle data imbalances for high and low resource languages that were used to train NLLB-200 is described in the paper. - Paper or other resource for more information NLLB Team et al, No Language Left Behind: Scaling Human-Centered Machine Translation, Arxiv, 2022 - Where to send questions or comments about the model: https://github.com/facebookresearch/fairseq/issues • Model performance measures: NLLB-200…
Open weights
cc-by-nc-4.0
1,024 tokens
transformers
hfname: spa-eng - sourcelanguages: spa - targetlanguages: eng - opusreadmeurl: https://github.com/Helsinki-NLP/Tatoeba-Challenge/tree/master/models/spa-eng/README.md - originalrepo: Tatoeba-Challenge - srcconstituents: {'spa'} - tgtconstituents: {'eng'} - srcmultilingual: False - tgtmultilingual: False - urlmodel: https://object.pouta.csc.fi/Tatoeba-MT-models/spa-eng/opus-2020-08-18.zip - urltestset: https://object.pouta.csc.fi/Tatoeba-MT-models/spa-eng/opus-2020-08-18.test.txt - srcalpha3: spa - tgtalpha3: eng - shortpair: es-en - chrF2score: 0.7390000000000001 - brevitypenalty: 0.9740000000000001 - reflen: 79376.0 - srcname: Spanish - tgtname: English - traindate: 2020-08-18 00:00:00…
Open weights
apache-2.0
512 tokens
transformers
This is the model card of NLLB-200's 3.3B variant. Here are the metrics for that particular checkpoint. - Information about training algorithms, parameters, fairness constraints or other applied approaches, and features. The exact training algorithm, data and the strategies to handle data imbalances for high and low resource languages that were used to train NLLB-200 is described in the paper. - Paper or other resource for more information NLLB Team et al, No Language Left Behind: Scaling Human-Centered Machine Translation, Arxiv, 2022 - Where to send questions or comments about the model: https://github.com/facebookresearch/fairseq/issues • Model performance measures: NLLB-200 model was…
Open weights
cc-by-nc-4.0
1,024 tokens
transformers
source languages: en; target languages: fr; OPUS readme: en-fr; dataset: opus; model: transformer-align; pre-processing: normalization + SentencePiece.
Open weights
apache-2.0
512 tokens
transformers
This model can be used for translation and text-to-text generation. CONTENT WARNING: Readers should be aware this section contains content that is disturbing, offensive, and can propagate historical and current stereotypes. Significant research has explored bias and fairness issues with language models (see, e.g., Sheng et al. (2021) and Bender et al. (2021)). Further details about the dataset for this model can be found in the OPUS readme: zho-eng helsinkigitsha: 480fcbe0ee1bf4774bcbe6226ad9f58e63f6c535 transformersgitsha: 2207e5d8cb224e954a7cba69fa4ac2309e9ff30b portmachine: brutasse porttime: 2020-08-21-14:41 srcmultilingual: False tgtmultilingual: False reflen: 82826.0 brevitypenalty…
Open weights
cc-by-4.0
512 tokens
transformers
hfname: eng-spa - sourcelanguages: eng - targetlanguages: spa - opusreadmeurl: https://github.com/Helsinki-NLP/Tatoeba-Challenge/tree/master/models/eng-spa/README.md - originalrepo: Tatoeba-Challenge - srcconstituents: {'eng'} - tgtconstituents: {'spa'} - srcmultilingual: False - tgtmultilingual: False - urlmodel: https://object.pouta.csc.fi/Tatoeba-MT-models/eng-spa/opus-2020-08-18.zip - urltestset: https://object.pouta.csc.fi/Tatoeba-MT-models/eng-spa/opus-2020-08-18.test.txt - srcalpha3: eng - tgtalpha3: spa - shortpair: en-es - chrF2score: 0.721 - brevitypenalty: 0.978 - reflen: 77311.0 - srcname: English - tgtname: Spanish - traindate: 2020-08-18 00:00:00 - srcalpha2: en - tgtalpha2…
Open weights
apache-2.0
512 tokens
transformers
Neural machine translation model for translating from English (en) to Turkish (tr). This model is part of the OPUS-MT project, an effort to make neural machine translation models widely available and accessible for many languages in the world. All models are originally trained using the amazing framework of Marian NMT, an efficient NMT implementation written in pure C++. The models have been converted to pyTorch using the transformers library by huggingface. Training data is taken from OPUS and training pipelines use the procedures of OPUS-MT-train. You can also use OPUS-MT models with the transformers pipelines, for example: The work is supported by the European Language Grid as pilot…
Open weights
cc-by-4.0
1,024 tokens
transformers
source languages: ar; target languages: en; OPUS readme: ar-en; dataset: opus; model: transformer-align; pre-processing: normalization + SentencePiece.
Open weights
apache-2.0
512 tokens
transformers
source languages: it; target languages: en; OPUS readme: it-en; dataset: opus; model: transformer-align; pre-processing: normalization + SentencePiece.
Open weights
apache-2.0
512 tokens
transformers
This is mbart-large-cc25, finetuned on wmtenro. It scores BLEU 28.1 without post processing and BLEU 38 with postprocessing. Instructions in romanianpostprocessing.md Original Code: https://github.com/pytorch/fairseq/tree/master/examples/mbart Docs: https://huggingface.co/transformers/master/modeldoc/mbart.html
Open weights
mit
611M parameters
1,024 tokens
transformers
a sentence initial language token is required in the form of >>id<< (id = valid target language ID) - hfname: eng-zho - sourcelanguages: eng - targetlanguages: zho - opusreadmeurl: https://github.com/Helsinki-NLP/Tatoeba-Challenge/tree/master/models/eng-zho/README.md - originalrepo: Tatoeba-Challenge - srcconstituents: {'eng'} - tgtconstituents: {'cmnHans', 'nan', 'nanHani', 'gan', 'yue', 'cmnKana', 'yueHani', 'wuuBopo', 'cmnLatn', 'yueHira', 'cmnHani', 'cjyHans', 'cmn', 'lzhHang', 'lzhHira', 'cmnHant', 'lzhBopo', 'zho', 'zhoHans', 'zhoHant', 'lzhHani', 'yueHang', 'wuu', 'yueKana', 'wuuLatn', 'yueBopo', 'cjyHant', 'yueHans', 'lzh', 'cmnHira', 'lzhYiii', 'lzhHans', 'cmnBopo', 'cmnHang'…
Open weights
apache-2.0
512 tokens
transformers
source languages: fr,frBE,frCA,frFR,wa,frp,oc,ca,rm,lld,fur,lij,lmo,es,esAR,esCL,esCO,esCR,esDO,esEC,esES,esGT,esHN,esMX,esNI,esPA,esPE,esPR,esSV,esUY,esVE,pt,ptbr,ptBR,ptPT,gl,lad,an,mwl,it,itIT,co,nap,scn,vec,sc,ro,la; target languages: en; OPUS readme: fr+frBE+frCA+frFR+wa+frp+oc+ca+rm+lld+fur+lij+lmo+es+esAR+esCL+esCO+esCR+esDO+esEC+esES+esGT+esHN+esMX+esNI+esPA+esPE+esPR+esSV+esUY+esVE+pt+ptbr+ptBR+ptPT+gl+lad+an+mwl+it+itIT+co+nap+scn+vec+sc+ro+la-en; dataset: opus; model: transformer; pre-processing: normalization + SentencePiece.
Open weights
apache-2.0
512 tokens
transformers
This is the model card of NLLB-200's 1.3B variant. Here are the metrics for that particular checkpoint. - Information about training algorithms, parameters, fairness constraints or other applied approaches, and features. The exact training algorithm, data and the strategies to handle data imbalances for high and low resource languages that were used to train NLLB-200 is described in the paper. - Paper or other resource for more information NLLB Team et al, No Language Left Behind: Scaling Human-Centered Machine Translation, Arxiv, 2022 - Where to send questions or comments about the model: https://github.com/facebookresearch/fairseq/issues • Model performance measures: NLLB-200 model was…
Open weights
cc-by-nc-4.0
1,024 tokens
transformers
hfname: mul-eng - sourcelanguages: mul - targetlanguages: eng - opusreadmeurl: https://github.com/Helsinki-NLP/Tatoeba-Challenge/tree/master/models/mul-eng/README.md - originalrepo: Tatoeba-Challenge - srcconstituents: {'sjnLatn', 'cat', 'nan', 'spa', 'ileLatn', 'pap', 'mwl', 'uzbLatn', 'mww', 'hil', 'lij', 'avkLatn', 'ladLatn', 'latLatn', 'bosLatn', 'oss', 'epo', 'ron', 'fry', 'cym', 'toiLatn', 'awa', 'swg', 'zsmLatn', 'zhoHant', 'gcfLatn', 'uzbCyrl', 'isl', 'lfnLatn', 'shsLatn', 'novLatn', 'bho', 'ltz', 'lzh', 'kurLatn', 'sun', 'arg', 'pesThaa', 'sqi', 'uigArab', 'csbLatn', 'fra', 'hat', 'livLatn', 'nonLatn', 'sco', 'cmnHans', 'pnb', 'roh', 'chv', 'ibo', 'bulLatn', 'amh', 'lfnCyrl'…
Open weights
apache-2.0
512 tokens
transformers
source languages: fr; target languages: es; OPUS readme: fr-es; dataset: opus; model: transformer-align; pre-processing: normalization + SentencePiece.
Open weights
apache-2.0
512 tokens
transformers
source languages: it; target languages: es; OPUS readme: it-es; dataset: opus; model: transformer-align; pre-processing: normalization + SentencePiece.
Open weights
apache-2.0
512 tokens
transformers
ProstT5 is a protein language model (pLM) which can translate between protein sequence and structure. ProstT5 (Protein structure-sequence T5) is based on ProtT5-XL-U50, a T5 model trained on encoding protein sequences using span corruption applied on billions of protein sequences. ProstT5 finetunes ProtT5-XL-U50 on translating between protein sequence and structure using 17M proteins with high-quality 3D structure predictions from the AlphaFoldDB. Protein structure is converted from 3D to 1D using the 3Di-tokens introduced by Foldseek. In a first step, ProstT5 learnt to represent the newly introduced 3Di-tokens by continuing the original span-denoising objective applied on 3Di- and amino…
Open weights
mit
transformers
source languages: tr; target languages: en; OPUS readme: tr-en; dataset: opus; model: transformer-align; pre-processing: normalization + SentencePiece.
Open weights
apache-2.0
512 tokens
transformers
hfname: eus-spa - sourcelanguages: eus - targetlanguages: spa - opusreadmeurl: https://github.com/Helsinki-NLP/Tatoeba-Challenge/tree/master/models/eus-spa/README.md - originalrepo: Tatoeba-Challenge - srcconstituents: {'eus'} - tgtconstituents: {'spa'} - srcmultilingual: False - tgtmultilingual: False - urlmodel: https://object.pouta.csc.fi/Tatoeba-MT-models/eus-spa/opus-2020-06-17.zip - urltestset: https://object.pouta.csc.fi/Tatoeba-MT-models/eus-spa/opus-2020-06-17.test.txt - srcalpha3: eus - tgtalpha3: spa - shortpair: eu-es - chrF2score: 0.6729999999999999 - brevitypenalty: 0.9640000000000001 - reflen: 12469.0 - srcname: Basque - tgtname: Spanish - traindate: 2020-06-17 - srcalpha2…
Open weights
apache-2.0
512 tokens
transformers
This is a LLaMA 3 Youko qlora fine-tune, created using a new version of the VNTL dataset. The purpose of this fine-tune is to improve performance of LLMs at translating Japanese visual novels to English. Unlike the previous version, this one doesn't includes the "chat mode". For this new version of VNTL 8B, I've rebuilt and expanded VNTL's dataset from the groud up, and I'm happy to say it performs really well, outperforming the previous version when it comes to accuracy and stability, it makes far fewer mistakes than it even when running at high temperatures (though I still recommend temperature 0 for the best accuracy). Some major changes in this version: - Switched to the default LLaMA3…
Open weights
llama3
Neural machine translation model for translating from Arabic (ar) to English (en). This model is part of the OPUS-MT project, an effort to make neural machine translation models widely available and accessible for many languages in the world. All models are originally trained using the amazing framework of Marian NMT, an efficient NMT implementation written in pure C++. The models have been converted to pyTorch using the transformers library by huggingface. Training data is taken from OPUS and training pipelines use the procedures of OPUS-MT-train. You can also use OPUS-MT models with the transformers pipelines, for example: The work is supported by the European Language Grid as pilot…
Open weights
cc-by-4.0
1,024 tokens
transformers
source languages: da; target languages: en; OPUS readme: da-en; dataset: opus; model: transformer-align; pre-processing: normalization + SentencePiece.
Open weights
apache-2.0
512 tokens
transformers
source languages: pl; target languages: en; OPUS readme: pl-en; dataset: opus; model: transformer-align; pre-processing: normalization + SentencePiece.
Open weights
apache-2.0
512 tokens
transformers
hfname: fin-eng - sourcelanguages: fin - targetlanguages: eng - opusreadmeurl: https://github.com/Helsinki-NLP/Tatoeba-Challenge/tree/master/models/fin-eng/README.md - originalrepo: Tatoeba-Challenge - srcconstituents: {'fin'} - tgtconstituents: {'eng'} - srcmultilingual: False - tgtmultilingual: False - urlmodel: https://object.pouta.csc.fi/Tatoeba-MT-models/fin-eng/opus-2020-08-05.zip - urltestset: https://object.pouta.csc.fi/Tatoeba-MT-models/fin-eng/opus-2020-08-05.test.txt - srcalpha3: fin - tgtalpha3: eng - shortpair: fi-en - chrF2score: 0.6970000000000001 - brevitypenalty: 0.99 - reflen: 74651.0 - srcname: Finnish - tgtname: English - traindate: 2020-08-05 - srcalpha2: fi…
Open weights
apache-2.0
512 tokens
transformers
source languages: ja; target languages: en; OPUS readme: ja-en; dataset: opus; model: transformer-align; pre-processing: normalization + SentencePiece.
Open weights
apache-2.0
512 tokens
transformers
source languages: bg; target languages: en; OPUS readme: bg-en; dataset: opus; model: transformer-align; pre-processing: normalization + SentencePiece.
Open weights
apache-2.0
512 tokens
transformers
Hunyuan Translation Model Version 1.5 includes a 1.8B translation model, HY-MT1.5-1.8B, and a 7B translation model, HY-MT1.5-7B. Both models focus on supporting mutual translation across 33 languages and incorporating 5 ethnic and dialect variations. Among them, HY-MT1.5-7B is an upgraded version of our WMT25 championship model, optimized for explanatory translation and mixed-language scenarios, with newly added support for terminology intervention, contextual translation, and formatted translation. Despite having less than one-third the parameters of HY-MT1.5-7B, HY-MT1.5-1.8B delivers translation performance comparable to its larger counterpart, achieving both high speed and high quality.…
Open weights
1.8B parameters
262,144 tokens
transformers
source languages: nl; target languages: fr; OPUS readme: nl-fr; dataset: opus; model: transformer-align; pre-processing: normalization + SentencePiece.
Open weights
apache-2.0
512 tokens
transformers
source languages: fr; target languages: de; OPUS readme: fr-de; dataset: opus; model: transformer-align; pre-processing: normalization + SentencePiece.
Open weights
apache-2.0
512 tokens
transformers
source languages: sv; target languages: en; OPUS readme: sv-en; dataset: opus; model: transformer-align; pre-processing: normalization + SentencePiece.
Open weights
apache-2.0
512 tokens
transformers
source languages: de; target languages: fr; OPUS readme: de-fr; dataset: opus; model: transformer-align; pre-processing: normalization + SentencePiece.
Open weights
apache-2.0
512 tokens
transformers
The developers of the Text-To-Text Transfer Transformer (T5) write: T5-11B is the checkpoint with 11 billion parameters. The developers write in a blog post that the model: See the blog post and research paper for further details. The model is pre-trained on the Colossal Clean Crawled Corpus (C4), which was developed and released in the context of the same research paper as T5. The model was pre-trained on a on a multi-task mixture of unsupervised (1.) and supervised tasks (2.). Thereby, the following datasets were being used for (1.) and (2.): 1. Datasets used for Unsupervised denoising objective: 2. Datasets used for Supervised text-to-text language modeling objective - CoLA Warstadt et…
Open weights
apache-2.0
transformers
a sentence initial language token is required in the form of >>id<< (id = valid target language ID) - hfname: eng-ara - sourcelanguages: eng - targetlanguages: ara - opusreadmeurl: https://github.com/Helsinki-NLP/Tatoeba-Challenge/tree/master/models/eng-ara/README.md - originalrepo: Tatoeba-Challenge - srcconstituents: {'eng'} - tgtconstituents: {'apc', 'ara', 'arqLatn', 'arq', 'afb', 'araLatn', 'apcLatn', 'arz'} - srcmultilingual: False - tgtmultilingual: False - urlmodel: https://object.pouta.csc.fi/Tatoeba-MT-models/eng-ara/opus-2020-07-03.zip - urltestset: https://object.pouta.csc.fi/Tatoeba-MT-models/eng-ara/opus-2020-07-03.test.txt - srcalpha3: eng - tgtalpha3: ara - shortpair: en-ar…
Open weights
apache-2.0
512 tokens
transformers
Unleashing the full potential of the previous sugoi 14B model, Sugoi 14B Ultra delivers near-double translation accuracy compared to its quantized predecessor—achieving a BLEU score of 21.38 vs 13.67. Its prompt-following skills rival those of Qwen 2.5 Base, especially when handling the bracket-heavy text commonly found in RPG Maker projects. - Key Improvements Nearly 2× BLEU score boost over previous quantized version (21.38 vs 13.67). Stronger prompt adherence, especially with RPGM-style bracketed text. - Ideal Use Cases Japanese → English translation—especially for game dialogue or RPG text. Interactive environments—works well with chat UIs like LM Studio. Must include a system prompt…
Open weights
apache-2.0
English ｜ 中文 Hy-MT2 is a family of “fast-thinking” multilingual translation models designed for complex real-world scenarios. It includes three model sizes: 1.8B, 7B, and 30B-A3B (MoE), all of which support translation among 33 languages and effectively follow translation instructions in multiple languages. For on-device deployment, AngelSlim 1.25-bit extreme quantization reduces the storage requirement of the 1.8B model to only 440 MB and improves inference speed by 1.5x. Multi-dimensional evaluations show that Hy-MT2 delivers outstanding performance across general, real-world business, domain-specific, and instruction-following translation tasks. The 7B and 30B-A3B models outperform…
Open weights
apache-2.0
30.1B parameters
262,144 tokens
transformers
Neural machine translation model for translating from English (en) to Bulgarian (bg). This model is part of the OPUS-MT project, an effort to make neural machine translation models widely available and accessible for many languages in the world. All models are originally trained using the amazing framework of Marian NMT, an efficient NMT implementation written in pure C++. The models have been converted to pyTorch using the transformers library by huggingface. Training data is taken from OPUS and training pipelines use the procedures of OPUS-MT-train. You can also use OPUS-MT models with the transformers pipelines, for example: The work is supported by the European Language Grid as pilot…
Open weights
cc-by-4.0
238M parameters
1,024 tokens
transformers
T5 is an encoder-decoder model pre-trained on a multi-task mixture of unsupervised and supervised tasks and for which each task is converted into a text-to-text format. For more information, please take a look at the original paper. Paper: Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Authors: Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J. Liu You can use this model with Transformers pipeline.
Open weights
apache-2.0
transformers
T5 is an encoder-decoder model pre-trained on a multi-task mixture of unsupervised and supervised tasks and for which each task is converted into a text-to-text format. For more information, please take a look at the original paper. Paper: Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Authors: Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J. Liu You can use this model with Transformers pipeline.
Open weights
apache-2.0
transformers
plT5 models are T5-based language models trained on Polish corpora. The models were optimized for the original T5 denoising target. plT5 was trained on six different corpora available for Polish language: The training dataset was tokenized into subwords using a sentencepiece unigram model with vocabulary size of 50k tokens. If you use this model, please cite the following paper: The model was trained by Machine Learning Research Team at Allegro and Linguistic Engineering Group at Institute of Computer Science, Polish Academy of Sciences. You can contact us at: [email protected]
Open weights
cc-by-4.0
transformers
plT5 models are T5-based language models trained on Polish corpora. The models were optimized for the original T5 denoising target. plT5 was trained on six different corpora available for Polish language: The training dataset was tokenized into subwords using a sentencepiece unigram model with vocabulary size of 50k tokens. If you use this model, please cite the following paper: The model was trained by Machine Learning Research Team at Allegro and Linguistic Engineering Group at Institute of Computer Science, Polish Academy of Sciences. You can contact us at: [email protected]
Open weights
cc-by-4.0
transformers
A Frontier Sparse Mixture-of-Experts (SMoE) Foundation Translation Model for 500+ Global Languages Mythos is an open-source foundation model family built by Adithyan AI. In this organization, we develop and open-source state-of-the-art Sparse Mixture-of-Experts (SMoE) language models, universal translation engines, parallel multilingual datasets, and ultra-efficient inference runtimes targeting 500+ languages. 100% Free and Open-Source: Released under the permissive Apache 2.0 license with zero paywalls, metered tokens, or subscription fees. Mythos2.0-2B was trained on a massive 16.5B sentence-pair parallel corpus covering 500+ languages and regional dialects across Africa, the Americas…
Access requested at publisher
apache-2.0
transformers
