source languages: nl; target languages: en; OPUS readme: nl-en; dataset: opus; model: transformer-align; pre-processing: normalization + SentencePiece.
Open-weight model · Translation
opus-mt-en-ar
by Helsinki-NLP Research Group Helsinki-NLP/opus-mt-en-ar
a sentence initial language token is required in the form of >>id<< (id = valid target language ID) - hfname: eng-ara - sourcelanguages: eng - targetlanguages: ara - opusreadmeurl…
Parameters—
Context512
Weights1.2 GB
Licenseapache-2.0
AccessOpen weights
Monthly Downloads27.3k
Model Card
By Helsinki-NLP Research Group, published under apache-2.0, revision 03087980e8ce.
a sentence initial language token is required in the form of >>id<< (id = valid target language ID) - hfname: eng-ara - sourcelanguages: eng - targetlanguages: ara - opusreadmeurl: https://github.com/Helsinki-NLP/Tatoeba-Challenge/tree/master/models/eng-ara/README.md - originalrepo: Tatoeba-Challenge - srcconstituents: {'eng'} - tgtconstituents: {'apc', 'ara', 'arqLatn', 'arq', 'afb', 'araLatn', 'apcLatn', 'arz'} - srcmultilingual: False - tgtmultilingual: False - urlmodel: https://object.pouta.csc.fi/Tatoeba-MT-models/eng-ara/opus-2020-07-03.zip - urltestset: https://object.pouta.csc.fi/Tatoeba-MT-models/eng-ara/opus-2020-07-03.test.txt - srcalpha3: eng - tgtalpha3: ara - shortpair: en-ar…
Read Helsinki-NLP Research Group's full model card
eng-ara
- source group: English
- target group: Arabic
-
OPUS readme: eng-ara
-
model: transformer
- source language(s): eng
- target language(s): acm afb apc apc_Latn ara ara_Latn arq arq_Latn ary arz
- model: transformer
- pre-processing: normalization + SentencePiece (spm32k,spm32k)
- a sentence initial language token is required in the form of
>>id<<(id = valid target language ID) - download original weights: opus-2020-07-03.zip
- test set translations: opus-2020-07-03.test.txt
- test set scores: opus-2020-07-03.eval.txt
Benchmarks
| testset | BLEU | chr-F |
|---|---|---|
| Tatoeba-test.eng.ara | 14.0 | 0.437 |
System Info:
-
hf_name: eng-ara
-
source_languages: eng
-
target_languages: ara
-
opus_readme_url: https://github.com/Helsinki-NLP/Tatoeba-Challenge/tree/master/models/eng-ara/README.md
-
original_repo: Tatoeba-Challenge
-
tags: ['translation']
-
languages: ['en', 'ar']
-
src_constituents: {'eng'}
-
tgt_constituents: {'apc', 'ara', 'arq_Latn', 'arq', 'afb', 'ara_Latn', 'apc_Latn', 'arz'}
-
src_multilingual: False
-
tgt_multilingual: False
-
prepro: normalization + SentencePiece (spm32k,spm32k)
-
url_model: https://object.pouta.csc.fi/Tatoeba-MT-models/eng-ara/opus-2020-07-03.zip
-
url_test_set: https://object.pouta.csc.fi/Tatoeba-MT-models/eng-ara/opus-2020-07-03.test.txt
-
src_alpha3: eng
-
tgt_alpha3: ara
-
short_pair: en-ar
-
chrF2_score: 0.43700000000000006
-
bleu: 14.0
-
brevity_penalty: 1.0
-
ref_len: 58935.0
-
src_name: English
-
tgt_name: Arabic
-
train_date: 2020-07-03
-
src_alpha2: en
-
tgt_alpha2: ar
-
prefer_old: False
-
long_pair: eng-ara
-
helsinki_git_sha: 480fcbe0ee1bf4774bcbe6226ad9f58e63f6c535
-
transformers_git_sha: 2207e5d8cb224e954a7cba69fa4ac2309e9ff30b
-
port_machine: brutasse
-
port_time: 2020-08-21-14:41
Configuration
- Architecture
- MarianMTModel
- Context length (tokens)
- 512
- Layers
- 6
- Vocabulary size
- 62,802
- Model type
- marian
Identity and Version
- Repository
- Helsinki-NLP/opus-mt-en-ar
- Publisher
- Helsinki-NLP Research Group
- Task
- Translation
- Modality
- Text
- Library
- transformers
- Parameters
- Not stated by the source
- Languages
- en, ar
- Revision
- 03087980e8ce753d64b3248ed0a912444545b840
- First published
- 2022-03-02
- Last updated
- 2023-08-16
Files and Weights
12 files, 1.2 GB in total. The weights are 3 files totalling 1.2 GB in bin, h5, ot.
Weights3 files · 1.2 GB
Configuration3 files · 2.8 KB
Tokenizer2 files · 2.1 MB
Documentation1 file · 2.3 KB
Other2 files · 1.7 MB
Repository1 file · 345 B
Every file
| File | Type | Size | SHA-256 |
|---|---|---|---|
| pytorch_model.bin | Weights | 307.6 MB | d6d027043454 |
| rust_model.ot | Weights | 564.8 MB | 724dbb6e24fb |
| tf_model.h5 | Weights | 308.1 MB | 582af41ec0a4 |
| config.json | Configuration | 1.4 KB | — |
| generation_config.json | Configuration | 293 B | — |
| metadata.json | Configuration | 1.1 KB | — |
| README.md | Documentation | 2.3 KB | — |
| source.spm | Other | 801.1 KB | — |
| target.spm | Other | 916.9 KB | — |
| .gitattributes | Repository | 345 B | — |
| tokenizer_config.json | Tokenizer | 44 B | — |
| vocab.json | Tokenizer | 2.1 MB | — |
License and Download
- License
- apache-2.0
- Access
- Open weights, no gate
- Download size
- 1.2 GB
Download from Helsinki-NLP Research Group
Released by Helsinki-NLP Research Group through its official repository on Hugging Face. Read the license.
Memory Requirements
| Precision | Weights in memory |
|---|---|
| As published | 1.2 GB |
Weights only, from the published parameter count; the key-value cache and runtime add to this.
Questions About opus-mt-en-ar
Can I use opus-mt-en-ar commercially?
Yes. opus-mt-en-ar is released under Apache License 2.0. The Apache License 2.0 is a permissive open-source license. It permits commercial use, modification and redistribution. It requires keeping the license and copyright notices and any NOTICE file, stating significant changes, and it includes an express patent grant from contributors.
What is opus-mt-en-ar's context length?
512 tokens, from the maximum position embeddings in its published configuration.
Similar Models
This is the model card of NLLB-200's distilled 600M variant. Here are the metrics for that particular checkpoint. - Information about training algorithms, parameters, fairness constraints or other applied approaches, and features. The exact training algorithm, data and the strategies to handle data imbalances for high and low resource languages that were used to train NLLB-200 is described in the paper. - Paper or other resource for more information NLLB Team et al, No Language Left Behind: Scaling Human-Centered Machine Translation, Arxiv, 2022 - Where to send questions or comments about the model: https://github.com/facebookresearch/fairseq/issues • Model performance measures: NLLB-200…
source languages: en; target languages: ru; OPUS readme: en-ru; dataset: opus; model: transformer-align; pre-processing: normalization + SentencePiece.
This model can be used for translation and text-to-text generation. CONTENT WARNING: Readers should be aware this section contains content that is disturbing, offensive, and can propagate historical and current stereotypes. Significant research has explored bias and fairness issues with language models (see, e.g., Sheng et al. (2021) and Bender et al. (2021)). Further details about the dataset for this model can be found in the OPUS readme: en-de
hfname: kor-eng - sourcelanguages: kor - targetlanguages: eng - opusreadmeurl: https://github.com/Helsinki-NLP/Tatoeba-Challenge/tree/master/models/kor-eng/README.md - originalrepo: Tatoeba-Challenge - srcconstituents: {'korHani', 'korHang', 'korLatn', 'kor'} - tgtconstituents: {'eng'} - srcmultilingual: False - tgtmultilingual: False - urlmodel: https://object.pouta.csc.fi/Tatoeba-MT-models/kor-eng/opus-2020-06-17.zip - urltestset: https://object.pouta.csc.fi/Tatoeba-MT-models/kor-eng/opus-2020-06-17.test.txt - srcalpha3: kor - tgtalpha3: eng - shortpair: ko-en - chrF2score: 0.588 - brevitypenalty: 0.9590000000000001 - reflen: 17711.0 - srcname: Korean - tgtname: English - traindate…
source languages: de; target languages: en; OPUS readme: de-en; dataset: opus; model: transformer-align; pre-processing: normalization + SentencePiece.