source languages: nl; target languages: en; OPUS readme: nl-en; dataset: opus; model: transformer-align; pre-processing: normalization + SentencePiece.
Open-weight model · Translation
opus-mt-ko-en
by Helsinki-NLP Research Group Helsinki-NLP/opus-mt-ko-en
hfname: kor-eng - sourcelanguages: kor - targetlanguages: eng - opusreadmeurl: https://github.com/Helsinki-NLP/Tatoeba-Challenge/tree/master/models/kor-eng/README.md - originalrepo: Tatoeba-Challenge - srcconstituents: {'korHani', 'korHang', 'korLatn', 'kor'}…
Parameters—
Context512
Weights624.7 MB
Licenseapache-2.0
AccessOpen weights
Monthly Downloads392.2k
Model Card
By Helsinki-NLP Research Group, published under apache-2.0, revision e42d1f41b661.
hfname: kor-eng - sourcelanguages: kor - targetlanguages: eng - opusreadmeurl: https://github.com/Helsinki-NLP/Tatoeba-Challenge/tree/master/models/kor-eng/README.md - originalrepo: Tatoeba-Challenge - srcconstituents: {'korHani', 'korHang', 'korLatn', 'kor'} - tgtconstituents: {'eng'} - srcmultilingual: False - tgtmultilingual: False - urlmodel: https://object.pouta.csc.fi/Tatoeba-MT-models/kor-eng/opus-2020-06-17.zip - urltestset: https://object.pouta.csc.fi/Tatoeba-MT-models/kor-eng/opus-2020-06-17.test.txt - srcalpha3: kor - tgtalpha3: eng - shortpair: ko-en - chrF2score: 0.588 - brevitypenalty: 0.9590000000000001 - reflen: 17711.0 - srcname: Korean - tgtname: English - traindate…
Read Helsinki-NLP Research Group's full model card
kor-eng
- source group: Korean
- target group: English
-
OPUS readme: kor-eng
-
model: transformer-align
- source language(s): kor kor_Hang kor_Latn
- target language(s): eng
- model: transformer-align
- pre-processing: normalization + SentencePiece (spm32k,spm32k)
- download original weights: opus-2020-06-17.zip
- test set translations: opus-2020-06-17.test.txt
- test set scores: opus-2020-06-17.eval.txt
Benchmarks
| testset | BLEU | chr-F |
|---|---|---|
| Tatoeba-test.kor.eng | 41.3 | 0.588 |
System Info:
-
hf_name: kor-eng
-
source_languages: kor
-
target_languages: eng
-
opus_readme_url: https://github.com/Helsinki-NLP/Tatoeba-Challenge/tree/master/models/kor-eng/README.md
-
original_repo: Tatoeba-Challenge
-
tags: ['translation']
-
languages: ['ko', 'en']
-
src_constituents: {'kor_Hani', 'kor_Hang', 'kor_Latn', 'kor'}
-
tgt_constituents: {'eng'}
-
src_multilingual: False
-
tgt_multilingual: False
-
prepro: normalization + SentencePiece (spm32k,spm32k)
-
url_model: https://object.pouta.csc.fi/Tatoeba-MT-models/kor-eng/opus-2020-06-17.zip
-
url_test_set: https://object.pouta.csc.fi/Tatoeba-MT-models/kor-eng/opus-2020-06-17.test.txt
-
src_alpha3: kor
-
tgt_alpha3: eng
-
short_pair: ko-en
-
chrF2_score: 0.588
-
bleu: 41.3
-
brevity_penalty: 0.9590000000000001
-
ref_len: 17711.0
-
src_name: Korean
-
tgt_name: English
-
train_date: 2020-06-17
-
src_alpha2: ko
-
tgt_alpha2: en
-
prefer_old: False
-
long_pair: kor-eng
-
helsinki_git_sha: 480fcbe0ee1bf4774bcbe6226ad9f58e63f6c535
-
transformers_git_sha: 2207e5d8cb224e954a7cba69fa4ac2309e9ff30b
-
port_machine: brutasse
-
port_time: 2020-08-21-14:41
Configuration
- Architecture
- MarianMTModel
- Context length (tokens)
- 512
- Layers
- 6
- Vocabulary size
- 65,001
- Model type
- marian
Identity and Version
- Repository
- Helsinki-NLP/opus-mt-ko-en
- Publisher
- Helsinki-NLP Research Group
- Task
- Translation
- Modality
- Text
- Library
- transformers
- Parameters
- Not stated by the source
- Languages
- ko, en
- Revision
- e42d1f41b66194e6d10512f8a27bebc1f4f5097e
- First published
- 2022-03-02
- Last updated
- 2023-08-16
Files and Weights
11 files, 628.0 MB in total. The weights are 2 files totalling 624.7 MB in bin, h5.
Weights2 files · 624.7 MB
Configuration3 files · 2.8 KB
Tokenizer2 files · 1.7 MB
Documentation1 file · 2.1 KB
Other2 files · 1.7 MB
Repository1 file · 345 B
Every file
| File | Type | Size | SHA-256 |
|---|---|---|---|
| pytorch_model.bin | Weights | 312.1 MB | 4b2209fbd0c5 |
| tf_model.h5 | Weights | 312.6 MB | 49d3e8aeba09 |
| config.json | Configuration | 1.4 KB | — |
| generation_config.json | Configuration | 293 B | — |
| metadata.json | Configuration | 1.1 KB | — |
| README.md | Documentation | 2.1 KB | — |
| source.spm | Other | 841.8 KB | — |
| target.spm | Other | 813.1 KB | — |
| .gitattributes | Repository | 345 B | — |
| tokenizer_config.json | Tokenizer | 44 B | — |
| vocab.json | Tokenizer | 1.7 MB | — |
License and Download
- License
- apache-2.0
- Access
- Open weights, no gate
- Download size
- 624.7 MB
Download from Helsinki-NLP Research Group
Released by Helsinki-NLP Research Group through its official repository on Hugging Face. Read the license.
Memory Requirements
| Precision | Weights in memory |
|---|---|
| As published | 624.7 MB |
Weights only, from the published parameter count; the key-value cache and runtime add to this.
Questions About opus-mt-ko-en
Can I use opus-mt-ko-en commercially?
Yes. opus-mt-ko-en is released under Apache License 2.0. The Apache License 2.0 is a permissive open-source license. It permits commercial use, modification and redistribution. It requires keeping the license and copyright notices and any NOTICE file, stating significant changes, and it includes an express patent grant from contributors.
What is opus-mt-ko-en's context length?
512 tokens, from the maximum position embeddings in its published configuration.
Similar Models
This is the model card of NLLB-200's distilled 600M variant. Here are the metrics for that particular checkpoint. - Information about training algorithms, parameters, fairness constraints or other applied approaches, and features. The exact training algorithm, data and the strategies to handle data imbalances for high and low resource languages that were used to train NLLB-200 is described in the paper. - Paper or other resource for more information NLLB Team et al, No Language Left Behind: Scaling Human-Centered Machine Translation, Arxiv, 2022 - Where to send questions or comments about the model: https://github.com/facebookresearch/fairseq/issues • Model performance measures: NLLB-200…
source languages: en; target languages: ru; OPUS readme: en-ru; dataset: opus; model: transformer-align; pre-processing: normalization + SentencePiece.
This model can be used for translation and text-to-text generation. CONTENT WARNING: Readers should be aware this section contains content that is disturbing, offensive, and can propagate historical and current stereotypes. Significant research has explored bias and fairness issues with language models (see, e.g., Sheng et al. (2021) and Bender et al. (2021)). Further details about the dataset for this model can be found in the OPUS readme: en-de
source languages: de; target languages: en; OPUS readme: de-en; dataset: opus; model: transformer-align; pre-processing: normalization + SentencePiece.
This model can be used for translation and text-to-text generation. CONTENT WARNING: Readers should be aware this section contains content that is disturbing, offensive, and can propagate historical and current stereotypes. Significant research has explored bias and fairness issues with language models (see, e.g., Sheng et al. (2021) and Bender et al. (2021)). Further details about the dataset for this model can be found in the OPUS readme: ru-en