SAVRN
Search Contact SAVRN

SAVRN Model Hub · Models by License

Open-Weight Models Under Creative Commons Attribution-ShareAlike 4.0

5 open-weight models released under Creative Commons Attribution-ShareAlike 4.0 in the SAVRN Model Hub, with Tohoku NLP, AUEB NLP Group and Language Media Processing Lab at Kyoto University publishing the most.

5Models
4Publishers
330M to 330MParameter range
1Licenses
YesCommercial use

What Creative Commons Attribution-ShareAlike 4.0 Allows

CC BY-SA 4.0 permits sharing and adapting, including commercially, with credit to the creator, and requires adaptations to be released under the same license. Read the license text.

SAVRN's Take

Five models in our catalog carry Creative Commons Attribution-ShareAlike 4.0. The terms are short: share and adapt the work, commercial use included, credit the creator, and release any adaptation under the same license. Running the published weights on your own racks asks only for the credit line, and serving an unmodified model to customers is the same story. Train on your own data and hand that checkpoint to anyone outside the building, though, and the ShareAlike term follows it. If the value of a deployment lives in a proprietary fine-tune, settle whether that checkpoint ever leaves your control before the training run starts.

Tohoku NLP publishes two of the five, and VISTEC-depa AI Research Institute of Thailand, AUEB NLP Group and the Language Media Processing Lab at Kyoto University publish one each. Four are fill-mask models with a 512-token context and one is a speech recognizer, so nobody comes here for a chat model.

The download counts favor the speech model. VISTEC-depa's wav2vec2-large-xlsr-53-th leads at 1,541,932 a month, about three times legal-bert-base-uncased at 507,966, pretrained on 12 GB of English legal text. Kyoto's deberta-v2-large-japanese-char-wwm follows at 364,338, then Tohoku's bert-base-japanese-whole-word-masking at 337,174 and bert-base-japanese at 207,165. The Kyoto model is the only one with a hardware line in the Index: 330M parameters, 0.8 GB at 16-bit, cheapest hosted on one MI300X at $1.85 an hour. At that size the hardware is not the decision. The license is.

Most Downloaded

ModelPublisherParametersLicenseMonthly downloadsCheapest GPUs at 16-bit
wav2vec2-large-xlsr-53-th VISTEC-depa AI Research Institute of Thailand cc-by-sa-4.0 1.5M
legal-bert-base-uncased AUEB NLP Group cc-by-sa-4.0 508k
deberta-v2-large-japanese-char-wwm Language Media Processing Lab at Kyoto University 330M cc-by-sa-4.0 364.3k 1x MI300X, $1.85/hr
bert-base-japanese-whole-word-masking Tohoku NLP cc-by-sa-4.0 337.2k
bert-base-japanese Tohoku NLP cc-by-sa-4.0 207.2k

All 5 Models

This is a Japanese DeBERTa V2 large model pre-trained on Japanese Wikipedia, the Japanese portion of CC-100, and the Japanese portion of OSCAR. This model is trained with character-level tokenization and whole word masking. You can use this model for masked language modeling as follows: You can also fine-tune this model on downstream tasks. There is no need to tokenize texts in advance, and you can give raw texts to the tokenizer. The texts are tokenized into character-level tokens by sentencepiece. We used the following corpora for pre-training: - Japanese Wikipedia (as of 20221020, 3.2GB, 27M sentences, 1.3M documents) - Japanese portion of CC-100 (85GB, 619M sentences, 66M documents)…

Open weights cc-by-sa-4.0 330M parameters 512 tokens transformers

Finetuning wav2vec2-large-xlsr-53 on Thai Common Voice 7.0 We finetune wav2vec2-large-xlsr-53 based on Fine-tuning Wav2Vec2 for English ASR using Thai examples of Common Voice Corpus 7.0. The notebooks and scripts can be found in vistec-ai/wav2vec2-large-xlsr-53-th. The pretrained model and processor can be found at airesearch/wav2vec2-large-xlsr-53-th. Add syllabletokenize, wordtokenize (PyThaiNLP) and deepcut tokenizers to eval.py from robust-speech-event Common Voice Corpus 7.0](https://commonvoice.mozilla.org/en/datasets) contains 133 validated hours of Thai (255 total hours) at 5GB. We pre-tokenize with pythainlp.tokenize.wordtokenize. We preprocess the dataset using cleaning rules…

Open weights cc-by-sa-4.0 transformers

LEGAL-BERT is a family of BERT models for the legal domain, intended to assist legal NLP research, computational law, and legal technology applications. To pre-train the different variations of LEGAL-BERT, we collected 12 GB of diverse English legal text from several fields (e.g., legislation, court cases, contracts) scraped from publicly available resources. Sub-domain variants (CONTRACTS-, EURLEX-, ECHR-) and/or general LEGAL-BERT perform better than using BERT out of the box for domain-specific tasks. A light-weight model (33% the size of BERT-BASE) pre-trained from scratch on legal data with competitive performance is also available. I. Chalkidis, M. Fergadiotis, P. Malakasiotis, N.…

Open weights cc-by-sa-4.0 512 tokens transformers

This is a BERT model pretrained on texts in the Japanese language. This version of the model processes input texts with word-level tokenization based on the IPA dictionary, followed by the WordPiece subword tokenization. Additionally, the model is trained with the whole word masking enabled for the masked language modeling (MLM) objective. The codes for the pretraining are available at cl-tohoku/bert-japanese. The model architecture is the same as the original BERT base model; 12 layers, 768 dimensions of hidden states, and 12 attention heads. The model is trained on Japanese Wikipedia as of September 1, 2019. To generate the training corpus, WikiExtractor is used to extract plain texts…

Open weights cc-by-sa-4.0 512 tokens transformers

Model · Fill mask

bert-base-japanese

Tohoku NLP

This is a BERT model pretrained on texts in the Japanese language. This version of the model processes input texts with word-level tokenization based on the IPA dictionary, followed by the WordPiece subword tokenization. The codes for the pretraining are available at cl-tohoku/bert-japanese. The model architecture is the same as the original BERT base model; 12 layers, 768 dimensions of hidden states, and 12 attention heads. The model is trained on Japanese Wikipedia as of September 1, 2019. To generate the training corpus, WikiExtractor is used to extract plain texts from a dump file of Wikipedia articles. The text files used for the training are 2.6GB in size, consisting of approximately…

Open weights cc-by-sa-4.0 512 tokens transformers

Questions

Can I use Creative Commons Attribution-ShareAlike 4.0 models commercially?

Yes. CC BY-SA 4.0 permits sharing and adapting, including commercially, with credit to the creator, and requires adaptations to be released under the same license.

Which Creative Commons Attribution-ShareAlike 4.0 models are most downloaded?

By monthly downloads reported by the Hugging Face Hub: deberta-v2-large-japanese-char-wwm (364.3k); wav2vec2-large-xlsr-53-th (1.5M); legal-bert-base-uncased (508k).

Other Licenses

See all