Search public pages, research tools, and SAVRN solutions.
Pretrained model on English language using a masked language modeling (MLM) objective. It was introduced in this paper and first released in this repository. This model is case-sensitive: it makes a difference between english and English.
What it takes to serve roberta-base (125M parameters): the memory its weights need at each precision, and the cheapest way to rent enough data-center GPUs to hold them.
| Precision | Weights | Memory needed | Cheapest setup | Per hour | Also fits |
|---|---|---|---|---|---|
| 16-bit | 0.2 GB | 0.3 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
| 8-bit | 0.1 GB | 0.1 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
| 4-bit | 0.1 GB | 0.1 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
Memory is the weights at that precision plus 20% for the runtime and a short context; a long context needs more. Prices are the lowest on-demand hourly rates in the SAVRN Index, read Sep 18, 2026.
Case matters here: roberta-base, from the Facebook AI community, tells english from English. Pretrained on raw English text from bookcorpus and wikipedia by masked language modeling, its listed task is fill mask: token prediction and what you build on it, not chat. At 16-bit its 125M parameters take 0.2 GB of weights and 0.3 GB to run, against a cheapest listed setup of one 192 GB MI300X at $1.85 per hour on-demand; on our floor that shares a card, never owns one.
MIT is short: commercial use, modification and redistribution are allowed, the one duty being to carry the copyright and permission notices along. Check provenance and packaging. The releasing team wrote no model card, so bookcorpus and wikipedia are all the page discloses on data. The repository is 2.82 GB across 13 files in safetensors, pytorch, jax, rust and tf; pull only the format your stack loads.
By Facebook AI community, published under mit, revision e2da8e2f811d.
Pretrained model on English language using a masked language modeling (MLM) objective. It was introduced in this paper and first released in this repository. This model is case-sensitive: it makes a difference between english and English.
Disclaimer: The team releasing RoBERTa did not write a model card for this model so this model card has been written by the Hugging Face team.
RoBERTa is a transformers model pretrained on a large corpus of English data in a self-supervised fashion. This means it was pretrained on the raw texts only, with no humans labelling them in any way (which is why it can use lots of publicly available data) with an automatic process to generate inputs and labels from those texts.
13 files, 2.8 GB in total. The weights are 5 files totalling 2.8 GB in bin, h5, msgpack, ot, safetensors.
| File | Type | Size | SHA-256 |
|---|---|---|---|
| flax_model.msgpack | Weights | 498.8 MB | 7647e7df15df |
| model.safetensors | Weights | 498.8 MB | 5bde1d28afb3 |
| pytorch_model.bin | Weights | 501.2 MB | 278b7a95739c |
| rust_model.ot | Weights | 655.6 MB | 9eab94d556cd |
| tf_model.h5 | Weights | 657.4 MB | 8ad49e51cdfa |
| config.json | Configuration | 481 B | — |
| README.md | Documentation | 9.1 KB | — |
| dict.txt | Other | 603.3 KB | — |
| .gitattributes | Repository | 445 B | — |
| merges.txt | Tokenizer | 456.3 KB | — |
| tokenizer.json | Tokenizer | 1.4 MB | — |
| tokenizer_config.json | Tokenizer | 25 B | — |
| vocab.json | Tokenizer | 898.8 KB | — |
Released by Facebook AI community through its official repository on Hugging Face. Read the license.
| Precision | Weights in memory |
|---|---|
| As published | 2.8 GB |
| 16-bit | 0.2 GB |
| 8-bit | 0.1 GB |
| 4-bit | 0.1 GB |
Weights only, from the published parameter count; the key-value cache and runtime add to this.
About 0.3 GB at 16-bit and 0.1 GB at 4-bit: the weights (125M parameters) plus a working margin. A long context needs more.
At 16-bit, 1x MI300X from $1.85 an hour; at 4-bit, 1x MI300X from $1.85 an hour, at the lowest on-demand prices the SAVRN Index lists.
Yes. roberta-base is released under MIT License. The MIT License is a short permissive license. It permits commercial use, modification and redistribution, provided the copyright notice and permission notice are included.
514 tokens, from the maximum position embeddings in its published configuration.
This is a microsoft/codebert-base-mlm model, trained for 1,000,000 steps (with batchsize=32) on Python code from the codeparrot/github-code-clean dataset, on the masked-language-modeling task. It is intended to be used in CodeBERTScore: https://github.com/neulab/code-bert-score, but can be used for any other model or task. If you use this model for research, please cite
This model is a distilled version of the BERT base multilingual model. The code for the distillation process can be found here. This model is cased: it does make a difference between english and English. The model is trained on the concatenation of Wikipedia in 104 different languages listed here. The model has 6 layers, 768 dimension and 12 heads, totalizing 134M parameters (compared to 177M parameters for mBERT-base). On average, this model, referred to as DistilmBERT, is twice as fast as mBERT-base. We encourage potential users of this model to check out the BERT base multilingual model card to learn more about usage, limitations and potential biases. You can use the raw model for either…
AraBERT is an Arabic pretrained language model based on Google's BERT architechture. AraBERT uses the same BERT-Base config. More details are available in the AraBERT Paper and in the AraBERT Meetup There are two versions of the model, AraBERTv0.1 and AraBERTv1, with the difference being that AraBERTv1 uses pre-segmented text where prefixes and suffixes were split using the Farasa Segmenter. We evaluate AraBERT models on different downstream tasks and compare them to mBERT), and other state of the art models (To the extent of our knowledge). The Tasks were Sentiment Analysis on 6 different datasets (HARD, ASTD-Balanced, ArsenTD-Lev, LABR), Named Entity Recognition with the ANERcorp, and…
CamemBERT is a state-of-the-art language model for French based on the RoBERTa model. It is now available on Hugging Face in 6 different versions with varying number of parameters, amount of pretraining data and pretraining data source domains. CamemBERT was trained and evaluated by Louis Martin\, Benjamin Muller\, Pedro Javier Ortiz Suárez\, Yoann Dupont, Laurent Romary, Éric Villemonte de la Clergerie, Djamé Seddah and Benoît Sagot. If you use our work, please cite
JuriBERT is a set of BERT models (tiny, mini, small and base) pre-trained from scratch on French legal-domain specific corpora. JuriBERT models are pretrained on 6.3GB of legal french raw text from two different sources: the first dataset is crawled from Légifrance and the other one consists of anonymized court’s decisions and the pleadings from the Court of Cassation (mémoires ampliatifs). The latter contains more than 100k long documents from different court cases. It is now on Hugging Face in four different versions with varying number of parameters. JuriBERT was trained and evaluated at École Polytechnique in collaboration with HEC Paris by Stella Douka, Hadi Abdine, Mihcalis…
Pretrained model on English language using a masked language modeling (MLM) objective. It was introduced in this paper and first released in this repository. This model is uncased: it does not make a difference between english and English. Disclaimer: The team releasing BERT did not write a model card for this model so this model card has been written by the Hugging Face team. BERT is a transformers model pretrained on a large corpus of English data in a self-supervised fashion. This means it was pretrained on the raw texts only, with no humans labeling them in any way (which is why it can use lots of publicly available data) with an automatic process to generate inputs and labels from…