# Fill Mask Models, Page 2: Open-Weight AI Source: https://savrn.com/models/tasks/fill-mask/page/2 Markdown alternate of the page above; the site index is https://savrn.com/llms.txt --- SAVRN Model Hub · Models by Task # Fill Mask Models 69 open-weight fill mask models in the SAVRN Model Hub, with Microsoft, BERT community and Facebook AI community publishing the most. 69Models 39Publishers 8M to 6.4BParameter range 6Licenses ## SAVRN's Take A fill-mask model reads a sentence with one token hidden and predicts what belongs in the gap. We count 55 of them, 29 indexed with hourly pricing, and the downloads pile onto a few names: bert-base-uncased at 47.2 million a month, xlm-roberta-base at 21.5 million, then roberta-base, distilbert-base-uncased and roberta-large between 6.5 and 8.0 million. The BERT community and Microsoft publish seven each, the Facebook AI community four. These are the lightest weights in the catalog: bert-base-uncased needs 0.3 GB at 16-bit, roberta-large 0.9 GB, xlm-roberta-base 0.7 GB. The cheapest indexed host for every entry is one MI300X at $1.85 an hour, so you are renting a whole accelerator for a few hundred megabytes of weights and should plan to pack several onto one device. At the ends of the range, the Kuleshov Group's Caduceus is 8M parameters and rounds to zero GB, while Biohub's ESMC-6B, a 6.4B protein language model, needs 15.2 GB at 16-bit and 3.8 GB at 4-bit. The top downloads stop at 512 or 514 tokens of context; ModernBERT-base from Answer.AI runs to 8,192 and ESMC-6B to 2,048. The license mix is as clean as it gets: MIT on 25 of the 55, Apache 2.0 on 19, CC BY-SA 4.0 on four, and seven with nothing stated, which stay off our racks until the publisher answers. Before committing, check whether you need cased or uncased handling, language coverage (bert-base-multilingual-uncased spans 102 languages, xlm-roberta-base 100), and whether a 512-token window fits your inputs. Only 28 publish a parameter count, so size from measured memory. SAVRN Research, 2026-09-18 ## Most Downloaded | Model | Publisher | Parameters | License | Monthly downloads | Cheapest GPUs at 16-bit | | --- | --- | --- | --- | --- | --- | | [bert-base-uncased](https://savrn.com/models/bert-base-uncased) | BERT community | 110M | apache-2.0 | 29.9M | 1x MI300X, $1.85/hr | | [xlm-roberta-base](https://savrn.com/models/xlm-roberta-base) | Facebook AI community | 279M | mit | 8.2M | 1x MI300X, $1.85/hr | | [roberta-large](https://savrn.com/models/roberta-large) | Facebook AI community | 355M | mit | 6.9M | 1x MI300X, $1.85/hr | | [roberta-base](https://savrn.com/models/roberta-base) | Facebook AI community | 125M | mit | 6.7M | 1x MI300X, $1.85/hr | | [distilbert-base-uncased](https://savrn.com/models/distilbert-base-uncased) | DistilBERT community | 67M | apache-2.0 | 6.4M | 1x MI300X, $1.85/hr | | [mdeberta-v3-base](https://savrn.com/models/mdeberta-v3-base) | Microsoft | — | mit | 5.6M | — | | [esm2_t33_650M_UR50D](https://savrn.com/models/esm2-t33-650m-ur50d) | AI at Meta | 652M | mit | 4.6M | 1x MI300X, $1.85/hr | | [ModernBERT-base](https://savrn.com/models/modernbert-base) | Answer.AI | 150M | apache-2.0 | 3.6M | 1x MI300X, $1.85/hr | | [bert-base-cased](https://savrn.com/models/bert-base-cased) | BERT community | 109M | apache-2.0 | 3.5M | 1x MI300X, $1.85/hr | | [deberta-v3-base](https://savrn.com/models/deberta-v3-base) | Microsoft | — | mit | 2.7M | — | ## Licenses | License | Models | Commercial use | | --- | --- | --- | | [mit](https://savrn.com/models/licenses/mit) | 27 | Yes | | [apache-2.0](https://savrn.com/models/licenses/apache-2-0) | 27 | Yes | | not stated | 9 | Not stated | | [cc-by-sa-4.0](https://savrn.com/models/licenses/cc-by-sa-4-0) | 4 | Yes | | [cc-by-nc-sa-4.0](https://savrn.com/models/licenses/cc-by-nc-sa-4-0) | 1 | Not without separate permission | | [cc-by-nc-4.0](https://savrn.com/models/licenses/cc-by-nc-4-0) | 1 | Not without separate permission | ## Who Publishes Them | Publisher | Models | | --- | --- | | [Microsoft](https://savrn.com/model-publishers/microsoft) | 7 | | [BERT community](https://savrn.com/model-publishers/google-bert) | 7 | | [Facebook AI community](https://savrn.com/model-publishers/facebookai) | 4 | | [Moussa Kamal Eddine](https://savrn.com/model-publishers/moussakam) | 3 | | [AI at Meta](https://savrn.com/model-publishers/facebook) | 3 | | [DistilBERT community](https://savrn.com/model-publishers/distilbert) | 3 | ## All 69 Models, Page 2 of 2 Model · Fill mask ### [hEVO-base-o](https://savrn.com/models/hevo-base-o) [Mulin Lab](https://savrn.com/model-publishers/mulinlab) hEVO-base-o is the base checkpoint in the hEvo public-data model family. It was independently pretrained on 1000 Genomes Project and Human Genome Diversity Project sequences using self-supervised masked language modeling. The checkpoint provenance records no target benchmark label supervision. Use a Python environment with PyTorch and the checkpoint's tested Transformers version: Access is gated for peer reviewers and requires manual approval by MulinLab. Sign in to Hugging Face, submit an access request on this page, and wait for approval. After approval, run hf auth login with your own read token; token=True below uses your locally stored credential. The model and tokenizer load… Access requested at publisher 87M parameters transformers [View model](https://savrn.com/models/hevo-base-o) Model · Fill mask ### [hEVO-large-o](https://savrn.com/models/hevo-large-o) [Mulin Lab](https://savrn.com/model-publishers/mulinlab) hEVO-large-o is the large checkpoint in the hEvo public-data model family. It was independently pretrained on 1000 Genomes Project and Human Genome Diversity Project sequences using self-supervised masked language modeling. The checkpoint provenance records no target benchmark label supervision. Use a Python environment with PyTorch and the checkpoint's tested Transformers version: Access is gated for peer reviewers and requires manual approval by MulinLab. Sign in to Hugging Face, submit an access request on this page, and wait for approval. After approval, run hf auth login with your own read token; token=True below uses your locally stored credential. The model and tokenizer load… Access requested at publisher 319M parameters transformers [View model](https://savrn.com/models/hevo-large-o) Model · Fill mask ### [albert-base-v2-persian-zwnj-shiraz-mlm](https://savrn.com/models/albert-base-v2-persian-zwnj-shiraz-mlm) [Shekar AI](https://savrn.com/model-publishers/shekar-ai) ALBERT-base-v2 adapted to Persian with a freshly trained 38,016-token SentencePiece unigram tokenizer. English Wikipedia supplies additional English text. ZWNJ (U+200C, Persian half-space) is preserved as a standalone token, ID 5. Initialized from albert/albert-base-v2, with word embeddings and the tied vocabulary head reinitialized for the new vocabulary. Trained with 15% masked-language modeling on normalized shekar-ai/Shiraz and English Wikipedia (target 10% of corpus characters). Persian text is normalized with shekar.Normalizer before tokenization; the tokenizer itself only normalizes whitespace. The tokenizer was trained from up to 6 million sampled corpus lines. validation gains… Open weights mit 12M parameters 512 tokens transformers [View model](https://savrn.com/models/albert-base-v2-persian-zwnj-shiraz-mlm) Model · Fill mask ### [distilroberta-base-finetuned-wikitext2](https://savrn.com/models/distilroberta-base-finetuned-wikitext2) [Pufhv98w](https://savrn.com/model-publishers/3809ksdfh) This model is a fine-tuned version of distilroberta-base on an unknown dataset. It achieves the following results on the evaluation set: The following hyperparameters were used during training: - learningrate: 2e-05 - trainbatchsize: 8 - evalbatchsize: 8 - lrschedulertype: linear - numepochs: 3.0 - Transformers 5.18.0 - Pytorch 2.11.0+cu130 - Datasets 4.8.5 - Tokenizers 0.23.2 Open weights apache-2.0 82M parameters 514 tokens transformers [View model](https://savrn.com/models/distilroberta-base-finetuned-wikitext2) Model · Fill mask ### [ModernBERT-Large-Instruct-Logician-v0.5](https://savrn.com/models/modernbert-large-instruct-logician-v0-5) [John Gorriceta](https://savrn.com/model-publishers/johngorri) This model is a fine-tuned version of ModernBERT optimized for logical reasoning, deductive analysis, and structure-based token prediction. It relies on the ModernBertForMaskedLM architecture, making it highly effective at filling in missing contextual logic tokens (fill-mask). - As an encoder-based Masked Language Model, it is not designed for long-form generative text (like ChatGPT). It excels at predicting masked tokens inside structured prompts. Open weights apache-2.0 396M parameters 8,192 tokens [View model](https://savrn.com/models/modernbert-large-instruct-logician-v0-5) Model · Fill mask ### [ModernBERT-Large-Instruct-Logician-v0.5-ONNX](https://savrn.com/models/modernbert-large-instruct-logician-v0-5-onnx) [John Gorriceta](https://savrn.com/model-publishers/johngorri) This is an ONNX version of JohnGorri/ModernBERT-Large-Instruct-Logician-v0.5. It was automatically converted and uploaded using this Hugging Face Space. See the pipeline documentation for fill-mask: https://huggingface.co/docs/transformers.js/api/pipelines#modulepipelines.FillMaskPipeline This model is a fine-tuned version of ModernBERT optimized for logical reasoning, deductive analysis, and structure-based token prediction. It relies on the ModernBertForMaskedLM architecture, making it highly effective at filling in missing contextual logic tokens (fill-mask). - As an encoder-based Masked Language Model, it is not designed for long-form generative text (like ChatGPT). It excels at… Open weights apache-2.0 8,192 tokens transformers.js [View model](https://savrn.com/models/modernbert-large-instruct-logician-v0-5-onnx) Model · Fill mask ### [my_awesome_eli5_mlm_model](https://savrn.com/models/my-awesome-eli5-mlm-model) [Ibrahim Samir Elmansoury](https://savrn.com/model-publishers/ibrahimmansoury) This model is a fine-tuned version of distilbert/distilroberta-base on an unknown dataset. It achieves the following results on the evaluation set: The following hyperparameters were used during training: - learningrate: 2e-05 - trainbatchsize: 8 - evalbatchsize: 8 - lrschedulertype: linear - numepochs: 3 - Transformers 5.17.0 - Pytorch 2.11.0+cu130 - Datasets 4.8.5 - Tokenizers 0.23.2 Open weights apache-2.0 82M parameters 514 tokens transformers [View model](https://savrn.com/models/my-awesome-eli5-mlm-model) Model · Fill mask ### [UltraBERT-150M-ONNX](https://savrn.com/models/ultrabert-150m-onnx) [Pastel Labs](https://savrn.com/model-publishers/pastel-labs) ONNX export of dmis-lab/UltraBERT-150M for ONNX Runtime and Transformers.js. All credit for the model goes to dmis-lab. The weights are theirs, under the same Apache-2.0 license. We only converted the format. Inputs are inputids and attentionmask, with dynamic batch size and sequence length. Output is lasthiddenstate plus logits from the masked-language-model head. The sliding-window (128 tokens) and full-attention layers are built from the attention mask inside the graph, so padding and any length work. We ran 20 English texts through the original PyTorch model and through each ONNX file, then compared mean-pooled last hidden states with cosine similarity. Fifteen texts are short (3 to 25… Open weights apache-2.0 transformers.js [View model](https://savrn.com/models/ultrabert-150m-onnx) Model · Fill mask ### [UltraBERT-380M-ONNX](https://savrn.com/models/ultrabert-380m-onnx) [Pastel Labs](https://savrn.com/model-publishers/pastel-labs) ONNX export of dmis-lab/UltraBERT-380M for ONNX Runtime and Transformers.js. All credit for the model goes to dmis-lab. The weights are theirs, under the same Apache-2.0 license. We only converted the format. Inputs are inputids and attentionmask, with dynamic batch size and sequence length. Output is lasthiddenstate plus logits from the masked-language-model head. The sliding-window (128 tokens) and full-attention layers are built from the attention mask inside the graph, so padding and any length work. We ran 20 English texts through the original PyTorch model and through each ONNX file, then compared mean-pooled last hidden states with cosine similarity. Fifteen texts are short (3 to 25… Open weights apache-2.0 transformers.js [View model](https://savrn.com/models/ultrabert-380m-onnx) ## Questions ### Which Fill mask models are most downloaded? By monthly downloads reported by the Hugging Face Hub: hEVO-base-o (3); hEVO-large-o (3). ## Other Tasks - [Text Generation](https://savrn.com/models/tasks/text-generation) 1598 - [Image and Text to Text](https://savrn.com/models/tasks/image-and-text-to-text) 598 - [Robotics](https://savrn.com/models/tasks/robotics) 410 - [Feature Extraction](https://savrn.com/models/tasks/feature-extraction) 405 - [Text Classification](https://savrn.com/models/tasks/text-classification) 336 - [Text to Image](https://savrn.com/models/tasks/text-to-image) 239 - [Speech Recognition](https://savrn.com/models/tasks/speech-recognition) 190 - [Text to Speech](https://savrn.com/models/tasks/text-to-speech) 149 - [Image Classification](https://savrn.com/models/tasks/image-classification) 140 - [Audio Classification](https://savrn.com/models/tasks/audio-classification) 130 - [Reinforcement Learning](https://savrn.com/models/tasks/reinforcement-learning) 105 - [Sentence Similarity](https://savrn.com/models/tasks/sentence-similarity) 103 [See all](https://savrn.com/models/tasks)