Fill Mask Models

Model · Fill mask

hEVO-base-o

Mulin Lab

hEVO-base-o is the base checkpoint in the hEvo public-data model family. It was independently pretrained on 1000 Genomes Project and Human Genome Diversity Project sequences using self-supervised masked language modeling. The checkpoint provenance records no target benchmark label supervision. Use a Python environment with PyTorch and the checkpoint's tested Transformers version: Access is gated for peer reviewers and requires manual approval by MulinLab. Sign in to Hugging Face, submit an access request on this page, and wait for approval. After approval, run hf auth login with your own read token; token=True below uses your locally stored credential. The model and tokenizer load…

Model · Fill mask

hEVO-large-o

Mulin Lab

hEVO-large-o is the large checkpoint in the hEvo public-data model family. It was independently pretrained on 1000 Genomes Project and Human Genome Diversity Project sequences using self-supervised masked language modeling. The checkpoint provenance records no target benchmark label supervision. Use a Python environment with PyTorch and the checkpoint's tested Transformers version: Access is gated for peer reviewers and requires manual approval by MulinLab. Sign in to Hugging Face, submit an access request on this page, and wait for approval. After approval, run hf auth login with your own read token; token=True below uses your locally stored credential. The model and tokenizer load…

Model · Fill mask

albert-base-v2-persian-zwnj-shiraz-mlm

Shekar AI

ALBERT-base-v2 adapted to Persian with a freshly trained 38,016-token SentencePiece unigram tokenizer. English Wikipedia supplies additional English text. ZWNJ (U+200C, Persian half-space) is preserved as a standalone token, ID 5. Initialized from albert/albert-base-v2, with word embeddings and the tied vocabulary head reinitialized for the new vocabulary. Trained with 15% masked-language modeling on normalized shekar-ai/Shiraz and English Wikipedia (target 10% of corpus characters). Persian text is normalized with shekar.Normalizer before tokenization; the tokenizer itself only normalizes whitespace. The tokenizer was trained from up to 6 million sampled corpus lines. validation gains…

This model is a fine-tuned version of distilroberta-base on an unknown dataset. It achieves the following results on the evaluation set: The following hyperparameters were used during training: - learningrate: 2e-05 - trainbatchsize: 8 - evalbatchsize: 8 - lrschedulertype: linear - numepochs: 3.0 - Transformers 5.18.0 - Pytorch 2.11.0+cu130 - Datasets 4.8.5 - Tokenizers 0.23.2

Model · Fill mask

ModernBERT-Large-Instruct-Logician-v0.5

John Gorriceta

This model is a fine-tuned version of ModernBERT optimized for logical reasoning, deductive analysis, and structure-based token prediction. It relies on the ModernBertForMaskedLM architecture, making it highly effective at filling in missing contextual logic tokens (fill-mask). - As an encoder-based Masked Language Model, it is not designed for long-form generative text (like ChatGPT). It excels at predicting masked tokens inside structured prompts.

Model · Fill mask

ModernBERT-Large-Instruct-Logician-v0.5-ONNX

John Gorriceta

This is an ONNX version of JohnGorri/ModernBERT-Large-Instruct-Logician-v0.5. It was automatically converted and uploaded using this Hugging Face Space. See the pipeline documentation for fill-mask: https://huggingface.co/docs/transformers.js/api/pipelines#modulepipelines.FillMaskPipeline This model is a fine-tuned version of ModernBERT optimized for logical reasoning, deductive analysis, and structure-based token prediction. It relies on the ModernBertForMaskedLM architecture, making it highly effective at filling in missing contextual logic tokens (fill-mask). - As an encoder-based Masked Language Model, it is not designed for long-form generative text (like ChatGPT). It excels at…

This model is a fine-tuned version of distilbert/distilroberta-base on an unknown dataset. It achieves the following results on the evaluation set: The following hyperparameters were used during training: - learningrate: 2e-05 - trainbatchsize: 8 - evalbatchsize: 8 - lrschedulertype: linear - numepochs: 3 - Transformers 5.17.0 - Pytorch 2.11.0+cu130 - Datasets 4.8.5 - Tokenizers 0.23.2

Model · Fill mask

UltraBERT-150M-ONNX

Pastel Labs

ONNX export of dmis-lab/UltraBERT-150M for ONNX Runtime and Transformers.js. All credit for the model goes to dmis-lab. The weights are theirs, under the same Apache-2.0 license. We only converted the format. Inputs are inputids and attentionmask, with dynamic batch size and sequence length. Output is lasthiddenstate plus logits from the masked-language-model head. The sliding-window (128 tokens) and full-attention layers are built from the attention mask inside the graph, so padding and any length work. We ran 20 English texts through the original PyTorch model and through each ONNX file, then compared mean-pooled last hidden states with cosine similarity. Fifteen texts are short (3 to 25…

Model · Fill mask

UltraBERT-380M-ONNX

Pastel Labs

ONNX export of dmis-lab/UltraBERT-380M for ONNX Runtime and Transformers.js. All credit for the model goes to dmis-lab. The weights are theirs, under the same Apache-2.0 license. We only converted the format. Inputs are inputids and attentionmask, with dynamic batch size and sequence length. Output is lasthiddenstate plus logits from the masked-language-model head. The sliding-window (128 tokens) and full-attention layers are built from the attention mask inside the graph, so padding and any length work. We ran 20 English texts through the original PyTorch model and through each ONNX file, then compared mean-pooled last hidden states with cosine similarity. Fifteen texts are short (3 to 25…