SAVRN
Search Contact SAVRN

Open-weight model · Zero-shot classification

ModernBERT-large-nli

by Tasksource tasksource/ModernBERT-large-nli

ModernBERT-large-nli is an open-weight model for zero-shot classification from Tasksource, released under Apache License 2.0. It has 396M parameters and a 2,048-token context. At 16-bit it needs about 1 GB of GPU memory, which fits on 1x MI300X from $1.85 an hour, at the lowest prices in the SAVRN Index. It draws 21.1k downloads a month.

This model is ModernBERT multi-task fine-tuned on tasksource NLI tasks, including MNLI, ANLI, SICK, WANLI, doc-nli, LingNLI, FOLIO, FOL-NLI, LogicNLI, Label-NLI and all datasets in the below table). This is the equivalent of an "instruct" version.

Parameters396M
Context2,048
Weights1.6 GB
Licenseapache-2.0
AccessOpen weights
Monthly Downloads21.1k

Runs On

What it takes to serve ModernBERT-large-nli (396M parameters): the memory its weights need at each precision, and the cheapest way to rent enough data-center GPUs to hold them.

PrecisionWeightsMemory neededCheapest setupPer hourAlso fits
16-bit 0.8 GB 1.0 GB 1x MI300X (192 GB)
Vultr
$1.85 1x H100 $1.99 · 1x MI325X $2.00
8-bit 0.4 GB 0.5 GB 1x MI300X (192 GB)
Vultr
$1.85 1x H100 $1.99 · 1x MI325X $2.00
4-bit 0.2 GB 0.2 GB 1x MI300X (192 GB)
Vultr
$1.85 1x H100 $1.99 · 1x MI325X $2.00

Memory is the weights at that precision plus 20% for the runtime and a short context; a long context needs more. Prices are the lowest on-demand hourly rates in the SAVRN Index, read Oct 13, 2026.

ModernBERT-large-nli on every accelerator the SAVRN Index prices, at every precision

Model Card

By Tasksource, published under apache-2.0, revision ca476cb923a8.

Model Card for Model ID

This model is ModernBERT multi-task fine-tuned on tasksource NLI tasks, including MNLI, ANLI, SICK, WANLI, doc-nli, LingNLI, FOLIO, FOL-NLI, LogicNLI, Label-NLI and all datasets in the below table). This is the equivalent of an "instruct" version. The model was trained for 200k steps on an Nvidia A30 GPU.

It is very good at reasoning tasks (better than llama 3.1 8B Instruct on ANLI and FOLIO), long context reasoning, sentiment analysis and zero-shot classification with new labels.

The following table shows model test accuracy. These are the scores for the same single transformer with different classification heads on top. Further gains can be obtained by fine-tuning on a single-task, e.g. SST, but it this checkpoint is great for zero-shot classification and natural language inference (contradiction/entailment/neutral classification).

Read the full model card (527 words)

Configuration

Architecture
ModernBertForSequenceClassification
Context length (tokens)
2,048
Layers
28
Hidden size
1,024
Feed-forward size
2,624
Attention heads
16
Vocabulary size
50,368
Stored precision
float32
Model type
modernbert

Identity and Version

Repository
tasksource/ModernBERT-large-nli
Publisher
Tasksource
Task
Zero-shot classification
Modality
Text
Library
transformers
Parameters
396M parameters
Languages
en
Revision
ca476cb923a8637073d4ceb0f19f7fc236e260d4
First published
2025-01-04
Last updated
2025-01-04

Files and Weights

7 files, 1.6 GB in total. The weights are 1 file totalling 1.6 GB in safetensors.

Weights1 file · 1.6 GB
Configuration2 files · 6.3 KB
Tokenizer2 files · 3.6 MB
Documentation1 file · 9.9 KB
Repository1 file · 1.5 KB
Every file
FileTypeSizeSHA-256
model.safetensorsWeights1.6 GB f56738ff6d17
config.jsonConfiguration5.6 KB —
special_tokens_map.jsonConfiguration694 B —
README.mdDocumentation9.9 KB —
.gitattributesRepository1.5 KB —
tokenizer.jsonTokenizer3.6 MB —
tokenizer_config.jsonTokenizer20.9 KB —

License and Download

License
apache-2.0
Access
Open weights, no gate
Download size
1.6 GB
Download from Tasksource

Released by Tasksource through its official repository on Hugging Face. Read the license.

Built From

Memory Requirements

PrecisionWeights in memory
As published1.6 GB
16-bit0.8 GB
8-bit0.4 GB
4-bit0.2 GB

Weights only, from the published parameter count; the key-value cache and runtime add to this.

Compare ModernBERT-large-nli

Questions About ModernBERT-large-nli

How much GPU memory does ModernBERT-large-nli need?

About 1 GB at 16-bit and 0.2 GB at 4-bit: the weights (396M parameters) plus a working margin. A long context needs more.

What is the cheapest GPU to run ModernBERT-large-nli on?

At 16-bit, 1x MI300X from $1.85 an hour; at 4-bit, 1x MI300X from $1.85 an hour, at the lowest on-demand prices the SAVRN Index lists.

Can I use ModernBERT-large-nli commercially?

Yes. ModernBERT-large-nli is released under Apache License 2.0. The Apache License 2.0 is a permissive open-source license. It permits commercial use, modification and redistribution. It requires keeping the license and copyright notices and any NOTICE file, stating significant changes, and it includes an express patent grant from contributors.

What is ModernBERT-large-nli's context length?

2,048 tokens, from the maximum position embeddings in its published configuration.

Similar Models

Model · Zero-shot classification

finecat-nli-l

Lee Miller

This model is a fine-tune of the excellent tasksource/ModernBERT-large-nli, trained on the dleemiller/FineCat-NLI dataset—a compilation of several high-quality NLI data sources with quality screening and reduction of easy samples in the training split. The training also incorporates logit distillation from MoritzLaurer/DeBERTa-v3-large-mnli-fever-anli-ling-wanli, a top-performing NLI model, particularly on ANLI benchmarks. \begin{equation} \mathcal{L} = \alpha \cdot \mathcal{L}{\text{CE}}(z^{(s)}, y) + \beta \cdot \mathcal{L}{\text{MSE}}(z^{(s)}, z^{(t)}) \end{equation} where \\(z^{(s)}\\) and \\(z^{(t)}\\) are the student and teacher logits, \\(y\\) are the ground truth labels, and…

Open weights 396M parameters 2,048 tokens sentence-transformers

Model · Zero-shot classification

finecat-nli-v2-l

Lee Miller

This model is a fine-tune of jhu-clsp/ettin-encoder-400m, trained on the dleemiller/FineCat-NLI-Reason dataset—a compilation of several high-quality NLI data sources with quality screening and reduction of easy samples in the training split—plus additional NLI, fact-verification and long-document sources (2.05M pairs in total). The training uses logit distillation from an NLI fine-tune of Qwen3.8-27B. \begin{equation} \mathcal{L} = \alpha \cdot \mathcal{L}{\text{CE}}(z^{(s)}, y) + \beta \cdot T^2 \cdot \mathrm{KL}\left(\sigma(z^{(t)}/T) \,\|\, \sigma(z^{(s)}/T)\right) \end{equation} where \\(z^{(s)}\\) and \\(z^{(t)}\\) are the student and teacher logits, \\(y\\) are the ground truth…

Open weights apache-2.0 396M parameters 7,999 tokens sentence-transformers

Model · Zero-shot classification

bart-large-mnli-yahoo-answers

Joe Davison

This model takes facebook/bart-large-mnli and fine-tunes it on Yahoo Answers topic classification. It can be used to predict whether a topic label can be assigned to a given sequence, whether or not the label has been seen before. You can play with an interactive demo of this zero-shot technique with this model, as well as the non-finetuned facebook/bart-large-mnli, here. This model was fine-tuned on topic classification and will perform best at zero-shot topic classification. Use hypothesistemplate="This text is about {}." as this is the template used during fine-tuning. For settings other than topic classification, you can use any model pre-trained on MNLI such as facebook/bart-large-mnli…

Open weights apache-2.0 407M parameters 1,024 tokens transformers

Model · Zero-shot classification

bart-large-mnli

AI at Meta

This is the checkpoint for bart-large after being trained on the MultiNLI (MNLI) dataset. - The bart-large model page - BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension Yin et al. proposed a method for using pre-trained NLI models as a ready-made zero-shot sequence classifiers. The method works by posing the sequence to be classified as the NLI premise and to construct a hypothesis from each candidate label. For example, if we want to evaluate whether a sequence belongs to the class "politics", we could construct a hypothesis of This text is about politics.. The probabilities for entailment and contradiction are then converted…

Open weights mit 407M parameters 1,024 tokens transformers

Model · Zero-shot classification

nitzotz

Netanel Elyasi

What it is. Nitzotz reads a Hebrew message and answers questions you type about it: pick one of several options, give a score on a scale, or say yes or no to a claim. For every answer it gives a probability you can trust, so you know when it is sure and when it is guessing. It does not write text, so it cannot make things up. It runs on a normal laptop, with no internet connection and no cost per question. What it is for. Deciding what to do with incoming messages: is this a scam, what kind of message is it, which department should get it, how urgent is it. It is not a chatbot and it is not built for long documents (see Limitations). In numbers. On 298 Hebrew messages it says correctly…

Open weights apache-2.0 384M parameters laya

Models in this series are designed for efficient zeroshot classification with the Hugging Face pipeline. These models can do classification without training data and run on both GPUs and CPUs. An overview of the latest zeroshot classifiers is available in my Zeroshot Classifier Collection. The main update of this zeroshot-v2.0 series of models is that several models are trained on fully commercially-friendly data for users with strict license requirements. These models can do one universal classification task: determine whether a hypothesis is "true" or "not true" given a text (entailment vs. notentailment). This task format is based on the Natural Language Inference task (NLI). The task is…

Open weights mit 435M parameters 512 tokens transformers