SAVRN
Search Contact SAVRN

Open-weight model · Zero-shot classification

nli-deberta-v3-base

by Sentence Transformers - Cross-Encoders cross-encoder/nli-deberta-v3-base

This model was trained using SentenceTransformers Cross-Encoder class. This model is based on microsoft/deberta-v3-base The model was trained on the SNLI and MultiNLI datasets.

Parameters184M
Context512
Weights5.9 GB
Licenseapache-2.0
AccessOpen weights
Monthly Downloads714.4k

Runs On

What it takes to serve nli-deberta-v3-base (184M parameters): the memory its weights need at each precision, and the cheapest way to rent enough data-center GPUs to hold them.

PrecisionWeightsMemory neededCheapest setupPer hourAlso fits
16-bit 0.4 GB 0.4 GB 1x MI300X (192 GB)
Vultr		 $1.85 1x H100 $1.99 · 1x MI325X $2.00
8-bit 0.2 GB 0.2 GB 1x MI300X (192 GB)
Vultr		 $1.85 1x H100 $1.99 · 1x MI325X $2.00
4-bit 0.1 GB 0.1 GB 1x MI300X (192 GB)
Vultr		 $1.85 1x H100 $1.99 · 1x MI325X $2.00

Memory is the weights at that precision plus 20% for the runtime and a short context; a long context needs more. Prices are the lowest on-demand hourly rates in the SAVRN Index, read Sep 18, 2026.

SAVRN's Notes on nli-deberta-v3-base

Sentence pairs go in, three scores come out: contradiction, entailment, neutral. That is the whole job of this cross-encoder, and it is why it turns up in zero-shot classification, where each label becomes a hypothesis the model tests against the text. At 184 million parameters it needs 0.4 GB at 16-bit and 0.1 GB at 4-bit, so it rides beside whatever larger models already occupy the GPU. The cheapest listed setup, one MI300X with 192 GB at $1.85 per hour, leaves almost all of that memory free.

Apache 2.0 allows commercial use, modification and redistribution, with notice obligations and a patent grant, so the license adds no deployment friction. Check the 512-token context: anything longer has to be chunked upstream. It derives from microsoft/deberta-v3-base and was trained on SNLI and MultiNLI, which tells you the kind of input it expects. It also ships in ONNX form.

SAVRN Research, 2026-09-18

Model Card

By Sentence Transformers - Cross-Encoders, published under apache-2.0, revision 6c749ce3425c.

Cross-Encoder for Natural Language Inference

This model was trained using SentenceTransformers Cross-Encoder class. This model is based on microsoft/deberta-v3-base

Training Data

The model was trained on the SNLI and MultiNLI datasets. For a given sentence pair, it will output three scores corresponding to the labels: contradiction, entailment, neutral.

Performance

  • Accuracy on SNLI-test dataset: 92.38
  • Accuracy on MNLI mismatched set: 90.04

For futher evaluation results, see SBERT.net - Pretrained Cross-Encoder.

Usage

Pre-trained models can be used like this:

from sentence_transformers import CrossEncoder
model = CrossEncoder('cross-encoder/nli-deberta-v3-base')
scores = model.predict([('A man is eating pizza', 'A man eats something'), ('A black race car starts up in front of a crowd of people.', 'A man is driving down a lonely road.')])

#Convert scores to labels
label_mapping = ['contradiction', 'entailment', 'neutral']
labels = [label_mapping[score_max] for score_max in scores.argmax(axis=1)]

Usage with Transformers AutoModel

You can use the model also directly with Transformers library (without SentenceTransformers library):

Read the full model card (254 words)

Configuration

Architecture
DebertaV2ForSequenceClassification
Context length (tokens)
512
Layers
12
Hidden size
768
Feed-forward size
3,072
Attention heads
12
Vocabulary size
128,100
Stored precision
float32
Model type
deberta-v2

Identity and Version

Repository
cross-encoder/nli-deberta-v3-base
Publisher
Sentence Transformers - Cross-Encoders
Task
Zero-shot classification
Modality
Text
Library
sentence-transformers
Parameters
184M parameters
Languages
en
Revision
6c749ce3425cd33b46d187e45b92bbf96ee12ec7
First published
2022-03-02
Last updated
2025-04-11

Files and Weights

20 files, 5.9 GB in total. The weights are 11 files totalling 5.9 GB in bin, onnx, safetensors.

Weights11 files · 5.9 GB
Configuration3 files · 1.4 KB
Tokenizer2 files · 8.7 MB
Documentation1 file · 2.9 KB
Other2 files · 2.5 MB
Repository1 file · 1.2 KB
Every file
FileTypeSizeSHA-256
model.safetensorsWeights737.7 MB d8148c6d49e0
onnx/model.onnxWeights738.6 MB 369ebf04fce8
onnx/model_O1.onnxWeights774.4 MB fbaee5c2dbfc
onnx/model_O2.onnxWeights774.3 MB 8c48d9940934
onnx/model_O3.onnxWeights774.3 MB e6c5190cc656
onnx/model_O4.onnxWeights387.5 MB 6d09dc7c7de2
onnx/model_qint8_arm64.onnxWeights244.4 MB cc4ef779fe3c
onnx/model_qint8_avx512.onnxWeights244.4 MB cc4ef779fe3c
onnx/model_qint8_avx512_vnni.onnxWeights244.4 MB cc4ef779fe3c
onnx/model_quint8_avx2.onnxWeights244.4 MB 36784d97274e
pytorch_model.binWeights737.8 MB 947f94dbf29b
added_tokens.jsonConfiguration26 B —
config.jsonConfiguration1.1 KB —
special_tokens_map.jsonConfiguration301 B —
README.mdDocumentation2.9 KB —
CESoftmaxAccuracyEvaluator_AllNLI-dev_results.csvOther682 B —
spm.modelOther2.5 MB c679fbf93643
.gitattributesRepository1.2 KB —
tokenizer.jsonTokenizer8.7 MB —
tokenizer_config.jsonTokenizer1.3 KB —

License and Download

License
apache-2.0
Access
Open weights, no gate
Download size
5.9 GB
Download from Sentence Transformers - Cross-Encoders

Released by Sentence Transformers - Cross-Encoders through its official repository on Hugging Face. Read the license.

Built From

Memory Requirements

PrecisionWeights in memory
As published5.9 GB
16-bit0.4 GB
8-bit0.2 GB
4-bit0.1 GB

Weights only, from the published parameter count; the key-value cache and runtime add to this.

Questions About nli-deberta-v3-base

How much GPU memory does nli-deberta-v3-base need?

About 0.4 GB at 16-bit and 0.1 GB at 4-bit: the weights (184M parameters) plus a working margin. A long context needs more.

What is the cheapest GPU to run nli-deberta-v3-base on?

At 16-bit, 1x MI300X from $1.85 an hour; at 4-bit, 1x MI300X from $1.85 an hour, at the lowest on-demand prices the SAVRN Index lists.

Can I use nli-deberta-v3-base commercially?

Yes. nli-deberta-v3-base is released under Apache License 2.0. The Apache License 2.0 is a permissive open-source license. It permits commercial use, modification and redistribution. It requires keeping the license and copyright notices and any NOTICE file, stating significant changes, and it includes an express patent grant from contributors.

What is nli-deberta-v3-base's context length?

512 tokens, from the maximum position embeddings in its published configuration.

Similar Models

This model was trained on the MultiNLI, Fever-NLI and Adversarial-NLI (ANLI) datasets, which comprise 763 913 NLI hypothesis-premise pairs. This base model outperforms almost all large models on the ANLI benchmark. The base model is DeBERTa-v3-base from Microsoft. The v3 variant of DeBERTa substantially outperforms previous versions of the model by including a different pre-training objective, see annex 11 of the original DeBERTa paper. For highest performance (but less speed), I recommend using https://huggingface.co/MoritzLaurer/DeBERTa-v3-large-mnli-fever-anli-ling-wanli. DeBERTa-v3-base-mnli-fever-anli was trained on the MultiNLI, Fever-NLI and Adversarial-NLI (ANLI) datasets, which…

Open weights mit 184M parameters 512 tokens transformers
View model

Model · Zero-shot classification

deberta-v3-base-tasksource-nli

Damien Sileo

NOTE Deprecated: use https://huggingface.co/tasksource/deberta-small-long-nli for longer context and better accuracy. This is DeBERTa-v3-base fine-tuned with multi-task learning on 600+ tasks of the tasksource collection. This checkpoint has strong zero-shot validation performance on many tasks (e.g. 70% on WNLI), and can be used for: - Zero-shot entailment-based classification for arbitrary labels [ZS]. - Natural language inference [NLI] - Hundreds of previous tasks with tasksource-adapters [TA]. - Further fine-tuning on a new task or tasksource task (classification, token classification or multiple-choice) [FT]. NLI training data of this model includes label-nli, a NLI dataset specially…

Open weights apache-2.0 184M parameters 512 tokens transformers
View model

This model was trained on the MultiNLI dataset, which consists of 392 702 NLI hypothesis-premise pairs. The base model is DeBERTa-v3-base from Microsoft. The v3 variant of DeBERTa substantially outperforms previous versions of the model by including a different pre-training objective, see annex 11 of the original DeBERTa paper. For a more powerful model, check out DeBERTa-v3-base-mnli-fever-anli which was trained on even more data. This model was trained on the MultiNLI dataset, which consists of 392 702 NLI hypothesis-premise pairs. DeBERTa-v3-base-mnli was trained using the Hugging Face trainer with the following hyperparameters. The model was evaluated using the matched test set and…

Open weights 184M parameters 512 tokens transformers
View model

Model · Zero-shot classification

deberta-base-long-nli

Tasksource

deberta-v3-base with context length of 1280 fine-tuned on tasksource for 250k steps. I oversampled long NLI tasks (ConTRoL, doc-nli). Training data include helpsteer v1/v2, logical reasoning tasks (FOLIO, FOL-nli, LogicNLI...), OASST, hh/rlhf, linguistics oriented NLI tasks, tasksource-dpo, fact verification tasks. This checkpoint has strong zero-shot validation performance on many tasks (e.g. 70% on WNLI), and can be used for: - Zero-shot entailment-based classification for arbitrary labels [ZS]. - Natural language inference [NLI] - Further fine-tuning on a new task or tasksource task (classification, token classification, reward modeling or multiple-choice) [FT]. Zero-shot GPT-4 scores…

Open weights apache-2.0 184M parameters 1,280 tokens transformers
View model

Models in this series are designed for efficient zeroshot classification with the Hugging Face pipeline. These models can do classification without training data and run on both GPUs and CPUs. An overview of the latest zeroshot classifiers is available in my Zeroshot Classifier Collection. The main update of this zeroshot-v2.0 series of models is that several models are trained on fully commercially-friendly data for users with strict license requirements. These models can do one universal classification task: determine whether a hypothesis is "true" or "not true" given a text (entailment vs. notentailment). This task format is based on the Natural Language Inference task (NLI). The task is…

Open weights mit 184M parameters 512 tokens transformers
View model

The model is designed for zero-shot classification with the Hugging Face pipeline. The model can do one universal classification task: determine whether a hypothesis is "true" or "not true" given a text (entailment vs. notentailment). This task format is based on the Natural Language Inference task (NLI). The task is so universal that any classification task can be reformulated into this task. A detailed description of how the model was trained and how it can be used is available in this paper. The model was trained on a mixture of 33 datasets and 387 classes that have been reformatted into this universal format. 1. Five NLI datasets with ~885k texts: "mnli", "anli", "fever", "wanli"…

Open weights mit 184M parameters 512 tokens transformers
View model