SAVRN
Search Contact SAVRN

Open-weight model · Zero-shot classification

nli-deberta-v3-xsmall

by Sentence Transformers - Cross-Encoders cross-encoder/nli-deberta-v3-xsmall

nli-deberta-v3-xsmall is an open-weight model for zero-shot classification from Sentence Transformers - Cross-Encoders, released under Apache License 2.0. It has 71M parameters and a 512-token context. At 16-bit it needs about 0.2 GB of GPU memory, which fits on 1x MI300X from $1.85 an hour, at the lowest prices in the SAVRN Index. It draws 69.9k downloads a month.

This model was trained using SentenceTransformers Cross-Encoder class. This model is based on microsoft/deberta-v3-xsmall The model was trained on the SNLI and MultiNLI datasets.

Parameters71M
Context512
Weights2.3 GB
Licenseapache-2.0
AccessOpen weights
Monthly Downloads69.9k

Runs On

What it takes to serve nli-deberta-v3-xsmall (71M parameters): the memory its weights need at each precision, and the cheapest way to rent enough data-center GPUs to hold them.

PrecisionWeightsMemory neededCheapest setupPer hourAlso fits
16-bit 0.1 GB 0.2 GB 1x MI300X (192 GB)
Vultr		 $1.85 1x H100 $1.99 · 1x MI325X $2.00
8-bit 0.1 GB 0.1 GB 1x MI300X (192 GB)
Vultr		 $1.85 1x H100 $1.99 · 1x MI325X $2.00
4-bit 0.0 GB 0.0 GB 1x MI300X (192 GB)
Vultr		 $1.85 1x H100 $1.99 · 1x MI325X $2.00

Memory is the weights at that precision plus 20% for the runtime and a short context; a long context needs more. Prices are the lowest on-demand hourly rates in the SAVRN Index, read Oct 9, 2026.

nli-deberta-v3-xsmall on every accelerator the SAVRN Index prices, at every precision

Model Card

By Sentence Transformers - Cross-Encoders, published under apache-2.0, revision a15087641532.

Cross-Encoder for Natural Language Inference

This model was trained using SentenceTransformers Cross-Encoder class. This model is based on microsoft/deberta-v3-xsmall

Training Data

The model was trained on the SNLI and MultiNLI datasets. For a given sentence pair, it will output three scores corresponding to the labels: contradiction, entailment, neutral.

Performance

  • Accuracy on SNLI-test dataset: 91.64
  • Accuracy on MNLI mismatched set: 87.77

For futher evaluation results, see SBERT.net - Pretrained Cross-Encoder.

Usage

Pre-trained models can be used like this:

from sentence_transformers import CrossEncoder
model = CrossEncoder('cross-encoder/nli-deberta-v3-xsmall')
scores = model.predict([('A man is eating pizza', 'A man eats something'), ('A black race car starts up in front of a crowd of people.', 'A man is driving down a lonely road.')])

#Convert scores to labels
label_mapping = ['contradiction', 'entailment', 'neutral']
labels = [label_mapping[score_max] for score_max in scores.argmax(axis=1)]

Usage with Transformers AutoModel

You can use the model also directly with Transformers library (without SentenceTransformers library):

Read the full model card (254 words)

Configuration

Architecture
DebertaV2ForSequenceClassification
Context length (tokens)
512
Layers
12
Hidden size
384
Feed-forward size
1,536
Attention heads
6
Vocabulary size
128,100
Stored precision
float32
Model type
deberta-v2

Identity and Version

Repository
cross-encoder/nli-deberta-v3-xsmall
Publisher
Sentence Transformers - Cross-Encoders
Task
Zero-shot classification
Modality
Text
Library
sentence-transformers
Parameters
71M parameters
Languages
en
Revision
a150876415327c80daeff35ca6f68f5ed8cf5c24
First published
2022-03-02
Last updated
2025-04-11

Files and Weights

20 files, 2.3 GB in total. The weights are 11 files totalling 2.3 GB in bin, onnx, safetensors.

Weights11 files · 2.3 GB
Configuration3 files · 1.4 KB
Tokenizer2 files · 8.7 MB
Documentation1 file · 2.9 KB
Other2 files · 2.5 MB
Repository1 file · 1.2 KB
Every file
FileTypeSizeSHA-256
model.safetensorsWeights283.4 MB 4e4fc4977f8d
onnx/model.onnxWeights284.2 MB 7105da41f625
onnx/model_O1.onnxWeights302.0 MB 67028fcdb307
onnx/model_O2.onnxWeights301.9 MB 41f436714a71
onnx/model_O3.onnxWeights301.9 MB e9c42b3c2931
onnx/model_O4.onnxWeights151.3 MB 7f3e118f3394
onnx/model_qint8_arm64.onnxWeights87.4 MB ef41fe8474f6
onnx/model_qint8_avx512.onnxWeights87.4 MB ef41fe8474f6
onnx/model_qint8_avx512_vnni.onnxWeights87.4 MB ef41fe8474f6
onnx/model_quint8_avx2.onnxWeights87.4 MB 21b14751a955
pytorch_model.binWeights283.4 MB 7bfaa3c5238c
added_tokens.jsonConfiguration26 B —
config.jsonConfiguration1.1 KB —
special_tokens_map.jsonConfiguration301 B —
README.mdDocumentation2.9 KB —
CESoftmaxAccuracyEvaluator_AllNLI-dev_results.csvOther681 B —
spm.modelOther2.5 MB c679fbf93643
.gitattributesRepository1.2 KB —
tokenizer.jsonTokenizer8.7 MB —
tokenizer_config.jsonTokenizer1.3 KB —

License and Download

License
apache-2.0
Access
Open weights, no gate
Download size
2.3 GB
Download from Sentence Transformers - Cross-Encoders

Released by Sentence Transformers - Cross-Encoders through its official repository on Hugging Face. Read the license.

Built From

  • Derived from microsoft/deberta-v3-xsmall
  • Quantized from microsoft/deberta-v3-xsmall
  • Trained on (disclosed) nyu-mll/multi_nli
  • Trained on (disclosed) stanfordnlp/snli

Memory Requirements

PrecisionWeights in memory
As published2.3 GB
16-bit0.1 GB
8-bit0.1 GB
4-bit0.0 GB

Weights only, from the published parameter count; the key-value cache and runtime add to this.

Built on This Model

Questions About nli-deberta-v3-xsmall

How much GPU memory does nli-deberta-v3-xsmall need?

About 0.2 GB at 16-bit and 0 GB at 4-bit: the weights (71M parameters) plus a working margin. A long context needs more.

What is the cheapest GPU to run nli-deberta-v3-xsmall on?

At 16-bit, 1x MI300X from $1.85 an hour; at 4-bit, 1x MI300X from $1.85 an hour, at the lowest on-demand prices the SAVRN Index lists.

Can I use nli-deberta-v3-xsmall commercially?

Yes. nli-deberta-v3-xsmall is released under Apache License 2.0. The Apache License 2.0 is a permissive open-source license. It permits commercial use, modification and redistribution. It requires keeping the license and copyright notices and any NOTICE file, stating significant changes, and it includes an express patent grant from contributors.

What is nli-deberta-v3-xsmall's context length?

512 tokens, from the maximum position embeddings in its published configuration.

Similar Models

This model was trained on 782 357 hypothesis-premise pairs from 4 NLI datasets: MultiNLI, Fever-NLI, LingNLI and ANLI. Note that the model was trained on binary NLI to predict either "entailment" or "not-entailment". This is specifically designed for zero-shot classification, where the difference between "neutral" and "contradiction" is irrelevant. The base model is DeBERTa-v3-xsmall from Microsoft. The v3 variant of DeBERTa substantially outperforms previous versions of the model by including a different pre-training objective, see the DeBERTa-V3 paper. For highest performance (but less speed), I recommend using…

Open weights mit 71M parameters 512 tokens transformers
View model

This model was fine-tuned using the same pipeline as described in the model card for MoritzLaurer/deberta-v3-large-zeroshot-v1.1-all-33 and in this paper. The foundation model is microsoft/deberta-v3-xsmall. The model only has 22 million backbone parameters and 128 million vocabulary parameters. The backbone parameters are the main parameters active during inference, providing a significant speedup over larger models. The model is 142 MB small. This model was trained to provide a small and highly efficient zeroshot option, especially for edge devices or in-browser use-cases with transformers.js. For usage instructions and other details refer to this model card…

Open weights mit 71M parameters 512 tokens transformers
View model

Model · Zero-shot classification

distilbert-base-uncased-mnli

Typeform

This is the uncased DistilBERT model fine-tuned on Multi-Genre Natural Language Inference (MNLI) dataset for the zero-shot classification task. This model can be used for text classification tasks. CONTENT WARNING: Readers should be aware this section contains content that is disturbing, offensive, and can propagate historical and current stereotypes. Significant research has explored bias and fairness issues with language models (see, e.g., Sheng et al. (2021) and Bender et al. (2021)). This model of DistilBERT-uncased is pretrained on the Multi-Genre Natural Language Inference (MultiNLI) corpus. It is a crowd-sourced collection of 433k sentence pairs annotated with textual entailment…

Open weights 67M parameters 512 tokens transformers
View model

This model was trained using SentenceTransformers Cross-Encoder class. The model was trained on the SNLI and MultiNLI datasets. For a given sentence pair, it will output three scores corresponding to the labels: contradiction, entailment, neutral. For evaluation results, see SBERT.net - Pretrained Cross-Encoder. Pre-trained models can be used like this: You can use the model also directly with Transformers library (without SentenceTransformers library): This model can also be used for zero-shot-classification

Open weights apache-2.0 82M parameters 514 tokens sentence-transformers
View model

This model was trained using SentenceTransformers Cross-Encoder class. The model was trained on the SNLI and MultiNLI datasets. For a given sentence pair, it will output three scores corresponding to the labels: contradiction, entailment, neutral. For evaluation results, see SBERT.net - Pretrained Cross-Encoder. Pre-trained models can be used like this: You can use the model also directly with Transformers library (without SentenceTransformers library): This model can also be used for zero-shot-classification

Open weights apache-2.0 82M parameters 514 tokens sentence-transformers
View model

multilingual - zero-shot-classification - text-classification - nli - pytorch - accuracy - multinli - xnli pipelinetag: zero-shot-classification candidatelabels: "politics, economy, entertainment, environment" This multilingual model can perform natural language inference (NLI) on 100+ languages and is therefore also suitable for multilingual zero-shot classification. The underlying multilingual-MiniLM-L6 model was created by Microsoft and was distilled from XLM-RoBERTa-large (see details in the original paper and newer information in this repo). The model was then fine-tuned on the XNLI dataset, which contains hypothesis-premise pairs from 15 languages, as well as the English MNLI dataset.…

Open weights mit 107M parameters 514 tokens transformers
View model