SAVRN
Search Contact SAVRN

Open-weight model · Zero-shot classification

bert-base-spanish-wwm-cased-xnli

by Recognai Recognai/bert-base-spanish-wwm-cased-xnli

UPDATE, 15.10.2021: Check out our new zero-shot classifiers, much more lightweight and even outperforming this one: zero-shot SELECTRA small and zero-shot SELECTRA medium.

Parameters110M
Context512
Weights1.3 GB
Licensemit
AccessOpen weights
Monthly Downloads3.6k

Runs On

What it takes to serve bert-base-spanish-wwm-cased-xnli (110M parameters): the memory its weights need at each precision, and the cheapest way to rent enough data-center GPUs to hold them.

PrecisionWeightsMemory neededCheapest setupPer hourAlso fits
16-bit 0.2 GB 0.3 GB 1x MI300X (192 GB)
Vultr		 $1.85 1x H100 $1.99 · 1x MI325X $2.00
8-bit 0.1 GB 0.1 GB 1x MI300X (192 GB)
Vultr		 $1.85 1x H100 $1.99 · 1x MI325X $2.00
4-bit 0.1 GB 0.1 GB 1x MI300X (192 GB)
Vultr		 $1.85 1x H100 $1.99 · 1x MI325X $2.00

Memory is the weights at that precision plus 20% for the runtime and a short context; a long context needs more. Prices are the lowest on-demand hourly rates in the SAVRN Index, read Sep 18, 2026.

Model Card

By Recognai, published under mit, revision e44f774ea40e.

UPDATE, 15.10.2021: Check out our new zero-shot classifiers, much more lightweight and even outperforming this one: zero-shot SELECTRA small and zero-shot SELECTRA medium. This model is a fine-tuned version of the spanish BERT model with the Spanish portion of the XNLI dataset. You can have a look at the training script for details of the training. You can use this model with Hugging Face's zero-shot-classification pipeline

Read Recognai's full model card

UPDATE, 15.10.2021: Check out our new zero-shot classifiers, much more lightweight and even outperforming this one: zero-shot SELECTRA small and zero-shot SELECTRA medium.

Model description

This model is a fine-tuned version of the spanish BERT model with the Spanish portion of the XNLI dataset. You can have a look at the training script for details of the training.

How to use

You can use this model with Hugging Face's zero-shot-classification pipeline:

from transformers import pipeline
classifier = pipeline("zero-shot-classification", 
                       model="Recognai/bert-base-spanish-wwm-cased-xnli")

classifier(
    "El autor se perfila, a los 50 años de su muerte, como uno de los grandes de su siglo",
    candidate_labels=["cultura", "sociedad", "economia", "salud", "deportes"],
    hypothesis_template="Este ejemplo es {}."
)
"""output
{'sequence': 'El autor se perfila, a los 50 años de su muerte, como uno de los grandes de su siglo',
 'labels': ['cultura', 'sociedad', 'economia', 'salud', 'deportes'],
 'scores': [0.38897448778152466,
  0.22997373342514038,
  0.1658431738615036,
  0.1205764189362526,
  0.09463217109441757]}
"""

Eval results

Accuracy for the test set:

XNLI-es
bert-base-spanish-wwm-cased-xnli 79.9%

Configuration

Architecture
BertForSequenceClassification
Context length (tokens)
512
Layers
12
Hidden size
768
Feed-forward size
3,072
Attention heads
12
Vocabulary size
31,002
Model type
bert

Identity and Version

Repository
Recognai/bert-base-spanish-wwm-cased-xnli
Publisher
Recognai
Task
Zero-shot classification
Modality
Text
Library
transformers
Parameters
110M parameters
Languages
es
Revision
e44f774ea40ee8202311d9e5448263f176e04796
First published
2022-03-02
Last updated
2023-07-14

Files and Weights

12 files, 1.3 GB in total. The weights are 4 files totalling 1.3 GB in bin, msgpack, safetensors.

Weights4 files · 1.3 GB
Configuration4 files · 63.9 KB
Tokenizer2 files · 242.6 KB
Documentation1 file · 2.1 KB
Repository1 file · 790 B
Every file
FileTypeSizeSHA-256
flax_model.msgpackWeights439.4 MB c4184bb3be20
model.safetensorsWeights439.4 MB 18edffe809d9
pytorch_model.binWeights439.5 MB d8f7f0e947cf
training_args.binWeights2.1 KB 622587664ec8
config.jsonConfiguration834 B —
special_tokens_map.jsonConfiguration112 B —
trainer_state.jsonConfiguration56.3 KB —
zeroshot_training_script.pyConfiguration6.6 KB —
README.mdDocumentation2.1 KB —
.gitattributesRepository790 B —
tokenizer_config.jsonTokenizer528 B —
vocab.txtTokenizer242.1 KB —

License and Download

License
mit
Access
Open weights, no gate
Download size
1.3 GB
Download from Recognai

Released by Recognai through its official repository on Hugging Face. Read the license.

Built From

  • Trained on (disclosed) xnli

Memory Requirements

PrecisionWeights in memory
As published1.3 GB
16-bit0.2 GB
8-bit0.1 GB
4-bit0.1 GB

Weights only, from the published parameter count; the key-value cache and runtime add to this.

Questions About bert-base-spanish-wwm-cased-xnli

How much GPU memory does bert-base-spanish-wwm-cased-xnli need?

About 0.3 GB at 16-bit and 0.1 GB at 4-bit: the weights (110M parameters) plus a working margin. A long context needs more.

What is the cheapest GPU to run bert-base-spanish-wwm-cased-xnli on?

At 16-bit, 1x MI300X from $1.85 an hour; at 4-bit, 1x MI300X from $1.85 an hour, at the lowest on-demand prices the SAVRN Index lists.

Can I use bert-base-spanish-wwm-cased-xnli commercially?

Yes. bert-base-spanish-wwm-cased-xnli is released under MIT License. The MIT License is a short permissive license. It permits commercial use, modification and redistribution, provided the copyright notice and permission notice are included.

What is bert-base-spanish-wwm-cased-xnli's context length?

512 tokens, from the maximum position embeddings in its published configuration.

Similar Models

multilingual - zero-shot-classification - text-classification - nli - pytorch - accuracy - multinli - xnli pipelinetag: zero-shot-classification candidatelabels: "politics, economy, entertainment, environment" This multilingual model can perform natural language inference (NLI) on 100+ languages and is therefore also suitable for multilingual zero-shot classification. The underlying multilingual-MiniLM-L6 model was created by Microsoft and was distilled from XLM-RoBERTa-large (see details in the original paper and newer information in this repo). The model was then fine-tuned on the XNLI dataset, which contains hypothesis-premise pairs from 15 languages, as well as the English MNLI dataset.…

Open weights mit 107M parameters 514 tokens transformers
View model

multilingual - zero-shot-classification - text-classification - nli - pytorch - accuracy - multinli - xnli pipelinetag: zero-shot-classification candidatelabels: "politics, economy, entertainment, environment" This multilingual model can perform natural language inference (NLI) on 100+ languages and is therefore also suitable for multilingual zero-shot classification. The underlying multilingual-MiniLM-L12 model was created by Microsoft and was distilled from XLM-RoBERTa-large (see details in the original paper and newer information in this repo). The model was then fine-tuned on the XNLI dataset, which contains hypothesis-premise pairs from 15 languages, as well as the English MNLI…

Open weights mit 118M parameters 514 tokens transformers
View model

Models in this series are designed for efficient zeroshot classification with the Hugging Face pipeline. These models can do classification without training data and run on both GPUs and CPUs. An overview of the latest zeroshot classifiers is available in my Zeroshot Classifier Collection. The main update of this zeroshot-v2.0 series of models is that several models are trained on fully commercially-friendly data for users with strict license requirements. These models can do one universal classification task: determine whether a hypothesis is "true" or "not true" given a text (entailment vs. notentailment). This task format is based on the Natural Language Inference task (NLI). The task is…

Open weights mit 125M parameters 514 tokens transformers
View model

Model · Zero-shot classification

nli-roberta-base

Sentence Transformers - Cross-Encoders

This model was trained using SentenceTransformers Cross-Encoder class. The model was trained on the SNLI and MultiNLI datasets. For a given sentence pair, it will output three scores corresponding to the labels: contradiction, entailment, neutral. For evaluation results, see SBERT.net - Pretrained Cross-Encoder. Pre-trained models can be used like this: You can use the model also directly with Transformers library (without SentenceTransformers library): This model can also be used for zero-shot-classification

Open weights apache-2.0 125M parameters 514 tokens sentence-transformers
View model

This model was trained using SentenceTransformers Cross-Encoder class. The model was trained on the SNLI and MultiNLI datasets. For a given sentence pair, it will output three scores corresponding to the labels: contradiction, entailment, neutral. For evaluation results, see SBERT.net - Pretrained Cross-Encoder. Pre-trained models can be used like this: You can use the model also directly with Transformers library (without SentenceTransformers library): This model can also be used for zero-shot-classification

Open weights apache-2.0 82M parameters 514 tokens sentence-transformers
View model

This model was trained using SentenceTransformers Cross-Encoder class. The model was trained on the SNLI and MultiNLI datasets. For a given sentence pair, it will output three scores corresponding to the labels: contradiction, entailment, neutral. For evaluation results, see SBERT.net - Pretrained Cross-Encoder. Pre-trained models can be used like this: You can use the model also directly with Transformers library (without SentenceTransformers library): This model can also be used for zero-shot-classification

Open weights apache-2.0 82M parameters 514 tokens sentence-transformers
View model