This multilingual model can perform natural language inference (NLI) on 100 languages and is therefore also suitable for multilingual zero-shot classification. The underlying mDeBERTa-v3-base model was pre-trained by Microsoft on the CC100 multilingual dataset with 100 languages. The model was then fine-tuned on the XNLI dataset and on the multilingual-NLI-26lang-2mil7 dataset. Both datasets contain more than 2.7 million hypothesis-premise pairs in 27 languages spoken by more than 4 billion people. As of December 2021, mDeBERTa-v3-base is the best performing multilingual base-sized transformer model introduced by Microsoft in this paper. This model was trained on the…
Open weights
mit
279M parameters
512 tokens
transformers
This model was trained on the MultiNLI, Fever-NLI and Adversarial-NLI (ANLI) datasets, which comprise 763 913 NLI hypothesis-premise pairs. This base model outperforms almost all large models on the ANLI benchmark. The base model is DeBERTa-v3-base from Microsoft. The v3 variant of DeBERTa substantially outperforms previous versions of the model by including a different pre-training objective, see annex 11 of the original DeBERTa paper. For highest performance (but less speed), I recommend using https://huggingface.co/MoritzLaurer/DeBERTa-v3-large-mnli-fever-anli-ling-wanli. DeBERTa-v3-base-mnli-fever-anli was trained on the MultiNLI, Fever-NLI and Adversarial-NLI (ANLI) datasets, which…
Open weights
mit
184M parameters
512 tokens
transformers
This multilingual model can perform natural language inference (NLI) on 100 languages and is therefore also suitable for multilingual zero-shot classification. The underlying model was pre-trained by Microsoft on the CC100 multilingual dataset. It was then fine-tuned on the XNLI dataset, which contains hypothesis-premise pairs from 15 languages, as well as the English MNLI dataset. As of December 2021, mDeBERTa-base is the best performing multilingual base-sized transformer model, introduced by Microsoft in this paper. If you are looking for a smaller, faster (but less performant) model, you can try multilingual-MiniLMv2-L6-mnli-xnli. This model was trained on the XNLI development dataset…
Open weights
mit
279M parameters
512 tokens
transformers
This model was fine-tuned on the MultiNLI, Fever-NLI, Adversarial-NLI (ANLI), LingNLI and WANLI datasets, which comprise 885 242 NLI hypothesis-premise pairs. This model is the best performing NLI model on the Hugging Face Hub as of 06.06.22 and can be used for zero-shot classification. It significantly outperforms all other large models on the ANLI benchmark. The foundation model is DeBERTa-v3-large from Microsoft. DeBERTa-v3 combines several recent innovations compared to classical Masked Language Models like BERT, RoBERTa etc., see the paper DeBERTa-v3-large-mnli-fever-anli-ling-wanli was trained on the MultiNLI, Fever-NLI, Adversarial-NLI (ANLI), LingNLI and WANLI datasets, which…
Open weights
mit
435M parameters
512 tokens
transformers
Models in this series are designed for efficient zeroshot classification with the Hugging Face pipeline. These models can do classification without training data and run on both GPUs and CPUs. An overview of the latest zeroshot classifiers is available in my Zeroshot Classifier Collection. The main update of this zeroshot-v2.0 series of models is that several models are trained on fully commercially-friendly data for users with strict license requirements. These models can do one universal classification task: determine whether a hypothesis is "true" or "not true" given a text (entailment vs. notentailment). This task format is based on the Natural Language Inference task (NLI). The task is…
Open weights
mit
435M parameters
512 tokens
transformers
Models in this series are designed for efficient zeroshot classification with the Hugging Face pipeline. These models can do classification without training data and run on both GPUs and CPUs. An overview of the latest zeroshot classifiers is available in my Zeroshot Classifier Collection. The main update of this zeroshot-v2.0 series of models is that several models are trained on fully commercially-friendly data for users with strict license requirements. These models can do one universal classification task: determine whether a hypothesis is "true" or "not true" given a text (entailment vs. notentailment). This task format is based on the Natural Language Inference task (NLI). The task is…
Open weights
mit
568M parameters
8,194 tokens
transformers
This model was fine-tuned using the same pipeline as described in the model card for MoritzLaurer/deberta-v3-large-zeroshot-v1.1-all-33 and in this paper. The foundation model is microsoft/deberta-v3-xsmall. The model only has 22 million backbone parameters and 128 million vocabulary parameters. The backbone parameters are the main parameters active during inference, providing a significant speedup over larger models. The model is 142 MB small. This model was trained to provide a small and highly efficient zeroshot option, especially for edge devices or in-browser use-cases with transformers.js. For usage instructions and other details refer to this model card…
Open weights
mit
71M parameters
512 tokens
transformers
This model was trained on 1.279.665 hypothesis-premise pairs from 8 NLI datasets: MultiNLI, Fever-NLI, LingNLI and DocNLI (which includes ANLI, QNLI, DUC, CNN/DailyMail, Curation). It is the only model in the model hub trained on 8 NLI datasets, including DocNLI with very long texts to learn long range reasoning. Note that the model was trained on binary NLI to predict either "entailment" or "not-entailment". The DocNLI merges the classes "neural" and "contradiction" into "not-entailment" to enable the inclusion of the DocNLI dataset. The base model is DeBERTa-v3-base from Microsoft. The v3 variant of DeBERTa substantially outperforms previous versions of the model by including a different…
Open weights
mit
184M parameters
512 tokens
transformers
multilingual - zero-shot-classification - text-classification - nli - pytorch - accuracy - multinli - xnli pipelinetag: zero-shot-classification candidatelabels: "politics, economy, entertainment, environment" This multilingual model can perform natural language inference (NLI) on 100+ languages and is therefore also suitable for multilingual zero-shot classification. The underlying multilingual-MiniLM-L12 model was created by Microsoft and was distilled from XLM-RoBERTa-large (see details in the original paper and newer information in this repo). The model was then fine-tuned on the XNLI dataset, which contains hypothesis-premise pairs from 15 languages, as well as the English MNLI…
Open weights
mit
118M parameters
514 tokens
transformers
The model is designed for zero-shot classification with the Hugging Face pipeline. The model should be substantially better at zero-shot classification than my other zero-shot models on the The model can do one universal task: determine whether a hypothesis is true or nottrue given a text (also called entailment vs. notentailment). This task format is based on the Natural Language Inference task (NLI). The task is so universal that any classification task can be reformulated into the task. The model was trained on a mixture of 27 tasks and 310 classes that have been reformatted into this universal format. 1. 26 classification tasks with ~400k texts: 'amazonpolarity', 'imdb', 'appreviews'…
Open weights
mit
435M parameters
512 tokens
transformers
Models in this series are designed for efficient zeroshot classification with the Hugging Face pipeline. These models can do classification without training data and run on both GPUs and CPUs. An overview of the latest zeroshot classifiers is available in my Zeroshot Classifier Collection. The main update of this zeroshot-v2.0 series of models is that several models are trained on fully commercially-friendly data for users with strict license requirements. These models can do one universal classification task: determine whether a hypothesis is "true" or "not true" given a text (entailment vs. notentailment). This task format is based on the Natural Language Inference task (NLI). The task is…
Open weights
mit
184M parameters
512 tokens
transformers
The model is designed for zero-shot classification with the Hugging Face pipeline. The model can do one universal classification task: determine whether a hypothesis is "true" or "not true" given a text (entailment vs. notentailment). This task format is based on the Natural Language Inference task (NLI). The task is so universal that any classification task can be reformulated into this task. A detailed description of how the model was trained and how it can be used is available in this paper. The model was trained on a mixture of 33 datasets and 387 classes that have been reformatted into this universal format. 1. Five NLI datasets with ~885k texts: "mnli", "anli", "fever", "wanli"…
Open weights
mit
184M parameters
512 tokens
transformers
This model was trained on 782 357 hypothesis-premise pairs from 4 NLI datasets: MultiNLI, Fever-NLI, LingNLI and ANLI. Note that the model was trained on binary NLI to predict either "entailment" or "not-entailment". This is specifically designed for zero-shot classification, where the difference between "neutral" and "contradiction" is irrelevant. The base model is DeBERTa-v3-xsmall from Microsoft. The v3 variant of DeBERTa substantially outperforms previous versions of the model by including a different pre-training objective, see the DeBERTa-V3 paper. For highest performance (but less speed), I recommend using…
Open weights
mit
71M parameters
512 tokens
transformers
multilingual - zero-shot-classification - text-classification - nli - pytorch - accuracy - multinli - xnli pipelinetag: zero-shot-classification candidatelabels: "politics, economy, entertainment, environment" This multilingual model can perform natural language inference (NLI) on 100+ languages and is therefore also suitable for multilingual zero-shot classification. The underlying multilingual-MiniLM-L6 model was created by Microsoft and was distilled from XLM-RoBERTa-large (see details in the original paper and newer information in this repo). The model was then fine-tuned on the XNLI dataset, which contains hypothesis-premise pairs from 15 languages, as well as the English MNLI dataset.…
Open weights
mit
107M parameters
514 tokens
transformers
Models in this series are designed for efficient zeroshot classification with the Hugging Face pipeline. These models can do classification without training data and run on both GPUs and CPUs. An overview of the latest zeroshot classifiers is available in my Zeroshot Classifier Collection. The main update of this zeroshot-v2.0 series of models is that several models are trained on fully commercially-friendly data for users with strict license requirements. These models can do one universal classification task: determine whether a hypothesis is "true" or "not true" given a text (entailment vs. notentailment). This task format is based on the Natural Language Inference task (NLI). The task is…
Open weights
mit
125M parameters
514 tokens
transformers
This model was fine-tuned using the same pipeline as described in the model card for MoritzLaurer/deberta-v3-large-zeroshot-v1.1-all-33 and in this paper. The foundation model is microsoft/xtremedistil-l6-h256-uncased. The model only has 22 million backbone parameters and 30 million vocabulary parameters. The backbone parameters are the main parameters active during inference, providing a significant speedup over larger models. The model is 25 MB small. This model was trained to provide a very small and highly efficient zeroshot option, especially for edge devices or in-browser use-cases with transformers.js. For usage instructions and other details refer to this model card…
Open weights
mit
13M parameters
512 tokens
transformers
The model is designed for zero-shot classification with the Hugging Face pipeline. The model can do one universal classification task: determine whether a hypothesis is "true" or "not true" given a text (entailment vs. notentailment). This task format is based on the Natural Language Inference task (NLI). The task is so universal that any classification task can be reformulated into this task. A detailed description of how the model was trained and how it can be used is available in this paper. The model was trained on a mixture of 33 datasets and 387 classes that have been reformatted into this universal format. 1. Five NLI datasets with ~885k texts: "mnli", "anli", "fever", "wanli"…
Open weights
mit
435M parameters
512 tokens
transformers
This model was trained on the MultiNLI dataset, which consists of 392 702 NLI hypothesis-premise pairs. The base model is DeBERTa-v3-base from Microsoft. The v3 variant of DeBERTa substantially outperforms previous versions of the model by including a different pre-training objective, see annex 11 of the original DeBERTa paper. For a more powerful model, check out DeBERTa-v3-base-mnli-fever-anli which was trained on even more data. This model was trained on the MultiNLI dataset, which consists of 392 702 NLI hypothesis-premise pairs. DeBERTa-v3-base-mnli was trained using the Hugging Face trainer with the following hyperparameters. The model was evaluated using the matched test set and…
Open weights
184M parameters
512 tokens
transformers
Models in this series are designed for efficient zeroshot classification with the Hugging Face pipeline. These models can do classification without training data and run on both GPUs and CPUs. An overview of the latest zeroshot classifiers is available in my Zeroshot Classifier Collection. The main update of this zeroshot-v2.0 series of models is that several models are trained on fully commercially-friendly data for users with strict license requirements. These models can do one universal classification task: determine whether a hypothesis is "true" or "not true" given a text (entailment vs. notentailment). This task format is based on the Natural Language Inference task (NLI). The task is…
Open weights
mit
435M parameters
512 tokens
transformers
