Models in this series are designed for efficient zeroshot classification with the Hugging Face pipeline. These models can do classification without training data and run on both GPUs and CPUs. An overview of the latest zeroshot classifiers is available in my Zeroshot Classifier Collection. The main update of this zeroshot-v2.0 series of models is that several models are trained on fully commercially-friendly data for users with strict license requirements. These models can do one universal classification task: determine whether a hypothesis is "true" or "not true" given a text (entailment vs. notentailment). This task format is based on the Natural Language Inference task (NLI). The task is…
Open weights
mit
184M parameters
512 tokens
transformers
The model is designed for zero-shot classification with the Hugging Face pipeline. The model can do one universal classification task: determine whether a hypothesis is "true" or "not true" given a text (entailment vs. notentailment). This task format is based on the Natural Language Inference task (NLI). The task is so universal that any classification task can be reformulated into this task. A detailed description of how the model was trained and how it can be used is available in this paper. The model was trained on a mixture of 33 datasets and 387 classes that have been reformatted into this universal format. 1. Five NLI datasets with ~885k texts: "mnli", "anli", "fever", "wanli"…
Open weights
mit
184M parameters
512 tokens
transformers
deberta-v3-base with context length of 1280 fine-tuned on tasksource for 250k steps. I oversampled long NLI tasks (ConTRoL, doc-nli). Training data include helpsteer v1/v2, logical reasoning tasks (FOLIO, FOL-nli, LogicNLI...), OASST, hh/rlhf, linguistics oriented NLI tasks, tasksource-dpo, fact verification tasks. This checkpoint has strong zero-shot validation performance on many tasks (e.g. 70% on WNLI), and can be used for: - Zero-shot entailment-based classification for arbitrary labels [ZS]. - Natural language inference [NLI] - Further fine-tuning on a new task or tasksource task (classification, token classification, reward modeling or multiple-choice) [FT]. Zero-shot GPT-4 scores…
Open weights
apache-2.0
184M parameters
1,280 tokens
transformers
This model was trained using SentenceTransformers Cross-Encoder class. This model is based on microsoft/deberta-v3-base The model was trained on the SNLI and MultiNLI datasets. For a given sentence pair, it will output three scores corresponding to the labels: contradiction, entailment, neutral. For futher evaluation results, see SBERT.net - Pretrained Cross-Encoder. Pre-trained models can be used like this: You can use the model also directly with Transformers library (without SentenceTransformers library): This model can also be used for zero-shot-classification
Open weights
apache-2.0
184M parameters
512 tokens
sentence-transformers
This model was trained on the MultiNLI, Fever-NLI and Adversarial-NLI (ANLI) datasets, which comprise 763 913 NLI hypothesis-premise pairs. This base model outperforms almost all large models on the ANLI benchmark. The base model is DeBERTa-v3-base from Microsoft. The v3 variant of DeBERTa substantially outperforms previous versions of the model by including a different pre-training objective, see annex 11 of the original DeBERTa paper. For highest performance (but less speed), I recommend using https://huggingface.co/MoritzLaurer/DeBERTa-v3-large-mnli-fever-anli-ling-wanli. DeBERTa-v3-base-mnli-fever-anli was trained on the MultiNLI, Fever-NLI and Adversarial-NLI (ANLI) datasets, which…
Open weights
mit
184M parameters
512 tokens
transformers
NOTE Deprecated: use https://huggingface.co/tasksource/deberta-small-long-nli for longer context and better accuracy. This is DeBERTa-v3-base fine-tuned with multi-task learning on 600+ tasks of the tasksource collection. This checkpoint has strong zero-shot validation performance on many tasks (e.g. 70% on WNLI), and can be used for: - Zero-shot entailment-based classification for arbitrary labels [ZS]. - Natural language inference [NLI] - Hundreds of previous tasks with tasksource-adapters [TA]. - Further fine-tuning on a new task or tasksource task (classification, token classification or multiple-choice) [FT]. NLI training data of this model includes label-nli, a NLI dataset specially…
Open weights
apache-2.0
184M parameters
512 tokens
transformers
