Damien Sileo

Model · Zero-shot classification

deberta-v3-base-tasksource-nli

Damien Sileo

NOTE Deprecated: use https://huggingface.co/tasksource/deberta-small-long-nli for longer context and better accuracy. This is DeBERTa-v3-base fine-tuned with multi-task learning on 600+ tasks of the tasksource collection. This checkpoint has strong zero-shot validation performance on many tasks (e.g. 70% on WNLI), and can be used for: - Zero-shot entailment-based classification for arbitrary labels [ZS]. - Natural language inference [NLI] - Hundreds of previous tasks with tasksource-adapters [TA]. - Further fine-tuning on a new task or tasksource task (classification, token classification or multiple-choice) [FT]. NLI training data of this model includes label-nli, a NLI dataset specially…

Model · Zero-shot classification

deberta-v3-large-tasksource-nli

Damien Sileo

DeBERTa-v3-large fine-tuned with multi-task learning on 600 tasks of the tasksource collection You can further fine-tune this model to use it for any classification or multiple-choice task. This checkpoint has strong zero-shot validation performance on many tasks (e.g. 77% on WNLI). The untuned model CLS embedding also has strong linear probing performance (90% on MNLI), due to the multitask training. This is the shared model with the MNLI classifier on top. Its encoder was trained on many datasets including bigbench, Anthropic rlhf, anli... alongside many NLI and classification tasks with a SequenceClassification heads while using only one shared encoder. Each task had a specific CLS…