This model is a conversion of MoritzLaurer/roberta-base-zeroshot-v2.0-c to ONNX format using the Optimum library.
Open-weight model · Zero-shot classification
distilbart-mnli-12-1
by Suraj Patil valhalla/distilbart-mnli-12-1
distilbart-mnli is the distilled version of bart-large-mnli created using the No Teacher Distillation technique proposed for BART summarisation by Huggingface, here. We just copy alternating layers from bart-large-mnli and finetune more on the same data.
Parameters—
Context1,024
Weights1.8 GB
License—
AccessOpen weights
Monthly Downloads37.3k
Model Card
distilbart-mnli is the distilled version of bart-large-mnli created using the No Teacher Distillation technique proposed for BART summarisation by Huggingface, here. We just copy alternating layers from bart-large-mnli and finetune more on the same data. This is a very simple and effective technique, as we can see the performance drop is very little. Detailed performace trade-offs will be posted in this sheet. If you want to train these models yourself, clone the distillbart-mnli repo and follow the steps below Clone and install transformers from source Download MNLI data Create student model Start fine-tuning You can find the logs of these trained models in this wandb project.
Excerpt from the card by Suraj Patil.
Configuration
- Architecture
- BartForSequenceClassification
- Context length (tokens)
- 1,024
- Layers
- 12
- Vocabulary size
- 50,265
- Model type
- bart
Identity and Version
- Repository
- valhalla/distilbart-mnli-12-1
- Publisher
- Suraj Patil
- Task
- Zero-shot classification
- Modality
- Text
- Library
- transformers
- Parameters
- Not stated by the source
- Languages
- jax
- Revision
- 506336d4214470e3b3b36021358daae28e25ceac
- First published
- 2022-03-02
- Last updated
- 2021-06-14
Files and Weights
9 files, 1.8 GB in total. The weights are 2 files totalling 1.8 GB in bin, msgpack.
Weights2 files · 1.8 GB
Configuration2 files · 2.2 KB
Tokenizer3 files · 1.4 MB
Documentation1 file · 2.4 KB
Repository1 file · 391 B
Every file
| File | Type | Size | SHA-256 |
|---|---|---|---|
| flax_model.msgpack | Weights | 890.3 MB | c1d623314815 |
| pytorch_model.bin | Weights | 890.4 MB | aa79ff59084a |
| config.json | Configuration | 1.4 KB | — |
| special_tokens_map.json | Configuration | 772 B | — |
| README.md | Documentation | 2.4 KB | — |
| .gitattributes | Repository | 391 B | — |
| merges.txt | Tokenizer | 456.3 KB | — |
| tokenizer_config.json | Tokenizer | 26 B | — |
| vocab.json | Tokenizer | 898.8 KB | — |
License and Download
- License
- Not stated by the source
- Access
- Open weights, no gate
- Download size
- 1.8 GB
Download from Suraj Patil
Released by Suraj Patil through its official repository on Hugging Face.
Built From
- Trained on (disclosed) mnli
Memory Requirements
| Precision | Weights in memory |
|---|---|
| As published | 1.8 GB |
Weights only, from the published parameter count; the key-value cache and runtime add to this.
Questions About distilbart-mnli-12-1
What is distilbart-mnli-12-1's context length?
1,024 tokens, from the maximum position embeddings in its published configuration.
Similar Models
https://huggingface.co/facebook/bart-large-mnli with ONNX weights to be compatible with Transformers.js. If you haven't already, you can install the Transformers.js JavaScript library from NPM using: Note: Having a separate repo for ONNX weights is intended to be a temporary solution until WebML gains more traction. If you would like to make your models web-ready, we recommend converting to ONNX using Optimum and structuring your repo like this one (with ONNX weights located in a subfolder named onnx).
distilbart-mnli is the distilled version of bart-large-mnli created using the No Teacher Distillation technique proposed for BART summarisation by Huggingface, here. We just copy alternating layers from bart-large-mnli and finetune more on the same data. This is a very simple and effective technique, as we can see the performance drop is very little. Detailed performace trade-offs will be posted in this sheet. If you want to train these models yourself, clone the distillbart-mnli repo and follow the steps below Clone and install transformers from source Download MNLI data Create student model Start fine-tuning You can find the logs of these trained models in this wandb project.
https://huggingface.co/typeform/mobilebert-uncased-mnli with ONNX weights to be compatible with Transformers.js. If you haven't already, you can install the Transformers.js JavaScript library from NPM using: Note: Having a separate repo for ONNX weights is intended to be a temporary solution until WebML gains more traction. If you would like to make your models web-ready, we recommend converting to ONNX using Optimum and structuring your repo like this one (with ONNX weights located in a subfolder named onnx).
https://huggingface.co/cross-encoder/nli-deberta-v3-xsmall with ONNX weights to be compatible with Transformers.js. If you haven't already, you can install the Transformers.js JavaScript library from NPM using: Note: Having a separate repo for ONNX weights is intended to be a temporary solution until WebML gains more traction. If you would like to make your models web-ready, we recommend converting to ONNX using Optimum and structuring your repo like this one (with ONNX weights located in a subfolder named onnx).
https://huggingface.co/typeform/distilbert-base-uncased-mnli with ONNX weights to be compatible with Transformers.js. If you haven't already, you can install the Transformers.js JavaScript library from NPM using: Note: Having a separate repo for ONNX weights is intended to be a temporary solution until WebML gains more traction. If you would like to make your models web-ready, we recommend converting to ONNX using Optimum and structuring your repo like this one (with ONNX weights located in a subfolder named onnx).