# xtremedistil-l6-h256-zeroshot-v1.1-all-33: Open-Weight Model Source: https://savrn.com/models/xtremedistil-l6-h256-zeroshot-v1-1-all-33 Markdown alternate of the page above; the site index is https://savrn.com/llms.txt --- ## Runs On What it takes to serve xtremedistil-l6-h256-zeroshot-v1.1-all-33 (13M parameters): the memory its weights need at each precision, and the cheapest way to rent enough data-center GPUs to hold them. | Precision | Weights | Memory needed | Cheapest setup | Per hour | Also fits | | --- | --- | --- | --- | --- | --- | | 16-bit | 0.0 GB | 0.0 GB | 1x [MI300X](https://savrn.com/ai-index/pricing/gpus/mi300x) (192 GB) Vultr | $1.85 | [1x H100](https://savrn.com/ai-index/pricing/gpus/h100) $1.99 · [1x MI325X](https://savrn.com/ai-index/pricing/gpus/mi325x) $2.00 | | 8-bit | 0.0 GB | 0.0 GB | 1x [MI300X](https://savrn.com/ai-index/pricing/gpus/mi300x) (192 GB) Vultr | $1.85 | [1x H100](https://savrn.com/ai-index/pricing/gpus/h100) $1.99 · [1x MI325X](https://savrn.com/ai-index/pricing/gpus/mi325x) $2.00 | | 4-bit | 0.0 GB | 0.0 GB | 1x [MI300X](https://savrn.com/ai-index/pricing/gpus/mi300x) (192 GB) Vultr | $1.85 | [1x H100](https://savrn.com/ai-index/pricing/gpus/h100) $1.99 · [1x MI325X](https://savrn.com/ai-index/pricing/gpus/mi325x) $2.00 | Memory is the weights at that precision plus 20% for the runtime and a short context; a long context needs more. Prices are the lowest on-demand hourly rates in the [SAVRN Index](https://savrn.com/ai-index/pricing/gpus), read Oct 24, 2026. [xtremedistil-l6-h256-zeroshot-v1.1-all-33 on every accelerator the SAVRN Index prices, at every precision](https://savrn.com/models/xtremedistil-l6-h256-zeroshot-v1-1-all-33/gpus) ## Model Card By Moritz Borrett-Laurer (formerly Laurer), published under mit, revision c07f66d9cbf7. This model was fine-tuned using the same pipeline as described in the model card for [MoritzLaurer/deberta-v3-large-zeroshot-v1.1-all-33](https://savrn.com/models/deberta-v3-large-zeroshot-v1-1-all-33) and in this [paper](https://savrn.com/papers/building-efficient-universal-classifiers-with-natural-language-inference). The foundation model is [microsoft/xtremedistil-l6-h256-uncased](https://huggingface.co/microsoft/xtremedistil-l6-h256-uncased). The model only has 22 million backbone parameters and 30 million vocabulary parameters. The backbone parameters are the main parameters active during inference, providing a significant speedup over larger models. The model is 25 MB small. This model was trained to provide a very small and highly efficient zeroshot option, especially for edge devices or in-browser use-cases with transformers.js. ### Usage and other details For usage instructions and other details refer to this model card [MoritzLaurer/deberta-v3-large-zeroshot-v1.1-all-33](https://savrn.com/models/deberta-v3-large-zeroshot-v1-1-all-33) and this [paper](https://savrn.com/papers/building-efficient-universal-classifiers-with-natural-language-inference). ### Metrics: I didn't not do zeroshot evaluation for this model to save time and compute. The table below shows standard accuracy for all datasets the model was trained on (note that the NLI datasets are binary). General takeaway: the model is much more efficient than its larger sisters, but it performs less well. [Read the full model card (274 words)](https://savrn.com/models/xtremedistil-l6-h256-zeroshot-v1-1-all-33/card) ## Configuration Architecture BertForSequenceClassification Context length (tokens) 512 Layers 6 Hidden size 256 Feed-forward size 1,024 Attention heads 8 Vocabulary size 30,522 Stored precision float16 Model type bert ## Identity and Version Repository MoritzLaurer/xtremedistil-l6-h256-zeroshot-v1.1-all-33 Publisher Moritz Borrett-Laurer (formerly Laurer) Task Zero-shot classification Modality Text Library transformers Parameters 13M parameters Languages en Revision c07f66d9cbf781191bee66edfe8ad7856f045781 First published 2024-01-10 Last updated 2025-01-16 ## Files and Weights 12 files, 116.3 MB in total. The weights are 5 files totalling 115.3 MB in bin, onnx, safetensors. Weights5 files · 115.3 MB Configuration2 files · 1.0 KB Tokenizer3 files · 943.3 KB Documentation1 file · 2.9 KB Repository1 file · 1.5 KB Every file | File | Type | Size | SHA-256 | | --- | --- | --- | --- | | model.safetensors | Weights | 25.5 MB | bf2d83426aa6 | | onnx/model.onnx | Weights | 51.1 MB | 89ca1f968301 | | onnx/model_quantized.onnx | Weights | 13.1 MB | 534df3a88f34 | | pytorch_model.bin | Weights | 25.5 MB | 98b31cbc8fab | | training_args.bin | Weights | 4.7 KB | e9fdf5b1ada2 | | config.json | Configuration | 882 B | — | | special_tokens_map.json | Configuration | 125 B | — | | README.md | Documentation | 2.9 KB | — | | .gitattributes | Repository | 1.5 KB | — | | tokenizer.json | Tokenizer | 711.4 KB | — | | tokenizer_config.json | Tokenizer | 366 B | — | | vocab.txt | Tokenizer | 231.5 KB | — | ## License and Download License mit Access Open weights, no gate Download size 115.3 MB [Download from Moritz Borrett-Laurer (formerly Laurer)](https://huggingface.co/MoritzLaurer/xtremedistil-l6-h256-zeroshot-v1.1-all-33) Released by Moritz Borrett-Laurer (formerly Laurer) through its official repository on Hugging Face. [Read the license](https://opensource.org/license/mit). ## Built From - Derived from microsoft/xtremedistil-l6-h256-uncased - Described by [arXiv:2312.17543](https://savrn.com/papers/building-efficient-universal-classifiers-with-natural-language-inference) - Quantized from microsoft/xtremedistil-l6-h256-uncased ## Memory Requirements | Precision | Weights in memory | | --- | --- | | As published | 115.3 MB | | 16-bit | 0.0 GB | | 8-bit | 0.0 GB | | 4-bit | 0.0 GB | Weights only, from the published parameter count; the key-value cache and runtime add to this. ## Questions About xtremedistil-l6-h256-zeroshot-v1.1-all-33 ### How much GPU memory does xtremedistil-l6-h256-zeroshot-v1.1-all-33 need? About 0 GB at 16-bit and 0 GB at 4-bit: the weights (13M parameters) plus a working margin. A long context needs more. ### What is the cheapest GPU to run xtremedistil-l6-h256-zeroshot-v1.1-all-33 on? At 16-bit, 1x MI300X from $1.85 an hour; at 4-bit, 1x MI300X from $1.85 an hour, at the lowest on-demand prices the SAVRN Index lists. ### Can I use xtremedistil-l6-h256-zeroshot-v1.1-all-33 commercially? Yes. xtremedistil-l6-h256-zeroshot-v1.1-all-33 is released under MIT License. The MIT License is a short permissive license. It permits commercial use, modification and redistribution, provided the copyright notice and permission notice are included. ### What is xtremedistil-l6-h256-zeroshot-v1.1-all-33's context length? 512 tokens, from the maximum position embeddings in its published configuration. ## Similar Models Model · Zero-shot classification ### [MoritzLaurer-roberta-base-zeroshot-v2.0-c-onnx](https://savrn.com/models/moritzlaurer-roberta-base-zeroshot-v2-0-c-onnx) [Protect AI](https://savrn.com/model-publishers/protectai) This model is a conversion of MoritzLaurer/roberta-base-zeroshot-v2.0-c to ONNX format using the Optimum library. Open weights apache-2.0 514 tokens transformers [View model](https://savrn.com/models/moritzlaurer-roberta-base-zeroshot-v2-0-c-onnx) Model · Zero-shot classification ### [bart-large-mnli](https://savrn.com/models/bart-large-mnli-2) [Joshua](https://savrn.com/model-publishers/xenova) https://huggingface.co/facebook/bart-large-mnli with ONNX weights to be compatible with Transformers.js. If you haven't already, you can install the Transformers.js JavaScript library from NPM using: Note: Having a separate repo for ONNX weights is intended to be a temporary solution until WebML gains more traction. If you would like to make your models web-ready, we recommend converting to ONNX using Optimum and structuring your repo like this one (with ONNX weights located in a subfolder named onnx). Open weights 1,024 tokens transformers.js [View model](https://savrn.com/models/bart-large-mnli-2) Model · Zero-shot classification ### [distilbart-mnli-12-3](https://savrn.com/models/distilbart-mnli-12-3) [Suraj Patil](https://savrn.com/model-publishers/valhalla) distilbart-mnli is the distilled version of bart-large-mnli created using the No Teacher Distillation technique proposed for BART summarisation by Huggingface, here. We just copy alternating layers from bart-large-mnli and finetune more on the same data. This is a very simple and effective technique, as we can see the performance drop is very little. Detailed performace trade-offs will be posted in this sheet. If you want to train these models yourself, clone the distillbart-mnli repo and follow the steps below Clone and install transformers from source Download MNLI data Create student model Start fine-tuning You can find the logs of these trained models in this wandb project. Open weights 1,024 tokens transformers [View model](https://savrn.com/models/distilbart-mnli-12-3) Model · Zero-shot classification ### [distilbart-mnli-12-1](https://savrn.com/models/distilbart-mnli-12-1) [Suraj Patil](https://savrn.com/model-publishers/valhalla) distilbart-mnli is the distilled version of bart-large-mnli created using the No Teacher Distillation technique proposed for BART summarisation by Huggingface, here. We just copy alternating layers from bart-large-mnli and finetune more on the same data. This is a very simple and effective technique, as we can see the performance drop is very little. Detailed performace trade-offs will be posted in this sheet. If you want to train these models yourself, clone the distillbart-mnli repo and follow the steps below Clone and install transformers from source Download MNLI data Create student model Start fine-tuning You can find the logs of these trained models in this wandb project. Open weights 1,024 tokens transformers [View model](https://savrn.com/models/distilbart-mnli-12-1) Model · Zero-shot classification ### [mobilebert-uncased-mnli](https://savrn.com/models/mobilebert-uncased-mnli) [Joshua](https://savrn.com/model-publishers/xenova) https://huggingface.co/typeform/mobilebert-uncased-mnli with ONNX weights to be compatible with Transformers.js. If you haven't already, you can install the Transformers.js JavaScript library from NPM using: Note: Having a separate repo for ONNX weights is intended to be a temporary solution until WebML gains more traction. If you would like to make your models web-ready, we recommend converting to ONNX using Optimum and structuring your repo like this one (with ONNX weights located in a subfolder named onnx). Open weights 512 tokens transformers.js [View model](https://savrn.com/models/mobilebert-uncased-mnli) Model · Zero-shot classification ### [nli-deberta-v3-xsmall](https://savrn.com/models/nli-deberta-v3-xsmall-2) [Joshua](https://savrn.com/model-publishers/xenova) https://huggingface.co/cross-encoder/nli-deberta-v3-xsmall with ONNX weights to be compatible with Transformers.js. If you haven't already, you can install the Transformers.js JavaScript library from NPM using: Note: Having a separate repo for ONNX weights is intended to be a temporary solution until WebML gains more traction. If you would like to make your models web-ready, we recommend converting to ONNX using Optimum and structuring your repo like this one (with ONNX weights located in a subfolder named onnx). Open weights 512 tokens transformers.js [View model](https://savrn.com/models/nli-deberta-v3-xsmall-2) ## Moritz Borrett-Laurer (formerly Laurer) [All models and datasets](https://savrn.com/model-publishers/moritzlaurer) ## Versions - [c07f66d9cbf7](https://savrn.com/models/xtremedistil-l6-h256-zeroshot-v1-1-all-33/versions/c07f66d9cbf7) · current 2026-09-18 ## Explore More - [All zero-shot classification models](https://savrn.com/models/tasks/zero-shot-classification) - [All models under mit](https://savrn.com/models/licenses/mit) - [Model comparisons](https://savrn.com/models/comparisons) - [The model directory](https://savrn.com/models) - [Open model prices by host](https://savrn.com/ai-index/pricing/open-models) ## Source - Repository metadata, read 2026-10-21. - [Hugging Face record](https://huggingface.co/MoritzLaurer/xtremedistil-l6-h256-zeroshot-v1.1-all-33) - [How the hub is built](https://savrn.com/model-hub/methodology)