This model is a fine-tuned version of distilbert/distilbert-base-uncased on an unknown dataset. It achieves the following results on the evaluation set: The following hyperparameters were used during training: - learningrate: 0.0001 - trainbatchsize: 32 - evalbatchsize: 32 - lrschedulertype: linear - numepochs: 10 - Transformers 5.17.0 - Pytorch 2.11.0+cu130 - Datasets 5.0.1 - Tokenizers 0.23.2
Open-weight model · Text classification
distilbert-base-uncased-finetuned-sst-2-english
by DistilBERT community distilbert/distilbert-base-uncased-finetuned-sst-2-english
distilbert-base-uncased-finetuned-sst-2-english is an open-weight model for text classification from DistilBERT community, released under Apache License 2.0. It has 67M parameters and a 512-token context. At 16-bit it needs about 0.2 GB of GPU memory, which fits on 1x MI300X from $1.85 an hour, at the lowest prices in the SAVRN Index. It draws 2.7M downloads a month.
This model is a fine-tune checkpoint of DistilBERT-base-uncased, fine-tuned on SST-2. This model reaches an accuracy of 91.3 on the dev set (for comparison, Bert bert-base-uncased version reaches an accuracy of 92.7).
Parameters67M
Context512
Weights1.3 GB
Licenseapache-2.0
AccessOpen weights
Monthly Downloads2.7M
Runs On
What it takes to serve distilbert-base-uncased-finetuned-sst-2-english (67M parameters): the memory its weights need at each precision, and the cheapest way to rent enough data-center GPUs to hold them.
| Precision | Weights | Memory needed | Cheapest setup | Per hour | Also fits |
|---|---|---|---|---|---|
| 16-bit | 0.1 GB | 0.2 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
| 8-bit | 0.1 GB | 0.1 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
| 4-bit | 0.0 GB | 0.0 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
Memory is the weights at that precision plus 20% for the runtime and a short context; a long context needs more. Prices are the lowest on-demand hourly rates in the SAVRN Index, read Oct 13, 2026.
SAVRN's Notes on distilbert-base-uncased-finetuned-sst-2-english
Sixty-seven million parameters, 512 tokens of context, one label per input: a classifier, not a generator, and the publisher's suggested use is topic classification. Quantizing is beside the point: 16-bit needs 0.2 GB, 8-bit 0.1 GB, the 4-bit row rounds to 0.0 GB, so run it at full precision. The cheapest Index setup is still a 192 GB MI300X at $1.85 an hour on-demand, a warehouse for a shoebox; in our facilities it shares a card with whatever else is running.
Apache 2.0 covers commercial use, modification and redistribution, notices kept, changes stated. The 512-token context means anything longer gets split upstream. It was trained on sst2 and glue, and the publisher reports 91.3 on the dev set against 92.7 for bert-base-uncased, the price of distillation. The repository holds 17 files in five formats, so pull only the one your stack loads.
Model Card
By DistilBERT community, published under apache-2.0, revision 714eb0fa89d2.
DistilBERT base uncased finetuned SST-2
Table of Contents
- Model Details
- How to Get Started With the Model
- Uses
- Risks, Limitations and Biases
- Training
Model Details
Model Description: This model is a fine-tune checkpoint of DistilBERT-base-uncased, fine-tuned on SST-2. This model reaches an accuracy of 91.3 on the dev set (for comparison, Bert bert-base-uncased version reaches an accuracy of 92.7). - Developed by: Hugging Face - Model Type: Text Classification - Language(s): English - License: Apache-2.0 - Parent Model: For more details about DistilBERT, we encourage users to check out this model card. - Resources for more information: - Model Documentation - DistilBERT paper
How to Get Started With the Model
Example of single-label classification:
import torch
from transformers import DistilBertTokenizer, DistilBertForSequenceClassification
tokenizer = DistilBertTokenizer.from_pretrained("distilbert-base-uncased-finetuned-sst-2-english")
model = DistilBertForSequenceClassification.from_pretrained("distilbert-base-uncased-finetuned-sst-2-english")
inputs = tokenizer("Hello, my dog is cute", return_tensors="pt")
with torch.no_grad():
logits = model(**inputs).logits
predicted_class_id = logits.argmax().item()
model.config.id2label[predicted_class_id]
Uses
Direct Use
Configuration
- Architecture
- DistilBertForSequenceClassification
- Context length (tokens)
- 512
- Vocabulary size
- 30,522
- Model type
- distilbert
Identity and Version
- Repository
- distilbert/distilbert-base-uncased-finetuned-sst-2-english
- Publisher
- DistilBERT community
- Task
- Text classification
- Modality
- Text
- Library
- transformers
- Parameters
- 67M parameters
- Languages
- en
- Revision
- 714eb0fa89d2f80546fda750413ed43d93601a13
- First published
- 2022-03-02
- Last updated
- 2023-12-19
Files and Weights
17 files, 1.3 GB in total. The weights are 5 files totalling 1.3 GB in bin, h5, onnx, ot, safetensors.
Weights5 files · 1.3 GB
Configuration4 files · 1.6 KB
Tokenizer5 files · 1.2 MB
Documentation1 file · 10.5 KB
Other1 file · 81.6 KB
Repository1 file · 399 B
Every file
| File | Type | Size | SHA-256 |
|---|---|---|---|
| model.safetensors | Weights | 267.8 MB | 7c3919835e44 |
| onnx/model.onnx | Weights | 268.0 MB | 252cf7048af9 |
| pytorch_model.bin | Weights | 267.8 MB | 60554cbd7781 |
| rust_model.ot | Weights | 267.8 MB | 9db97da21b97 |
| tf_model.h5 | Weights | 267.9 MB | b44df675bb34 |
| config.json | Configuration | 629 B | — |
| onnx/added_tokens.json | Configuration | 82 B | — |
| onnx/config.json | Configuration | 735 B | — |
| onnx/special_tokens_map.json | Configuration | 125 B | — |
| README.md | Documentation | 10.5 KB | — |
| map.jpeg | Other | 81.6 KB | — |
| .gitattributes | Repository | 399 B | — |
| onnx/tokenizer.json | Tokenizer | 711.4 KB | — |
| onnx/tokenizer_config.json | Tokenizer | 1.3 KB | — |
| onnx/vocab.txt | Tokenizer | 231.5 KB | — |
| tokenizer_config.json | Tokenizer | 48 B | — |
| vocab.txt | Tokenizer | 231.5 KB | — |
License and Download
- License
- apache-2.0
- Access
- Open weights, no gate
- Download size
- 1.3 GB
Download from DistilBERT community
Released by DistilBERT community through its official repository on Hugging Face. Read the license.
Built From
- Described by arXiv:1910.01108
- Trained on (disclosed) glue
- Trained on (disclosed) sst2
Evaluations
Each result is shown as reported, with the conditions its reporter stated. None is a SAVRN measurement. A comparison lines two results up only when their configuration, unit and setup are all stated and identical.
| Benchmark | Conditions | Result | Reported by | Revision | Date |
|---|---|---|---|---|---|
| glue | Configuration sst2Task Text ClassificationMetric AUCComparison conditions not established | 0.971663 | distilbert Publisher reported |
Evaluated revision not stated | — |
| glue | Configuration sst2Task Text ClassificationMetric AccuracyComparison conditions not established | 0.91055 | distilbert Publisher reported |
Evaluated revision not stated | — |
| glue | Configuration sst2Task Text ClassificationMetric F1Comparison conditions not established | 0.913717 | distilbert Publisher reported |
Evaluated revision not stated | — |
| glue | Configuration sst2Task Text ClassificationMetric PrecisionComparison conditions not established | 0.897826 | distilbert Publisher reported |
Evaluated revision not stated | — |
| glue | Configuration sst2Task Text ClassificationMetric RecallComparison conditions not established | 0.93018 | distilbert Publisher reported |
Evaluated revision not stated | — |
| glue | Configuration sst2Task Text ClassificationMetric lossComparison conditions not established | 0.390139 | distilbert Publisher reported |
Evaluated revision not stated | — |
| sst2 | Configuration defaultTask Text ClassificationMetric AccuracyComparison conditions not established | 0.988552 | distilbert Publisher reported |
Evaluated revision not stated | — |
| sst2 | Configuration defaultTask Text ClassificationMetric F1 MacroComparison conditions not established | 0.988402 | distilbert Publisher reported |
Evaluated revision not stated | — |
| sst2 | Configuration defaultTask Text ClassificationMetric F1 MicroComparison conditions not established | 0.988552 | distilbert Publisher reported |
Evaluated revision not stated | — |
| sst2 | Configuration defaultTask Text ClassificationMetric F1 WeightedComparison conditions not established | 0.988555 | distilbert Publisher reported |
Evaluated revision not stated | — |
| sst2 | Configuration defaultTask Text ClassificationMetric Precision MacroComparison conditions not established | 0.988197 | distilbert Publisher reported |
Evaluated revision not stated | — |
| sst2 | Configuration defaultTask Text ClassificationMetric Precision MicroComparison conditions not established | 0.988552 | distilbert Publisher reported |
Evaluated revision not stated | — |
| sst2 | Configuration defaultTask Text ClassificationMetric Precision WeightedComparison conditions not established | 0.988564 | distilbert Publisher reported |
Evaluated revision not stated | — |
| sst2 | Configuration defaultTask Text ClassificationMetric Recall MacroComparison conditions not established | 0.988615 | distilbert Publisher reported |
Evaluated revision not stated | — |
| sst2 | Configuration defaultTask Text ClassificationMetric Recall MicroComparison conditions not established | 0.988552 | distilbert Publisher reported |
Evaluated revision not stated | — |
| sst2 | Configuration defaultTask Text ClassificationMetric Recall WeightedComparison conditions not established | 0.988552 | distilbert Publisher reported |
Evaluated revision not stated | — |
| sst2 | Configuration defaultTask Text ClassificationMetric lossComparison conditions not established | 0.0406526 | distilbert Publisher reported |
Evaluated revision not stated | — |
Memory Requirements
| Precision | Weights in memory |
|---|---|
| As published | 1.3 GB |
| 16-bit | 0.1 GB |
| 8-bit | 0.1 GB |
| 4-bit | 0.0 GB |
Weights only, from the published parameter count; the key-value cache and runtime add to this.
Compare distilbert-base-uncased-finetuned-sst-2-english
Questions About distilbert-base-uncased-finetuned-sst-2-english
How much GPU memory does distilbert-base-uncased-finetuned-sst-2-english need?
About 0.2 GB at 16-bit and 0 GB at 4-bit: the weights (67M parameters) plus a working margin. A long context needs more.
What is the cheapest GPU to run distilbert-base-uncased-finetuned-sst-2-english on?
At 16-bit, 1x MI300X from $1.85 an hour; at 4-bit, 1x MI300X from $1.85 an hour, at the lowest on-demand prices the SAVRN Index lists.
Can I use distilbert-base-uncased-finetuned-sst-2-english commercially?
Yes. distilbert-base-uncased-finetuned-sst-2-english is released under Apache License 2.0. The Apache License 2.0 is a permissive open-source license. It permits commercial use, modification and redistribution. It requires keeping the license and copyright notices and any NOTICE file, stating significant changes, and it includes an express patent grant from contributors.
What is distilbert-base-uncased-finetuned-sst-2-english's context length?
512 tokens, from the maximum position embeddings in its published configuration.
Similar Models
This model is a fine-tuned version of distilbert-base-uncased on the None dataset. The following hyperparameters were used during training: - learningrate: 3e-05 - trainbatchsize: 8 - evalbatchsize: 8 - lrschedulertype: linear - numepochs: 10 - Transformers 5.16.1 - Pytorch 2.11.0+cu128 - Datasets 5.0.1 - Tokenizers 0.23.1
This is the model card of a transformers model that has been pushed on the Hub. Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. Use the code below to get started with the model. Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).
This model is a fine-tuned version of distilbert/distilbert-base-uncased on an unknown dataset. It achieves the following results on the evaluation set: The following hyperparameters were used during training: - learningrate: 0.0001 - trainbatchsize: 32 - evalbatchsize: 32 - lrschedulertype: linear - numepochs: 10 - Transformers 5.16.1 - Pytorch 2.11.0+cu128 - Datasets 4.8.5 - Tokenizers 0.23.1
This model is a fine-tuned version of distilbert-base-uncased on the None dataset. It achieves the following results on the evaluation set: The following hyperparameters were used during training: - learningrate: 2e-05 - trainbatchsize: 32 - evalbatchsize: 64 - lrschedulertype: linear - lrschedulerwarmupratio: 0.1 - numepochs: 3 - Transformers 4.57.6 - Pytorch 2.14.0+cu130 - Datasets 5.0.1 - Tokenizers 0.22.2
This model is a fine-tuned version of distilbert-base-uncased on an unknown dataset. It achieves the following results on the evaluation set: The following hyperparameters were used during training: - learningrate: 0.0001 - trainbatchsize: 32 - evalbatchsize: 32 - lrschedulertype: linear - numepochs: 10 - Transformers 5.18.0 - Pytorch 2.11.0+cpu - Datasets 5.0.1 - Tokenizers 0.23.2