SAVRN
Search Contact SAVRN

Open-weight model · Text classification

distilbert-imdb

by Leandro von Werra lvwerra/distilbert-imdb

This model is a fine-tuned version of distilbert-base-uncased on the imdb dataset (training notebook is here).

Parameters
Context512
Weights267.9 MB
Licenseapache-2.0
AccessOpen weights
Monthly Downloads373.8k

Model Card

By Leandro von Werra, published under apache-2.0, revision 0fc02cd68445.

This model is a fine-tuned version of distilbert-base-uncased on the imdb dataset (training notebook is here). It achieves the following results on the evaluation set: The following hyperparameters were used during training: - learningrate: 5e-05 - trainbatchsize: 16 - evalbatchsize: 16 - lrschedulertype: linear - numepochs: 1 - Transformers 4.15.0 - Pytorch 1.10.0+cu111 - Datasets 1.17.0 - Tokenizers 0.10.3

Read Leandro von Werra's full model card

This model is a fine-tuned version of distilbert-base-uncased on the imdb dataset (training notebook is here). It achieves the following results on the evaluation set: - Loss: 0.1903 - Accuracy: 0.928

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training: - learning_rate: 5e-05 - train_batch_size: 16 - eval_batch_size: 16 - seed: 42 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 - lr_scheduler_type: linear - num_epochs: 1

Training results

Training Loss Epoch Step Validation Loss Accuracy
0.2195 1.0 1563 0.1903 0.928

Framework versions

  • Transformers 4.15.0
  • Pytorch 1.10.0+cu111
  • Datasets 1.17.0
  • Tokenizers 0.10.3

Configuration

Architecture
DistilBertForSequenceClassification
Context length (tokens)
512
Vocabulary size
30,522
Stored precision
float32
Model type
distilbert

Identity and Version

Repository
lvwerra/distilbert-imdb
Publisher
Leandro von Werra
Task
Text classification
Modality
Text
Library
transformers
Parameters
Not stated by the source
Languages
Not stated by the source
Revision
0fc02cd68445b599a9cb2da2368050e7fb31d29a
First published
2022-03-02
Last updated
2023-01-25

Files and Weights

18 files, 268.6 MB in total. The weights are 2 files totalling 267.9 MB in bin.

Weights2 files · 267.9 MB
Configuration2 files · 847 B
Tokenizer3 files · 698.0 KB
Documentation1 file · 1.7 KB
Other8 files · 39.2 KB
Repository2 files · 1.2 KB
Every file
FileTypeSizeSHA-256
pytorch_model.binWeights267.9 MB 2ede11721eca
training_args.binWeights2.9 KB a78ec6374b76
config.jsonConfiguration735 B
special_tokens_map.jsonConfiguration112 B
README.mdDocumentation1.7 KB
distilbert-imdb-training.ipynbOther6.5 KB
runs/Dec30_16-56-30_3603c37d6fc6/1640883891.1614847/events.out.tfevents.1640883891.3603c37d6fc6.75.1Other4.7 KB a5828c9712ee
runs/Dec30_16-56-30_3603c37d6fc6/events.out.tfevents.1640883891.3603c37d6fc6.75.0Other3.1 KB 3748dfd52722
runs/Dec30_17-07-51_3603c37d6fc6/1640884114.6640563/events.out.tfevents.1640884114.3603c37d6fc6.75.3Other4.7 KB 359761f26cd8
runs/Dec30_17-07-51_3603c37d6fc6/1640884728.1701791/events.out.tfevents.1640884728.3603c37d6fc6.75.4Other4.7 KB cf81463fb3b1
runs/Dec30_17-07-51_3603c37d6fc6/events.out.tfevents.1640884114.3603c37d6fc6.75.2Other6.4 KB 35112ddd1b90
runs/Dec30_17-18-53_3603c37d6fc6/1640884742.6121824/events.out.tfevents.1640884742.3603c37d6fc6.75.6Other4.7 KB 82b8ec666b46
runs/Dec30_17-18-53_3603c37d6fc6/events.out.tfevents.1640884742.3603c37d6fc6.75.5Other4.3 KB ae7d69ce92f6
.gitattributesRepository1.2 KB
.gitignoreRepository13 B
tokenizer.jsonTokenizer466.1 KB
tokenizer_config.jsonTokenizer333 B
vocab.txtTokenizer231.5 KB

License and Download

License
apache-2.0
Access
Open weights, no gate
Download size
267.9 MB
Download from Leandro von Werra

Released by Leandro von Werra through its official repository on Hugging Face. Read the license.

Built From

  • Trained on (disclosed) imdb

Evaluations

Each result is shown as reported, with the conditions its reporter stated. None is a SAVRN measurement. A comparison lines two results up only when their configuration, unit and setup are all stated and identical.

BenchmarkConditionsResultReported byRevisionDate
imdb Task Text ClassificationMetric AccuracyComparison conditions not established 0.928 lvwerra
Publisher reported
Evaluated revision not stated

Memory Requirements

PrecisionWeights in memory
As published267.9 MB

Weights only, from the published parameter count; the key-value cache and runtime add to this.

Questions About distilbert-imdb

Can I use distilbert-imdb commercially?

Yes. distilbert-imdb is released under Apache License 2.0. The Apache License 2.0 is a permissive open-source license. It permits commercial use, modification and redistribution. It requires keeping the license and copyright notices and any NOTICE file, stating significant changes, and it includes an express patent grant from contributors.

What is distilbert-imdb's context length?

512 tokens, from the maximum position embeddings in its published configuration.

Similar Models

Model · Text classification

finbert

Prosus AI

FinBERT is a pre-trained NLP model to analyze sentiment of financial text. It is built by further training the BERT language model in the finance domain, using a large financial corpus and thereby fine-tuning it for financial sentiment classification. Financial PhraseBank by Malo et al. (2014) is used for fine-tuning. For more details, please see the paper FinBERT: Financial Sentiment Analysis with Pre-trained Language Models and our related blog post on Medium. The model will give softmax outputs for three labels: positive, negative or neutral. About Prosus Prosus is a global consumer internet group and one of the largest technology investors in the world. Operating and investing globally…

Open weights 512 tokens transformers

Model · Text classification

twitter-roberta-base-sentiment-latest

Cardiff NLP

This is a RoBERTa-base model trained on ~124M tweets from January 2018 to December 2021, and finetuned for sentiment analysis with the TweetEval benchmark. The original Twitter-based RoBERTa model can be found here and the original reference paper is TweetEval. This model is suitable for English. 0 -> Negative; 1 -> Neutral; 2 -> Positive This sentiment analysis model has been integrated into TweetNLP. You can access the demo here.

Open weights cc-by-4.0 514 tokens transformers

Model · Text classification

ms-marco-MiniLM-L-6-v2

Joshua

https://huggingface.co/cross-encoder/ms-marco-MiniLM-L-6-v2 with ONNX weights to be compatible with Transformers.js. If you haven't already, you can install the Transformers.js JavaScript library from NPM using: Note: Having a separate repo for ONNX weights is intended to be a temporary solution until WebML gains more traction. If you would like to make your models web-ready, we recommend converting to ONNX using Optimum and structuring your repo like this one (with ONNX weights located in a subfolder named onnx).

Open weights 512 tokens transformers.js

Model · Text classification

twitter-xlm-roberta-base-sentiment

Cardiff NLP

This is a multilingual XLM-roBERTa-base model trained on ~198M tweets and finetuned for sentiment analysis. The sentiment fine-tuning was done on 8 languages (Ar, En, Fr, De, Hi, It, Sp, Pt) but it can be used for more languages (see paper for details). This model has been integrated into the TweetNLP library.

Open weights 514 tokens transformers

Model · Text classification

finbert-tone

Yi

FinBERT is a BERT model pre-trained on financial communication text. The purpose is to enhance financial NLP research and practice. It is trained on the following three financial communication corpus. The total corpora size is 4.9B tokens. More technical details on FinBERT: Click Link This released finbert-tone model is the FinBERT model fine-tuned on 10,000 manually annotated (positive, negative, neutral) sentences from analyst reports. This model achieves superior performance on financial tone analysis task. If you are simply interested in using FinBERT for financial tone analysis, give it a try. If you use the model in your academic work, please cite the following paper: Huang, Allen H.…

Open weights 512 tokens transformers

Model · Text classification

emotion-english-distilroberta-base

Hartmann

With this model, you can classify emotions in English text data. The model was trained on 6 diverse datasets (see Appendix below) and predicts Ekman's 6 basic emotions, plus a neutral class: 1) anger 2) disgust 3) fear 4) joy 5) neutral 6) sadness 7) surprise The model is a fine-tuned checkpoint of DistilRoBERTa-base. For a 'non-distilled' emotion model, please refer to the model card of the RoBERTa-large version. a) Run emotion model with 3 lines of code on single text example using Hugging Face's pipeline command on Google Colab: b) Run emotion model on multiple examples and full datasets (e.g.,.csv files) on Google Colab: Please reach out to [email protected] if you have any…

Open weights 514 tokens transformers