This is the model card of a transformers model that has been pushed on the Hub. Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. Use the code below to get started with the model. Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).
cryptobert is an open-weight model for text classification from Mikolaj Kulakowski, released under MIT License. It has 125M parameters and a 514-token context. At 16-bit it needs about 0.3 GB of GPU memory, which fits on 1x MI300X from $1.85 an hour, at the lowest prices in the SAVRN Index. It draws 490.6k downloads a month.
For academic reference, cite the following paper: https://ieeexplore.ieee.org/document/10223689 CryptoBERT is a pre-trained NLP model to analyse the language and sentiments of cryptocurrency-related social media posts and messages.
Parameters125M
Context514
Weights997.3 MB
Licensemit
AccessOpen weights
Monthly Downloads490.6k
Runs On
What it takes to serve cryptobert (125M parameters): the memory its weights need at each precision, and the cheapest way to rent enough data-center GPUs to hold them.
| Precision | Weights | Memory needed | Cheapest setup | Per hour | Also fits |
|---|---|---|---|---|---|
| 16-bit | 0.2 GB | 0.3 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
| 8-bit | 0.1 GB | 0.1 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
| 4-bit | 0.1 GB | 0.1 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
Memory is the weights at that precision plus 20% for the runtime and a short context; a long context needs more. Prices are the lowest on-demand hourly rates in the SAVRN Index, read Oct 9, 2026.
cryptobert on every accelerator the SAVRN Index prices, at every precision
SAVRN's Notes on cryptobert
Three labels, Bearish, Neutral and Bullish, applied to short social media posts about cryptocurrency. That is the scope of this 125-million-parameter RoBERTa classifier, built by continuing to train vinai's bertweet-base on more than 3.2 million crypto-related posts. Half-precision weights are 0.2 GB and it needs 0.3 GB to run, against 192 GB on the single MI300X we list cheapest at $1.85 per hour. Score posts in batches on a shared GPU rather than renting a card for this alone.
MIT is a short permissive license: commercial use, modification and redistribution, with the notices carried along. Our check would be on the data. The file lists ElKulako/stocktwits-crypto as the training set, and the 514-token context fits a post, not a thread. Confirm your own feed resembles a corpus assembled for a June 2022 release; the weights were last touched in May 2025 and the labels are fixed at three.
Model Card
By Mikolaj Kulakowski, published under mit, revision 9e37c910fe87.
For academic reference, cite the following paper: https://ieeexplore.ieee.org/document/10223689
CryptoBERT
CryptoBERT is a pre-trained NLP model to analyse the language and sentiments of cryptocurrency-related social media posts and messages. It was built by further training the vinai's bertweet-base language model on the cryptocurrency domain, using a corpus of over 3.2M unique cryptocurrency-related social media posts. (A research paper with more details will follow soon.)
Classification Training
The model was trained on the following labels: "Bearish" : 0, "Neutral": 1, "Bullish": 2
CryptoBERT's sentiment classification head was fine-tuned on a balanced dataset of 2M labelled StockTwits posts, sampled from ElKulako/stocktwits-crypto.
CryptoBERT was trained with a max sequence length of 128. Technically, it can handle sequences of up to 514 tokens, however, going beyond 128 is not recommended.
Classification Example
Configuration
- Architecture
- RobertaForSequenceClassification
- Context length (tokens)
- 514
- Layers
- 12
- Hidden size
- 768
- Feed-forward size
- 3,072
- Attention heads
- 12
- Vocabulary size
- 50,265
- Stored precision
- float32
- Model type
- roberta
Identity and Version
- Repository
- ElKulako/cryptobert
- Publisher
- Mikolaj Kulakowski
- Task
- Text classification
- Modality
- Text
- Library
- transformers
- Parameters
- 125M parameters
- Languages
- en
- Revision
- 9e37c910fe87727cb842a9ac55c6388256fe0f15
- First published
- 2022-06-20
- Last updated
- 2025-05-26
Files and Weights
10 files, 1.0 GB in total. The weights are 2 files totalling 997.3 MB in bin, safetensors.
Weights2 files · 997.3 MB
Configuration2 files · 1.9 KB
Tokenizer4 files · 3.4 MB
Documentation1 file · 3.7 KB
Repository1 file · 1.2 KB
Every file
| File | Type | Size | SHA-256 |
|---|---|---|---|
| model.safetensors | Weights | 498.6 MB | fe34ba09b3d7 |
| pytorch_model.bin | Weights | 498.7 MB | 80aaa3c6d754 |
| config.json | Configuration | 932 B | — |
| special_tokens_map.json | Configuration | 957 B | — |
| README.md | Documentation | 3.7 KB | — |
| .gitattributes | Repository | 1.2 KB | — |
| merges.txt | Tokenizer | 456.4 KB | — |
| tokenizer.json | Tokenizer | 2.1 MB | — |
| tokenizer_config.json | Tokenizer | 1.3 KB | — |
| vocab.json | Tokenizer | 798.3 KB | — |
License and Download
- License
- mit
- Access
- Open weights, no gate
- Download size
- 997.3 MB
Download from Mikolaj Kulakowski
Released by Mikolaj Kulakowski through its official repository on Hugging Face. Read the license.
Built From
- Trained on (disclosed) ElKulako/stocktwits-crypto
Memory Requirements
| Precision | Weights in memory |
|---|---|
| As published | 997.3 MB |
| 16-bit | 0.2 GB |
| 8-bit | 0.1 GB |
| 4-bit | 0.1 GB |
Weights only, from the published parameter count; the key-value cache and runtime add to this.
Questions About cryptobert
How much GPU memory does cryptobert need?
About 0.3 GB at 16-bit and 0.1 GB at 4-bit: the weights (125M parameters) plus a working margin. A long context needs more.
What is the cheapest GPU to run cryptobert on?
At 16-bit, 1x MI300X from $1.85 an hour; at 4-bit, 1x MI300X from $1.85 an hour, at the lowest on-demand prices the SAVRN Index lists.
Can I use cryptobert commercially?
Yes. cryptobert is released under MIT License. The MIT License is a short permissive license. It permits commercial use, modification and redistribution, provided the copyright notice and permission notice are included.
What is cryptobert's context length?
514 tokens, from the maximum position embeddings in its published configuration.
Similar Models
This is the model card of a transformers model that has been pushed on the Hub. Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. Use the code below to get started with the model. Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).
Model trained from roberta-base on the goemotions dataset for multi-label classification. A version of this model in ONNX format (including an INT8 quantized ONNX version) is now available at https://huggingface.co/SamLowe/roberta-base-goemotions-onnx. These are faster for inference, esp for smaller batch sizes, massively reduce the size of the dependencies required for inference, make inference of the model more multi-platform, and in the case of the quantized version reduce the model file/download size by 75% whilst retaining almost all the accuracy if you only need inference. goemotions is based on Reddit data and has 28 labels. It is a multi-label dataset where one or multiple labels…
Base model: indobenchmark/indobert-base-p1; Input: title, description; Tokenization: maxlength=256, truncation=True; Output: Single label, with confidence score for each label..
castorini/afribertalarge fine-tuned on the Kiswahili subset of AfriSenti (Muhammad et al., 2023) to classify tweets as positive / neutral / negative. best epoch selected on validation macro-F1 (seed 42). Demo: https://huggingface.co/spaces/RubaxTyra/kiswahili-sentiment-demo Limitations: trained on only 1,810 tweets; emojis and punctuation were removed from the training data; struggles with sarcasm, slang/Sheng and with negative news vs negative opinion.
An open-weights language model for contextually-aware end-of-utterance (EOU) detection in voice AI applications. The model predicts whether a user has finished speaking based on the semantic content of their transcribed speech, providing a critical complement to voice activity detection (VAD) systems. Traditional voice agents rely on voice activity detection (VAD) to determine when a user has finished speaking. VAD works by detecting the presence or absence of speech in an audio signal and applying a silence timer. While effective for detecting pauses, VAD lacks language understanding and frequently causes false positives. For example, a user who says "I need to think about that for a…