# cryptobert by Mikolaj Kulakowski: Open-Weight Model Source: https://savrn.com/models/cryptobert Markdown alternate of the page above; the site index is https://savrn.com/llms.txt --- ## Runs On What it takes to serve cryptobert (125M parameters): the memory its weights need at each precision, and the cheapest way to rent enough data-center GPUs to hold them. | Precision | Weights | Memory needed | Cheapest setup | Per hour | Also fits | | --- | --- | --- | --- | --- | --- | | 16-bit | 0.2 GB | 0.3 GB | 1x [MI300X](https://savrn.com/ai-index/pricing/gpus/mi300x) (192 GB) Vultr | $1.85 | [1x H100](https://savrn.com/ai-index/pricing/gpus/h100) $1.99 · [1x MI325X](https://savrn.com/ai-index/pricing/gpus/mi325x) $2.00 | | 8-bit | 0.1 GB | 0.1 GB | 1x [MI300X](https://savrn.com/ai-index/pricing/gpus/mi300x) (192 GB) Vultr | $1.85 | [1x H100](https://savrn.com/ai-index/pricing/gpus/h100) $1.99 · [1x MI325X](https://savrn.com/ai-index/pricing/gpus/mi325x) $2.00 | | 4-bit | 0.1 GB | 0.1 GB | 1x [MI300X](https://savrn.com/ai-index/pricing/gpus/mi300x) (192 GB) Vultr | $1.85 | [1x H100](https://savrn.com/ai-index/pricing/gpus/h100) $1.99 · [1x MI325X](https://savrn.com/ai-index/pricing/gpus/mi325x) $2.00 | Memory is the weights at that precision plus 20% for the runtime and a short context; a long context needs more. Prices are the lowest on-demand hourly rates in the [SAVRN Index](https://savrn.com/ai-index/pricing/gpus), read Oct 26, 2026. [cryptobert on every accelerator the SAVRN Index prices, at every precision](https://savrn.com/models/cryptobert/gpus) ## SAVRN's Notes on cryptobert Three labels, Bearish, Neutral and Bullish, applied to short social media posts about cryptocurrency. That is the scope of this 125-million-parameter RoBERTa classifier, built by continuing to train vinai's bertweet-base on more than 3.2 million crypto-related posts. Half-precision weights are 0.2 GB and it needs 0.3 GB to run, against 192 GB on the single MI300X we list cheapest at $1.85 per hour. Score posts in batches on a shared GPU rather than renting a card for this alone. MIT is a short permissive license: commercial use, modification and redistribution, with the notices carried along. Our check would be on the data. The file lists ElKulako/stocktwits-crypto as the training set, and the 514-token context fits a post, not a thread. Confirm your own feed resembles a corpus assembled for a June 2022 release; the weights were last touched in May 2025 and the labels are fixed at three. SAVRN Research, 2026-09-18 ## Model Card By Mikolaj Kulakowski, published under mit, revision 9e37c910fe87. For academic reference, cite the following paper: https://ieeexplore.ieee.org/document/10223689 ### CryptoBERT CryptoBERT is a pre-trained NLP model to analyse the language and sentiments of cryptocurrency-related social media posts and messages. It was built by further training the [vinai's bertweet-base](https://savrn.com/models/bertweet-base) language model on the cryptocurrency domain, using a corpus of over 3.2M unique cryptocurrency-related social media posts. (A research paper with more details will follow soon.) ### Classification Training The model was trained on the following labels: "Bearish" : 0, "Neutral": 1, "Bullish": 2 CryptoBERT's sentiment classification head was fine-tuned on a balanced dataset of 2M labelled StockTwits posts, sampled from [ElKulako/stocktwits-crypto](https://huggingface.co/datasets/ElKulako/stocktwits-crypto). CryptoBERT was trained with a max sequence length of 128. Technically, it can handle sequences of up to 514 tokens, however, going beyond 128 is not recommended. ### Classification Example [Read the full model card (416 words)](https://savrn.com/models/cryptobert/card) ## Configuration Architecture RobertaForSequenceClassification Context length (tokens) 514 Layers 12 Hidden size 768 Feed-forward size 3,072 Attention heads 12 Vocabulary size 50,265 Stored precision float32 Model type roberta ## Identity and Version Repository ElKulako/cryptobert Publisher Mikolaj Kulakowski Task Text classification Modality Text Library transformers Parameters 125M parameters Languages en Revision 9e37c910fe87727cb842a9ac55c6388256fe0f15 First published 2022-06-20 Last updated 2025-05-26 ## Files and Weights 10 files, 1.0 GB in total. The weights are 2 files totalling 997.3 MB in bin, safetensors. Weights2 files · 997.3 MB Configuration2 files · 1.9 KB Tokenizer4 files · 3.4 MB Documentation1 file · 3.7 KB Repository1 file · 1.2 KB Every file | File | Type | Size | SHA-256 | | --- | --- | --- | --- | | model.safetensors | Weights | 498.6 MB | fe34ba09b3d7 | | pytorch_model.bin | Weights | 498.7 MB | 80aaa3c6d754 | | config.json | Configuration | 932 B | — | | special_tokens_map.json | Configuration | 957 B | — | | README.md | Documentation | 3.7 KB | — | | .gitattributes | Repository | 1.2 KB | — | | merges.txt | Tokenizer | 456.4 KB | — | | tokenizer.json | Tokenizer | 2.1 MB | — | | tokenizer_config.json | Tokenizer | 1.3 KB | — | | vocab.json | Tokenizer | 798.3 KB | — | ## License and Download License mit Access Open weights, no gate Download size 997.3 MB [Download from Mikolaj Kulakowski](https://huggingface.co/ElKulako/cryptobert) Released by Mikolaj Kulakowski through its official repository on Hugging Face. [Read the license](https://opensource.org/license/mit). ## Built From - Trained on (disclosed) ElKulako/stocktwits-crypto ## Memory Requirements | Precision | Weights in memory | | --- | --- | | As published | 997.3 MB | | 16-bit | 0.2 GB | | 8-bit | 0.1 GB | | 4-bit | 0.1 GB | Weights only, from the published parameter count; the key-value cache and runtime add to this. ## Questions About cryptobert ### How much GPU memory does cryptobert need? About 0.3 GB at 16-bit and 0.1 GB at 4-bit: the weights (125M parameters) plus a working margin. A long context needs more. ### What is the cheapest GPU to run cryptobert on? At 16-bit, 1x MI300X from $1.85 an hour; at 4-bit, 1x MI300X from $1.85 an hour, at the lowest on-demand prices the SAVRN Index lists. ### Can I use cryptobert commercially? Yes. cryptobert is released under MIT License. The MIT License is a short permissive license. It permits commercial use, modification and redistribution, provided the copyright notice and permission notice are included. ### What is cryptobert's context length? 514 tokens, from the maximum position embeddings in its published configuration. ## Similar Models Model · Text classification ### [sentiment-analysis-deploy](https://savrn.com/models/sentiment-analysis-deploy) [GM](https://savrn.com/model-publishers/gm84) Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. Use the code below to get started with the model. Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019). Access requested at publisher 125M parameters [View model](https://savrn.com/models/sentiment-analysis-deploy) Model · Text classification ### [roberta-nli-final](https://savrn.com/models/roberta-nli-final) [Jiahui](https://savrn.com/model-publishers/sjh0929) Open weights 125M parameters transformers [View model](https://savrn.com/models/roberta-nli-final) Model · Text classification ### [sawyer-llama-reward](https://savrn.com/models/sawyer-llama-reward-2) [Josel Olat](https://savrn.com/model-publishers/selayy) This is the model card of a transformers model that has been pushed on the Hub. Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. Use the code below to get started with the model. Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019). Open weights 125M parameters 514 tokens transformers [View model](https://savrn.com/models/sawyer-llama-reward-2) Model · Text classification ### [sawyer-reward-v3](https://savrn.com/models/sawyer-reward-v3) [Miguel Isaac Beraquit](https://savrn.com/model-publishers/migeruaizakku) This is the model card of a transformers model that has been pushed on the Hub. Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. Use the code below to get started with the model. Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019). Open weights 125M parameters 514 tokens transformers [View model](https://savrn.com/models/sawyer-reward-v3) Model · Text classification ### [roberta-base-go_emotions](https://savrn.com/models/roberta-base-go-emotions) [Sam Lowe](https://savrn.com/model-publishers/samlowe) Model trained from roberta-base on the goemotions dataset for multi-label classification. A version of this model in ONNX format (including an INT8 quantized ONNX version) is now available at https://huggingface.co/SamLowe/roberta-base-goemotions-onnx. These are faster for inference, esp for smaller batch sizes, massively reduce the size of the dependencies required for inference, make inference of the model more multi-platform, and in the case of the quantized version reduce the model file/download size by 75% whilst retaining almost all the accuracy if you only need inference. goemotions is based on Reddit data and has 28 labels. It is a multi-label dataset where one or multiple labels… Open weights mit 125M parameters 514 tokens transformers [View model](https://savrn.com/models/roberta-base-go-emotions) Model · Text classification ### [careconnect-indobert](https://savrn.com/models/careconnect-indobert) [Dafa Ghani](https://savrn.com/model-publishers/kangdap) Base model: indobenchmark/indobert-base-p1; Input: title, description; Tokenization: maxlength=256, truncation=True; Output: Single label, with confidence score for each label.. Open weights mit 124M parameters 512 tokens [View model](https://savrn.com/models/careconnect-indobert) ## Mikolaj Kulakowski [All models and datasets](https://savrn.com/model-publishers/elkulako) ## Versions - [9e37c910fe87](https://savrn.com/models/cryptobert/versions/9e37c910fe87) · current 2026-09-18 ## Explore More - [All text classification models](https://savrn.com/models/tasks/text-classification) - [All models under mit](https://savrn.com/models/licenses/mit) - [Model comparisons](https://savrn.com/models/comparisons) - [The model directory](https://savrn.com/models) - [Open model prices by host](https://savrn.com/ai-index/pricing/open-models) ## Source - Repository metadata, read 2026-10-20. - [Hugging Face record](https://huggingface.co/ElKulako/cryptobert) - [How the hub is built](https://savrn.com/model-hub/methodology)