This is a LinkTransformer model. At its core this model this is a sentence transformer model sentence-transformers model- it just wraps around the class. It is designed for quick and easy record linkage (entity-matching) through the LinkTransformer package. The tasks include clustering, deduplication, linking, aggregation and more. Notwithstanding that, it can be used for any sentence similarity task within the sentence-transformers framework as well. It maps sentences & paragraphs to a 768 dimensional dense vector space and can be used for tasks like clustering or semantic search. Take a look at the documentation of sentence-transformers if you want to use this model for more than what we…
Open-weight model · Sentence similarity
all-distilroberta-v1
by Sentence Transformers sentence-transformers/all-distilroberta-v1
This is a sentence-transformers model: It maps sentences & paragraphs to a 768 dimensional dense vector space and can be used for tasks like clustering or semantic search.
Parameters82M
Context514
Weights3.2 GB
Licenseapache-2.0
AccessOpen weights
Monthly Downloads2.7M
Runs On
What it takes to serve all-distilroberta-v1 (82M parameters): the memory its weights need at each precision, and the cheapest way to rent enough data-center GPUs to hold them.
| Precision | Weights | Memory needed | Cheapest setup | Per hour | Also fits |
|---|---|---|---|---|---|
| 16-bit | 0.2 GB | 0.2 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
| 8-bit | 0.1 GB | 0.1 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
| 4-bit | 0.0 GB | 0.0 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
Memory is the weights at that precision plus 20% for the runtime and a short context; a long context needs more. Prices are the lowest on-demand hourly rates in the SAVRN Index, read Sep 18, 2026.
SAVRN's Notes on all-distilroberta-v1
How much GPU does an 82M-parameter embedding model need? Almost none: 0.2 GB of weights at 16-bit, 0.1 GB at 8-bit, and the cheapest setup on our Index is one MI300X with 192 GB at $1.85 an hour, a card this model leaves nearly empty, so share it with other work. What you get is a 768-dimensional vector for each sentence or paragraph, meant for semantic search and clustering, inside a 514-token context, so anything longer gets chunked first.
Eight training sets are on record, among them s2orc, ms_marco, code_search_net, stackexchange_xml and eli5; ask whether your corpus looks like them. Apache 2.0 means shipping it inside a commercial product, modifying it and redistributing it are allowed once the license and notices are kept and changes stated. Download only the format you need: the 3.19 GB across 30 files spans safetensors, onnx, openvino, pytorch and rust, and one is enough.
Model Card
By Sentence Transformers, published under apache-2.0, revision 842eaed40bee.
This is a sentence-transformers model: It maps sentences & paragraphs to a 768 dimensional dense vector space and can be used for tasks like clustering or semantic search.
Usage (Sentence-Transformers)
Using this model becomes easy when you have sentence-transformers installed:
pip install -U sentence-transformers
Then you can use the model like this:
from sentence_transformers import SentenceTransformer
sentences = ["This is an example sentence", "Each sentence is converted"]
model = SentenceTransformer('sentence-transformers/all-distilroberta-v1')
embeddings = model.encode(sentences)
print(embeddings)
Usage (HuggingFace Transformers)
Without sentence-transformers, you can use the model like this: First, you pass your input through the transformer model, then you have to apply the right pooling-operation on-top of the contextualized word embeddings.
Configuration
- Architecture
- RobertaForMaskedLM
- Context length (tokens)
- 514
- Layers
- 6
- Hidden size
- 768
- Feed-forward size
- 3,072
- Attention heads
- 12
- Vocabulary size
- 50,265
- Model type
- roberta
Identity and Version
- Repository
- sentence-transformers/all-distilroberta-v1
- Publisher
- Sentence Transformers
- Task
- Sentence similarity
- Modality
- Text
- Library
- sentence-transformers
- Parameters
- 82M parameters
- Languages
- en
- Revision
- 842eaed40bee4d61673a81c92d5689a8fed7a09f
- First published
- 2022-03-02
- Last updated
- 2025-03-06
Files and Weights
30 files, 3.2 GB in total. The weights are 14 files totalling 3.2 GB in bin, onnx, ot, safetensors.
Weights14 files · 3.2 GB
Configuration8 files · 30.4 KB
Tokenizer4 files · 2.6 MB
Documentation1 file · 10.1 KB
Other2 files · 589.6 KB
Repository1 file · 791 B
Every file
| File | Type | Size | SHA-256 |
|---|---|---|---|
| model.safetensors | Weights | 328.5 MB | 3c739fad12f3 |
| onnx/model.onnx | Weights | 326.2 MB | e29a4c305dba |
| onnx/model_O1.onnx | Weights | 326.2 MB | 0aa8e69c5185 |
| onnx/model_O2.onnx | Weights | 326.1 MB | aa55f453a76f |
| onnx/model_O3.onnx | Weights | 326.1 MB | c2fa562c0651 |
| onnx/model_O4.onnx | Weights | 163.1 MB | 02f627f41a8b |
| onnx/model_qint8_arm64.onnx | Weights | 82.2 MB | c9e6677186f1 |
| onnx/model_qint8_avx512.onnx | Weights | 82.2 MB | c9e6677186f1 |
| onnx/model_qint8_avx512_vnni.onnx | Weights | 82.2 MB | c9e6677186f1 |
| onnx/model_quint8_avx2.onnx | Weights | 82.2 MB | 0672eaf82896 |
| openvino/openvino_model.bin | Weights | 326.1 MB | 212d7661f0a7 |
| openvino/openvino_model_qint8_quantized.bin | Weights | 82.2 MB | 308b23332854 |
| pytorch_model.bin | Weights | 328.5 MB | b90e424e0e82 |
| rust_model.ot | Weights | 328.5 MB | d1a9901b8d3b |
| 1_Pooling/config.json | Configuration | 190 B | — |
| config.json | Configuration | 653 B | — |
| config_sentence_transformers.json | Configuration | 116 B | — |
| data_config.json | Configuration | 15.7 KB | — |
| modules.json | Configuration | 349 B | — |
| sentence_bert_config.json | Configuration | 53 B | — |
| special_tokens_map.json | Configuration | 239 B | — |
| train_script.py | Configuration | 13.1 KB | — |
| README.md | Documentation | 10.1 KB | — |
| openvino/openvino_model.xml | Other | 216.5 KB | — |
| openvino/openvino_model_qint8_quantized.xml | Other | 373.1 KB | — |
| .gitattributes | Repository | 791 B | — |
| merges.txt | Tokenizer | 456.4 KB | — |
| tokenizer.json | Tokenizer | 1.4 MB | — |
| tokenizer_config.json | Tokenizer | 333 B | — |
| vocab.json | Tokenizer | 798.3 KB | — |
License and Download
- License
- apache-2.0
- Access
- Open weights, no gate
- Download size
- 3.2 GB
Download from Sentence Transformers
Released by Sentence Transformers through its official repository on Hugging Face. Read the license.
Built From
- Described by arXiv:1704.05179
- Described by arXiv:1810.09305
- Described by arXiv:1904.06472
- Described by arXiv:2102.07033
- Described by arXiv:2104.08727
- Trained on (disclosed) code_search_net
- Trained on (disclosed) eli5
- Trained on (disclosed) embedding-data/PAQ_pairs
- Trained on (disclosed) embedding-data/QQP
- Trained on (disclosed) embedding-data/SPECTER
- Trained on (disclosed) embedding-data/WikiAnswers
- Trained on (disclosed) embedding-data/altlex
- Trained on (disclosed) embedding-data/flickr30k-captions
- Trained on (disclosed) embedding-data/sentence-compression
- Trained on (disclosed) embedding-data/simple-wiki
- Trained on (disclosed) flax-sentence-embeddings/stackexchange_xml
- Trained on (disclosed) gooaq
- Trained on (disclosed) ms_marco
- Trained on (disclosed) multi_nli
- Trained on (disclosed) natural_questions
- Trained on (disclosed) s2orc
- Trained on (disclosed) search_qa
- Trained on (disclosed) snli
- Trained on (disclosed) trivia_qa
- Trained on (disclosed) wikihow
- Trained on (disclosed) yahoo_answers_topics
Memory Requirements
| Precision | Weights in memory |
|---|---|
| As published | 3.2 GB |
| 16-bit | 0.2 GB |
| 8-bit | 0.1 GB |
| 4-bit | 0.0 GB |
Weights only, from the published parameter count; the key-value cache and runtime add to this.
Compare all-distilroberta-v1
Questions About all-distilroberta-v1
How much GPU memory does all-distilroberta-v1 need?
About 0.2 GB at 16-bit and 0 GB at 4-bit: the weights (82M parameters) plus a working margin. A long context needs more.
What is the cheapest GPU to run all-distilroberta-v1 on?
At 16-bit, 1x MI300X from $1.85 an hour; at 4-bit, 1x MI300X from $1.85 an hour, at the lowest on-demand prices the SAVRN Index lists.
Can I use all-distilroberta-v1 commercially?
Yes. all-distilroberta-v1 is released under Apache License 2.0. The Apache License 2.0 is a permissive open-source license. It permits commercial use, modification and redistribution. It requires keeping the license and copyright notices and any NOTICE file, stating significant changes, and it includes an express patent grant from contributors.
What is all-distilroberta-v1's context length?
514 tokens, from the maximum position embeddings in its published configuration.
Similar Models
This is a LinkTransformer model. At its core this model this is a sentence transformer model sentence-transformers model- it just wraps around the class. It is designed for quick and easy record linkage (entity-matching) through the LinkTransformer package. The tasks include clustering, deduplication, linking, aggregation and more. Notwithstanding that, it can be used for any sentence similarity task within the sentence-transformers framework as well. It maps sentences & paragraphs to a 768 dimensional dense vector space and can be used for tasks like clustering or semantic search. Take a look at the documentation of sentence-transformers if you want to use this model for more than what we…
This is a LinkTransformer model. At its core this model this is a sentence transformer model sentence-transformers model- it just wraps around the class. It is designed for quick and easy record linkage (entity-matching) through the LinkTransformer package. The tasks include clustering, deduplication, linking, aggregation and more. Notwithstanding that, it can be used for any sentence similarity task within the sentence-transformers framework as well. It maps sentences & paragraphs to a 768 dimensional dense vector space and can be used for tasks like clustering or semantic search. Take a look at the documentation of sentence-transformers if you want to use this model for more than what we…
This is a CoSENT(Cosine Sentence) model: shibing624/text2vec-base-chinese. It maps sentences to a 768 dimensional dense vector space and can be used for tasks like sentence embeddings, text matching or semantic search. For an automated evaluation of this model, see the Evaluation Benchmark: text2vec - chinese text matching task： - 结果评测指标：spearman系数 - shibing624/text2vec-base-chinese模型，是用CoSENT方法训练，基于hfl/chinese-macbert-base在中文STS-B数据训练得到，并在中文STS-B测试集评估达到较好效果，运行examples/trainingsuptextmatchingmodel.py代码可训练模型，模型文件已经上传HF model hub，中文通用语义匹配任务推荐使用…
This is a sentence-transformers model finetuned from BAAI/bge-base-en-v1.5. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more. First install the Sentence Transformers library: Then you can load this model and run inference. Approximate statistics based on the first 45 samples: Approximate statistics based on the first 5 samples: - evalstrategy: steps - perdevicetrainbatchsize: 16 - perdeviceevalbatchsize: 16 - learningrate: 3e-06 - maxsteps: 24 - warmupratio: 0.1 - batchsampler: noduplicates - overwriteoutputdir: False - dopredict: False…
This is a PubMedBERT-base model fined-tuned using sentence-transformers. It maps sentences & paragraphs to a 768 dimensional dense vector space and can be used for tasks like clustering or semantic search. The training dataset was generated using a random sample of PubMed title-abstract pairs along with similar title pairs. PubMedBERT Embeddings produces higher quality embeddings than generalized models for medical literature. Further fine-tuning for a medical subdomain will result in even better performance. This model can be used to build embeddings databases with txtai for semantic search and/or as a knowledge source for retrieval augmented generation (RAG). Alternatively, the model can…