SAVRN Model Hub · Comparisons
all-distilroberta-v1 vs pubmedbert-base-embeddings
All-distilroberta-v1 has 82M parameters and pubmedbert-base-embeddings has 109M parameters; both are released under Apache License 2.0; at 16-bit, all-distilroberta-v1 needs about 0.2 GB (1x MI300X from $1.85 an hour) and pubmedbert-base-embeddings about 0.3 GB (1x MI300X from $1.85 an hour).
| Field | all-distilroberta-v1 sentence-transformers/all-distilroberta-v1 | pubmedbert-base-embeddings NeuML/pubmedbert-base-embeddings |
|---|---|---|
| Publisher | Sentence Transformers | NeuML |
| Task | Sentence similarity | Sentence similarity |
| Modality | Text | Text |
| Parameters, as reported | 82M parameters | 109M parameters |
| Architecture | RobertaForMaskedLM | BertModel |
| Library | sentence-transformers | sentence-transformers |
| Context length | 514 tokens | 512 tokens |
| Repository size | 3.2 GB | 876.9 MB |
| Artifact formats | safetensors, onnx, openvino, pytorch, rust | safetensors, pytorch |
| License | apache-2.0 | apache-2.0 |
| Access | Open weights, no gate | Open weights, no gate |
| Memory at 16-bit (weights and margin) | 0.2 GB | 0.3 GB |
| Cheapest GPUs at 16-bit, per hour | 1x MI300X, $1.85 | 1x MI300X, $1.85 |
| Memory at 4-bit (weights and margin) | 0 GB | 0.1 GB |
| Cheapest GPUs at 4-bit, per hour | 1x MI300X, $1.85 | 1x MI300X, $1.85 |
| Revision viewed | 842eaed40bee | b79526d6ef36 |
| Downloads reported by the hub | 2.7M | 938k |
| Last observed | 2026-09-18 | 2026-09-18 |
An evaluation row appears only where at least two of these models report the same benchmark with the same stated configuration, metric, unit and setup. Different evaluators stay named in each cell. Values are shown as reported: no unit conversion, no ranking.
SAVRN's Notes on all-distilroberta-v1
How much GPU does an 82M-parameter embedding model need? Almost none: 0.2 GB of weights at 16-bit, 0.1 GB at 8-bit, and the cheapest setup on our Index is one MI300X with 192 GB at $1.85 an hour, a card this model leaves nearly empty, so share it with other work. What you get is a 768-dimensional vector for each sentence or paragraph, meant for semantic search and clustering, inside a 514-token context, so anything longer gets chunked first.
Eight training sets are on record, among them s2orc, ms_marco, code_search_net, stackexchange_xml and eli5; ask whether your corpus looks like them. Apache 2.0 means shipping it inside a commercial product, modifying it and redistributing it are allowed once the license and notices are kept and changes stated. Download only the format you need: the 3.19 GB across 30 files spans safetensors, onnx, openvino, pytorch and rust, and one is enough.
SAVRN's Notes on pubmedbert-base-embeddings
Medical literature search is where we would put this one. NeuML fine-tuned a PubMedBERT base with sentence-transformers on PubMed title and abstract pairs, and the output is a 768-dimensional vector per sentence or paragraph, built for clustering and semantic search over medical text. The 109M parameters need 0.3 GB at 16-bit, and the stored precision is float32, so even the full-width copy stays under 1 GB. The lowest-cost host in our data is one MI300X at $1.85 an hour, far more card than an embedding job this size needs. Share it with something bigger.
Apache 2.0 covers commercial use, modification and redistribution, with notices kept and significant changes stated. Two things to check: the 512-token context, so anything longer gets chunked before embedding, and the parent model, Microsoft's BiomedBERT base trained on abstracts and full text, which carries its own terms.
Questions
Which is larger, all-distilroberta-v1 or pubmedbert-base-embeddings?
pubmedbert-base-embeddings (109M parameters) is larger than all-distilroberta-v1 (82M parameters), by the parameter counts their publishers report.
Which is cheaper to run, all-distilroberta-v1 or pubmedbert-base-embeddings?
At 4-bit, all-distilroberta-v1 fits on 1x MI300X from $1.85 an hour and pubmedbert-base-embeddings on 1x MI300X from $1.85 an hour, at the lowest on-demand prices the SAVRN Index lists.
Can I use all-distilroberta-v1 commercially?
Yes. all-distilroberta-v1 is released under Apache License 2.0. The Apache License 2.0 is a permissive open-source license. It permits commercial use, modification and redistribution. It requires keeping the license and copyright notices and any NOTICE file, stating significant changes, and it includes an express patent grant from contributors.
Can I use pubmedbert-base-embeddings commercially?
Yes. pubmedbert-base-embeddings is released under Apache License 2.0. The Apache License 2.0 is a permissive open-source license. It permits commercial use, modification and redistribution. It requires keeping the license and copyright notices and any NOTICE file, stating significant changes, and it includes an express patent grant from contributors.