SAVRN
Search Contact SAVRN

Open-weight model · Sentence similarity

pubmedbert-base-embeddings

by NeuML NeuML/pubmedbert-base-embeddings

This is a PubMedBERT-base model fined-tuned using sentence-transformers. It maps sentences & paragraphs to a 768 dimensional dense vector space and can be used for tasks like clustering or semantic search.

Parameters109M
Context512
Weights875.9 MB
Licenseapache-2.0
AccessOpen weights
Monthly Downloads938k

Runs On

What it takes to serve pubmedbert-base-embeddings (109M parameters): the memory its weights need at each precision, and the cheapest way to rent enough data-center GPUs to hold them.

PrecisionWeightsMemory neededCheapest setupPer hourAlso fits
16-bit 0.2 GB 0.3 GB 1x MI300X (192 GB)
Vultr		 $1.85 1x H100 $1.99 · 1x MI325X $2.00
8-bit 0.1 GB 0.1 GB 1x MI300X (192 GB)
Vultr		 $1.85 1x H100 $1.99 · 1x MI325X $2.00
4-bit 0.1 GB 0.1 GB 1x MI300X (192 GB)
Vultr		 $1.85 1x H100 $1.99 · 1x MI325X $2.00

Memory is the weights at that precision plus 20% for the runtime and a short context; a long context needs more. Prices are the lowest on-demand hourly rates in the SAVRN Index, read Sep 18, 2026.

SAVRN's Notes on pubmedbert-base-embeddings

Medical literature search is where we would put this one. NeuML fine-tuned a PubMedBERT base with sentence-transformers on PubMed title and abstract pairs, and the output is a 768-dimensional vector per sentence or paragraph, built for clustering and semantic search over medical text. The 109M parameters need 0.3 GB at 16-bit, and the stored precision is float32, so even the full-width copy stays under 1 GB. The lowest-cost host in our data is one MI300X at $1.85 an hour, far more card than an embedding job this size needs. Share it with something bigger.

Apache 2.0 covers commercial use, modification and redistribution, with notices kept and significant changes stated. Two things to check: the 512-token context, so anything longer gets chunked before embedding, and the parent model, Microsoft's BiomedBERT base trained on abstracts and full text, which carries its own terms.

SAVRN Research, 2026-09-18

Model Card

By NeuML, published under apache-2.0, revision b79526d6ef36.

PubMedBERT Embeddings

This is a PubMedBERT-base model fined-tuned using sentence-transformers. It maps sentences & paragraphs to a 768 dimensional dense vector space and can be used for tasks like clustering or semantic search. The training dataset was generated using a random sample of PubMed title-abstract pairs along with similar title pairs.

PubMedBERT Embeddings produces higher quality embeddings than generalized models for medical literature. Further fine-tuning for a medical subdomain will result in even better performance.

Usage (txtai)

This model can be used to build embeddings databases with txtai for semantic search and/or as a knowledge source for retrieval augmented generation (RAG).

import txtai

embeddings = txtai.Embeddings(path="neuml/pubmedbert-base-embeddings", content=True)
embeddings.index(documents())

# Run a query
embeddings.search("query to run")

Usage (Sentence-Transformers)

Alternatively, the model can be loaded with sentence-transformers.

from sentence_transformers import SentenceTransformer
sentences = ["This is an example sentence", "Each sentence is converted"]

model = SentenceTransformer("neuml/pubmedbert-base-embeddings")
embeddings = model.encode(sentences)
print(embeddings)

Usage (Hugging Face Transformers)

Read the full model card (492 words)

Configuration

Architecture
BertModel
Context length (tokens)
512
Layers
12
Hidden size
768
Feed-forward size
3,072
Attention heads
12
Vocabulary size
30,522
Stored precision
float32
Model type
bert

Identity and Version

Repository
NeuML/pubmedbert-base-embeddings
Publisher
NeuML
Task
Sentence similarity
Modality
Text
Library
sentence-transformers
Parameters
109M parameters
Languages
en
Revision
b79526d6ef3645e0df4530322e266f24c829f5ef
First published
2023-10-18
Last updated
2026-04-21

Files and Weights

15 files, 876.9 MB in total. The weights are 2 files totalling 875.9 MB in bin, safetensors.

Weights2 files · 875.9 MB
Configuration7 files · 1.5 KB
Tokenizer3 files · 933.7 KB
Documentation1 file · 6.3 KB
Other1 file · 301 B
Repository1 file · 1.5 KB
Every file
FileTypeSizeSHA-256
model.safetensorsWeights438.0 MB 929ddc16369b
pytorch_model.binWeights438.0 MB 0bdb9787bcb6
1_Pooling/config.jsonConfiguration190 B —
added_tokens.jsonConfiguration74 B —
config.jsonConfiguration667 B —
config_sentence_transformers.jsonConfiguration123 B —
modules.jsonConfiguration229 B —
sentence_bert_config.jsonConfiguration53 B —
special_tokens_map.jsonConfiguration125 B —
README.mdDocumentation6.3 KB —
similarity_evaluation_results.csvOther301 B —
.gitattributesRepository1.5 KB —
tokenizer.jsonTokenizer706.3 KB —
tokenizer_config.jsonTokenizer1.3 KB —
vocab.txtTokenizer226.2 KB —

License and Download

License
apache-2.0
Access
Open weights, no gate
Download size
875.9 MB
Download from NeuML

Released by NeuML through its official repository on Hugging Face. Read the license.

Built From

Memory Requirements

PrecisionWeights in memory
As published875.9 MB
16-bit0.2 GB
8-bit0.1 GB
4-bit0.1 GB

Weights only, from the published parameter count; the key-value cache and runtime add to this.

Compare pubmedbert-base-embeddings

Questions About pubmedbert-base-embeddings

How much GPU memory does pubmedbert-base-embeddings need?

About 0.3 GB at 16-bit and 0.1 GB at 4-bit: the weights (109M parameters) plus a working margin. A long context needs more.

What is the cheapest GPU to run pubmedbert-base-embeddings on?

At 16-bit, 1x MI300X from $1.85 an hour; at 4-bit, 1x MI300X from $1.85 an hour, at the lowest on-demand prices the SAVRN Index lists.

Can I use pubmedbert-base-embeddings commercially?

Yes. pubmedbert-base-embeddings is released under Apache License 2.0. The Apache License 2.0 is a permissive open-source license. It permits commercial use, modification and redistribution. It requires keeping the license and copyright notices and any NOTICE file, stating significant changes, and it includes an express patent grant from contributors.

What is pubmedbert-base-embeddings's context length?

512 tokens, from the maximum position embeddings in its published configuration.

Similar Models

This is a sentence-transformers model finetuned from BAAI/bge-base-en-v1.5. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more. First install the Sentence Transformers library: Then you can load this model and run inference. Approximate statistics based on the first 45 samples: Approximate statistics based on the first 5 samples: - evalstrategy: steps - perdevicetrainbatchsize: 16 - perdeviceevalbatchsize: 16 - learningrate: 3e-06 - maxsteps: 24 - warmupratio: 0.1 - batchsampler: noduplicates - overwriteoutputdir: False - dopredict: False…

Open weights 109M parameters 512 tokens sentence-transformers
View model

Model · Sentence similarity

e5-base-v2

Liang Wang

Liang Wang, Nan Yang, Xiaolong Huang, Binxing Jiao, Linjun Yang, Daxin Jiang, Rangan Majumder, Furu Wei, arXiv 2022 This model has 12 layers and the embedding size is 768. Below is an example to encode queries and passages from the MS-MARCO passage ranking dataset. Please refer to our paper at https://arxiv.org/pdf/2212.03533.pdf. Check out unilm/e5 to reproduce evaluation results on the BEIR and MTEB benchmark. Below is an example for usage with sentencetransformers. Package requirements pip install sentencetransformers~=2.2.2 1. Do I need to add the prefix "query: " and "passage: " to input texts? Yes, this is how the model is trained, otherwise you will see a performance degradation.…

Open weights mit 109M parameters 512 tokens sentence-transformers
View model

Model · Sentence similarity

e5-base

Liang Wang

News (May 2023): please switch to e5-base-v2, which has better performance and same method of usage. Liang Wang, Nan Yang, Xiaolong Huang, Binxing Jiao, Linjun Yang, Daxin Jiang, Rangan Majumder, Furu Wei, arXiv 2022 This model has 12 layers and the embedding size is 768. Below is an example to encode queries and passages from the MS-MARCO passage ranking dataset. Please refer to our paper at https://arxiv.org/pdf/2212.03533.pdf. Check out unilm/e5 to reproduce evaluation results on the BEIR and MTEB benchmark. Below is an example for usage with sentencetransformers. Package requirements pip install sentencetransformers~=2.2.2 1. Do I need to add the prefix "query: " and "passage: " to…

Open weights mit 109M parameters 512 tokens sentence-transformers
View model

Model · Sentence similarity

lt-wikidata-comp-en

Dell Research Harvard

This is a LinkTransformer model. At its core this model this is a sentence transformer model sentence-transformers model- it just wraps around the class. It is designed for quick and easy record linkage (entity-matching) through the LinkTransformer package. The tasks include clustering, deduplication, linking, aggregation and more. Notwithstanding that, it can be used for any sentence similarity task within the sentence-transformers framework as well. It maps sentences & paragraphs to a 768 dimensional dense vector space and can be used for tasks like clustering or semantic search. Take a look at the documentation of sentence-transformers if you want to use this model for more than what we…

Open weights 109M parameters 514 tokens sentence-transformers
View model

Model · Sentence similarity

lt-un-data-fine-fine-en

Dell Research Harvard

This is a LinkTransformer model. At its core this model this is a sentence transformer model sentence-transformers model- it just wraps around the class. It is designed for quick and easy record linkage (entity-matching) through the LinkTransformer package. The tasks include clustering, deduplication, linking, aggregation and more. Notwithstanding that, it can be used for any sentence similarity task within the sentence-transformers framework as well. It maps sentences & paragraphs to a 768 dimensional dense vector space and can be used for tasks like clustering or semantic search. Take a look at the documentation of sentence-transformers if you want to use this model for more than what we…

Open weights 109M parameters 514 tokens sentence-transformers
View model

Model · Sentence similarity

names-linking-model

Dell Research Harvard

This is a LinkTransformer model. At its core this model this is a sentence transformer model sentence-transformers model- it just wraps around the class. It is designed for quick and easy record linkage (entity-matching) through the LinkTransformer package. The tasks include clustering, deduplication, linking, aggregation and more. Notwithstanding that, it can be used for any sentence similarity task within the sentence-transformers framework as well. It maps sentences & paragraphs to a 768 dimensional dense vector space and can be used for tasks like clustering or semantic search. Take a look at the documentation of sentence-transformers if you want to use this model for more than what we…

Open weights 109M parameters 514 tokens sentence-transformers
View model