Skip to page content
SAVRN
Search Plan a project

Open-weight model · Sentence similarity

simcse-bert-base-snli-unsup

by Julio De Aquino pyrawn/simcse-bert-base-snli-unsup

simcse-bert-base-snli-unsup is an open-weight model for sentence similarity from Julio De Aquino. It has 109M parameters and a 512-token context. At 16-bit it needs about 0.3 GB of GPU memory, which fits on 1x MI300X from $1.85 an hour, at the lowest prices in the SAVRN Index.

Unsupervised SimCSE (Gao, Yao & Chen, EMNLP 2021, Eq. 1) on top of bert-base-uncased, trained on sentences from a 100k-record SNLI subset as a course assignment (U2T02, Trends in Data Science).

Parameters109M
Context512
Weights438.0 MB
License—
AccessOpen weights
Monthly Downloads—

Runs On

What it takes to serve simcse-bert-base-snli-unsup (109M parameters): the memory its weights need at each precision, and the cheapest way to rent enough data-center GPUs to hold them.

PrecisionWeightsMemory neededCheapest setupPer hourAlso fits
16-bit 0.2 GB 0.3 GB 1x MI300X (192 GB)
Vultr
$1.85 1x H100 $1.99 · 1x MI325X $2.00
8-bit 0.1 GB 0.1 GB 1x MI300X (192 GB)
Vultr
$1.85 1x H100 $1.99 · 1x MI325X $2.00
4-bit 0.1 GB 0.1 GB 1x MI300X (192 GB)
Vultr
$1.85 1x H100 $1.99 · 1x MI325X $2.00

Memory is the weights at that precision plus 20% for the runtime and a short context; a long context needs more. Prices are the lowest on-demand hourly rates in the SAVRN Index, read Oct 18, 2026.

simcse-bert-base-snli-unsup on every accelerator the SAVRN Index prices, at every precision

Model Card

Unsupervised SimCSE (Gao, Yao & Chen, EMNLP 2021, Eq. 1) on top of bert-base-uncased, trained on sentences from a 100k-record SNLI subset as a course assignment (U2T02, Trends in Data Science). It maps a sentence to a 768-dimensional, L2-normalized vector; compare vectors with cosine similarity. Team: Julio Cesar De Aquino Castellanos, Gustavo Alexander Fuentes Marin, Valeria Nicol Hernandez Leon, Ricardo Daniel Horta Sanchez, Jose Angel Pech Xool, Lorena Danae Perez Lopez. This is a reproduction under a reduced budget, not the paper's released checkpoint: it is 8.5 Spearman points below the paper on STS-B test (see Limitations). The supervised companion model is…

Excerpt from the card by Julio De Aquino.

Configuration

Architecture
BertModel
Context length (tokens)
512
Layers
12
Hidden size
768
Feed-forward size
3,072
Attention heads
12
Vocabulary size
30,522
Model type
bert

Identity and Version

Repository
pyrawn/simcse-bert-base-snli-unsup
Publisher
Julio De Aquino
Task
Sentence similarity
Modality
Text
Library
sentence-transformers
Parameters
109M parameters
Languages
en
Revision
49a48bc73518c1e7068772fac078fd18c6246dc4
First published
2026-10-03
Last updated
2026-10-03

Files and Weights

11 files, 438.7 MB in total. The weights are 1 file totalling 438.0 MB in safetensors.

Weights1 file · 438.0 MB
Configuration6 files · 1.9 KB
Tokenizer2 files · 712.2 KB
Documentation1 file · 6.6 KB
Repository1 file · 1.5 KB
Every file
FileTypeSizeSHA-256
model.safetensorsWeights438.0 MB aadc92f7f819
1_Pooling/config.jsonConfiguration89 B —
2_Normalize/config.jsonConfiguration97 B —
config.jsonConfiguration746 B —
config_sentence_transformers.jsonConfiguration282 B —
modules.jsonConfiguration413 B —
sentence_bert_config.jsonConfiguration241 B —
README.mdDocumentation6.6 KB —
.gitattributesRepository1.5 KB —
tokenizer.jsonTokenizer711.6 KB —
tokenizer_config.jsonTokenizer541 B —

License and Download

License
Not stated by the source
Access
Open weights, no gate
Download size
438.0 MB
Download from Julio De Aquino

Released by Julio De Aquino through its official repository on Hugging Face.

Built From

Evaluations

Each result is shown as reported, with the conditions its reporter stated. None is a SAVRN measurement. A comparison lines two results up only when their configuration, unit and setup are all stated and identical.

BenchmarkConditionsResultReported byRevisionDate
STS-B (test) Task Semantic Textual SimilarityMetric Spearman (cosine) x100Comparison conditions not established 68.33 pyrawn
Publisher reported
Evaluated revision not stated —

Memory Requirements

PrecisionWeights in memory
As published438.0 MB
16-bit0.2 GB
8-bit0.1 GB
4-bit0.1 GB

Weights only, from the published parameter count; the key-value cache and runtime add to this.

Questions About simcse-bert-base-snli-unsup

How much GPU memory does simcse-bert-base-snli-unsup need?

About 0.3 GB at 16-bit and 0.1 GB at 4-bit: the weights (109M parameters) plus a working margin. A long context needs more.

What is the cheapest GPU to run simcse-bert-base-snli-unsup on?

At 16-bit, 1x MI300X from $1.85 an hour; at 4-bit, 1x MI300X from $1.85 an hour, at the lowest on-demand prices the SAVRN Index lists.

What is simcse-bert-base-snli-unsup's context length?

512 tokens, from the maximum position embeddings in its published configuration.

Similar Models

This is a sentence-transformers model finetuned from BAAI/bge-base-en-v1.5. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more. First install the Sentence Transformers library: Then you can load this model and run inference. Approximate statistics based on the first 45 samples: Approximate statistics based on the first 5 samples: - evalstrategy: steps - perdevicetrainbatchsize: 16 - perdeviceevalbatchsize: 16 - learningrate: 3e-06 - maxsteps: 24 - warmupratio: 0.1 - batchsampler: noduplicates - overwriteoutputdir: False - dopredict: False…

Open weights 109M parameters 512 tokens sentence-transformers

Model · Sentence similarity

pubmedbert-base-embeddings

NeuML

This is a PubMedBERT-base model fined-tuned using sentence-transformers. It maps sentences & paragraphs to a 768 dimensional dense vector space and can be used for tasks like clustering or semantic search. The training dataset was generated using a random sample of PubMed title-abstract pairs along with similar title pairs. PubMedBERT Embeddings produces higher quality embeddings than generalized models for medical literature. Further fine-tuning for a medical subdomain will result in even better performance. This model can be used to build embeddings databases with txtai for semantic search and/or as a knowledge source for retrieval augmented generation (RAG). Alternatively, the model can…

Open weights apache-2.0 109M parameters 512 tokens sentence-transformers

Model · Sentence similarity

vandf-rxnorm-biencoder

Kevin Venanzi

Maps a VA National Drug File (VANDF) drug string to the RxNorm clinical drug confidence so you can auto-accept the sure cases and route the rest to review. Published tools such as RxMap normalize to the ingredient level (IN/MIN). This model targets the full clinical drug, where a miss on strength or dose form is a wrong answer. Method, hypotheses, the full sweep, and the error Mapper loads the encoder, trainconfig.json (input preprocessing), calibration.json (score → probability), and candidates.parquet (the 27,287 active RxNorm SCD/SBD names it searches) from this repo. The default acceptance threshold (0.92) was chosen on validation for 99% precision on a population that includes real…

Open weights apache-2.0 109M parameters 512 tokens sentence-transformers

Model · Sentence similarity

vandf-rxnorm-biencoder-all

Kevin Venanzi

The recipe of kvenanzi/vandf-rxnorm-biencoder, trained on all 14,369 VA National Drug File (VANDF) strings that have a clinical drug in RxNorm, so that every ingredient family in the file is in training. It maps a VANDF drug string to the RxNorm clinical drug it names (SCD or SBD: ingredient, strength, and dose form), with a calibrated confidence for routing uncertain strings to review. This model has no test set of its own: every VA string was used for training. Its expected accuracy comes from seven-fold cross-validation of the recipe, and a comparison with the first model on strings neither model trained on (below). Write-up: https://withinnoise.dev/blog/posts/vandf-rxnorm-interventions/…

Open weights apache-2.0 109M parameters 512 tokens sentence-transformers

Model · Sentence similarity

finetuned-medical-bert

Fico Filbert

This is a sentence-transformers model finetuned from google-bert/bert-base-uncased. It maps inputs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, classification, clustering, and more. First install the Sentence Transformers library: Then you can load this model and run inference. Approximate statistics based on the first 100 samples: - perdevicetrainbatchsize: 16 - perdeviceevalbatchsize: 16 - multidatasetbatchsampler: roundrobin - dopredict: False - predictionlossonly: True - perdevicetrainbatchsize: 16 - perdeviceevalbatchsize: 16 - gradientaccumulationsteps: 1 - evalaccumulationsteps: None…

Open weights 109M parameters 512 tokens sentence-transformers

Model · Sentence similarity

simcse-bert-uncased-unsup

Russel Ku

Unsupervised SimCSE sentence encoder trained from bert-base-uncased on a 100k-record subset of SNLI, as a course replication of Gao, Yao & Chen (EMNLP 2021). It maps an English sentence to a 768-dimensional vector; compare sentences with cosine similarity. Team (Universidad Politécnica de Yucatán, U2T02): Acosta Castellanos Bianca Alexandra, Canche Chuc Angel Rivaldo, Ku Aguilar Russel, Sanchez Novelo Damian, Velasco Martin Jonathan Abisai. 165,528 unique sentences (premises and hypotheses, whitespace-stripped) from snlitrain100k.jsonl, a 100k-record subset of the SNLI train split. Labels are not used: each sentence is its own positive, the two views differ only by dropout. The paper…

Open weights apache-2.0 109M parameters 512 tokens sentence-transformers