ms-marco-MiniLM-L12-v2: Open-Weight Model

Runs On

What it takes to serve ms-marco-MiniLM-L12-v2 (33M parameters): the memory its weights need at each precision, and the cheapest way to rent enough data-center GPUs to hold them.

Precision	Weights	Memory needed	Cheapest setup	Per hour	Also fits
16-bit	0.1 GB	0.1 GB	1x MI300X (192 GB) Vultr	$1.85	1x H100 $1.99 · 1x MI325X $2.00
8-bit	0.0 GB	0.0 GB	1x MI300X (192 GB) Vultr	$1.85	1x H100 $1.99 · 1x MI325X $2.00
4-bit	0.0 GB	0.0 GB	1x MI300X (192 GB) Vultr	$1.85	1x H100 $1.99 · 1x MI325X $2.00

Memory is the weights at that precision plus 20% for the runtime and a short context; a long context needs more. Prices are the lowest on-demand hourly rates in the SAVRN Index, read Sep 18, 2026.

SAVRN's Notes on ms-marco-MiniLM-L12-v2

Put this behind the search box, after retrieval. A cross-encoder scores a query against each candidate passage, so you pull candidates with something like Elasticsearch and let this model order them. Trained on the MS MARCO passage ranking task, it carries 33 million parameters and needs 0.1 GB of memory at 16-bit. A 192 GB MI300X at $1.85 an hour, the cheapest fit we list, is a mismatch of scale, so this rides on a card already doing other work.

The license is Apache 2.0, with the usual obligations: keep the license and NOTICE file attached and state significant changes. Start with the 512 token limit: query plus passage must fit inside it, so long passages get chunked before scoring. Second, lineage: it derives from microsoft/MiniLM-L12-H384-uncased and trained on sentence-transformers/msmarco, so test ranking on your own documents. It ships in safetensors, ONNX, OpenVINO, PyTorch and JAX formats.

SAVRN Research, 2026-09-18

Model Card

By Sentence Transformers - Cross-Encoders, published under apache-2.0, revision 7b0235231ca2.

Cross-Encoder for MS Marco

This model was trained on the MS Marco Passage Ranking task.

The model can be used for Information Retrieval: Given a query, encode the query will all possible passages (e.g. retrieved with ElasticSearch). Then sort the passages in a decreasing order. See SBERT.net Retrieve & Re-rank for more details. The training code is available here: SBERT.net Training MS Marco

Usage with SentenceTransformers

The usage is easy when you have SentenceTransformers installed. Then you can use the pre-trained models like this:

from sentence_transformers import CrossEncoder

model = CrossEncoder('cross-encoder/ms-marco-MiniLM-L12-v2')
scores = model.predict([
    ("How many people live in Berlin?", "Berlin had a population of 3,520,031 registered inhabitants in an area of 891.82 square kilometers."),
    ("How many people live in Berlin?", "Berlin is well known for its museums."),
])
print(scores)
# [ 9.218911  -4.0780287]

Usage with Transformers

Read the full model card (315 words)

Configuration

Architecture: BertForSequenceClassification
Context length (tokens): 512
Layers: 12
Hidden size: 384
Feed-forward size: 1,536
Attention heads: 12
Vocabulary size: 30,522
Model type: bert

Identity and Version

Repository: cross-encoder/ms-marco-MiniLM-L12-v2
Publisher: Sentence Transformers - Cross-Encoders
Task: Text ranking
Modality: Other
Library: sentence-transformers
Parameters: 33M parameters
Languages: en
Revision: 7b0235231ca2674cb8ca8f022859a6eba2b1c968
First published: 2022-03-02
Last updated: 2025-08-29

Files and Weights

23 files, 1.3 GB in total. The weights are 14 files totalling 1.3 GB in bin, msgpack, onnx, safetensors.

Weights14 files · 1.3 GB

Configuration2 files · 923 B

Tokenizer3 files · 944.2 KB

Documentation1 file · 3.7 KB

Other2 files · 1.1 MB

Repository1 file · 790 B

Every file

File	Type	Size	SHA-256
flax_model.msgpack	Weights	133.4 MB	f5544d476410
model.safetensors	Weights	133.5 MB	1ed84b90cdf3
onnx/model.onnx	Weights	133.7 MB	2ac389ab6abe
onnx/model_O1.onnx	Weights	133.6 MB	c3a5524ece92
onnx/model_O2.onnx	Weights	133.5 MB	39748ee92c8b
onnx/model_O3.onnx	Weights	133.5 MB	7bced017a0a0
onnx/model_O4.onnx	Weights	66.9 MB	cb2d497a5f40
onnx/model_qint8_arm64.onnx	Weights	34.3 MB	148a402605f6
onnx/model_qint8_avx512.onnx	Weights	34.3 MB	148a402605f6
onnx/model_qint8_avx512_vnni.onnx	Weights	34.3 MB	148a402605f6
onnx/model_quint8_avx2.onnx	Weights	34.3 MB	0a9906ae940e
openvino/openvino_model.bin	Weights	133.4 MB	8538376778c5
openvino/openvino_model_qint8_quantized.bin	Weights	34.0 MB	81c1d9df505f
pytorch_model.bin	Weights	133.5 MB	207bb14d184b
config.json	Configuration	791 B	—
special_tokens_map.json	Configuration	132 B	—
README.md	Documentation	3.7 KB	—
openvino/openvino_model.xml	Other	382.4 KB	—
openvino/openvino_model_qint8_quantized.xml	Other	702.6 KB	—
.gitattributes	Repository	790 B	—
tokenizer.json	Tokenizer	711.4 KB	—
tokenizer_config.json	Tokenizer	1.3 KB	—
vocab.txt	Tokenizer	231.5 KB	—

License and Download

License: apache-2.0
Access: Open weights, no gate
Download size: 1.3 GB

Download from Sentence Transformers - Cross-Encoders

Released by Sentence Transformers - Cross-Encoders through its official repository on Hugging Face. Read the license.

Built From

Derived from microsoft/MiniLM-L12-H384-uncased
Quantized from microsoft/MiniLM-L12-H384-uncased
Trained on (disclosed) sentence-transformers/msmarco

Memory Requirements

Precision	Weights in memory
As published	1.3 GB
16-bit	0.1 GB
8-bit	0.0 GB
4-bit	0.0 GB

Weights only, from the published parameter count; the key-value cache and runtime add to this.

Built on This Model

Quantized fromms-marco-MiniLM-L6-v2
Derived fromms-marco-MiniLM-L6-v2
Quantized fromms-marco-MiniLM-L4-v2
Derived fromms-marco-MiniLM-L4-v2
Quantized fromms-marco-MiniLM-L2-v2
Derived fromms-marco-MiniLM-L2-v2

Compare ms-marco-MiniLM-L12-v2

Questions About ms-marco-MiniLM-L12-v2

How much GPU memory does ms-marco-MiniLM-L12-v2 need?

About 0.1 GB at 16-bit and 0 GB at 4-bit: the weights (33M parameters) plus a working margin. A long context needs more.

What is the cheapest GPU to run ms-marco-MiniLM-L12-v2 on?

At 16-bit, 1x MI300X from $1.85 an hour; at 4-bit, 1x MI300X from $1.85 an hour, at the lowest on-demand prices the SAVRN Index lists.

Can I use ms-marco-MiniLM-L12-v2 commercially?

Yes. ms-marco-MiniLM-L12-v2 is released under Apache License 2.0. The Apache License 2.0 is a permissive open-source license. It permits commercial use, modification and redistribution. It requires keeping the license and copyright notices and any NOTICE file, stating significant changes, and it includes an express patent grant from contributors.

What is ms-marco-MiniLM-L12-v2's context length?

512 tokens, from the maximum position embeddings in its published configuration.

Similar Models

Model · Text ranking

ms-marco-MiniLM-L6-v2

Sentence Transformers - Cross-Encoders

This model was trained on the MS Marco Passage Ranking task. The model can be used for Information Retrieval: Given a query, encode the query with all possible passages (e.g. retrieved with ElasticSearch). Then sort the passages in a decreasing order. See SBERT.net Retrieve & Re-rank for more details. The training code is available here: SBERT.net Training MS Marco The usage is easy when you have SentenceTransformers installed. Then you can use the pre-trained models like this: In the following table, we provide various pre-trained Cross-Encoders together with their performance on the TREC Deep Learning 2019 and the MS Marco Passage Reranking dataset.

Open weights apache-2.0 23M parameters 512 tokens sentence-transformers

View model

Model · Text ranking

ms-marco-MiniLM-L4-v2

Sentence Transformers - Cross-Encoders

This model was trained on the MS Marco Passage Ranking task. The model can be used for Information Retrieval: Given a query, encode the query will all possible passages (e.g. retrieved with ElasticSearch). Then sort the passages in a decreasing order. See SBERT.net Retrieve & Re-rank for more details. The training code is available here: SBERT.net Training MS Marco The usage is easy when you have SentenceTransformers installed. Then you can use the pre-trained models like this: In the following table, we provide various pre-trained Cross-Encoders together with their performance on the TREC Deep Learning 2019 and the MS Marco Passage Reranking dataset.

Open weights apache-2.0 19M parameters 512 tokens sentence-transformers

View model

Model · Text ranking

ms-marco-MiniLM-L2-v2

Sentence Transformers - Cross-Encoders

This model was trained on the MS Marco Passage Ranking task. The model can be used for Information Retrieval: Given a query, encode the query will all possible passages (e.g. retrieved with ElasticSearch). Then sort the passages in a decreasing order. See SBERT.net Retrieve & Re-rank for more details. The training code is available here: SBERT.net Training MS Marco The usage is easy when you have SentenceTransformers installed. Then you can use the pre-trained models like this: In the following table, we provide various pre-trained Cross-Encoders together with their performance on the TREC Deep Learning 2019 and the MS Marco Passage Reranking dataset.

Open weights apache-2.0 16M parameters 512 tokens sentence-transformers

View model

Model · Text ranking

ms-marco-TinyBERT-L2-v2

Sentence Transformers - Cross-Encoders

This model was trained on the MS Marco Passage Ranking task. The model can be used for Information Retrieval: Given a query, encode the query will all possible passages (e.g. retrieved with ElasticSearch). Then sort the passages in a decreasing order. See SBERT.net Retrieve & Re-rank for more details. The training code is available here: SBERT.net Training MS Marco The usage becomes easier when you have SentenceTransformers installed. Then, you can use the pre-trained models like this: In the following table, we provide various pre-trained Cross-Encoders together with their performance on the TREC Deep Learning 2019 and the MS Marco Passage Reranking dataset.

Open weights apache-2.0 4M parameters 512 tokens sentence-transformers

View model

Model · Text ranking

rerank-indonesia-Q8_0-GGUF

Radhi Fadlillah

This model was converted to GGUF format from madebyaris/rerank-indonesia using llama.cpp via the ggml.ai's GGUF-my-repo space. Refer to the original model card for more details on the model. Install llama.cpp through brew (works on Mac and Linux) Invoke the llama.cpp server or the CLI. Note: You can also use this checkpoint directly through the usage steps listed in the Llama.cpp repo as well. Step 2: Move into the llama.cpp folder and build it with LLAMACURL=1 flag along with other hardware-specific flags (for ex: LLAMACUDA=1 for Nvidia GPUs on Linux).

Open weights apache-2.0 sentence-transformers

View model

Model · Text ranking

mxbai-rerank-xsmall-v1

Mixedbread

This is the smallest model in our family of powerful reranker models. You can learn more about the models in our blog post. Currently, the best way to use our models is with the most recent version of sentence-transformers. pip install -U sentence-transformers Let's say you have a query, and you want to rerank a set of documents. You can do that with only one line of code: Install transformers.js npm i @xenova/transformers Let's say you have a query, and you want to rerank a set of documents. In JavaScript, you need to add a function: You can use the large model via our API as follows: The API comes with additional features, such as a continous trained reranker! Check out the docs for more…

Open weights apache-2.0 71M parameters 512 tokens transformers

View model