tinyroberta-squad2 by Deepset: Open-Weight Model

Runs On

What it takes to serve tinyroberta-squad2 (82M parameters): the memory its weights need at each precision, and the cheapest way to rent enough data-center GPUs to hold them.

Precision	Weights	Memory needed	Cheapest setup	Per hour	Also fits
16-bit	0.2 GB	0.2 GB	1x MI300X (192 GB) Vultr	$1.85	1x H100 $1.99 · 1x MI325X $2.00
8-bit	0.1 GB	0.1 GB	1x MI300X (192 GB) Vultr	$1.85	1x H100 $1.99 · 1x MI325X $2.00
4-bit	0.0 GB	0.0 GB	1x MI300X (192 GB) Vultr	$1.85	1x H100 $1.99 · 1x MI325X $2.00

Memory is the weights at that precision plus 20% for the runtime and a short context; a long context needs more. Prices are the lowest on-demand hourly rates in the SAVRN Index, read Sep 18, 2026.

Model Card

By Deepset, published under cc-by-4.0, revision 12b287c9df67.

tinyroberta for Extractive QA

This is the distilled version of the deepset/roberta-base-squad2 model. This model has a comparable prediction quality and runs at twice the speed of the base model.

Overview

Language model: tinyroberta-squad2
Language: English
Downstream-task: Extractive QA
Training data: SQuAD 2.0
Eval data: SQuAD 2.0
Code: See an example extractive QA pipeline built with Haystack
Infrastructure: 4x Tesla v100

Hyperparameters

batch_size = 96
n_epochs = 4
base_LM_model = "deepset/tinyroberta-squad2-step1"
max_seq_len = 384
learning_rate = 3e-5
lr_schedule = LinearWarmup
warmup_proportion = 0.2
doc_stride = 128
max_query_length = 64
distillation_loss_weight = 0.75
temperature = 1.5
teacher = "deepset/robert-large-squad2"

Distillation

Read the full model card (491 words)

Configuration

Architecture: RobertaForQuestionAnswering
Context length (tokens): 514
Layers: 6
Hidden size: 768
Feed-forward size: 3,072
Attention heads: 12
Vocabulary size: 50,265
Stored precision: float32
Model type: roberta

Identity and Version

Repository: deepset/tinyroberta-squad2
Publisher: Deepset
Task: Question answering
Modality: Text
Library: transformers
Parameters: 82M parameters
Languages: en
Revision: 12b287c9df677e28b07f0a023850dba68c997dbf
First published: 2022-03-02
Last updated: 2024-09-26

Files and Weights

10 files, 654.9 MB in total. The weights are 2 files totalling 652.3 MB in bin, safetensors.

Weights2 files · 652.3 MB

Configuration2 files · 1.1 KB

Tokenizer4 files · 2.6 MB

Documentation1 file · 9.4 KB

Repository1 file · 1.2 KB

Every file

File	Type	Size	SHA-256
model.safetensors	Weights	326.1 MB	39955cbf5f63
pytorch_model.bin	Weights	326.2 MB	20d4e188516b
config.json	Configuration	835 B	—
special_tokens_map.json	Configuration	239 B	—
README.md	Documentation	9.4 KB	—
.gitattributes	Repository	1.2 KB	—
merges.txt	Tokenizer	456.4 KB	—
tokenizer.json	Tokenizer	1.4 MB	—
tokenizer_config.json	Tokenizer	383 B	—
vocab.json	Tokenizer	798.3 KB	—

License and Download

License: cc-by-4.0
Access: Open weights, no gate
Download size: 652.3 MB

Download from Deepset

Released by Deepset through its official repository on Hugging Face. Read the license.

Built From

Described by arXiv:1909.10351
Trained on (disclosed) squad_v2

Evaluations

Each result is shown as reported, with the conditions its reporter stated. None is a SAVRN measurement. A comparison lines two results up only when their configuration, unit and setup are all stated and identical.

Benchmark	Conditions	Result	Reported by	Revision	Date
adversarial_qa	Configuration adversarialQATask Question AnsweringMetric Exact MatchComparison conditions not established	25.967	deepset Publisher reported	Evaluated revision not stated	—
adversarial_qa	Configuration adversarialQATask Question AnsweringMetric F1Comparison conditions not established	37.006	deepset Publisher reported	Evaluated revision not stated	—
squad	Configuration plain_textTask Question AnsweringMetric Exact MatchComparison conditions not established	83.86	deepset Publisher reported	Evaluated revision not stated	—
squad	Configuration plain_textTask Question AnsweringMetric F1Comparison conditions not established	90.752	deepset Publisher reported	Evaluated revision not stated	—
squad_adversarial	Configuration AddOneSentTask Question AnsweringMetric Exact MatchComparison conditions not established	76.329	deepset Publisher reported	Evaluated revision not stated	—
squad_adversarial	Configuration AddOneSentTask Question AnsweringMetric F1Comparison conditions not established	83.292	deepset Publisher reported	Evaluated revision not stated	—
squad_v2	Configuration squad_v2Task Question AnsweringMetric Exact MatchComparison conditions not established	78.8627	deepset Publisher reported	Evaluated revision not stated	—
squad_v2	Configuration squad_v2Task Question AnsweringMetric F1Comparison conditions not established	82.0355	deepset Publisher reported	Evaluated revision not stated	—
squadshifts amazon	Configuration amazonTask Question AnsweringMetric Exact MatchComparison conditions not established	63.915	deepset Publisher reported	Evaluated revision not stated	—
squadshifts amazon	Configuration amazonTask Question AnsweringMetric F1Comparison conditions not established	78.395	deepset Publisher reported	Evaluated revision not stated	—
squadshifts new_wiki	Configuration new_wikiTask Question AnsweringMetric Exact MatchComparison conditions not established	80.297	deepset Publisher reported	Evaluated revision not stated	—
squadshifts new_wiki	Configuration new_wikiTask Question AnsweringMetric F1Comparison conditions not established	89.808	deepset Publisher reported	Evaluated revision not stated	—
squadshifts nyt	Configuration nytTask Question AnsweringMetric Exact MatchComparison conditions not established	80.149	deepset Publisher reported	Evaluated revision not stated	—
squadshifts nyt	Configuration nytTask Question AnsweringMetric F1Comparison conditions not established	88.321	deepset Publisher reported	Evaluated revision not stated	—
squadshifts reddit	Configuration redditTask Question AnsweringMetric Exact MatchComparison conditions not established	66.959	deepset Publisher reported	Evaluated revision not stated	—
squadshifts reddit	Configuration redditTask Question AnsweringMetric F1Comparison conditions not established	79.3	deepset Publisher reported	Evaluated revision not stated	—

Memory Requirements

Precision	Weights in memory
As published	652.3 MB
16-bit	0.2 GB
8-bit	0.1 GB
4-bit	0.0 GB

Weights only, from the published parameter count; the key-value cache and runtime add to this.

Questions About tinyroberta-squad2

How much GPU memory does tinyroberta-squad2 need?

About 0.2 GB at 16-bit and 0 GB at 4-bit: the weights (82M parameters) plus a working margin. A long context needs more.

What is the cheapest GPU to run tinyroberta-squad2 on?

At 16-bit, 1x MI300X from $1.85 an hour; at 4-bit, 1x MI300X from $1.85 an hour, at the lowest on-demand prices the SAVRN Index lists.

Can I use tinyroberta-squad2 commercially?

Yes. tinyroberta-squad2 is released under Creative Commons Attribution 4.0. CC BY 4.0 permits sharing and adapting the work, including commercially, provided the creator is credited and changes are indicated.

What is tinyroberta-squad2's context length?

514 tokens, from the maximum position embeddings in its published configuration.

Similar Models

Model · Question answering

distilbert-base-uncased-distilled-squad

DistilBERT community

The DistilBERT model was proposed in the blog post Smaller, faster, cheaper, lighter: Introducing DistilBERT, adistilled version of BERT, and the paper DistilBERT, adistilled version of BERT: smaller, faster, cheaper and lighter. DistilBERT is a small, fast, cheap and light Transformer model trained by distilling BERT base. It has 40% less parameters than bert-base-uncased, runs 60% faster while preserving over 95% of BERT's performances as measured on the GLUE language understanding benchmark. This model is a fine-tune checkpoint of DistilBERT-base-uncased, fine-tuned using (a second step of) knowledge distillation on SQuAD v1.1. - See this repository for more about Distil\ (a class of…

Open weights apache-2.0 66M parameters 512 tokens transformers

View model

Model · Question answering

distilbert-base-cased-distilled-squad

DistilBERT community

The DistilBERT model was proposed in the blog post Smaller, faster, cheaper, lighter: Introducing DistilBERT, adistilled version of BERT, and the paper DistilBERT, adistilled version of BERT: smaller, faster, cheaper and lighter. DistilBERT is a small, fast, cheap and light Transformer model trained by distilling BERT base. It has 40% less parameters than bert-base-uncased, runs 60% faster while preserving over 95% of BERT's performances as measured on the GLUE language understanding benchmark. This model is a fine-tune checkpoint of DistilBERT-base-cased, fine-tuned using (a second step of) knowledge distillation on SQuAD v1.1. - See this repository for more about Distil\ (a class of…

Open weights apache-2.0 65M parameters 512 tokens transformers

View model

Model · Question answering

bert-base-cased-squad2

Deepset

This is a BERT base cased model trained on SQuAD v2 Haystack is an AI orchestration framework to build customizable, production-ready LLM applications. You can use this model in Haystack to do extractive question answering on documents. To load and run the model with Haystack: For a complete example with an extractive question answering pipeline that scales over many documents, check out the corresponding Haystack tutorial. deepset is the company behind the production-ready open-source AI framework Haystack. We also have a Discord community open to everyone!

Open weights cc-by-4.0 108M parameters 512 tokens transformers

View model

Model · Question answering

electra-base-squad2

Deepset

Evaluated on the SQuAD 2.0 dev set with the official eval script. Haystack is an AI orchestration framework to build customizable, production-ready LLM applications. You can use this model in Haystack to do extractive question answering on documents. To load and run the model with Haystack: For a complete example with an extractive question answering pipeline that scales over many documents, check out the corresponding Haystack tutorial. Vaishali Pal vaishali.pal [at] deepset.ai Timo Möller: timo.moeller [at] deepset.ai deepset is the company behind the production-ready open-source AI framework Haystack. We also have a Discord community open to everyone!

Open weights cc-by-4.0 109M parameters 512 tokens transformers

View model

Model · Question answering

bert-base-uncased-squad2

Deepset

Haystack is an AI orchestration framework to build customizable, production-ready LLM applications. You can use this model in Haystack to do extractive question answering on documents. To load and run the model with Haystack: For a complete example with an extractive question answering pipeline that scales over many documents, check out the corresponding Haystack tutorial. - Timo Möller: timo.moeller [at] deepset.ai deepset is the company behind the production-ready open-source AI framework Haystack. We also have a Discord community open to everyone!

Open weights cc-by-4.0 109M parameters 512 tokens transformers

View model

Model · Question answering

gelectra-base-germanquad

Deepset

We trained a German question answering model with a gelectra-base model as its basis. - The dataset is GermanQuAD, a new, German language dataset, which we hand-annotated and published online. - The training dataset is one-way annotated and contains 11518 questions and 11518 answers, while the test dataset is three-way annotated so that there are 2204 questions and with 2204·3−76 = 6536answers, because we removed 76 wrong answers. See https://deepset.ai/germanquad for more details and dataset download in SQuAD format. Haystack is an AI orchestration framework to build customizable, production-ready LLM applications. You can use this model in Haystack to do extractive question answering on…

Open weights mit 109M parameters 512 tokens transformers

View model