SAVRN
Search Contact SAVRN

Open-weight model · Question answering

deberta-v3-base-squad2

by Deepset deepset/deberta-v3-base-squad2

This is the deberta-v3-base model, fine-tuned using the SQuAD2.0 dataset. It's been trained on question-answer pairs, including unanswerable questions, for the task of Question Answering.

Parameters184M
Context512
Weights1.5 GB
Licensecc-by-4.0
AccessOpen weights
Monthly Downloads247.6k

Runs On

What it takes to serve deberta-v3-base-squad2 (184M parameters): the memory its weights need at each precision, and the cheapest way to rent enough data-center GPUs to hold them.

PrecisionWeightsMemory neededCheapest setupPer hourAlso fits
16-bit 0.4 GB 0.4 GB 1x MI300X (192 GB)
Vultr		 $1.85 1x H100 $1.99 · 1x MI325X $2.00
8-bit 0.2 GB 0.2 GB 1x MI300X (192 GB)
Vultr		 $1.85 1x H100 $1.99 · 1x MI325X $2.00
4-bit 0.1 GB 0.1 GB 1x MI300X (192 GB)
Vultr		 $1.85 1x H100 $1.99 · 1x MI325X $2.00

Memory is the weights at that precision plus 20% for the runtime and a short context; a long context needs more. Prices are the lowest on-demand hourly rates in the SAVRN Index, read Sep 18, 2026.

Model Card

By Deepset, published under cc-by-4.0, revision eea39c60cc30.

deberta-v3-base for Extractive QA

This is the deberta-v3-base model, fine-tuned using the SQuAD2.0 dataset. It's been trained on question-answer pairs, including unanswerable questions, for the task of Question Answering.

Overview

Language model: deberta-v3-base
Language: English
Downstream-task: Extractive QA
Training data: SQuAD 2.0
Eval data: SQuAD 2.0
Code: See an example extractive QA pipeline built with Haystack
Infrastructure: 1x NVIDIA A10G

Hyperparameters

batch_size = 12
n_epochs = 4
base_LM_model = "deberta-v3-base"
max_seq_len = 512
learning_rate = 2e-5
lr_schedule = LinearWarmup
warmup_proportion = 0.2
doc_stride = 128
max_query_length = 64

Usage

In Haystack

Haystack is an AI orchestration framework to build customizable, production-ready LLM applications. You can use this model in Haystack to do extractive question answering on documents. To load and run the model with Haystack:

Read the full model card (381 words)

Configuration

Architecture
DebertaV2ForQuestionAnswering
Context length (tokens)
512
Layers
12
Hidden size
768
Feed-forward size
3,072
Attention heads
12
Vocabulary size
128,100
Stored precision
float32
Model type
deberta-v2

Identity and Version

Repository
deepset/deberta-v3-base-squad2
Publisher
Deepset
Task
Question answering
Modality
Text
Library
transformers
Parameters
184M parameters
Languages
en
Revision
eea39c60cc305c2e4a9504f5ff117294bebb42db
First published
2022-07-22
Last updated
2024-09-24

Files and Weights

10 files, 1.5 GB in total. The weights are 2 files totalling 1.5 GB in bin, safetensors.

Weights2 files · 1.5 GB
Configuration3 files · 1.2 KB
Tokenizer2 files · 8.6 MB
Documentation1 file · 8.7 KB
Other1 file · 2.5 MB
Repository1 file · 1.5 KB
Every file
FileTypeSizeSHA-256
model.safetensorsWeights735.4 MB aff8759009b7
pytorch_model.binWeights735.4 MB 8a2477929551
added_tokens.jsonConfiguration23 B —
config.jsonConfiguration992 B —
special_tokens_map.jsonConfiguration173 B —
README.mdDocumentation8.7 KB —
spm.modelOther2.5 MB c679fbf93643
.gitattributesRepository1.5 KB —
tokenizer.jsonTokenizer8.6 MB 7fb827d15550
tokenizer_config.jsonTokenizer379 B —

License and Download

License
cc-by-4.0
Access
Open weights, no gate
Download size
1.5 GB
Download from Deepset

Released by Deepset through its official repository on Hugging Face. Read the license.

Built From

Evaluations

Each result is shown as reported, with the conditions its reporter stated. None is a SAVRN measurement. A comparison lines two results up only when their configuration, unit and setup are all stated and identical.

BenchmarkConditionsResultReported byRevisionDate
adversarial_qa Configuration adversarialQATask Question AnsweringMetric Exact MatchComparison conditions not established 30.733 deepset
Publisher reported		 Evaluated revision not stated —
adversarial_qa Configuration adversarialQATask Question AnsweringMetric F1Comparison conditions not established 44.099 deepset
Publisher reported		 Evaluated revision not stated —
squad Configuration plain_textTask Question AnsweringMetric Exact MatchComparison conditions not established 84.9678 deepset
Publisher reported		 Evaluated revision not stated —
squad Configuration plain_textTask Question AnsweringMetric F1Comparison conditions not established 92.2777 deepset
Publisher reported		 Evaluated revision not stated —
squad_adversarial Configuration AddOneSentTask Question AnsweringMetric Exact MatchComparison conditions not established 79.295 deepset
Publisher reported		 Evaluated revision not stated —
squad_adversarial Configuration AddOneSentTask Question AnsweringMetric F1Comparison conditions not established 86.609 deepset
Publisher reported		 Evaluated revision not stated —
squad_v2 Configuration squad_v2Task Question AnsweringMetric Exact MatchComparison conditions not established 83.8248 deepset
Publisher reported		 Evaluated revision not stated —
squad_v2 Configuration squad_v2Task Question AnsweringMetric F1Comparison conditions not established 87.41 deepset
Publisher reported		 Evaluated revision not stated —
squadshifts amazon Configuration amazonTask Question AnsweringMetric Exact MatchComparison conditions not established 68.68 deepset
Publisher reported		 Evaluated revision not stated —
squadshifts amazon Configuration amazonTask Question AnsweringMetric F1Comparison conditions not established 83.832 deepset
Publisher reported		 Evaluated revision not stated —
squadshifts new_wiki Configuration new_wikiTask Question AnsweringMetric Exact MatchComparison conditions not established 80.171 deepset
Publisher reported		 Evaluated revision not stated —
squadshifts new_wiki Configuration new_wikiTask Question AnsweringMetric F1Comparison conditions not established 90.452 deepset
Publisher reported		 Evaluated revision not stated —
squadshifts nyt Configuration nytTask Question AnsweringMetric Exact MatchComparison conditions not established 81.57 deepset
Publisher reported		 Evaluated revision not stated —
squadshifts nyt Configuration nytTask Question AnsweringMetric F1Comparison conditions not established 90.644 deepset
Publisher reported		 Evaluated revision not stated —
squadshifts reddit Configuration redditTask Question AnsweringMetric Exact MatchComparison conditions not established 66.99 deepset
Publisher reported		 Evaluated revision not stated —
squadshifts reddit Configuration redditTask Question AnsweringMetric F1Comparison conditions not established 80.231 deepset
Publisher reported		 Evaluated revision not stated —

Memory Requirements

PrecisionWeights in memory
As published1.5 GB
16-bit0.4 GB
8-bit0.2 GB
4-bit0.1 GB

Weights only, from the published parameter count; the key-value cache and runtime add to this.

Questions About deberta-v3-base-squad2

How much GPU memory does deberta-v3-base-squad2 need?

About 0.4 GB at 16-bit and 0.1 GB at 4-bit: the weights (184M parameters) plus a working margin. A long context needs more.

What is the cheapest GPU to run deberta-v3-base-squad2 on?

At 16-bit, 1x MI300X from $1.85 an hour; at 4-bit, 1x MI300X from $1.85 an hour, at the lowest on-demand prices the SAVRN Index lists.

Can I use deberta-v3-base-squad2 commercially?

Yes. deberta-v3-base-squad2 is released under Creative Commons Attribution 4.0. CC BY 4.0 permits sharing and adapting the work, including commercially, provided the creator is credited and changes are indicated.

What is deberta-v3-base-squad2's context length?

512 tokens, from the maximum position embeddings in its published configuration.

Similar Models

This model was created by Google and fine-tuned on XQuAD like data for multilingual (11 different languages) Q&A downstream task. Deepmind XQuAD As the dataset is based on SQuAD v1.1, there are no unanswerable questions in the data. We chose this setting so that models can focus on cross-lingual transfer. We show the average number of tokens per paragraph, question, and answer for each language in the table below. The statistics were obtained using Jieba for Chinese and the Moses tokenizer for the other languages. As XQuAD is just an evaluation dataset, I used Data augmentation techniques (scraping, neural machine translation, etc) to obtain more samples and split the dataset in order to…

Open weights 178M parameters 512 tokens transformers
View model

Model · Question answering

flan-t5-base-squad2

Sebastian Husch Lee

This is the flan-t5-base model, fine-tuned using the SQuAD2.0 dataset. It's been trained on question-answer pairs, including unanswerable questions, for the task of Extractive Question Answering. UPDATE: With transformers version 4.31.0 the useremotecode=True is no longer necessary. NOTE: The token must be manually added to the beginning of the question for this model to work properly. It uses the token to be able to make "no answer" predictions. The t5 tokenizer does not automatically add this special token which is why it is added manually. The following hyperparameters were used during training: - learningrate: 2e-05 - trainbatchsize: 16 - evalbatchsize: 8 - gradientaccumulationsteps: 6…

Open weights mit 223M parameters transformers
View model

Model · Question answering

roberta-base-squad2

Deepset

This is the roberta-base model, fine-tuned using the SQuAD2.0 dataset. It's been trained on question-answer pairs, including unanswerable questions, for the task of Extractive Question Answering. We have also released a distilled version of this model called deepset/tinyroberta-squad2. It has a comparable prediction quality and runs at twice the speed of deepset/roberta-base-squad2. Haystack is an AI orchestration framework to build customizable, production-ready LLM applications. You can use this model in Haystack to do extractive question answering on documents. To load and run the model with Haystack: For a complete example with an extractive question answering pipeline that scales over…

Open weights cc-by-4.0 124M parameters 514 tokens transformers
View model

Model · Question answering

roberta-base-squad2-distilled

Deepset

haystack's distillation feature was used for training. deepset/roberta-large-squad2 was used as the teacher model. Haystack is an AI orchestration framework to build customizable, production-ready LLM applications. You can use this model in Haystack to do extractive question answering on documents. To load and run the model with Haystack: For a complete example with an extractive question answering pipeline that scales over many documents, check out the corresponding Haystack tutorial. Timo Möller: [email protected] deepset is the company behind the production-ready open-source AI framework Haystack. We also have a Discord community open to everyone!

Open weights mit 124M parameters 514 tokens transformers
View model

Model · Question answering

roberta-base-cuad

Taner Akdeniz

This model is the fine-tuned version of "RoBERTa Base" using CUAD dataset https://huggingface.co/datasets/cuad For the use of the model with CUAD: https://github.com/marshmellow77/cuad-demo and https://huggingface.co/spaces/akdeniz27/contract-understanding-atticus-dataset-demo

Open weights 124M parameters 514 tokens transformers
View model