# roberta-large-squad2 by Deepset: Open-Weight Model Source: https://savrn.com/models/roberta-large-squad2 Markdown alternate of the page above; the site index is https://savrn.com/llms.txt --- ## Runs On What it takes to serve roberta-large-squad2 (354M parameters): the memory its weights need at each precision, and the cheapest way to rent enough data-center GPUs to hold them. | Precision | Weights | Memory needed | Cheapest setup | Per hour | Also fits | | --- | --- | --- | --- | --- | --- | | 16-bit | 0.7 GB | 0.9 GB | 1x [MI300X](https://savrn.com/ai-index/pricing/gpus/mi300x) (192 GB) Vultr | $1.85 | [1x H100](https://savrn.com/ai-index/pricing/gpus/h100) $1.99 · [1x MI325X](https://savrn.com/ai-index/pricing/gpus/mi325x) $2.00 | | 8-bit | 0.4 GB | 0.4 GB | 1x [MI300X](https://savrn.com/ai-index/pricing/gpus/mi300x) (192 GB) Vultr | $1.85 | [1x H100](https://savrn.com/ai-index/pricing/gpus/h100) $1.99 · [1x MI325X](https://savrn.com/ai-index/pricing/gpus/mi325x) $2.00 | | 4-bit | 0.2 GB | 0.2 GB | 1x [MI300X](https://savrn.com/ai-index/pricing/gpus/mi300x) (192 GB) Vultr | $1.85 | [1x H100](https://savrn.com/ai-index/pricing/gpus/h100) $1.99 · [1x MI325X](https://savrn.com/ai-index/pricing/gpus/mi325x) $2.00 | Memory is the weights at that precision plus 20% for the runtime and a short context; a long context needs more. Prices are the lowest on-demand hourly rates in the [SAVRN Index](https://savrn.com/ai-index/pricing/gpus), read Oct 23, 2026. [roberta-large-squad2 on every accelerator the SAVRN Index prices, at every precision](https://savrn.com/models/roberta-large-squad2/gpus) ## Model Card By Deepset, published under cc-by-4.0, revision 78fb38a59ea3. ### roberta-large for Extractive QA This is the [roberta-large](https://huggingface.co/roberta-large) model, fine-tuned using the [SQuAD2.0](https://huggingface.co/datasets/squad_v2) dataset. It's been trained on question-answer pairs, including unanswerable questions, for the task of Question Answering. ### Overview Language model: roberta-large Language: English Downstream-task: Extractive QA Training data: SQuAD 2.0 Eval data: SQuAD 2.0 Code: See [an example extractive QA pipeline built with Haystack](https://haystack.deepset.ai/tutorials/34_extractive_qa_pipeline) Infrastructure: 4x Tesla v100 ### Hyperparameters ``` base_LM_model = "roberta-large" ``` ### Using a distilled model instead Please note that we have also released a distilled version of this model called [deepset/roberta-base-squad2-distilled](https://savrn.com/models/roberta-base-squad2-distilled). The distilled model has a comparable prediction quality and runs at twice the speed of the large model. ### Usage #### In Haystack Haystack is an AI orchestration framework to build customizable, production-ready LLM applications. You can use this model in Haystack to do extractive question answering on documents. To load and run the model with [Haystack](https://github.com/deepset-ai/haystack/): [Read the full model card (393 words)](https://savrn.com/models/roberta-large-squad2/card) ## Configuration Architecture RobertaForQuestionAnswering Context length (tokens) 514 Layers 24 Hidden size 1,024 Feed-forward size 4,096 Attention heads 16 Vocabulary size 50,265 Model type roberta ## Identity and Version Repository deepset/roberta-large-squad2 Publisher Deepset Task Question answering Modality Text Library transformers Parameters 354M parameters Languages en Revision 78fb38a59ea3cb6902e04d96da93efc87aeeff76 First published 2022-03-02 Last updated 2024-09-26 ## Files and Weights 10 files, 4.3 GB in total. The weights are 3 files totalling 4.3 GB in bin, msgpack, safetensors. Weights3 files · 4.3 GB Configuration2 files · 935 B Tokenizer3 files · 1.3 MB Documentation1 file · 7.7 KB Repository1 file · 790 B Every file | File | Type | Size | SHA-256 | | --- | --- | --- | --- | | flax_model.msgpack | Weights | 1.4 GB | 17d19dd0f991 | | model.safetensors | Weights | 1.4 GB | 63ff07ecdc76 | | pytorch_model.bin | Weights | 1.4 GB | f0a98cec67b9 | | config.json | Configuration | 696 B | — | | special_tokens_map.json | Configuration | 239 B | — | | README.md | Documentation | 7.7 KB | — | | .gitattributes | Repository | 790 B | — | | merges.txt | Tokenizer | 456.4 KB | — | | tokenizer_config.json | Tokenizer | 1.2 KB | — | | vocab.json | Tokenizer | 798.3 KB | — | ## License and Download License cc-by-4.0 Access Open weights, no gate Download size 4.3 GB [Download from Deepset](https://huggingface.co/deepset/roberta-large-squad2) Released by Deepset through its official repository on Hugging Face. [Read the license](https://creativecommons.org/licenses/by/4.0/). ## Built From - Derived from [FacebookAI/roberta-large](https://savrn.com/models/roberta-large) - Trained on (disclosed) squad_v2 ## Evaluations Each result is shown as reported, with the conditions its reporter stated. None is a SAVRN measurement. A comparison lines two results up only when their configuration, unit and setup are all stated and identical. | Benchmark | Conditions | Result | Reported by | Revision | Date | | --- | --- | --- | --- | --- | --- | | adversarial_qa | Configuration adversarialQATask Question AnsweringMetric Exact MatchComparison conditions not established | 35.9 | [deepset](https://huggingface.co/deepset/roberta-large-squad2) Publisher reported | Evaluated revision not stated | — | | adversarial_qa | Configuration adversarialQATask Question AnsweringMetric F1Comparison conditions not established | 48.923 | [deepset](https://huggingface.co/deepset/roberta-large-squad2) Publisher reported | Evaluated revision not stated | — | | squad | Configuration plain_textTask Question AnsweringMetric Exact MatchComparison conditions not established | 87.162 | [deepset](https://huggingface.co/deepset/roberta-large-squad2) Publisher reported | Evaluated revision not stated | — | | squad | Configuration plain_textTask Question AnsweringMetric F1Comparison conditions not established | 93.603 | [deepset](https://huggingface.co/deepset/roberta-large-squad2) Publisher reported | Evaluated revision not stated | — | | squad_adversarial | Configuration AddOneSentTask Question AnsweringMetric Exact MatchComparison conditions not established | 81.142 | [deepset](https://huggingface.co/deepset/roberta-large-squad2) Publisher reported | Evaluated revision not stated | — | | squad_adversarial | Configuration AddOneSentTask Question AnsweringMetric F1Comparison conditions not established | 87.099 | [deepset](https://huggingface.co/deepset/roberta-large-squad2) Publisher reported | Evaluated revision not stated | — | | squad_v2 | Configuration squad_v2Task Question AnsweringMetric Exact MatchComparison conditions not established | 85.168 | [deepset](https://huggingface.co/deepset/roberta-large-squad2) Publisher reported | Evaluated revision not stated | — | | squad_v2 | Configuration squad_v2Task Question AnsweringMetric F1Comparison conditions not established | 88.349 | [deepset](https://huggingface.co/deepset/roberta-large-squad2) Publisher reported | Evaluated revision not stated | — | | squadshifts amazon | Configuration amazonTask Question AnsweringMetric Exact MatchComparison conditions not established | 72.453 | [deepset](https://huggingface.co/deepset/roberta-large-squad2) Publisher reported | Evaluated revision not stated | — | | squadshifts amazon | Configuration amazonTask Question AnsweringMetric F1Comparison conditions not established | 86.325 | [deepset](https://huggingface.co/deepset/roberta-large-squad2) Publisher reported | Evaluated revision not stated | — | | squadshifts new_wiki | Configuration new_wikiTask Question AnsweringMetric Exact MatchComparison conditions not established | 82.338 | [deepset](https://huggingface.co/deepset/roberta-large-squad2) Publisher reported | Evaluated revision not stated | — | | squadshifts new_wiki | Configuration new_wikiTask Question AnsweringMetric F1Comparison conditions not established | 91.974 | [deepset](https://huggingface.co/deepset/roberta-large-squad2) Publisher reported | Evaluated revision not stated | — | | squadshifts nyt | Configuration nytTask Question AnsweringMetric Exact MatchComparison conditions not established | 84.352 | [deepset](https://huggingface.co/deepset/roberta-large-squad2) Publisher reported | Evaluated revision not stated | — | | squadshifts nyt | Configuration nytTask Question AnsweringMetric F1Comparison conditions not established | 92.645 | [deepset](https://huggingface.co/deepset/roberta-large-squad2) Publisher reported | Evaluated revision not stated | — | | squadshifts reddit | Configuration redditTask Question AnsweringMetric Exact MatchComparison conditions not established | 74.722 | [deepset](https://huggingface.co/deepset/roberta-large-squad2) Publisher reported | Evaluated revision not stated | — | | squadshifts reddit | Configuration redditTask Question AnsweringMetric F1Comparison conditions not established | 86.86 | [deepset](https://huggingface.co/deepset/roberta-large-squad2) Publisher reported | Evaluated revision not stated | — | ## Memory Requirements | Precision | Weights in memory | | --- | --- | | As published | 4.3 GB | | 16-bit | 0.7 GB | | 8-bit | 0.4 GB | | 4-bit | 0.2 GB | Weights only, from the published parameter count; the key-value cache and runtime add to this. ## Questions About roberta-large-squad2 ### How much GPU memory does roberta-large-squad2 need? About 0.9 GB at 16-bit and 0.2 GB at 4-bit: the weights (354M parameters) plus a working margin. A long context needs more. ### What is the cheapest GPU to run roberta-large-squad2 on? At 16-bit, 1x MI300X from $1.85 an hour; at 4-bit, 1x MI300X from $1.85 an hour, at the lowest on-demand prices the SAVRN Index lists. ### Can I use roberta-large-squad2 commercially? Yes. roberta-large-squad2 is released under Creative Commons Attribution 4.0. CC BY 4.0 permits sharing and adapting the work, including commercially, provided the creator is credited and changes are indicated. ### What is roberta-large-squad2's context length? 514 tokens, from the maximum position embeddings in its published configuration. ## Similar Models Model · Question answering ### [bert-large-uncased-whole-word-masking-squad2](https://savrn.com/models/bert-large-uncased-whole-word-masking-squad2) [Deepset](https://savrn.com/model-publishers/deepset) This is a berta-large model, fine-tuned using the SQuAD2.0 dataset for the task of question answering. Haystack is an AI orchestration framework to build customizable, production-ready LLM applications. You can use this model in Haystack to do extractive question answering on documents. To load and run the model with Haystack: For a complete example with an extractive question answering pipeline that scales over many documents, check out the corresponding Haystack tutorial. deepset is the company behind the production-ready open-source AI framework Haystack. We also have a Discord community open to everyone! Open weights cc-by-4.0 335M parameters 512 tokens transformers [View model](https://savrn.com/models/bert-large-uncased-whole-word-masking-squad2) Model · Question answering ### [bert-large-uncased-whole-word-masking-finetuned-squad](https://savrn.com/models/bert-large-uncased-whole-word-masking-finetuned-squad) [BERT community](https://savrn.com/model-publishers/google-bert) Pretrained model on English language using a masked language modeling (MLM) objective. It was introduced in this paper and first released in this repository. This model is uncased: it does not make a difference between english and English. Differently to other BERT models, this model was trained with a new technique: Whole Word Masking. In this case, all of the tokens corresponding to a word are masked at once. The overall masking rate remains the same. The training is identical -- each masked WordPiece token is predicted independently. After pre-training, this model was fine-tuned on the SQuAD dataset with one of our fine-tuning scripts. See below for more information regarding this… Open weights apache-2.0 335M parameters 512 tokens transformers [View model](https://savrn.com/models/bert-large-uncased-whole-word-masking-finetuned-squad) Model · Question answering ### [gelectra-large-germanquad](https://savrn.com/models/gelectra-large-germanquad) [Deepset](https://savrn.com/model-publishers/deepset) We trained a German question answering model with a gelectra-large model as its basis. - The dataset is GermanQuAD, a new, German language dataset, which we hand-annotated and published online. - The training dataset is one-way annotated and contains 11518 questions and 11518 answers, while the test dataset is three-way annotated so that there are 2204 questions and with 2204·3−76 = 6536 answers, because we removed 76 wrong answers. See https://deepset.ai/germanquad for more details and dataset download in SQuAD format. Haystack is an AI orchestration framework to build customizable, production-ready LLM applications. You can use this model in Haystack to do extractive question answering on… Open weights mit 335M parameters 512 tokens transformers [View model](https://savrn.com/models/gelectra-large-germanquad) Model · Question answering ### [bert-large-cased-whole-word-masking-finetuned-squad](https://savrn.com/models/bert-large-cased-whole-word-masking-finetuned-squad) [BERT community](https://savrn.com/model-publishers/google-bert) Pretrained model on English language using a masked language modeling (MLM) objective. It was introduced in this paper and first released in this repository. This model is cased: it makes a difference between english and English. Differently to other BERT models, this model was trained with a new technique: Whole Word Masking. In this case, all of the tokens corresponding to a word are masked at once. The overall masking rate remains the same. The training is identical -- each masked WordPiece token is predicted independently. After pre-training, this model was fine-tuned on the SQuAD dataset with one of our fine-tuning scripts. See below for more information regarding this fine-tuning.… Open weights apache-2.0 334M parameters 512 tokens transformers [View model](https://savrn.com/models/bert-large-cased-whole-word-masking-finetuned-squad) Model · Question answering ### [mdeberta-v3-base-squad2](https://savrn.com/models/mdeberta-v3-base-squad2) [Tim](https://savrn.com/model-publishers/timpal0l) It has been finetuned for 3 epochs on SQuAD2.0. DeBERTa improves the BERT and RoBERTa models using disentangled attention and enhanced mask decoder. With those two improvements, DeBERTa out perform RoBERTa on a majority of NLU tasks with 80GB training data. In DeBERTa V3, we further improved the efficiency of DeBERTa using ELECTRA-Style pre-training with Gradient Disentangled Embedding Sharing. Compared to DeBERTa, our V3 version significantly improves the model performance on downstream tasks. You can find more technique details about the new model from our paper. Please check the official repository for more implementation details and updates. mDeBERTa is multilingual version of DeBERTa… Open weights mit 278M parameters 512 tokens transformers [View model](https://savrn.com/models/mdeberta-v3-base-squad2) Model · Question answering ### [xlm-roberta-base-squad2](https://savrn.com/models/xlm-roberta-base-squad2) [Deepset](https://savrn.com/model-publishers/deepset) Haystack is an AI orchestration framework to build customizable, production-ready LLM applications. You can use this model in Haystack to do extractive question answering on documents. To load and run the model with Haystack: For a complete example with an extractive question answering pipeline that scales over many documents, check out the corresponding Haystack tutorial. Evaluated on the SQuAD 2.0 dev set with the official eval script. "exact": 33.67279167589108 "total": 4517 "exact": 48.739495798319325 "total": 1190 Timo Möller: timo.moeller [at] deepset.ai deepset is the company behind the production-ready open-source AI framework Haystack. We also have a Discord community open to… Open weights cc-by-4.0 277M parameters 514 tokens transformers [View model](https://savrn.com/models/xlm-roberta-base-squad2) ## Deepset [All models and datasets](https://savrn.com/model-publishers/deepset) ## Versions - [78fb38a59ea3](https://savrn.com/models/roberta-large-squad2/versions/78fb38a59ea3) · current 2026-09-18 ## Explore More - [All question answering models](https://savrn.com/models/tasks/question-answering) - [All models under cc-by-4.0](https://savrn.com/models/licenses/cc-by-4-0) - [Model comparisons](https://savrn.com/models/comparisons) - [The model directory](https://savrn.com/models) - [Open model prices by host](https://savrn.com/ai-index/pricing/open-models) ## Source - Repository metadata, read 2026-10-21. - [Hugging Face record](https://huggingface.co/deepset/roberta-large-squad2) - [How the hub is built](https://savrn.com/model-hub/methodology)