distilbert-base-cased-distilled-squad

Model · Question answering

distilbert-base-uncased-distilled-squad

DistilBERT community

The DistilBERT model was proposed in the blog post Smaller, faster, cheaper, lighter: Introducing DistilBERT, adistilled version of BERT, and the paper DistilBERT, adistilled version of BERT: smaller, faster, cheaper and lighter. DistilBERT is a small, fast, cheap and light Transformer model trained by distilling BERT base. It has 40% less parameters than bert-base-uncased, runs 60% faster while preserving over 95% of BERT's performances as measured on the GLUE language understanding benchmark. This model is a fine-tune checkpoint of DistilBERT-base-uncased, fine-tuned using (a second step of) knowledge distillation on SQuAD v1.1. - See this repository for more about Distil\ (a class of…

Model · Question answering

tinyroberta-squad2

Deepset

This is the distilled version of the deepset/roberta-base-squad2 model. This model has a comparable prediction quality and runs at twice the speed of the base model. This model was distilled using the TinyBERT approach described in this paper and implemented in haystack. Firstly, we have performed intermediate layer distillation with roberta-base as the teacher which resulted in deepset/tinyroberta-6l-768d. Secondly, we have performed task-specific distillation with deepset/roberta-base-squad2 as the teacher for further intermediate layer distillation on an augmented version of SQuADv2 and then with deepset/roberta-large-squad2 as the teacher for prediction layer distillation. Haystack is…

Model · Question answering

bert-medium-squad2-distilled

Deepset

Haystack version 1.x distillation feature was used for training. deepset/bert-large-uncased-whole-word-masking-squad2 was used as the teacher model. Haystack is an AI orchestration framework to build customizable, production-ready LLM applications. You can use this model in Haystack to do extractive question answering on documents. To load and run the model with Haystack: For a complete example with an extractive question answering pipeline that scales over many documents, check out the corresponding Haystack tutorial. - Timo Möller: timo.moeller [at] deepset.ai deepset is the company behind the production-ready open-source AI framework Haystack. We also have a Discord community open to…

Model · Question answering

minilm-uncased-squad2

Deepset

Haystack is an AI orchestration framework to build customizable, production-ready LLM applications. You can use this model in Haystack to do extractive question answering on documents. To load and run the model with Haystack: For a complete example with an extractive question answering pipeline that scales over many documents, check out the corresponding Haystack tutorial. Evaluated on the SQuAD 2.0 dev set with the official eval script. Timo Möller: [email protected] deepset is the company behind the production-ready open-source AI framework Haystack. We also have a Discord community open to everyone!

Model · Question answering

mobilebert-uncased-squad-v2

Qingqing Cao

MobileBERT is a thin version of BERTLARGE, while equipped with bottleneck structures and a carefully designed balance between self-attentions and feed-forward networks. This model was fine-tuned from the HuggingFace checkpoint google/mobilebert-uncased on SQuAD2.0. CPU: Intel(R) Core(TM) i7-6800K CPU @ 3.40GHz Memory: 32 GiB GPUs: 2 GeForce GTX 1070, each with 8GiB memory GPU driver: 418.87.01, CUDA: 10.1 It took about 3.5 hours to finish. Note that the above results didn't involve any hyperparameter search.

Model · Question answering

mobilebert-uncased-squad-v1

Qingqing Cao

MobileBERT is a thin version of BERTLARGE, while equipped with bottleneck structures and a carefully designed balance between self-attentions and feed-forward networks. This model was fine-tuned from the HuggingFace checkpoint google/mobilebert-uncased on SQuAD1.1. CPU: Intel(R) Core(TM) i7-6800K CPU @ 3.40GHz Memory: 32 GiB GPUs: 2 GeForce GTX 1070, each with 8GiB memory GPU driver: 418.87.01, CUDA: 10.1 It took about 3 hours to finish. Note that the above results didn't involve any hyperparameter search.