Stanford Question Answering Dataset (SQuAD) is a reading comprehension dataset, consisting of questions posed by crowdworkers on a set of Wikipedia articles, where the answer to every question is a segment of text, or span, from the corresponding reading…
Dataset Card
By Pranav R, published under cc-by-sa-4.0, revision 7b6d24c440a3.
Dataset Card for SQuAD
Table of Contents
- Dataset Card for "squad"
- Table of Contents
- Dataset Description
- Dataset Summary
- Supported Tasks and Leaderboards
- Languages
- Dataset Structure
- Data Instances
- plain_text
- Data Fields
- plain_text
- Data Splits
- Dataset Creation
- Curation Rationale
- Source Data
- Initial Data Collection and Normalization
- Who are the source language producers?
- Annotations
- Annotation process
- Who are the annotators?
- Personal and Sensitive Information
- Considerations for Using the Data
- Social Impact of Dataset
- Discussion of Biases
- Other Known Limitations
- Additional Information
- Dataset Curators
- Licensing Information
- Citation Information
- Contributions
Dataset Description
- Homepage: https://rajpurkar.github.io/SQuAD-explorer/
- Repository: More Information Needed
- Paper: https://arxiv.org/abs/1606.05250
- Point of Contact: More Information Needed
Dataset Summary
Stanford Question Answering Dataset (SQuAD) is a reading comprehension dataset, consisting of questions posed by crowdworkers on a set of Wikipedia articles, where the answer to every question is a segment of text, or span, from the corresponding reading passage, or the question might be unanswerable.
SQuAD 1.1 contains 100,000+ question-answer pairs on 500+ articles.
Structure
plain_text 98,169 rows
| Split | Rows | Size |
|---|---|---|
| train | 87,599 | 25.0 MB |
| validation | 10,570 | 3.2 MB |
Details
- Repository
- rajpurkar/squad
- Publisher
- Pranav R
- Task category
- Question answering
- Tags
- Not stated by the source
- Size category
- 10K<n<100K
- Languages
- en
- Revision
- 7b6d24c440a36b6815f21b70d25016731768db1f
- Last updated
- 2024-03-04
SAVRN's Notes on squad
SQuAD is 98,169 crowd-written question-answer pairs posed over Wikipedia articles, 16.3 MB under CC BY-SA 4.0, published here by Pranav Rajpurkar and last touched in March 2024. It remains the archetypal extractive-QA fine-tuning target: every answer is a span of the source passage, so the task reduces to boundary selection rather than generation. The inbound list on our page makes the usage pattern explicit — bert-base-cased-squad-extractive-qa, bert-base-squad-qa and similar BERT-era extractive models all disclose training on it.
CC BY-SA 4.0 means attribution plus share-alike on adapted material — workable for research, but clear it before a commercial deployment builds on a SQuAD fine-tune. Two practical warnings. Every answer is a span of a Wikipedia passage, so any model pretrained heavily on Wikipedia has seen the context; and the benchmark has been saturated since the BERT era, so expect ceiling-level scores that no longer discriminate between systems. Use it to validate an extractive-QA pipeline, not to prove capability.
Files
4 files, 16.3 MB in total.
Every file
| File | Type | Size | SHA-256 |
|---|---|---|---|
| plain_text/train-00000-of-00001.parquet | Data | 14.5 MB | ea7f52bac024 |
| plain_text/validation-00000-of-00001.parquet | Data | 1.8 MB | 8c6646d36bd5 |
| README.md | Documentation | 7.6 KB | — |
| .gitattributes | Repository | 1.2 KB | — |
License and Download
- License
- cc-by-sa-4.0
- Access
- No access gate
Released by Pranav R through its official repository on Hugging Face. Read the license.
Models Trained on This Dataset
- Trained on (disclosed)bert-base-cased-squad-extractive-qa
- Trained on (disclosed)Extractive-QA_BERT_body
- Trained on (disclosed)bert-base-squad-qa