OpenBookQA aims to promote research in advanced question-answering, probing a deeper understanding of both the topic (with salient facts summarized as an open book, also provided with the dataset) and the language it is expressed in.
Dataset Card
OpenBookQA aims to promote research in advanced question-answering, probing a deeper understanding of both the topic (with salient facts summarized as an open book, also provided with the dataset) and the language it is expressed in. In particular, it contains questions that require multi-step reasoning, use of additional common and commonsense knowledge, and rich text comprehension. OpenBookQA is a new kind of question-answering dataset modeled after open book exams for assessing human understanding of a subject. An example of 'train' looks as follows: An example of 'train' looks as follows: The data fields are the same among all splits. - id: a string feature. - questionstem: a string…
Excerpt from the card by Ai2, licensed unknown.
Structure
additional 5,957 rows
| Split | Rows | Size |
|---|---|---|
| train | 4,957 | 1.3 MB |
| validation | 500 | 136.0 KB |
| test | 500 | 130.8 KB |
main 5,957 rows
| Split | Rows | Size |
|---|---|---|
| train | 4,957 | 937.0 KB |
| validation | 500 | 95.4 KB |
| test | 500 | 91.8 KB |
Details
- Repository
- allenai/openbookqa
- Publisher
- Ai2
- Task category
- Question answering
- Tags
- Not stated by the source
- Size category
- 1K<n<10K
- Languages
- en
- Revision
- 388097ea7776314e93a529163e0fea805b8a6454
- Last updated
- 2024-01-04
SAVRN's Notes on openbookqa
OpenBookQA: 11,914 four-way multiple-choice science questions in 1.4 MB, two configs listed, main and additional, each at 5,957 rows with 4,957 train, 500 validation and 500 test rows, the additional config adding fact1, humanScore and clarity annotation fields. 487.6k monthly downloads and one inbound model on our page, kev-8b, published by Ai2 as a question-answering benchmark set, last updated 2024-01-04.
The license column reads unknown, which is the story: the page states no terms, so a commercial training run holds no documented rights until you verify the upstream license yourself. As an eval it is small and aging; use it as a secondary commonsense check next to ARC, and do not expect modern models to blink. For anything that ships, solve the license question first.
Files
8 files, 1.4 MB in total.
Every file
| File | Type | Size | SHA-256 |
|---|---|---|---|
| additional/test-00000-of-00001.parquet | Data | 72.5 KB | 33b318ea8e23 |
| additional/train-00000-of-00001.parquet | Data | 635.4 KB | d16d719e87ef |
| additional/validation-00000-of-00001.parquet | Data | 75.9 KB | 92e5e68e4da7 |
| main/test-00000-of-00001.parquet | Data | 55.5 KB | cd5483e366da |
| main/train-00000-of-00001.parquet | Data | 495.8 KB | 98148f8a54e6 |
| main/validation-00000-of-00001.parquet | Data | 58.2 KB | 35370b9cfee8 |
| README.md | Documentation | 9.1 KB | — |
| .gitattributes | Repository | 1.2 KB | — |
License and Download
- License
- unknown
- Access
- No access gate
Released by Ai2 through its official repository on Hugging Face.
Models Trained on This Dataset
- Trained on (disclosed)kev-8b