CommonsenseQA is a new multiple-choice question answering dataset that requires different types of commonsense knowledge to predict the correct answers. It contains 12,102 questions with one correct answer and four distractor answers.
Dataset Card
By Tel Aviv University, published under mit, revision 94630fe30dad.
Dataset Card for "commonsense_qa"
Table of Contents
- Table of Contents
- Dataset Description
- Dataset Summary
- Supported Tasks and Leaderboards
- Languages
- Dataset Structure
- Data Instances
- Data Fields
- Data Splits
- Dataset Creation
- Curation Rationale
- Source Data
- Annotations
- Personal and Sensitive Information
- Considerations for Using the Data
- Social Impact of Dataset
- Discussion of Biases
- Other Known Limitations
- Additional Information
- Dataset Curators
- Licensing Information
- Citation Information
- Contributions
Dataset Description
- Homepage: https://www.tau-nlp.org/commonsenseqa
- Repository: https://github.com/jonathanherzig/commonsenseqa
- Paper: https://arxiv.org/abs/1811.00937
- Point of Contact: More Information Needed
- Size of downloaded dataset files: 4.68 MB
- Size of the generated dataset: 2.18 MB
- Total amount of disk used: 6.86 MB
Dataset Summary
CommonsenseQA is a new multiple-choice question answering dataset that requires different types of commonsense knowledge to predict the correct answers . It contains 12,102 questions with one correct answer and four distractor answers. The dataset is provided in two major training/validation/testing set splits: "Random split" which is the main evaluation split, and "Question token split", see paper for details.
Structure
default 12,102 rows
| Split | Rows | Size |
|---|---|---|
| train | 9,741 | 2.2 MB |
| validation | 1,221 | 274.3 KB |
| test | 1,140 | 257.4 KB |
Details
- Repository
- tau/commonsense_qa
- Publisher
- Tel Aviv University
- Task category
- Question answering
- Tags
- Not stated by the source
- Size category
- 1K<n<10K
- Languages
- en
- Revision
- 94630fe30dad47192a8546eb75f094926d47e155
- Last updated
- 2024-01-04
SAVRN's Notes on commonsense_qa
CommonsenseQA from Tel Aviv University is a 12,102-question, five-way multiple-choice benchmark testing commonsense reasoning: one correct answer, four distractors, 1.6 MB in five files under the MIT license as recorded on the hub, downloaded 308.4k times a month and unchanged since January 2024. The kev-8b model on our hub discloses training on it, which is typical for a set this size — cheap, canonical, and easy to fold into instruction-tuning mixes.
That ubiquity is the risk: benchmark-overfit pressure is high on a 12,102-question set, so keep the official validation split untouched and report honestly. MIT makes this one of the cleanest small QA sets for commercial fine-tuning — no attribution choreography, no NC clause. Who should use it: anyone fine-tuning small models for multiple-choice commonsense, or building eval suites; who should not: teams needing factuality coverage beyond everyday reasoning.
Files
5 files, 1.6 MB in total.
Every file
| File | Type | Size | SHA-256 |
|---|---|---|---|
| data/test-00000-of-00001.parquet | Data | 151.2 KB | 19efe93223d1 |
| data/train-00000-of-00001.parquet | Data | 1.2 MB | b0449767ed98 |
| data/validation-00000-of-00001.parquet | Data | 160.2 KB | bdbd9bf9cc4d |
| README.md | Documentation | 7.4 KB | — |
| .gitattributes | Repository | 1.2 KB | — |
License and Download
- License
- mit
- Access
- No access gate
Released by Tel Aviv University through its official repository on Hugging Face. Read the license.
Models Trained on This Dataset
- Trained on (disclosed)shingi-27b
- Trained on (disclosed)hopper
- Trained on (disclosed)kev-8b