TruthfulQA is a benchmark to measure whether a language model is truthful in generating answers to questions. The benchmark comprises 817 questions that span 38 categories, including health, law, finance and politics.
Rows1,634
Configurations2
Size504.8 KB
Licenseapache-2.0
AccessPublicly accessible
Monthly Downloads204.1k
Dataset Card
By TruthfulQA, published under apache-2.0, revision 741b8276f2d1.
Dataset Card for truthful_qa
Table of Contents
- Dataset Card for truthful_qa
- Table of Contents
- Dataset Description
- Dataset Summary
- Supported Tasks and Leaderboards
- Languages
- Dataset Structure
- Data Instances
- generation
- multiple_choice
- Data Fields
- generation
- multiple_choice
- Data Splits
- Dataset Creation
- Curation Rationale
- Source Data
- Initial Data Collection and Normalization
- Who are the source language producers?
- Annotations
- Annotation process
- Who are the annotators?
- Personal and Sensitive Information
- Considerations for Using the Data
- Social Impact of Dataset
- Discussion of Biases
- Other Known Limitations
- Additional Information
- Dataset Curators
- Licensing Information
- Citation Information
- Contributions
Dataset Description
- Homepage: [Needs More Information]
- Repository: https://github.com/sylinrl/TruthfulQA
- Paper: https://arxiv.org/abs/2109.07958
- Leaderboard: [Needs More Information]
- Point of Contact: [Needs More Information]
Dataset Summary
Structure
generation 817 rows
| Split | Rows | Size |
|---|---|---|
| validation | 817 | 473.4 KB |
typestringcategorystringquestionstringbest_answerstringcorrect_answersSequenceincorrect_answersSequencesourcestring
multiple_choice 817 rows
| Split | Rows | Size |
|---|---|---|
| validation | 817 | 610.3 KB |
questionstringmc1_targetsvaluemc2_targetsvalue
Details
- Repository
- truthfulqa/truthful_qa
- Publisher
- TruthfulQA
- Task category
- Multiple choice
- Tags
- Not stated by the source
- Size category
- n<1K
- Languages
- en
- Revision
- 741b8276f2d1982aa3d5b832d3ee81ed3b896490
- Last updated
- 2024-01-04
Files
4 files, 504.8 KB in total.
Data2 files · 493.7 KB
Documentation1 file · 9.6 KB
Repository1 file · 1.6 KB
Every file
| File | Type | Size | SHA-256 |
|---|---|---|---|
| generation/validation-00000-of-00001.parquet | Data | 222.6 KB | dfb1004b8ab8 |
| multiple_choice/validation-00000-of-00001.parquet | Data | 271.0 KB | 23f08e230ca4 |
| README.md | Documentation | 9.6 KB | — |
| .gitattributes | Repository | 1.6 KB | — |
License and Download
- License
- apache-2.0
- Access
- No access gate
Download from TruthfulQA
Released by TruthfulQA through its official repository on Hugging Face. Read the license.