MMLU-Pro dataset is a more robust and challenging massive multi-task understanding dataset tailored to more rigorously benchmark large language models' capabilities. This dataset contains 12K complex questions across various disciplines.
Rows12,102
Configurations1
Size4.2 MB
Licensemit
AccessPublicly accessible
Monthly Downloads246k
Dataset Card
By TIGER-Lab, published under mit, revision b189ec765aa7.
MMLU-Pro Dataset
MMLU-Pro dataset is a more robust and challenging massive multi-task understanding dataset tailored to more rigorously benchmark large language models' capabilities. This dataset contains 12K complex questions across various disciplines.
|Github | Leaderboard | Paper |
What's New
Structure
default 12,102 rows
| Split | Rows | Size |
|---|---|---|
| test | 12,032 | 7.9 MB |
| validation | 70 | 61.9 KB |
question_idint64questionstringoptionslistanswerstringanswer_indexint64cot_contentstringcategorystringsrcstring
Details
- Repository
- TIGER-Lab/MMLU-Pro
- Publisher
- TIGER-Lab
- Task category
- Question answering
- Tags
- evaluation
- Size category
- 10K<n<100K
- Languages
- en
- Revision
- b189ec765aa7ed75c8acfea42df31fdae71f97be
- Last updated
- 2026-05-02
Files
7 files, 4.2 MB in total.
Data2 files · 4.2 MB
Documentation1 file · 11.7 KB
Other3 files · 6.3 KB
Repository1 file · 2.3 KB
Every file
| File | Type | Size | SHA-256 |
|---|---|---|---|
| data/test-00000-of-00001.parquet | Data | 4.1 MB | 0e24a191921c |
| data/validation-00000-of-00001.parquet | Data | 42.9 KB | 139423c23722 |
| README.md | Documentation | 11.7 KB | — |
| eval.yaml | Other | 590 B | — |
| run_claude3.py | Other | 2.7 KB | — |
| run_gpt4o.py | Other | 3.0 KB | — |
| .gitattributes | Repository | 2.3 KB | — |
License and Download
- License
- mit
- Access
- No access gate
Download from TIGER-Lab
Released by TIGER-Lab through its official repository on Hugging Face. Read the license.