# MMLU-Pro by TIGER-Lab: AI Training Dataset
Source: https://savrn.com/datasets/mmlu-pro
Markdown alternate of the page above; the site index is https://savrn.com/llms.txt

---

## Dataset Card

By TIGER-Lab, published under mit, revision b189ec765aa7.

### MMLU-Pro Dataset

MMLU-Pro dataset is a more robust and challenging massive multi-task understanding dataset tailored to more rigorously benchmark large language models' capabilities. This dataset contains 12K complex questions across various disciplines.

|[Github](https://github.com/TIGER-AI-Lab/MMLU-Pro) | [Leaderboard](https://huggingface.co/spaces/TIGER-Lab/MMLU-Pro) | [Paper](https://arxiv.org/abs/2406.01574) |

### What's New

[Read the full dataset card (1,124 words)](https://savrn.com/datasets/mmlu-pro/card)

## Structure

### default 12,102 rows

| Split | Rows | Size |
| --- | --- | --- |
| test | 12,032 | 7.9 MB |
| validation | 70 | 61.9 KB |

question_idint64questionstringoptionslistanswerstringanswer_indexint64cot_contentstringcategorystringsrcstring

## Details

Repository

TIGER-Lab/MMLU-Pro

Publisher

TIGER-Lab

Task category

Question answering

Tags

evaluation

Size category

10K<n<100K

Languages

en

Revision

b189ec765aa7ed75c8acfea42df31fdae71f97be

Last updated

2026-05-02

## SAVRN's Notes on MMLU-Pro

MMLU-Pro is 12,102 questions in 4.2 MB under MIT terms, but the split layout tells you what it is: 12,032 rows of test and only 70 rows of validation. This is an evaluation set, published by TIGER-Lab as a harder, more option-dense successor to MMLU, and the near-empty validation split leaves almost no room for prompt development or calibration without touching the test data itself.

MIT licensing removes friction for commercial eval pipelines, and at 4.2 MB it costs nothing to mirror internally. The discipline that matters is contamination: this is a headline benchmark with a fixed public test split, so any model that has seen it in training will overstate capability, and reviewers increasingly check for exactly that. The page was last updated May 2026 — verify the revision when comparing scores against published results.

SAVRN Research

## Files

7 files, 4.2 MB in total.

Data2 files · 4.2 MB

Documentation1 file · 11.7 KB

Other3 files · 6.3 KB

Repository1 file · 2.3 KB

Every file

| File | Type | Size | SHA-256 |
| --- | --- | --- | --- |
| data/test-00000-of-00001.parquet | Data | 4.1 MB | 0e24a191921c |
| data/validation-00000-of-00001.parquet | Data | 42.9 KB | 139423c23722 |
| README.md | Documentation | 11.7 KB | — |
| eval.yaml | Other | 590 B | — |
| run_claude3.py | Other | 2.7 KB | — |
| run_gpt4o.py | Other | 3.0 KB | — |
| .gitattributes | Repository | 2.3 KB | — |

## License and Download

License

mit

Access

No access gate

[Download from TIGER-Lab](https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro)

Released by TIGER-Lab through its official repository on Hugging Face. [Read the license](https://opensource.org/license/mit).

## Models Trained on This Dataset

- Trained on (disclosed)[ProKope-7B](https://savrn.com/models/prokope-7b)

## TIGER-Lab

[All models and datasets](https://savrn.com/model-publishers/tiger-lab)

## Explore More

- [The dataset directory](https://savrn.com/datasets)
- [The model directory](https://savrn.com/models)
- [Research papers](https://savrn.com/papers)

## Source

- Repository metadata, read 2026-10-19.
- [Hugging Face record](https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro)
- [How the hub is built](https://savrn.com/model-hub/methodology)