# mmlu by Center for AI Safety: AI Training Dataset
Source: https://savrn.com/datasets/mmlu
Markdown alternate of the page above; the site index is https://savrn.com/llms.txt

---

## Dataset Card

By Center for AI Safety, published under mit, revision c30699e8356d.

### Dataset Card for MMLU

### Table of Contents

- Table of Contents
- Dataset Description
- Dataset Summary
- Supported Tasks and Leaderboards
- Languages
- Dataset Structure
- Data Instances
- Data Fields
- Data Splits
- Dataset Creation
- Curation Rationale
- Source Data
- Annotations
- Personal and Sensitive Information
- Considerations for Using the Data
- Social Impact of Dataset
- Discussion of Biases
- Other Known Limitations
- Additional Information
- Dataset Curators
- Licensing Information
- Citation Information
- Contributions

### Dataset Description

- Repository: https://github.com/hendrycks/test
- Paper: https://arxiv.org/abs/2009.03300

#### Dataset Summary

[Measuring Massive Multitask Language Understanding](https://savrn.com/papers/measuring-massive-multitask-language-understanding) by [Dan Hendrycks](https://people.eecs.berkeley.edu/~hendrycks/), [Collin Burns](http://collinpburns.com), [Steven Basart](https://stevenbas.art), Andy Zou, Mantas Mazeika, [Dawn Song](https://people.eecs.berkeley.edu/~dawnsong/), and [Jacob Steinhardt](https://www.stat.berkeley.edu/~jsteinhardt/) (ICLR 2021).

[Read the full dataset card (654 words)](https://savrn.com/datasets/mmlu/card)

## Structure

### abstract_algebra 116 rows

| Split | Rows | Size |
| --- | --- | --- |
| test | 100 | 21.3 KB |
| validation | 11 | 2.2 KB |
| dev | 5 | 919 B |

questionstringsubjectstringchoicesSequenceanswerClassLabel

### all 115,700 rows

| Split | Rows | Size |
| --- | --- | --- |
| test | 14,042 | 4.1 MB |
| validation | 1,531 | 613.5 KB |
| dev | 285 | 125.4 KB |
| auxiliary_train | 99,842 | 123.4 MB |

questionstringsubjectstringchoicesSequenceanswerClassLabel

### anatomy 154 rows

| Split | Rows | Size |
| --- | --- | --- |
| test | 135 | 34.6 KB |
| validation | 14 | 3.3 KB |
| dev | 5 | 1.0 KB |

questionstringsubjectstringchoicesSequenceanswerClassLabel

### astronomy 173 rows

| Split | Rows | Size |
| --- | --- | --- |
| test | 152 | 48.8 KB |
| validation | 16 | 5.2 KB |
| dev | 5 | 2.1 KB |

questionstringsubjectstringchoicesSequenceanswerClassLabel

### auxiliary_train 99,842 rows

| Split | Rows | Size |
| --- | --- | --- |
| train | 99,842 | 123.4 MB |

trainvalue

### business_ethics 116 rows

| Split | Rows | Size |
| --- | --- | --- |
| test | 100 | 35.2 KB |
| validation | 11 | 3.2 KB |
| dev | 5 | 2.3 KB |

questionstringsubjectstringchoicesSequenceanswerClassLabel

### clinical_knowledge 299 rows

| Split | Rows | Size |
| --- | --- | --- |
| test | 265 | 68.6 KB |
| validation | 29 | 7.3 KB |
| dev | 5 | 1.3 KB |

questionstringsubjectstringchoicesSequenceanswerClassLabel

### college_biology 165 rows

| Split | Rows | Size |
| --- | --- | --- |
| test | 144 | 51.5 KB |
| validation | 16 | 5.1 KB |
| dev | 5 | 1.6 KB |

questionstringsubjectstringchoicesSequenceanswerClassLabel

### college_chemistry 113 rows

| Split | Rows | Size |
| --- | --- | --- |
| test | 100 | 26.8 KB |
| validation | 8 | 2.5 KB |
| dev | 5 | 1.4 KB |

questionstringsubjectstringchoicesSequenceanswerClassLabel

### college_computer_science 116 rows

| Split | Rows | Size |
| --- | --- | --- |
| test | 100 | 45.4 KB |
| validation | 11 | 5.0 KB |
| dev | 5 | 2.9 KB |

questionstringsubjectstringchoicesSequenceanswerClassLabel

### college_mathematics 116 rows

| Split | Rows | Size |
| --- | --- | --- |
| test | 100 | 27.0 KB |
| validation | 11 | 2.9 KB |
| dev | 5 | 1.6 KB |

questionstringsubjectstringchoicesSequenceanswerClassLabel

### college_medicine 200 rows

| Split | Rows | Size |
| --- | --- | --- |
| test | 173 | 85.9 KB |
| validation | 22 | 8.3 KB |
| dev | 5 | 1.8 KB |

questionstringsubjectstringchoicesSequenceanswerClassLabel

## Details

Repository

cais/mmlu

Publisher

Center for AI Safety

Task category

Question answering

Tags

Not stated by the source

Size category

10K<n<100K

Languages

en

Revision

c30699e8356da336a370243923dbaf21066bb9fe

Last updated

2024-03-08

## SAVRN's Notes on mmlu

MMLU lists 217,110 rows across 12 shown configs, from abstract_algebra to auxiliary_train, in 270 MB and 181 files, MIT-licensed, 779.7k monthly downloads, published by the Center for AI Safety as a question-answering benchmark. The config that matters is all, 115,700 rows with a 14,042-row test split, and the only trainable slice is auxiliary_train at 99,842 rows, itself assembled from other exams' practice material.

Everyone uses it; nobody should train on the test set. Our page shows no models trained on MMLU, which matches its role as an eval. The 14k-question test set has been mined by model makers for years, so assume your base model has seen it and report it as a sanity check next to fresher benchmarks. Commercially the MIT license is fine; the risk is credibility, not legality.

SAVRN Research

## Files

181 files, 270.0 MB in total.

Data178 files · 270.0 MB

Documentation1 file · 53.2 KB

Other1 file · 5.7 KB

Repository1 file · 2.3 KB

Every file

| File | Type | Size | SHA-256 |
| --- | --- | --- | --- |
| abstract_algebra/dev-00000-of-00001.parquet | Data | 3.5 KB | e778f6090366 |
| abstract_algebra/test-00000-of-00001.parquet | Data | 10.0 KB | 92338cdcdb43 |
| abstract_algebra/validation-00000-of-00001.parquet | Data | 3.7 KB | b66df8764498 |
| all/auxiliary_train-00000-of-00001.parquet | Data | 47.5 MB | 5126bb35f06c |
| all/dev-00000-of-00001.parquet | Data | 76.5 KB | 2b19bde1ed8c |
| all/test-00000-of-00001.parquet | Data | 3.5 MB | 74a41822ce7d |
| all/validation-00000-of-00001.parquet | Data | 408.4 KB | 66cdf0b090cc |
| anatomy/dev-00000-of-00001.parquet | Data | 3.5 KB | 1774b4af192a |
| anatomy/test-00000-of-00001.parquet | Data | 20.1 KB | cb88f5920b67 |
| anatomy/validation-00000-of-00001.parquet | Data | 5.3 KB | f99c5b6b26a8 |
| astronomy/dev-00000-of-00001.parquet | Data | 4.9 KB | 73b897f92f1f |
| astronomy/test-00000-of-00001.parquet | Data | 28.3 KB | bcdfe1afcb94 |
| astronomy/validation-00000-of-00001.parquet | Data | 6.1 KB | 98b9cdb9327b |
| auxiliary_train/train-00000-of-00001.parquet | Data | 47.5 MB | ae6662576ef9 |
| business_ethics/dev-00000-of-00001.parquet | Data | 5.0 KB | 494ea1d948ca |
| business_ethics/test-00000-of-00001.parquet | Data | 21.6 KB | 7910a4b8236c |
| business_ethics/validation-00000-of-00001.parquet | Data | 5.1 KB | 3a899aef15e3 |
| clinical_knowledge/dev-00000-of-00001.parquet | Data | 3.7 KB | 105b492234b3 |
| clinical_knowledge/test-00000-of-00001.parquet | Data | 40.5 KB | 7bd3de302db3 |
| clinical_knowledge/validation-00000-of-00001.parquet | Data | 7.5 KB | 2337cba28303 |
| college_biology/dev-00000-of-00001.parquet | Data | 4.3 KB | 0cf3cbb4fde7 |
| college_biology/test-00000-of-00001.parquet | Data | 31.8 KB | 5b6920ddbbb4 |
| college_biology/validation-00000-of-00001.parquet | Data | 6.9 KB | f5010d257812 |
| college_chemistry/dev-00000-of-00001.parquet | Data | 4.0 KB | 4c689a238a27 |
| college_chemistry/test-00000-of-00001.parquet | Data | 17.9 KB | f012f90fc3ae |
| college_chemistry/validation-00000-of-00001.parquet | Data | 4.9 KB | e394d9086108 |
| college_computer_science/dev-00000-of-00001.parquet | Data | 6.8 KB | 6b35dffa1635 |
| college_computer_science/test-00000-of-00001.parquet | Data | 28.1 KB | 08b79916057e |
| college_computer_science/validation-00000-of-00001.parquet | Data | 6.3 KB | 967090233faa |
| college_mathematics/dev-00000-of-00001.parquet | Data | 5.2 KB | 46ee5c84b379 |
| college_mathematics/test-00000-of-00001.parquet | Data | 16.6 KB | f57b57e992d0 |
| college_mathematics/validation-00000-of-00001.parquet | Data | 5.0 KB | e668d0830970 |
| college_medicine/dev-00000-of-00001.parquet | Data | 4.8 KB | 85bf7f27b795 |
| college_medicine/test-00000-of-00001.parquet | Data | 42.5 KB | f21b3166975d |
| college_medicine/validation-00000-of-00001.parquet | Data | 9.0 KB | 4b5a104770d7 |
| college_physics/dev-00000-of-00001.parquet | Data | 4.5 KB | eec8b9851f44 |
| college_physics/test-00000-of-00001.parquet | Data | 18.6 KB | 81b8eafc776b |
| college_physics/validation-00000-of-00001.parquet | Data | 6.4 KB | 79f20c0d67a0 |
| computer_security/dev-00000-of-00001.parquet | Data | 4.3 KB | b491f6df6ed1 |
| computer_security/test-00000-of-00001.parquet | Data | 19.1 KB | a6ad5774acf5 |
| computer_security/validation-00000-of-00001.parquet | Data | 6.7 KB | 1eaed4c8c709 |
| conceptual_physics/dev-00000-of-00001.parquet | Data | 4.0 KB | 1503db271848 |
| conceptual_physics/test-00000-of-00001.parquet | Data | 25.0 KB | c66a861048a2 |
| conceptual_physics/validation-00000-of-00001.parquet | Data | 6.0 KB | c6df7c8fd416 |
| data.tar | Data | 166.2 MB | bec563ba4bac |
| dataset_infos.json | Data | 138.3 KB | — |
| econometrics/dev-00000-of-00001.parquet | Data | 4.5 KB | 5edd31ddb8ce |
| econometrics/test-00000-of-00001.parquet | Data | 24.5 KB | 63501e9de9f8 |
| econometrics/validation-00000-of-00001.parquet | Data | 7.0 KB | 52aceec5650c |
| electrical_engineering/dev-00000-of-00001.parquet | Data | 4.1 KB | 92416acbafa7 |
| electrical_engineering/test-00000-of-00001.parquet | Data | 17.6 KB | 3df176cabbd1 |
| electrical_engineering/validation-00000-of-00001.parquet | Data | 5.1 KB | f03b430fd953 |
| elementary_mathematics/dev-00000-of-00001.parquet | Data | 4.5 KB | 03b0ff28b651 |
| elementary_mathematics/test-00000-of-00001.parquet | Data | 41.1 KB | 9cb645300084 |
| elementary_mathematics/validation-00000-of-00001.parquet | Data | 9.4 KB | a33f741dae37 |
| formal_logic/dev-00000-of-00001.parquet | Data | 4.8 KB | 6928410ade5a |
| formal_logic/test-00000-of-00001.parquet | Data | 21.5 KB | a9f917e01461 |
| formal_logic/validation-00000-of-00001.parquet | Data | 6.6 KB | b5ae1527ce33 |
| global_facts/dev-00000-of-00001.parquet | Data | 3.6 KB | ee23f3bd9dcc |
| global_facts/test-00000-of-00001.parquet | Data | 11.5 KB | 9e4b5a22ad1c |
| global_facts/validation-00000-of-00001.parquet | Data | 4.2 KB | 48b84607d614 |
| high_school_biology/dev-00000-of-00001.parquet | Data | 4.9 KB | 4ae1c2d737b6 |
| high_school_biology/test-00000-of-00001.parquet | Data | 62.7 KB | 73600cc5c803 |
| high_school_biology/validation-00000-of-00001.parquet | Data | 10.6 KB | 5c7a4d5bbb0a |
| high_school_chemistry/dev-00000-of-00001.parquet | Data | 4.2 KB | 9ff0a3200962 |
| high_school_chemistry/test-00000-of-00001.parquet | Data | 33.3 KB | fde2345147be |
| high_school_chemistry/validation-00000-of-00001.parquet | Data | 8.3 KB | 0caed5473675 |
| high_school_computer_science/dev-00000-of-00001.parquet | Data | 6.5 KB | 431ad5ab2038 |
| high_school_computer_science/test-00000-of-00001.parquet | Data | 27.3 KB | 6e1b210896af |
| high_school_computer_science/validation-00000-of-00001.parquet | Data | 5.3 KB | 1e41b804c5d8 |
| high_school_european_history/dev-00000-of-00001.parquet | Data | 22.2 KB | 61cc946b0082 |
| high_school_european_history/test-00000-of-00001.parquet | Data | 142.5 KB | 2bcf46d76656 |
| high_school_european_history/validation-00000-of-00001.parquet | Data | 31.6 KB | 38bd62d9100b |
| high_school_geography/dev-00000-of-00001.parquet | Data | 3.9 KB | d3a732b15dc6 |
| high_school_geography/test-00000-of-00001.parquet | Data | 28.2 KB | d083f0a37e98 |
| high_school_geography/validation-00000-of-00001.parquet | Data | 6.2 KB | d1c34639bf3b |
| high_school_government_and_politics/dev-00000-of-00001.parquet | Data | 4.5 KB | 3877a26ca2e8 |
| high_school_government_and_politics/test-00000-of-00001.parquet | Data | 40.2 KB | 54bb3fcc3987 |
| high_school_government_and_politics/validation-00000-of-00001.parquet | Data | 8.3 KB | 28f197e5a0ab |
| high_school_macroeconomics/dev-00000-of-00001.parquet | Data | 4.0 KB | 6aa2bf96a9cf |
| high_school_macroeconomics/test-00000-of-00001.parquet | Data | 54.8 KB | a0f2d6a33352 |
| high_school_macroeconomics/validation-00000-of-00001.parquet | Data | 9.9 KB | 3219f345d00b |
| high_school_mathematics/dev-00000-of-00001.parquet | Data | 4.5 KB | 73034b3a7694 |
| high_school_mathematics/test-00000-of-00001.parquet | Data | 33.7 KB | 310628511fac |
| high_school_mathematics/validation-00000-of-00001.parquet | Data | 7.0 KB | 841cc40a133f |
| high_school_microeconomics/dev-00000-of-00001.parquet | Data | 3.8 KB | a8ae561bb6bf |
| high_school_microeconomics/test-00000-of-00001.parquet | Data | 38.8 KB | 8a55dc2fa10f |
| high_school_microeconomics/validation-00000-of-00001.parquet | Data | 7.2 KB | 5dce9d16182e |
| high_school_physics/dev-00000-of-00001.parquet | Data | 4.6 KB | 3b6fb5f8d1f1 |
| high_school_physics/test-00000-of-00001.parquet | Data | 33.0 KB | 48fc84060e4c |
| high_school_physics/validation-00000-of-00001.parquet | Data | 8.0 KB | 696ee856dae6 |
| high_school_psychology/dev-00000-of-00001.parquet | Data | 5.2 KB | ada3aebdb980 |
| high_school_psychology/test-00000-of-00001.parquet | Data | 92.8 KB | 9df4a744eb6f |
| high_school_psychology/validation-00000-of-00001.parquet | Data | 15.2 KB | 964f33208874 |
| high_school_statistics/dev-00000-of-00001.parquet | Data | 6.1 KB | 3b7c5d8b0c6c |
| high_school_statistics/test-00000-of-00001.parquet | Data | 58.0 KB | 4922828bda3a |
| high_school_statistics/validation-00000-of-00001.parquet | Data | 10.9 KB | cb74547d5d96 |
| high_school_us_history/dev-00000-of-00001.parquet | Data | 17.8 KB | dcc43231bf78 |
| high_school_us_history/test-00000-of-00001.parquet | Data | 154.9 KB | 7e3c242fa3b0 |
| high_school_us_history/validation-00000-of-00001.parquet | Data | 27.3 KB | 73b682d8cc00 |
| high_school_world_history/dev-00000-of-00001.parquet | Data | 10.2 KB | 5c37e72b49e3 |
| high_school_world_history/test-00000-of-00001.parquet | Data | 201.6 KB | abc9b0182e7c |
| high_school_world_history/validation-00000-of-00001.parquet | Data | 38.5 KB | e7647086c3e3 |
| human_aging/dev-00000-of-00001.parquet | Data | 3.7 KB | 7d5ef3e4af1f |
| human_aging/test-00000-of-00001.parquet | Data | 31.2 KB | 69b143d394fd |
| human_aging/validation-00000-of-00001.parquet | Data | 6.3 KB | bacec25cb2c8 |
| human_sexuality/dev-00000-of-00001.parquet | Data | 4.1 KB | e2f15b5006b8 |
| human_sexuality/test-00000-of-00001.parquet | Data | 23.2 KB | 49efe58e1311 |
| human_sexuality/validation-00000-of-00001.parquet | Data | 5.3 KB | 0fdf548c599d |
| international_law/dev-00000-of-00001.parquet | Data | 5.0 KB | a74c7a33d187 |
| international_law/test-00000-of-00001.parquet | Data | 29.5 KB | 05da704a004f |
| international_law/validation-00000-of-00001.parquet | Data | 7.1 KB | 32b9266ab1a3 |
| jurisprudence/dev-00000-of-00001.parquet | Data | 4.0 KB | 1291e1cbdaaa |
| jurisprudence/test-00000-of-00001.parquet | Data | 23.3 KB | 0f302db5178c |
| jurisprudence/validation-00000-of-00001.parquet | Data | 6.2 KB | 8ef2d3dcda61 |
| logical_fallacies/dev-00000-of-00001.parquet | Data | 4.1 KB | cba7736ad6b6 |
| logical_fallacies/test-00000-of-00001.parquet | Data | 23.0 KB | 87eefcf75ec8 |
| logical_fallacies/validation-00000-of-00001.parquet | Data | 6.5 KB | 042b2502c81a |
| machine_learning/dev-00000-of-00001.parquet | Data | 5.2 KB | b8882136a1a6 |
| machine_learning/test-00000-of-00001.parquet | Data | 19.7 KB | cd3d1ac01f11 |
| machine_learning/validation-00000-of-00001.parquet | Data | 6.2 KB | 090d14c981b1 |
| management/dev-00000-of-00001.parquet | Data | 3.6 KB | 772b281c0322 |
| management/test-00000-of-00001.parquet | Data | 14.7 KB | b5579485f3b8 |
| management/validation-00000-of-00001.parquet | Data | 4.5 KB | 3db139ecc101 |
| marketing/dev-00000-of-00001.parquet | Data | 4.3 KB | a00bdec21ca5 |
| marketing/test-00000-of-00001.parquet | Data | 37.3 KB | 3c975f1b0dc6 |
| marketing/validation-00000-of-00001.parquet | Data | 8.2 KB | 844b008134e3 |
| medical_genetics/dev-00000-of-00001.parquet | Data | 3.8 KB | a3e66d3e1619 |
| medical_genetics/test-00000-of-00001.parquet | Data | 16.4 KB | f5535475f3af |
| medical_genetics/validation-00000-of-00001.parquet | Data | 5.6 KB | b1e5b8381e2f |
| miscellaneous/dev-00000-of-00001.parquet | Data | 3.4 KB | b3a42ee21922 |
| miscellaneous/test-00000-of-00001.parquet | Data | 98.6 KB | 5665dde94d34 |
| miscellaneous/validation-00000-of-00001.parquet | Data | 13.2 KB | 00724923c0ea |
| moral_disputes/dev-00000-of-00001.parquet | Data | 4.4 KB | 7efc2d7512a7 |
| moral_disputes/test-00000-of-00001.parquet | Data | 60.9 KB | fc2a4270481c |
| moral_disputes/validation-00000-of-00001.parquet | Data | 10.7 KB | c88b8e7cd4a5 |
| moral_scenarios/dev-00000-of-00001.parquet | Data | 5.1 KB | c31c54bb7f8c |
| moral_scenarios/test-00000-of-00001.parquet | Data | 89.8 KB | e21cc9d8f5a1 |
| moral_scenarios/validation-00000-of-00001.parquet | Data | 14.9 KB | 8b222953f05e |
| nutrition/dev-00000-of-00001.parquet | Data | 5.0 KB | 4727ff1fb935 |
| nutrition/test-00000-of-00001.parquet | Data | 55.0 KB | 7ba291c21774 |
| nutrition/validation-00000-of-00001.parquet | Data | 9.0 KB | 044ca489a676 |
| philosophy/dev-00000-of-00001.parquet | Data | 4.2 KB | 40bd9dee29bb |
| philosophy/test-00000-of-00001.parquet | Data | 48.6 KB | 51a6715bca76 |
| philosophy/validation-00000-of-00001.parquet | Data | 9.2 KB | 6883bbc1930b |
| prehistory/dev-00000-of-00001.parquet | Data | 4.6 KB | 9f5ee161a94b |
| prehistory/test-00000-of-00001.parquet | Data | 54.3 KB | f4cf1e103e64 |
| prehistory/validation-00000-of-00001.parquet | Data | 9.9 KB | 7ef498da9ba7 |
| professional_accounting/dev-00000-of-00001.parquet | Data | 4.9 KB | 03ab25053042 |
| professional_accounting/test-00000-of-00001.parquet | Data | 69.5 KB | 1fe6da7a3a86 |
| professional_accounting/validation-00000-of-00001.parquet | Data | 12.9 KB | c2cf2ba25cde |
| professional_law/dev-00000-of-00001.parquet | Data | 15.1 KB | cb7bf53b1bcf |
| professional_law/test-00000-of-00001.parquet | Data | 1.0 MB | 81cfdb63639f |
| professional_law/validation-00000-of-00001.parquet | Data | 115.6 KB | d64b01ffb220 |
| professional_medicine/dev-00000-of-00001.parquet | Data | 8.4 KB | 899036c8f5d8 |
| professional_medicine/test-00000-of-00001.parquet | Data | 124.9 KB | 8c1c18b16c98 |
| professional_medicine/validation-00000-of-00001.parquet | Data | 19.9 KB | f218bc4485cb |
| professional_psychology/dev-00000-of-00001.parquet | Data | 4.7 KB | c61d4fb24acc |
| professional_psychology/test-00000-of-00001.parquet | Data | 132.6 KB | 91529edc6760 |
| professional_psychology/validation-00000-of-00001.parquet | Data | 22.1 KB | 85a3859b7186 |
| public_relations/dev-00000-of-00001.parquet | Data | 4.4 KB | 7dd78ceeccd2 |
| public_relations/test-00000-of-00001.parquet | Data | 20.6 KB | 9ce4ff437b2b |
| public_relations/validation-00000-of-00001.parquet | Data | 6.4 KB | 38c2f887dbd9 |
| security_studies/dev-00000-of-00001.parquet | Data | 7.5 KB | c7ab02234d03 |
| security_studies/test-00000-of-00001.parquet | Data | 114.1 KB | 38477f527d22 |
| security_studies/validation-00000-of-00001.parquet | Data | 18.7 KB | 65866e2efc98 |
| sociology/dev-00000-of-00001.parquet | Data | 4.2 KB | 4abb7ebab6f1 |
| sociology/test-00000-of-00001.parquet | Data | 43.9 KB | 0cdb1dab92e7 |
| sociology/validation-00000-of-00001.parquet | Data | 8.4 KB | aadeb1fdd88e |
| us_foreign_policy/dev-00000-of-00001.parquet | Data | 4.2 KB | da27347e8868 |
| us_foreign_policy/test-00000-of-00001.parquet | Data | 19.5 KB | 8051dfc5f49b |
| us_foreign_policy/validation-00000-of-00001.parquet | Data | 5.3 KB | e1149eaf277f |
| virology/dev-00000-of-00001.parquet | Data | 3.9 KB | 2790f42d02f0 |
| virology/test-00000-of-00001.parquet | Data | 27.3 KB | c59ea23f72b4 |
| virology/validation-00000-of-00001.parquet | Data | 7.0 KB | fcc8b6cd99a0 |
| world_religions/dev-00000-of-00001.parquet | Data | 3.3 KB | 5676e31ed09b |
| world_religions/test-00000-of-00001.parquet | Data | 18.9 KB | b59bdb0cb4b1 |
| world_religions/validation-00000-of-00001.parquet | Data | 4.9 KB | 769666ee1ad8 |
| README.md | Documentation | 53.2 KB | — |
| hendrycks_test.py | Other | 5.7 KB | — |
| .gitattributes | Repository | 2.3 KB | — |

## License and Download

License

mit

Access

No access gate

[Download from Center for AI Safety](https://huggingface.co/datasets/cais/mmlu)

Released by Center for AI Safety through its official repository on Hugging Face. [Read the license](https://opensource.org/license/mit).

## Models Trained on This Dataset

- Trained on (disclosed)[shingi-27b](https://savrn.com/models/shingi-27b)
- Trained on (disclosed)[hopper](https://savrn.com/models/hopper)

## Center for AI Safety

[All models and datasets](https://savrn.com/model-publishers/cais)

## Explore More

- [The dataset directory](https://savrn.com/datasets)
- [The model directory](https://savrn.com/models)
- [Research papers](https://savrn.com/papers)

## Source

- Repository metadata, read 2026-10-19.
- [Hugging Face record](https://huggingface.co/datasets/cais/mmlu)
- [How the hub is built](https://savrn.com/model-hub/methodology)