# blimp by NYU Machine Learning for Language: Dataset
Source: https://savrn.com/datasets/blimp
Markdown alternate of the page above; the site index is https://savrn.com/llms.txt

---

## Dataset Card

By NYU Machine Learning for Language, published under cc-by-4.0, revision 877fba0801ff.

### Dataset Card for "blimp"

### Table of Contents

- Dataset Description
- Dataset Summary
- Supported Tasks and Leaderboards
- Languages
- Dataset Structure
- Data Instances
- Data Fields
- Data Splits
- Dataset Creation
- Curation Rationale
- Source Data
- Annotations
- Personal and Sensitive Information
- Considerations for Using the Data
- Social Impact of Dataset
- Discussion of Biases
- Other Known Limitations
- Additional Information
- Dataset Curators
- Licensing Information
- Citation Information
- Contributions

### Dataset Description

- Homepage:
- Repository: https://github.com/alexwarstadt/blimp
- Paper: [BLiMP: The Benchmark of Linguistic Minimal Pairs for English](https://doi.org/10.1162/tacl_a_00321)
- Paper: https://arxiv.org/abs/1912.00582
- Point of Contact: [More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)
- Size of downloaded dataset files: 29.58 MB
- Size of the generated dataset: 11.45 MB
- Total amount of disk used: 41.03 MB

#### Dataset Summary

BLiMP is a challenge set for evaluating what language models (LMs) know about major grammatical phenomena in English. BLiMP consists of 67 sub-datasets, each containing 1000 minimal pairs isolating specific contrasts in syntax, morphology, or semantics. The data is automatically generated according to expert-crafted grammars.

#### Supported Tasks and Leaderboards

[More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)

#### Languages

[Read the full dataset card (1,083 words)](https://savrn.com/datasets/blimp/card)

## Structure

### adjunct_island 1,000 rows

| Split | Rows | Size |
| --- | --- | --- |
| train | 1,000 | 166.0 KB |

sentence_goodstringsentence_badstringfieldstringlinguistics_termstringUIDstringsimple_LM_methodboolone_prefix_methodbooltwo_prefix_methodboollexically_identicalboolpair_idint32

### anaphor_gender_agreement 1,000 rows

| Split | Rows | Size |
| --- | --- | --- |
| train | 1,000 | 131.0 KB |

sentence_goodstringsentence_badstringfieldstringlinguistics_termstringUIDstringsimple_LM_methodboolone_prefix_methodbooltwo_prefix_methodboollexically_identicalboolpair_idint32

### anaphor_number_agreement 1,000 rows

| Split | Rows | Size |
| --- | --- | --- |
| train | 1,000 | 140.0 KB |

sentence_goodstringsentence_badstringfieldstringlinguistics_termstringUIDstringsimple_LM_methodboolone_prefix_methodbooltwo_prefix_methodboollexically_identicalboolpair_idint32

### animate_subject_passive 1,000 rows

| Split | Rows | Size |
| --- | --- | --- |
| train | 1,000 | 144.5 KB |

sentence_goodstringsentence_badstringfieldstringlinguistics_termstringUIDstringsimple_LM_methodboolone_prefix_methodbooltwo_prefix_methodboollexically_identicalboolpair_idint32

### animate_subject_trans 1,000 rows

| Split | Rows | Size |
| --- | --- | --- |
| train | 1,000 | 127.9 KB |

sentence_goodstringsentence_badstringfieldstringlinguistics_termstringUIDstringsimple_LM_methodboolone_prefix_methodbooltwo_prefix_methodboollexically_identicalboolpair_idint32

### causative 1,000 rows

| Split | Rows | Size |
| --- | --- | --- |
| train | 1,000 | 122.9 KB |

sentence_goodstringsentence_badstringfieldstringlinguistics_termstringUIDstringsimple_LM_methodboolone_prefix_methodbooltwo_prefix_methodboollexically_identicalboolpair_idint32

### complex_NP_island 1,000 rows

| Split | Rows | Size |
| --- | --- | --- |
| train | 1,000 | 199.1 KB |

sentence_goodstringsentence_badstringfieldstringlinguistics_termstringUIDstringsimple_LM_methodboolone_prefix_methodbooltwo_prefix_methodboollexically_identicalboolpair_idint32

### coordinate_structure_constraint_complex_left_branch 1,000 rows

| Split | Rows | Size |
| --- | --- | --- |
| train | 1,000 | 211.0 KB |

sentence_goodstringsentence_badstringfieldstringlinguistics_termstringUIDstringsimple_LM_methodboolone_prefix_methodbooltwo_prefix_methodboollexically_identicalboolpair_idint32

### coordinate_structure_constraint_object_extraction 1,000 rows

| Split | Rows | Size |
| --- | --- | --- |
| train | 1,000 | 171.8 KB |

sentence_goodstringsentence_badstringfieldstringlinguistics_termstringUIDstringsimple_LM_methodboolone_prefix_methodbooltwo_prefix_methodboollexically_identicalboolpair_idint32

### determiner_noun_agreement_1 1,000 rows

| Split | Rows | Size |
| --- | --- | --- |
| train | 1,000 | 156.2 KB |

sentence_goodstringsentence_badstringfieldstringlinguistics_termstringUIDstringsimple_LM_methodboolone_prefix_methodbooltwo_prefix_methodboollexically_identicalboolpair_idint32

### determiner_noun_agreement_2 1,000 rows

| Split | Rows | Size |
| --- | --- | --- |
| train | 1,000 | 156.3 KB |

sentence_goodstringsentence_badstringfieldstringlinguistics_termstringUIDstringsimple_LM_methodboolone_prefix_methodbooltwo_prefix_methodboollexically_identicalboolpair_idint32

### determiner_noun_agreement_irregular_1 1,000 rows

| Split | Rows | Size |
| --- | --- | --- |
| train | 1,000 | 164.6 KB |

sentence_goodstringsentence_badstringfieldstringlinguistics_termstringUIDstringsimple_LM_methodboolone_prefix_methodbooltwo_prefix_methodboollexically_identicalboolpair_idint32

## Details

Repository

nyu-mll/blimp

Publisher

NYU Machine Learning for Language

Task category

Text classification

Tags

Not stated by the source

Size category

10K<n<100K

Languages

en

Revision

877fba0801ffb7cbd8c39c1ff314a46f053f6036

Last updated

2024-01-23

## SAVRN's Notes on blimp

BLiMP is a challenge set for English grammar knowledge, built as minimal pairs isolating contrasts in syntax, morphology and semantics. The card describes 67 sub-datasets of 1,000 pairs each, while the page metadata shows 12,000 rows across 12 configurations and 4.0 MB, so verify which subset your loader is returning before quoting results. Downloads run 153.3k a month, largely as an eval fixture.

CC-BY-4.0 allows commercial use with attribution, and the entire set is 4.0 MB, so cost is irrelevant. The discipline that matters is eval hygiene: minimal-pair sets of this size memorize fast, and any fine-tuning on them voids your benchmark comparison. Keep it strictly in the evaluation slot, report the version hash in your results, and never let it leak into pretraining corpora.

SAVRN Research

## Files

69 files, 4.0 MB in total.

Data67 files · 3.9 MB

Documentation1 file · 58.6 KB

Repository1 file · 1.2 KB

Every file

| File | Type | Size | SHA-256 |
| --- | --- | --- | --- |
| adjunct_island/train-00000-of-00001.parquet | Data | 62.2 KB | 094b1a6c4d8e |
| anaphor_gender_agreement/train-00000-of-00001.parquet | Data | 39.2 KB | 3c3f1d8d9e6a |
| anaphor_number_agreement/train-00000-of-00001.parquet | Data | 41.5 KB | 5dc16d19b9b3 |
| animate_subject_passive/train-00000-of-00001.parquet | Data | 47.3 KB | d4c957edec77 |
| animate_subject_trans/train-00000-of-00001.parquet | Data | 49.7 KB | ef2f412c3f47 |
| causative/train-00000-of-00001.parquet | Data | 49.0 KB | cfec48a03e32 |
| complex_NP_island/train-00000-of-00001.parquet | Data | 78.2 KB | 4a01cfd34626 |
| coordinate_structure_constraint_complex_left_branch/train-00000-of-00001.parquet | Data | 67.9 KB | b776239fe670 |
| coordinate_structure_constraint_object_extraction/train-00000-of-00001.parquet | Data | 51.6 KB | 55d830a519e5 |
| determiner_noun_agreement_1/train-00000-of-00001.parquet | Data | 49.9 KB | eac75d95c9fa |
| determiner_noun_agreement_2/train-00000-of-00001.parquet | Data | 49.5 KB | 17520554fff0 |
| determiner_noun_agreement_irregular_1/train-00000-of-00001.parquet | Data | 47.3 KB | b73d7c301e81 |
| determiner_noun_agreement_irregular_2/train-00000-of-00001.parquet | Data | 47.4 KB | 73c404f947d0 |
| determiner_noun_agreement_with_adj_2/train-00000-of-00001.parquet | Data | 56.3 KB | 732f6dc785f2 |
| determiner_noun_agreement_with_adj_irregular_1/train-00000-of-00001.parquet | Data | 54.4 KB | 939567391e6e |
| determiner_noun_agreement_with_adj_irregular_2/train-00000-of-00001.parquet | Data | 54.1 KB | e71c6a9baf9c |
| determiner_noun_agreement_with_adjective_1/train-00000-of-00001.parquet | Data | 55.7 KB | dd61dd23aa32 |
| distractor_agreement_relational_noun/train-00000-of-00001.parquet | Data | 59.6 KB | 2e0cef46676c |
| distractor_agreement_relative_clause/train-00000-of-00001.parquet | Data | 77.9 KB | c23c03ee033d |
| drop_argument/train-00000-of-00001.parquet | Data | 40.0 KB | 4acf04febbe9 |
| ellipsis_n_bar_1/train-00000-of-00001.parquet | Data | 92.8 KB | 58edad00d53c |
| ellipsis_n_bar_2/train-00000-of-00001.parquet | Data | 98.9 KB | 7506bd80abb5 |
| existential_there_object_raising/train-00000-of-00001.parquet | Data | 76.6 KB | 1c63d5b77b5d |
| existential_there_quantifiers_1/train-00000-of-00001.parquet | Data | 51.6 KB | ac2e197d5530 |
| existential_there_quantifiers_2/train-00000-of-00001.parquet | Data | 52.1 KB | df307122fda8 |
| existential_there_subject_raising/train-00000-of-00001.parquet | Data | 59.5 KB | 1e281bf8a80a |
| expletive_it_object_raising/train-00000-of-00001.parquet | Data | 88.6 KB | 45a0a957726a |
| inchoative/train-00000-of-00001.parquet | Data | 39.8 KB | cb452cd6ee37 |
| intransitive/train-00000-of-00001.parquet | Data | 42.4 KB | f04f66e0a30a |
| irregular_past_participle_adjectives/train-00000-of-00001.parquet | Data | 36.7 KB | 77d65967bbf6 |
| irregular_past_participle_verbs/train-00000-of-00001.parquet | Data | 37.3 KB | f6acd480c04c |
| irregular_plural_subject_verb_agreement_1/train-00000-of-00001.parquet | Data | 50.7 KB | b4b07609019d |
| irregular_plural_subject_verb_agreement_2/train-00000-of-00001.parquet | Data | 42.7 KB | 66a1bf6bba6d |
| left_branch_island_echo_question/train-00000-of-00001.parquet | Data | 50.5 KB | 41a535903bc7 |
| left_branch_island_simple_question/train-00000-of-00001.parquet | Data | 50.3 KB | 8c718b6f37cb |
| matrix_question_npi_licensor_present/train-00000-of-00001.parquet | Data | 51.9 KB | a7ed0834104a |
| npi_present_1/train-00000-of-00001.parquet | Data | 52.0 KB | cca3e48e8a08 |
| npi_present_2/train-00000-of-00001.parquet | Data | 51.7 KB | 44cdf2e5c796 |
| only_npi_licensor_present/train-00000-of-00001.parquet | Data | 51.4 KB | 8e26a57bc8d8 |
| only_npi_scope/train-00000-of-00001.parquet | Data | 85.0 KB | c5c14f42c274 |
| passive_1/train-00000-of-00001.parquet | Data | 53.9 KB | 3bcb3d739721 |
| passive_2/train-00000-of-00001.parquet | Data | 40.5 KB | ee29e69ca55e |
| principle_A_c_command/train-00000-of-00001.parquet | Data | 67.9 KB | 6d18116ede2c |
| principle_A_case_1/train-00000-of-00001.parquet | Data | 61.1 KB | 6615f7f1572b |
| principle_A_case_2/train-00000-of-00001.parquet | Data | 56.4 KB | e3a13d2ab24f |
| principle_A_domain_1/train-00000-of-00001.parquet | Data | 59.1 KB | 5efcec2af824 |
| principle_A_domain_2/train-00000-of-00001.parquet | Data | 58.5 KB | 5de4982ab8ba |
| principle_A_domain_3/train-00000-of-00001.parquet | Data | 52.9 KB | 25227d39f616 |
| principle_A_reconstruction/train-00000-of-00001.parquet | Data | 44.5 KB | bd57fef2d8e9 |
| regular_plural_subject_verb_agreement_1/train-00000-of-00001.parquet | Data | 49.5 KB | 602d82e663e7 |
| regular_plural_subject_verb_agreement_2/train-00000-of-00001.parquet | Data | 43.4 KB | 47b11c839024 |
| sentential_negation_npi_licensor_present/train-00000-of-00001.parquet | Data | 54.8 KB | 2f2e30c84624 |
| sentential_negation_npi_scope/train-00000-of-00001.parquet | Data | 90.2 KB | 06afb479723e |
| sentential_subject_island/train-00000-of-00001.parquet | Data | 56.7 KB | a096e1ea8ff2 |
| superlative_quantifiers_1/train-00000-of-00001.parquet | Data | 48.5 KB | eba2e172058a |
| superlative_quantifiers_2/train-00000-of-00001.parquet | Data | 50.5 KB | 76eadb284f7c |
| tough_vs_raising_1/train-00000-of-00001.parquet | Data | 44.8 KB | defbb5c0cb9b |
| tough_vs_raising_2/train-00000-of-00001.parquet | Data | 61.5 KB | 01d9c6a6b0df |
| transitive/train-00000-of-00001.parquet | Data | 55.1 KB | 1c92b2316478 |
| wh_island/train-00000-of-00001.parquet | Data | 52.8 KB | 63e455edde12 |
| wh_questions_object_gap/train-00000-of-00001.parquet | Data | 70.0 KB | 5107d0b8a405 |
| wh_questions_subject_gap/train-00000-of-00001.parquet | Data | 71.6 KB | a58be1f30174 |
| wh_questions_subject_gap_long_distance/train-00000-of-00001.parquet | Data | 98.9 KB | ca30473b9569 |
| wh_vs_that_no_gap/train-00000-of-00001.parquet | Data | 71.7 KB | 92fc7e67b32b |
| wh_vs_that_no_gap_long_distance/train-00000-of-00001.parquet | Data | 95.5 KB | e1aa6fd1c740 |
| wh_vs_that_with_gap/train-00000-of-00001.parquet | Data | 60.3 KB | c905f68e4432 |
| wh_vs_that_with_gap_long_distance/train-00000-of-00001.parquet | Data | 84.1 KB | c2adf0cb1714 |
| README.md | Documentation | 58.6 KB | — |
| .gitattributes | Repository | 1.2 KB | — |

## License and Download

License

cc-by-4.0

Access

No access gate

[Download from NYU Machine Learning for Language](https://huggingface.co/datasets/nyu-mll/blimp)

Released by NYU Machine Learning for Language through its official repository on Hugging Face. [Read the license](https://creativecommons.org/licenses/by/4.0/).

## NYU Machine Learning for Language

[All models and datasets](https://savrn.com/model-publishers/nyu-mll)

## Explore More

- [The dataset directory](https://savrn.com/datasets)
- [The model directory](https://savrn.com/models)
- [Research papers](https://savrn.com/papers)

## Source

- Repository metadata, read 2026-10-19.
- [Hugging Face record](https://huggingface.co/datasets/nyu-mll/blimp)
- [How the hub is built](https://savrn.com/model-hub/methodology)