The Adversarial Natural Language Inference (ANLI) is a new large-scale NLI benchmark dataset, The dataset is collected via an iterative, adversarial human-and-model-in-the-loop procedure.
Dataset Card
The Adversarial Natural Language Inference (ANLI) is a new large-scale NLI benchmark dataset, The dataset is collected via an iterative, adversarial human-and-model-in-the-loop procedure. ANLI is much more difficult than its predecessors including SNLI and MNLI. It contains three rounds. Each round has train/dev/test splits. English An example of 'trainr2' looks as follows. The data fields are the same among all splits. - uid: a string feature. - premise: a string feature. - hypothesis: a string feature. - label: a classification label, with possible values including entailment (0), neutral (1), contradiction (2). - reason: a string feature. Thanks to @thomwolf, @easonnie, @lhoestq…
Excerpt from the card by AI at Meta, licensed cc-by-nc-4.0.
Structure
plain_text 169,265 rows
| Split | Rows | Size |
|---|---|---|
| train_r1 | 16,946 | 7.6 MB |
| dev_r1 | 1,000 | 573.6 KB |
| test_r1 | 1,000 | 575.0 KB |
| train_r2 | 45,460 | 20.0 MB |
| dev_r2 | 1,000 | 556.2 KB |
| test_r2 | 1,000 | 572.8 KB |
| train_r3 | 100,459 | 67.1 MB |
| dev_r3 | 1,200 | 673.1 KB |
| test_r3 | 1,200 | 666.9 KB |
Details
- Repository
- facebook/anli
- Publisher
- AI at Meta
- Task category
- Text classification
- Tags
- Not stated by the source
- Size category
- 100K<n<1M
- Languages
- en
- Revision
- 8e4813d81f46d313dac7892e1c28076917cfcdf9
- Last updated
- 2023-12-21
SAVRN's Notes on anli
ANLI is adversarial natural language inference: 169,265 rows, 26.3 MB, 11 files, built over three rounds with each round carrying its own train, dev and test splits, where dev and test sit at 1,000 rows per round. The inbound list on our hub shows the actual training use: a wall of NLI encoders, including the DeBERTa-v3 mnli-fever-anli family and ModernBERT-base and ModernBERT-large nli variants.
The license is CC-BY-NC-4.0, which bars commercial use: fine for research fine-tuning and evaluation, off-limits for a product training run. At 181.1k downloads a month, it remains the standard hard NLI benchmark, and its adversarial human-and-model-in-the-loop construction is exactly why it still separates models. Storage is a non-issue at 26.3 MB in English; the NC clause is the decision point.
Files
11 files, 26.3 MB in total.
Every file
| File | Type | Size | SHA-256 |
|---|---|---|---|
| plain_text/dev_r1-00000-of-00001.parquet | Data | 351.5 KB | 72e27463177b |
| plain_text/dev_r2-00000-of-00001.parquet | Data | 350.6 KB | 43e4673665de |
| plain_text/dev_r3-00000-of-00001.parquet | Data | 434.0 KB | 61775ec09351 |
| plain_text/test_r1-00000-of-00001.parquet | Data | 353.4 KB | c4a3d304c467 |
| plain_text/test_r2-00000-of-00001.parquet | Data | 361.5 KB | df5daccdd562 |
| plain_text/test_r3-00000-of-00001.parquet | Data | 434.6 KB | 3232c4217979 |
| plain_text/train_r1-00000-of-00001.parquet | Data | 3.1 MB | de2d038ae67f |
| plain_text/train_r2-00000-of-00001.parquet | Data | 6.5 MB | 209f4a15bf77 |
| plain_text/train_r3-00000-of-00001.parquet | Data | 14.3 MB | c1d3f614d673 |
| README.md | Documentation | 8.0 KB | — |
| .gitattributes | Repository | 1.2 KB | — |
License and Download
- License
- cc-by-nc-4.0
- Access
- No access gate
Released by AI at Meta through its official repository on Hugging Face. Read the license.
Models Trained on This Dataset
- Trained on (disclosed)mDeBERTa-v3-base-xnli-multilingual-nli-2mil7
- Trained on (disclosed)DeBERTa-v3-base-mnli-fever-anli
- Trained on (disclosed)DeBERTa-v3-large-mnli-fever-anli-ling-wanli
- Trained on (disclosed)ModernBERT-base-nli
- Trained on (disclosed)ModernBERT-large-nli
- Trained on (disclosed)DeBERTa-v3-xsmall-mnli-fever-anli-ling-binary
- Trained on (disclosed)deberta-small-long-nli
- Trained on (disclosed)deberta-base-long-nli