IFEval

Dataset Card for IFEval

Dataset Description

• Repository: https://github.com/google-research/google-research/tree/master/instruction_following_eval
• Paper: https://huggingface.co/papers/2311.07911
• Leaderboard: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard
• Point of Contact: Le Hou

Dataset Summary

This dataset contains the prompts used in the Instruction-Following Eval (IFEval) benchmark for large language models. It contains around 500 "verifiable instructions" such as "write in more than 400 words" and "mention the keyword of AI at least 3 times" which can be verified by heuristics. To load the dataset, run:

from datasets import load_dataset

ifeval = load_dataset("google/IFEval")

Supported Tasks and Leaderboards

The IFEval dataset is designed for evaluating chat or instruction fine-tuned language models and is one of the core benchmarks used in the Open LLM Leaderboard.

Languages

The data in IFEval are in English (BCP-47 en).

Dataset Structure

Data Instances

An example of the train split looks as follows: