This dataset contains a subset of 500 problems from the MATH benchmark that OpenAI created in their Let's Verify Step by Step paper. See their GitHub repo for the source file: https://github.com/openai/prm800k/tree/main?tab=readme-ov-file#math-splits
Dataset Card
This dataset contains a subset of 500 problems from the MATH benchmark that OpenAI created in their Let's Verify Step by Step paper. See their GitHub repo for the source file: https://github.com/openai/prm800k/tree/main?tab=readme-ov-file#math-splits
Excerpt from the card by Hugging Face H4.
Structure
default 500 rows
| Split | Rows | Size |
|---|---|---|
| test | 500 | 400.3 KB |
Details
- Repository
- HuggingFaceH4/MATH-500
- Publisher
- Hugging Face H4
- Task category
- Text generation
- Tags
- Not stated by the source
- Size category
- Not stated by the source
- Languages
- en
- Revision
- 6e4ed1a2a79af7d8630a6b768ec859cb5af4d3be
- Last updated
- 2025-12-15
SAVRN's Notes on MATH-500
MATH-500 is exactly what the name says: 500 problems, 450.3 KB, one config, cut from the MATH benchmark by OpenAI for the 'Let's Verify Step by Step' paper and hosted by Hugging Face H4. The inbound list shows what it is used for — math RLVR and small reasoning models (MoA-400M, MoA-150M, next-1b, next-12b-i1, and a family of Qwen2.5-3B Sheldon-RLVR checkpoints) all disclose training against it.
It is an evaluation set, and its smallness is the point: 500 problems is cheap to run at every checkpoint, which is why it shows up in so many training runs. Two cautions. The license field on our page reads 'not stated by the source', so a commercial run should clear the underlying MATH terms rather than assume. And because it is a fixed public subset of MATH test items, any model trained on MATH-family data needs an overlap check before its MATH-500 scores mean anything.
Files
4 files, 450.3 KB in total.
Every file
| File | Type | Size | SHA-256 |
|---|---|---|---|
| test.jsonl | Data | 446.6 KB | — |
| README.md | Documentation | 412 B | — |
| eval.yaml | Other | 949 B | — |
| .gitattributes | Repository | 2.4 KB | — |
License and Download
- License
- Not stated by the source
- Access
- No access gate
Released by Hugging Face H4 through its official repository on Hugging Face.
Models Trained on This Dataset
- Trained on (disclosed)MoA-400M
- Trained on (disclosed)MoA-150M
- Trained on (disclosed)next-1b
- Trained on (disclosed)next-12b-i1-GGUF
- Trained on (disclosed)Qwen2.5-3B-Instruct-Sheldon-RLVR-math-v1
- Trained on (disclosed)Qwen2.5-3B-Instruct-Sheldon-RLVR-math-v1-LoRA
- Trained on (disclosed)Qwen2.5-3B-Instruct-Sheldon-RLVR-math-v2-fullft