OpenMathInstruct-2 is a math instruction tuning dataset with 14M problem-solution pairs generated using the Llama3.1-405B-Instruct model.
Dataset Card
By NVIDIA, published under cc-by-4.0, revision 469216e3f46f.
OpenMathInstruct-2 is a math instruction tuning dataset with 14M problem-solution pairs generated using the Llama3.1-405B-Instruct model.
The training set problems of GSM8K and MATH are used for constructing the dataset in the following ways: - Solution augmentation: Generating chain-of-thought solutions for training set problems in GSM8K and MATH. - Problem-Solution augmentation: Generating new problems, followed by solutions for these new problems.
OpenMathInstruct-2 dataset contains the following fields:
- problem: Original problem from either the GSM8K or MATH training set or augmented problem from these training sets.
- generated_solution: Synthetically generated solution.
- expected_answer: For problems in the training set, it is the ground-truth answer provided in the datasets. For augmented problems, it is the majority-voting answer.
- problem_source: Whether the problem is taken directly from GSM8K or MATH or is an augmented version derived from either dataset.
Structure
default 21,972,791 rows
| Split | Rows | Size |
|---|---|---|
| train | 13,972,791 | 15.4 GB |
| train_1M | 1,000,000 | 1.3 GB |
| train_2M | 2,000,000 | 2.8 GB |
| train_5M | 5,000,000 | 6.5 GB |
Details
- Repository
- nvidia/OpenMathInstruct-2
- Publisher
- NVIDIA
- Task category
- Question answering
- Tags
- math, nvidia
- Size category
- 10M<n<100M
- Languages
- en
- Revision
- 469216e3f46f4dacf476b382e192485ea51a143e
- Last updated
- 2024-11-25
SAVRN's Notes on OpenMathInstruct-2
OpenMathInstruct-2 is synthetic math instruction data at production scale: 21,972,791 rows, 12.6 GB across 59 files, 146k monthly downloads, generated with Llama 3.1 405B Instruct, task category question answering. Per the card, the problems come from GSM8K and MATH training sets via solution augmentation and problem-solution augmentation, with expected answers taken from ground truth for originals and majority-vote answers for the augmented problems.
CC-BY-4.0 permits commercial use with attribution, and the data is English. Two flags before you commit: because it is built on GSM8K and MATH training problems, do not report scores on either benchmark from models trained here, and the majority-vote labeling on augmented problems means some answer noise is structural, not accidental. Disclosed inbound models on our hub include FinAI and cagliostro-v3.
Files
59 files, 12.6 GB in total.
Every file
| File | Type | Size | SHA-256 |
|---|---|---|---|
| data/train-00000-of-00032.parquet | Data | 236.6 MB | 55bae5fa27c8 |
| data/train-00001-of-00032.parquet | Data | 236.9 MB | 4c1a2103a979 |
| data/train-00002-of-00032.parquet | Data | 236.6 MB | 29bd4017239b |
| data/train-00003-of-00032.parquet | Data | 236.8 MB | 619c696a8e43 |
| data/train-00004-of-00032.parquet | Data | 236.7 MB | e8d6d88a1cca |
| data/train-00005-of-00032.parquet | Data | 236.6 MB | 84781d52cab8 |
| data/train-00006-of-00032.parquet | Data | 236.8 MB | 85e196aac429 |
| data/train-00007-of-00032.parquet | Data | 236.8 MB | 279590a0ac58 |
| data/train-00008-of-00032.parquet | Data | 237.0 MB | bc6dbc7a97df |
| data/train-00009-of-00032.parquet | Data | 236.7 MB | fe38efc9a128 |
| data/train-00010-of-00032.parquet | Data | 237.0 MB | 574debcf9111 |
| data/train-00011-of-00032.parquet | Data | 237.0 MB | 750480cc24d2 |
| data/train-00012-of-00032.parquet | Data | 236.5 MB | 338117e09231 |
| data/train-00013-of-00032.parquet | Data | 236.7 MB | 363739df048e |
| data/train-00014-of-00032.parquet | Data | 236.7 MB | 9653648f9e65 |
| data/train-00015-of-00032.parquet | Data | 237.3 MB | 46aa107221b4 |
| data/train-00016-of-00032.parquet | Data | 236.8 MB | fc022c921698 |
| data/train-00017-of-00032.parquet | Data | 236.5 MB | 424a92182ed1 |
| data/train-00018-of-00032.parquet | Data | 236.7 MB | 57f3ce262c96 |
| data/train-00019-of-00032.parquet | Data | 236.5 MB | 9ff9798ed581 |
| data/train-00020-of-00032.parquet | Data | 236.8 MB | a9cbd3619965 |
| data/train-00021-of-00032.parquet | Data | 236.8 MB | 5f644f9d77b1 |
| data/train-00022-of-00032.parquet | Data | 236.8 MB | b546f1f094d8 |
| data/train-00023-of-00032.parquet | Data | 236.7 MB | 4ba1d104be88 |
| data/train-00024-of-00032.parquet | Data | 236.7 MB | dfbce7391b52 |
| data/train-00025-of-00032.parquet | Data | 236.9 MB | 08e7ebce1ecc |
| data/train-00026-of-00032.parquet | Data | 236.7 MB | 3e6545137ea2 |
| data/train-00027-of-00032.parquet | Data | 236.3 MB | 6d46ac4f285f |
| data/train-00028-of-00032.parquet | Data | 236.8 MB | b9378a52c74b |
| data/train-00029-of-00032.parquet | Data | 236.8 MB | 93aee9bae8fe |
| data/train-00030-of-00032.parquet | Data | 236.6 MB | 2191af67a930 |
| data/train-00031-of-00032.parquet | Data | 236.8 MB | b7fc689fb28a |
| data/train_1M-00000-of-00003.parquet | Data | 212.9 MB | 93700f39cdc8 |
| data/train_1M-00001-of-00003.parquet | Data | 213.2 MB | c208b3cda829 |
| data/train_1M-00002-of-00003.parquet | Data | 212.9 MB | 1a42cf7139b6 |
| data/train_2M-00000-of-00006.parquet | Data | 217.1 MB | 0e5070d547da |
| data/train_2M-00001-of-00006.parquet | Data | 217.0 MB | f535f5868188 |
| data/train_2M-00002-of-00006.parquet | Data | 216.9 MB | 3b9e9306db9e |
| data/train_2M-00003-of-00006.parquet | Data | 217.2 MB | 0b758d1e798f |
| data/train_2M-00004-of-00006.parquet | Data | 217.2 MB | 478feb390152 |
| data/train_2M-00005-of-00006.parquet | Data | 217.4 MB | cbec946a2d11 |
| data/train_5M-00000-of-00014.parquet | Data | 222.4 MB | b6c45a7eb504 |
| data/train_5M-00001-of-00014.parquet | Data | 222.1 MB | 73b9d40ac366 |
| data/train_5M-00002-of-00014.parquet | Data | 221.8 MB | 8313fde2b1cf |
| data/train_5M-00003-of-00014.parquet | Data | 221.9 MB | 2a5183be98b8 |
| data/train_5M-00004-of-00014.parquet | Data | 222.1 MB | 1e22f77711e0 |
| data/train_5M-00005-of-00014.parquet | Data | 222.0 MB | 69c9ea15ac79 |
| data/train_5M-00006-of-00014.parquet | Data | 222.3 MB | a293ab952250 |
| data/train_5M-00007-of-00014.parquet | Data | 222.6 MB | 5a4c2b9fe337 |
| data/train_5M-00008-of-00014.parquet | Data | 222.1 MB | 0b2d598714a1 |
| data/train_5M-00009-of-00014.parquet | Data | 222.4 MB | 7d07b4847af4 |
| data/train_5M-00010-of-00014.parquet | Data | 221.9 MB | 34168a7b268c |
| data/train_5M-00011-of-00014.parquet | Data | 222.3 MB | 7dc69d30990e |
| data/train_5M-00012-of-00014.parquet | Data | 222.4 MB | 3518f76d3d95 |
| data/train_5M-00013-of-00014.parquet | Data | 221.8 MB | 5c15c5617b51 |
| README.md | Documentation | 6.4 KB | — |
| SFT Data Diagram 1.jpg | Other | 269.0 KB | d9fa9ade48cb |
| scaling_plot.jpg | Other | 172.0 KB | dfccb18ebb30 |
| .gitattributes | Repository | 3.3 KB | — |
License and Download
- License
- cc-by-4.0
- Access
- No access gate
Released by NVIDIA through its official repository on Hugging Face. Read the license.
Models Trained on This Dataset
- Trained on (disclosed)FinAI
- Trained on (disclosed)cagliostro-v3