Rows—
Configurations—
Size1.4 TB
License—
AccessPublicly accessible
Monthly Downloads196
Dataset Card
The publisher has not written a card for this dataset.
Details
- Repository
- Diamegs/pre-training-tokens
- Publisher
- Johannes Schwab
- Task category
- Not stated by the source
- Tags
- Not stated by the source
- Size category
- Not stated by the source
- Languages
- Not stated by the source
- Revision
- 1853b1fb6d29d927013a321b326afd4bdda1736f
- Last updated
- 2026-10-04
Files
75 files, 1.4 TB in total.
Data2 files · 1.7 GB
Other72 files · 1.4 TB
Repository1 file · 2.5 KB
Every file
| File | Type | Size | SHA-256 |
|---|---|---|---|
| croissant.json | Data | 4.7 KB | — |
| ft-data/post_training_dataset.tar.gz | Data | 1.7 GB | 4bfef7ea5019 |
| pre-training-tokens/2013-05.bin | Other | 16.0 GB | 4a893bf4ef4f |
| pre-training-tokens/2013-06.bin | Other | 16.0 GB | 0d8b6f2ae588 |
| pre-training-tokens/2013-12.bin | Other | 96.0 GB | e67e60a7fe13 |
| pre-training-tokens/2014-03.bin | Other | 48.0 GB | b5631f3ca02a |
| pre-training-tokens/2014-04.bin | Other | 16.0 GB | d4823a420e2a |
| pre-training-tokens/2014-07.bin | Other | 48.0 GB | 447d51a3e172 |
| pre-training-tokens/2014-08.bin | Other | 16.0 GB | e7d4a06150c7 |
| pre-training-tokens/2014-09.bin | Other | 16.0 GB | d7625dc26453 |
| pre-training-tokens/2014-10.bin | Other | 16.0 GB | 0f242939fa3b |
| pre-training-tokens/2014-11.bin | Other | 16.0 GB | 261d71448816 |
| pre-training-tokens/2014-12.bin | Other | 16.0 GB | b100160ecfb1 |
| pre-training-tokens/2015-01.bin | Other | 16.0 GB | e29fd38babfb |
| pre-training-tokens/2015-02.bin | Other | 16.0 GB | ed817405d8ac |
| pre-training-tokens/2015-03.bin | Other | 16.0 GB | 209daa01c4bf |
| pre-training-tokens/2015-04.bin | Other | 16.0 GB | e1fd7608a66d |
| pre-training-tokens/2015-05.bin | Other | 16.0 GB | 2e15b3384f7c |
| pre-training-tokens/2015-06.bin | Other | 16.0 GB | 89725b71b4e8 |
| pre-training-tokens/2015-07.bin | Other | 16.0 GB | da67e05c3a46 |
| pre-training-tokens/2015-08.bin | Other | 16.0 GB | 63064520020e |
| pre-training-tokens/2015-10.bin | Other | 32.0 GB | 3e8bbdfa7d39 |
| pre-training-tokens/2015-11.bin | Other | 16.0 GB | 618056e03e75 |
| pre-training-tokens/2016-02.bin | Other | 48.0 GB | 1e89bae5642a |
| pre-training-tokens/2016-04.bin | Other | 32.0 GB | c4fc607b28b9 |
| pre-training-tokens/2016-05.bin | Other | 16.0 GB | 7791630d2112 |
| pre-training-tokens/2016-06.bin | Other | 16.0 GB | b8aa84f64aae |
| pre-training-tokens/2016-07.bin | Other | 16.0 GB | b1ce68d6e7a5 |
| pre-training-tokens/2016-08.bin | Other | 16.0 GB | bb31dd004f2e |
| pre-training-tokens/2016-09.bin | Other | 16.0 GB | 1dd41dc280d4 |
| pre-training-tokens/2016-10.bin | Other | 16.0 GB | 4b8ee2e93490 |
| pre-training-tokens/2016-12.bin | Other | 32.0 GB | b010ccc79f48 |
| pre-training-tokens/2017-01.bin | Other | 16.0 GB | 4711a70b001f |
| pre-training-tokens/2017-02.bin | Other | 16.0 GB | cd17eaacd5f9 |
| pre-training-tokens/2017-03.bin | Other | 16.0 GB | 7d4a2fde58ed |
| pre-training-tokens/2017-04.bin | Other | 16.0 GB | 4f7ef75391f2 |
| pre-training-tokens/2017-05.bin | Other | 16.0 GB | 998629d80de1 |
| pre-training-tokens/2017-06.bin | Other | 16.0 GB | 928c8b67673d |
| pre-training-tokens/2017-07.bin | Other | 16.0 GB | a118a95872cc |
| pre-training-tokens/2017-08.bin | Other | 16.0 GB | 590584cfa19d |
| pre-training-tokens/2017-09.bin | Other | 16.0 GB | 4be4c5d71b36 |
| pre-training-tokens/2017-10.bin | Other | 16.0 GB | 5d450d0e8950 |
| pre-training-tokens/2017-11.bin | Other | 16.0 GB | 21419421160c |
| pre-training-tokens/2017-12.bin | Other | 16.0 GB | 298af99e3dc3 |
| pre-training-tokens/2018-01.bin | Other | 16.0 GB | 3b51d989b8b4 |
| pre-training-tokens/2018-02.bin | Other | 16.0 GB | 067cbe1e4de6 |
| pre-training-tokens/2018-03.bin | Other | 16.0 GB | 60f9b5c329d8 |
| pre-training-tokens/2018-04.bin | Other | 16.0 GB | 4a8bd89f7643 |
| pre-training-tokens/2018-05.bin | Other | 16.0 GB | 785e0dd4477f |
| pre-training-tokens/2018-06.bin | Other | 16.0 GB | b624d979b282 |
| pre-training-tokens/2018-07.bin | Other | 16.0 GB | f57be60fe647 |
| pre-training-tokens/2018-08.bin | Other | 16.0 GB | a2c135f2ba26 |
| pre-training-tokens/2018-09.bin | Other | 16.0 GB | b6dba5b37ebc |
| pre-training-tokens/2018-10.bin | Other | 16.0 GB | f224fb85e592 |
| pre-training-tokens/2018-11.bin | Other | 16.0 GB | be7f3e3649b4 |
| pre-training-tokens/2018-12.bin | Other | 16.0 GB | b833410142ee |
| pre-training-tokens/2019-01.bin | Other | 16.0 GB | 04c43231fdea |
| pre-training-tokens/2019-02.bin | Other | 16.0 GB | c67875ee9438 |
| pre-training-tokens/2019-03.bin | Other | 16.0 GB | 4c234e2f417a |
| pre-training-tokens/2019-04.bin | Other | 16.0 GB | a3d82b55f78b |
| pre-training-tokens/2019-05.bin | Other | 16.0 GB | b35b8a30754a |
| pre-training-tokens/2019-06.bin | Other | 16.0 GB | d9f66ed81cd5 |
| pre-training-tokens/2019-07.bin | Other | 16.0 GB | 4e202cff9af7 |
| pre-training-tokens/2019-08.bin | Other | 16.0 GB | 36fe519eae97 |
| pre-training-tokens/2019-09.bin | Other | 16.0 GB | 10c4285f2ba7 |
| pre-training-tokens/2019-10.bin | Other | 16.0 GB | f364eca87e69 |
| pre-training-tokens/2019-11.bin | Other | 16.0 GB | e62c1b08a8ea |
| pre-training-tokens/2019-12.bin | Other | 16.0 GB | 7b9ae6067c87 |
| pre-training-tokens/2020-01.bin | Other | 16.0 GB | 475a0da02570 |
| pre-training-tokens/2020-02.bin | Other | 16.0 GB | 5670c681f7d4 |
| pre-training-tokens/2020-03.bin | Other | 16.0 GB | 3bf177d5d5c9 |
| pre-training-tokens/2020-04.bin | Other | 16.0 GB | dcab6fff3701 |
| pre-training-tokens/2020-05.bin | Other | 16.0 GB | 6e7c8c2a7909 |
| pre-training-tokens/2020-06.bin | Other | 16.0 GB | b52dce8828d1 |
| .gitattributes | Repository | 2.5 KB | — |
License and Download
- License
- Not stated by the source
- Access
- No access gate
Download from Johannes Schwab
Released by Johannes Schwab through its official repository on Hugging Face.