This public pilot contains packed uint16 token IDs for pretraining the Tiny Base 300M, 2,048-context language model. It contains 50,000,000 training, 1,000,000 validation, and 1,000,000 test tokens. It does not redistribute raw documents.
Dataset Card
This public pilot contains packed uint16 token IDs for pretraining the Tiny Base 300M, 2,048-context language model. It contains 50,000,000 training, 1,000,000 validation, and 1,000,000 test tokens. It does not redistribute raw documents. train.bin, validation.bin, and test.bin are little-endian uint16 streams. Use tokenizer.json with the Hugging Face tokenizers library. manifest.json records exact token counts, split policy, source revisions, mixture, and SHA-256 checksums; settings.json records preparation settings. The deterministic split is SHA-256 of normalized text modulo 1,000: 0–9 test, 10–19 validation, and 20–999 training. The mixture is 70% FineWeb-Edu, 25% English Wikipedia, and…
Excerpt from the card by BJY, licensed other.
Details
- Repository
- helloboy91/tiny-base-300m-data
- Publisher
- BJY
- Task category
- Not stated by the source
- Tags
- pretraining, tokenized, english
- Size category
- n<1GB
- Languages
- en
- Revision
- 385b4eaec80f628d96bc08d77b02109bd76f3527
- Last updated
- 2026-09-18
Files
8 files, 106.3 MB in total.
Every file
| File | Type | Size | SHA-256 |
|---|---|---|---|
| manifest.json | Data | 1.6 KB | — |
| settings.json | Data | 649 B | — |
| README.md | Documentation | 1.8 KB | — |
| test.bin | Other | 2.0 MB | e85fb2f775c5 |
| train.bin | Other | 100.0 MB | 8f18a017f3bb |
| validation.bin | Other | 2.0 MB | 252bf6986e18 |
| .gitattributes | Repository | 2.5 KB | — |
| tokenizer.json | Tokenizer | 2.3 MB | — |
License and Download
- License
- other
- Access
- No access gate
Released by BJY through its official repository on Hugging Face.