Skip to page content
SAVRN
Search Plan a project

Open-weight model · Text generation

TeenyTinyStories-1M

by YF ableian/TeenyTinyStories-1M

TeenyTinyStories-1M is an open-weight model for text generation from YF, released under Apache License 2.0. It has 4M parameters and a 2,048-token context. At 16-bit it needs about 0 GB of GPU memory, which fits on 1x MI300X from $1.85 an hour, at the lowest prices in the SAVRN Index.

GPT-Neo storyteller (nominal 1M; actual 3,745,984 parameters including the 50,257-token embedding) trained on TeenyTinyStories (TTS): 2,491,490 stories (498.5M tokens with EOS) generated by roneneldan/TinyStories-Instruct-33M.

Parameters4M
Context2,048
Weights15.0 MB
Licenseapache-2.0
AccessOpen weights
Monthly Downloads—

Runs On

What it takes to serve TeenyTinyStories-1M (4M parameters): the memory its weights need at each precision, and the cheapest way to rent enough data-center GPUs to hold them.

PrecisionWeightsMemory neededCheapest setupPer hourAlso fits
16-bit 0.0 GB 0.0 GB 1x MI300X (192 GB)
Vultr
$1.85 1x H100 $1.99 · 1x MI325X $2.00
8-bit 0.0 GB 0.0 GB 1x MI300X (192 GB)
Vultr
$1.85 1x H100 $1.99 · 1x MI325X $2.00
4-bit 0.0 GB 0.0 GB 1x MI300X (192 GB)
Vultr
$1.85 1x H100 $1.99 · 1x MI325X $2.00

Memory is the weights at that precision plus 20% for the runtime and a short context; a long context needs more. Prices are the lowest on-demand hourly rates in the SAVRN Index, read Oct 18, 2026.

TeenyTinyStories-1M on every accelerator the SAVRN Index prices, at every precision

Model Card

By YF, published under apache-2.0, revision 5bcd2553ae2b.

GPT-Neo storyteller (nominal 1M; actual 3,745,984 parameters including the 50,257-token embedding) trained on TeenyTinyStories (TTS): 2,491,490 stories (498.5M tokens with EOS) generated by roneneldan/TinyStories-Instruct-33M. batch 1280 x 512, constant LR 5e-4, weight decay 0.1, betas (0.9, 0.95), clip 1.0. - TS-valid capped to stories of at most 400 tokens (20,711 stories): 2.369 / ppl 10.69 Research on small-scale synthetic pretraining data. Generates simple children's stories; sample at temperature 0.8-1.0. See the TeenyTinyStories paper for the corpus recipe and judge evaluations. Apache-2.0.

Read YF's full model card

GPT-Neo storyteller (nominal 1M; actual 3,745,984 parameters including the 50,257-token embedding) trained on TeenyTinyStories (TTS): 2,491,490 stories (498.5M tokens with EOS) generated by roneneldan/TinyStories-Instruct-33M.

Training

  • Architecture: GPT-Neo, 8 layers, hidden 64, 16 heads, window 256, context 512, fp32.
  • Budget: 20 epochs over written train tokens (15,212 steps x 655,360 tokens = 9.97B tokens), batch 1280 x 512, constant LR 5e-4, weight decay 0.1, betas (0.9, 0.95), clip 1.0.
  • Tokenizer: EleutherAI/gpt-neo-125M (EOS 50256; one EOS per story).

Held-out loss (nats, full files)

  • TTS-val (20,000 stories): 1.987 / ppl 7.29
  • TS-valid capped to stories of at most 400 tokens (20,711 stories): 2.369 / ppl 10.69
  • TS-valid full (21,990 stories): 2.385 / ppl 10.86

Intended use

Research on small-scale synthetic pretraining data. Generates simple children's stories; sample at temperature 0.8-1.0. See the TeenyTinyStories paper for the corpus recipe and judge evaluations.

License

Apache-2.0.

Configuration

Architecture
GPTNeoForCausalLM
Context length (tokens)
2,048
Hidden size
64
Feed-forward size
256
Vocabulary size
50,257
Model type
gpt_neo

Identity and Version

Repository
ableian/TeenyTinyStories-1M
Publisher
YF
Task
Text generation
Modality
Text
Library
transformers
Parameters
4M parameters
Languages
gpt-neo
Revision
5bcd2553ae2b11935b1f760595f3b424205c3916
First published
2026-10-11
Last updated
2026-10-11

Files and Weights

7 files, 18.6 MB in total. The weights are 1 file totalling 15.0 MB in safetensors.

Weights1 file · 15.0 MB
Configuration2 files · 1.1 KB
Tokenizer2 files · 3.6 MB
Documentation1 file · 1.2 KB
Repository1 file · 1.5 KB
Every file
FileTypeSizeSHA-256
model.safetensorsWeights15.0 MB 12462c4f4736
config.jsonConfiguration868 B —
generation_config.jsonConfiguration203 B —
README.mdDocumentation1.2 KB —
.gitattributesRepository1.5 KB —
tokenizer.jsonTokenizer3.6 MB —
tokenizer_config.jsonTokenizer356 B —

License and Download

License
apache-2.0
Access
Open weights, no gate
Download size
15.0 MB
Download from YF

Released by YF through its official repository on Hugging Face. Read the license.

Memory Requirements

PrecisionWeights in memory
As published15.0 MB
16-bit0.0 GB
8-bit0.0 GB
4-bit0.0 GB

Weights only, from the published parameter count; the key-value cache and runtime add to this.

Questions About TeenyTinyStories-1M

How much GPU memory does TeenyTinyStories-1M need?

About 0 GB at 16-bit and 0 GB at 4-bit: the weights (4M parameters) plus a working margin. A long context needs more.

What is the cheapest GPU to run TeenyTinyStories-1M on?

At 16-bit, 1x MI300X from $1.85 an hour; at 4-bit, 1x MI300X from $1.85 an hour, at the lowest on-demand prices the SAVRN Index lists.

Can I use TeenyTinyStories-1M commercially?

Yes. TeenyTinyStories-1M is released under Apache License 2.0. The Apache License 2.0 is a permissive open-source license. It permits commercial use, modification and redistribution. It requires keeping the license and copyright notices and any NOTICE file, stating significant changes, and it includes an express patent grant from contributors.

What is TeenyTinyStories-1M's context length?

2,048 tokens, from the maximum position embeddings in its published configuration.

Similar Models

Model · Text generation

Ornith-1.0-9B

Ornith

Aloha! Today, we are releasing Ornith-1.0, a self-improving family of open-source models for agentic coding. This model card documents Ornith-1.0-9B, the most lightweight member of the Ornith family, designed for efficient single-GPU deployment. Ornith-1.0-9B is a dense ~9B model (≈19 GB in bf16), so it serves comfortably on a single 80GB GPU. The recipes below stand up an OpenAI-compatible server; add --tensor-parallel-size / --tp if you want to shard across more GPUs. For a quick local test (or to script offline generation), load the model directly with Transformers. Make sure you have a recent release installed — see the Transformers installation guide; Ornith-1.0-9B requires…

Open weights mit 1M parameters 262,144 tokens transformers

Model · Text generation

Mangaba-Nano-1M

Mangaba AI

Um modelo de linguagem de 1,4 milhão de parâmetros, treinado do zero em português do Brasil pela Mangaba AI. Relatório técnico completo (dados, arquitetura, Muon, ablações, controles e destilação): https://huggingface.co/spaces/mangaba-ai/mangaba-nano-artigo Não há modelo base: os pesos começaram aleatórios e o tokenizer também foi treinado do zero sobre o mesmo corpus. O treino rodou nas GPUs NVIDIA L40S da própria Mangaba, em datacenter no Brasil. É um modelo de pesquisa e de borda: cabe em 5,6 MB, roda em qualquer CPU e serve como base para estudar escalas minúsculas, fazer ajuste fino em tarefas estreitas (classificação, autocompletar, extração simples) ou rodar em microcontroladores e…

Open weights apache-2.0 1M parameters 512 tokens transformers

This package uses the standard Transformers Llama causal-language-model architecture with OpenWALDO's schema-1 byte tokenizer. Load the tokenizer with trustremotecode=True. BOM.json inventories every release file and EU-BOM.json contains the EU GPAI training-content disclosure mapping.

Open weights 820,736 parameters 512 tokens transformers

This package uses the standard Transformers Llama causal-language-model architecture with OpenWALDO's schema-1 byte tokenizer. Load the tokenizer with trustremotecode=True. BOM.json inventories every release file and EU-BOM.json contains the EU GPAI training-content disclosure mapping.

Open weights 820,736 parameters 512 tokens transformers

This package uses the standard Transformers Llama causal-language-model architecture with OpenWALDO's schema-1 byte tokenizer. Load the tokenizer with trustremotecode=True. BOM.json inventories every release file and EU-BOM.json contains the EU GPAI training-content disclosure mapping.

Open weights 820,736 parameters 512 tokens transformers

This package uses the standard Transformers Llama causal-language-model architecture with OpenWALDO's schema-1 byte tokenizer. Load the tokenizer with trustremotecode=True. BOM.json inventories every release file and EU-BOM.json contains the EU GPAI training-content disclosure mapping.

Open weights 820,736 parameters 512 tokens transformers