This llama model was trained 2x faster with Unsloth and Huggingface's TRL library.
gpt2 is an open-weight model for text generation from OpenAI community, released under MIT License. It has 137M parameters. At 16-bit it needs about 0.3 GB of GPU memory, which fits on 1x MI300X from $1.85 an hour, at the lowest prices in the SAVRN Index. It draws 16.1M downloads a month.
Test the whole generation capabilities here: https://transformer.huggingface.co/doc/gpt2-large Pretrained model on English language using a causal language modeling (CLM) objective. It was introduced in and first released at this page. model.
Parameters137M
Context—
Weights5.6 GB
Licensemit
AccessOpen weights
Monthly Downloads16.1M
Runs On
What it takes to serve gpt2 (137M parameters): the memory its weights need at each precision, and the cheapest way to rent enough data-center GPUs to hold them.
| Precision | Weights | Memory needed | Cheapest setup | Per hour | Also fits |
|---|---|---|---|---|---|
| 16-bit | 0.3 GB | 0.3 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
| 8-bit | 0.1 GB | 0.2 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
| 4-bit | 0.1 GB | 0.1 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
Memory is the weights at that precision plus 20% for the runtime and a short context; a long context needs more. Prices are the lowest on-demand hourly rates in the SAVRN Index, read Oct 16, 2026.
gpt2 on every accelerator the SAVRN Index prices, at every precision
SAVRN's Notes on gpt2
Seven formats ship in this repository: safetensors, ONNX, PyTorch, TFLite, JAX, Rust and TensorFlow. That is why 26 files add up to 5,632,417,295 bytes for a 137M-parameter text generator, and why a mirror should hold only the format your serving stack loads. It was pretrained with a causal language modeling objective on raw English text, no human labeling. At 16-bit, weights and working memory are each 0.3 GB; on the cheapest setup we list, an MI300X with 192 GB at $1.85 per hour, the model is a rounding error and you size the card for whatever runs beside it.
MIT asks one thing in return for commercial use, modification and redistribution: the copyright and permission notices travel with every copy. Before committing, confirm the context window, which the listing does not state, and know that it records no reported evaluations and no per-token host price on the SAVRN Index.
Model Card
By OpenAI community, published under mit, revision 607a30d783df.
Test the whole generation capabilities here: https://transformer.huggingface.co/doc/gpt2-large
Pretrained model on English language using a causal language modeling (CLM) objective. It was introduced in this paper and first released at this page.
Disclaimer: The team releasing GPT-2 also wrote a model card for their model. Content from this model card has been written by the Hugging Face team to complete the information they provided and give specific examples of bias.
Model description
GPT-2 is a transformers model pretrained on a very large corpus of English data in a self-supervised fashion. This means it was pretrained on the raw texts only, with no humans labelling them in any way (which is why it can use lots of publicly available data) with an automatic process to generate inputs and labels from those texts. More precisely, it was trained to guess the next word in sentences.
Configuration
- Architecture
- GPT2LMHeadModel
- Vocabulary size
- 50,257
- Model type
- gpt2
Identity and Version
- Repository
- openai-community/gpt2
- Publisher
- OpenAI community
- Task
- Text generation
- Modality
- Text
- Library
- transformers
- Parameters
- 137M parameters
- Languages
- en
- Revision
- 607a30d783dfa663caf39e06633721c8d4cfcd7e
- First published
- 2022-03-02
- Last updated
- 2024-02-19
Files and Weights
26 files, 5.6 GB in total. The weights are 11 files totalling 5.6 GB in bin, h5, msgpack, onnx, ot, safetensors, tflite.
Weights11 files · 5.6 GB
Configuration5 files · 1.9 KB
Tokenizer8 files · 6.2 MB
Documentation1 file · 8.1 KB
Repository1 file · 445 B
Every file
| File | Type | Size | SHA-256 |
|---|---|---|---|
| 64-8bits.tflite | Weights | 125.2 MB | c966da3b7469 |
| 64-fp16.tflite | Weights | 248.3 MB | 1ceafd82e733 |
| 64.tflite | Weights | 495.8 MB | cfcd510b239d |
| flax_model.msgpack | Weights | 497.8 MB | 192e8257ae9e |
| model.safetensors | Weights | 548.1 MB | 248dfc391186 |
| onnx/decoder_model.onnx | Weights | 653.7 MB | e3fc9615868f |
| onnx/decoder_model_merged.onnx | Weights | 655.2 MB | e6fc046fe5a7 |
| onnx/decoder_with_past_model.onnx | Weights | 653.7 MB | 570d958241de |
| pytorch_model.bin | Weights | 548.1 MB | 7c5d3f4b8b76 |
| rust_model.ot | Weights | 702.5 MB | adf0adedbf40 |
| tf_model.h5 | Weights | 497.9 MB | d08c1307f7df |
| config.json | Configuration | 665 B | — |
| generation_config.json | Configuration | 124 B | — |
| onnx/config.json | Configuration | 879 B | — |
| onnx/generation_config.json | Configuration | 119 B | — |
| onnx/special_tokens_map.json | Configuration | 99 B | — |
| README.md | Documentation | 8.1 KB | — |
| .gitattributes | Repository | 445 B | — |
| merges.txt | Tokenizer | 456.3 KB | — |
| onnx/merges.txt | Tokenizer | 456.3 KB | — |
| onnx/tokenizer.json | Tokenizer | 2.1 MB | — |
| onnx/tokenizer_config.json | Tokenizer | 234 B | — |
| onnx/vocab.json | Tokenizer | 798.2 KB | — |
| tokenizer.json | Tokenizer | 1.4 MB | — |
| tokenizer_config.json | Tokenizer | 26 B | — |
| vocab.json | Tokenizer | 1.0 MB | — |
License and Download
- License
- mit
- Access
- Open weights, no gate
- Download size
- 5.6 GB
Download from OpenAI community
Released by OpenAI community through its official repository on Hugging Face. Read the license.
Memory Requirements
| Precision | Weights in memory |
|---|---|
| As published | 5.6 GB |
| 16-bit | 0.3 GB |
| 8-bit | 0.1 GB |
| 4-bit | 0.1 GB |
Weights only, from the published parameter count; the key-value cache and runtime add to this.
Built on This Model
- Derived fromgpt2-small-smoltalk
- Derived fromgpt2-small-ag-news
Compare gpt2
Questions About gpt2
How much GPU memory does gpt2 need?
About 0.3 GB at 16-bit and 0.1 GB at 4-bit: the weights (137M parameters) plus a working margin. A long context needs more.
What is the cheapest GPU to run gpt2 on?
At 16-bit, 1x MI300X from $1.85 an hour; at 4-bit, 1x MI300X from $1.85 an hour, at the lowest on-demand prices the SAVRN Index lists.
Can I use gpt2 commercially?
Yes. gpt2 is released under MIT License. The MIT License is a short permissive license. It permits commercial use, modification and redistribution, provided the copyright notice and permission notice are included.
Similar Models
This llama model was trained 2x faster with Unsloth and Huggingface's TRL library.
SmolLM2 is a family of compact language models available in three size: 135M, 360M, and 1.7B parameters. They are capable of solving a wide range of tasks while being lightweight enough to run on-device. More details in our paper: https://arxiv.org/abs/2502.02737 SmolLM2 demonstrates significant advances over its predecessor SmolLM1, particularly in instruction following, knowledge, reasoning. The 135M model was trained on 2 trillion tokens using a diverse dataset combination: FineWeb-Edu, DCLM, The Stack, along with new filtered datasets we curated and will release soon. We developed the instruct version through supervised fine-tuning (SFT) using a combination of public datasets and our…
SmolLM2 is a family of compact language models available in three size: 135M, 360M, and 1.7B parameters. They are capable of solving a wide range of tasks while being lightweight enough to run on-device. More details in our paper https://arxiv.org/abs/2502.02737 SmolLM2 demonstrates significant advances over its predecessor SmolLM1, particularly in instruction following, knowledge, reasoning. The 135M model was trained on 2 trillion tokens using a diverse dataset combination: FineWeb-Edu, DCLM, The Stack, along with new filtered datasets we curated and will release soon. We developed the instruct version through supervised fine-tuning (SFT) using a combination of public datasets and our own…
This model is a fine-tuned version of HuggingFaceTB/SmolLM2-135M-Instruct on an unknown dataset. The following hyperparameters were used during training: - learningrate: 2e-05 - trainbatchsize: 1 - evalbatchsize: 1 - gradientaccumulationsteps: 4 - totaltrainbatchsize: 4 - lrschedulertype: linear - numepochs: 1 - Transformers 5.17.0 - Pytorch 2.11.0+cu130 - Datasets 4.8.5 - Tokenizers 0.23.2
This model is a fine-tuned version of HuggingFaceTB/SmolLM2-135M on an unknown dataset. It achieves the following results on the evaluation set: The following hyperparameters were used during training: - learningrate: 5e-05 - trainbatchsize: 4 - evalbatchsize: 8 - lrschedulertype: linear - trainingsteps: 1000 - Transformers 4.46.3 - Pytorch 2.11.0+cu130 - Datasets 4.8.5 - Tokenizers 0.20.3