# gpt2 by OpenAI community: Open-Weight Model Source: https://savrn.com/models/gpt2 Markdown alternate of the page above; the site index is https://savrn.com/llms.txt --- ## Runs On What it takes to serve gpt2 (137M parameters): the memory its weights need at each precision, and the cheapest way to rent enough data-center GPUs to hold them. | Precision | Weights | Memory needed | Cheapest setup | Per hour | Also fits | | --- | --- | --- | --- | --- | --- | | 16-bit | 0.3 GB | 0.3 GB | 1x [MI300X](https://savrn.com/ai-index/pricing/gpus/mi300x) (192 GB) Vultr | $1.85 | [1x H100](https://savrn.com/ai-index/pricing/gpus/h100) $1.99 · [1x MI325X](https://savrn.com/ai-index/pricing/gpus/mi325x) $2.00 | | 8-bit | 0.1 GB | 0.2 GB | 1x [MI300X](https://savrn.com/ai-index/pricing/gpus/mi300x) (192 GB) Vultr | $1.85 | [1x H100](https://savrn.com/ai-index/pricing/gpus/h100) $1.99 · [1x MI325X](https://savrn.com/ai-index/pricing/gpus/mi325x) $2.00 | | 4-bit | 0.1 GB | 0.1 GB | 1x [MI300X](https://savrn.com/ai-index/pricing/gpus/mi300x) (192 GB) Vultr | $1.85 | [1x H100](https://savrn.com/ai-index/pricing/gpus/h100) $1.99 · [1x MI325X](https://savrn.com/ai-index/pricing/gpus/mi325x) $2.00 | Memory is the weights at that precision plus 20% for the runtime and a short context; a long context needs more. Prices are the lowest on-demand hourly rates in the [SAVRN Index](https://savrn.com/ai-index/pricing/gpus), read Oct 23, 2026. [gpt2 on every accelerator the SAVRN Index prices, at every precision](https://savrn.com/models/gpt2/gpus) ## SAVRN's Notes on gpt2 Seven formats ship in this repository: safetensors, ONNX, PyTorch, TFLite, JAX, Rust and TensorFlow. That is why 26 files add up to 5,632,417,295 bytes for a 137M-parameter text generator, and why a mirror should hold only the format your serving stack loads. It was pretrained with a causal language modeling objective on raw English text, no human labeling. At 16-bit, weights and working memory are each 0.3 GB; on the cheapest setup we list, an MI300X with 192 GB at $1.85 per hour, the model is a rounding error and you size the card for whatever runs beside it. MIT asks one thing in return for commercial use, modification and redistribution: the copyright and permission notices travel with every copy. Before committing, confirm the context window, which the listing does not state, and know that it records no reported evaluations and no per-token host price on the SAVRN Index. SAVRN Research, 2026-09-18 ## Model Card By OpenAI community, published under mit, revision 607a30d783df. Test the whole generation capabilities here: https://transformer.huggingface.co/doc/gpt2-large Pretrained model on English language using a causal language modeling (CLM) objective. It was introduced in [this paper](https://d4mucfpksywv.cloudfront.net/better-language-models/language_models_are_unsupervised_multitask_learners.pdf) and first released at [this page](https://openai.com/blog/better-language-models/). Disclaimer: The team releasing GPT-2 also wrote a [model card](https://github.com/openai/gpt-2/blob/master/model_card.md) for their model. Content from this model card has been written by the Hugging Face team to complete the information they provided and give specific examples of bias. ### Model description GPT-2 is a transformers model pretrained on a very large corpus of English data in a self-supervised fashion. This means it was pretrained on the raw texts only, with no humans labelling them in any way (which is why it can use lots of publicly available data) with an automatic process to generate inputs and labels from those texts. More precisely, it was trained to guess the next word in sentences. [Read the full model card (1,065 words)](https://savrn.com/models/gpt2/card) ## Configuration Architecture GPT2LMHeadModel Vocabulary size 50,257 Model type gpt2 ## Identity and Version Repository openai-community/gpt2 Publisher OpenAI community Task Text generation Modality Text Library transformers Parameters 137M parameters Languages en Revision 607a30d783dfa663caf39e06633721c8d4cfcd7e First published 2022-03-02 Last updated 2024-02-19 ## Files and Weights 26 files, 5.6 GB in total. The weights are 11 files totalling 5.6 GB in bin, h5, msgpack, onnx, ot, safetensors, tflite. Weights11 files · 5.6 GB Configuration5 files · 1.9 KB Tokenizer8 files · 6.2 MB Documentation1 file · 8.1 KB Repository1 file · 445 B Every file | File | Type | Size | SHA-256 | | --- | --- | --- | --- | | 64-8bits.tflite | Weights | 125.2 MB | c966da3b7469 | | 64-fp16.tflite | Weights | 248.3 MB | 1ceafd82e733 | | 64.tflite | Weights | 495.8 MB | cfcd510b239d | | flax_model.msgpack | Weights | 497.8 MB | 192e8257ae9e | | model.safetensors | Weights | 548.1 MB | 248dfc391186 | | onnx/decoder_model.onnx | Weights | 653.7 MB | e3fc9615868f | | onnx/decoder_model_merged.onnx | Weights | 655.2 MB | e6fc046fe5a7 | | onnx/decoder_with_past_model.onnx | Weights | 653.7 MB | 570d958241de | | pytorch_model.bin | Weights | 548.1 MB | 7c5d3f4b8b76 | | rust_model.ot | Weights | 702.5 MB | adf0adedbf40 | | tf_model.h5 | Weights | 497.9 MB | d08c1307f7df | | config.json | Configuration | 665 B | — | | generation_config.json | Configuration | 124 B | — | | onnx/config.json | Configuration | 879 B | — | | onnx/generation_config.json | Configuration | 119 B | — | | onnx/special_tokens_map.json | Configuration | 99 B | — | | README.md | Documentation | 8.1 KB | — | | .gitattributes | Repository | 445 B | — | | merges.txt | Tokenizer | 456.3 KB | — | | onnx/merges.txt | Tokenizer | 456.3 KB | — | | onnx/tokenizer.json | Tokenizer | 2.1 MB | — | | onnx/tokenizer_config.json | Tokenizer | 234 B | — | | onnx/vocab.json | Tokenizer | 798.2 KB | — | | tokenizer.json | Tokenizer | 1.4 MB | — | | tokenizer_config.json | Tokenizer | 26 B | — | | vocab.json | Tokenizer | 1.0 MB | — | ## License and Download License mit Access Open weights, no gate Download size 5.6 GB [Download from OpenAI community](https://huggingface.co/openai-community/gpt2) Released by OpenAI community through its official repository on Hugging Face. [Read the license](https://opensource.org/license/mit). ## Memory Requirements | Precision | Weights in memory | | --- | --- | | As published | 5.6 GB | | 16-bit | 0.3 GB | | 8-bit | 0.1 GB | | 4-bit | 0.1 GB | Weights only, from the published parameter count; the key-value cache and runtime add to this. ## Built on This Model - Derived from[gpt2-small-smoltalk](https://savrn.com/models/gpt2-small-smoltalk) - Derived from[gpt2-small-ag-news](https://savrn.com/models/gpt2-small-ag-news) ## Compare gpt2 - [gpt2 vs macbert4csc-base-chinese](https://savrn.com/models/compare/gpt2-vs-macbert4csc-base-chinese) - [gpt2 vs SmolLM2-135M](https://savrn.com/models/compare/gpt2-vs-smollm2-135m) - [gpt2 vs SmolLM2-135M-Instruct](https://savrn.com/models/compare/gpt2-vs-smollm2-135m-instruct) ## Questions About gpt2 ### How much GPU memory does gpt2 need? About 0.3 GB at 16-bit and 0.1 GB at 4-bit: the weights (137M parameters) plus a working margin. A long context needs more. ### What is the cheapest GPU to run gpt2 on? At 16-bit, 1x MI300X from $1.85 an hour; at 4-bit, 1x MI300X from $1.85 an hour, at the lowest on-demand prices the SAVRN Index lists. ### Can I use gpt2 commercially? Yes. gpt2 is released under MIT License. The MIT License is a short permissive license. It permits commercial use, modification and redistribution, provided the copyright notice and permission notice are included. ## Similar Models Model · Text generation ### [Smol-GPT-Fp8](https://savrn.com/models/smol-gpt-fp8) [Gggff123](https://savrn.com/model-publishers/gggff123) This llama model was trained 2x faster with Unsloth and Huggingface's TRL library. Open weights apache-2.0 135M parameters 8,192 tokens transformers [View model](https://savrn.com/models/smol-gpt-fp8) Model · Text generation ### [Smol-GPT-RAW](https://savrn.com/models/smol-gpt-raw) [Gggff123](https://savrn.com/model-publishers/gggff123) This llama model was trained 2x faster with Unsloth and Huggingface's TRL library. Open weights apache-2.0 135M parameters 8,192 tokens transformers [View model](https://savrn.com/models/smol-gpt-raw) Model · Text generation ### [SmolLM2-135M](https://savrn.com/models/smollm2-135m) [Hugging Face Smol Models Research](https://savrn.com/model-publishers/huggingfacetb) SmolLM2 is a family of compact language models available in three size: 135M, 360M, and 1.7B parameters. They are capable of solving a wide range of tasks while being lightweight enough to run on-device. More details in our paper: https://arxiv.org/abs/2502.02737 SmolLM2 demonstrates significant advances over its predecessor SmolLM1, particularly in instruction following, knowledge, reasoning. The 135M model was trained on 2 trillion tokens using a diverse dataset combination: FineWeb-Edu, DCLM, The Stack, along with new filtered datasets we curated and will release soon. We developed the instruct version through supervised fine-tuning (SFT) using a combination of public datasets and our… Open weights apache-2.0 135M parameters 8,192 tokens transformers [View model](https://savrn.com/models/smollm2-135m) Model · Text generation ### [SmolLM2-135M-Instruct](https://savrn.com/models/smollm2-135m-instruct) [Hugging Face Smol Models Research](https://savrn.com/model-publishers/huggingfacetb) SmolLM2 is a family of compact language models available in three size: 135M, 360M, and 1.7B parameters. They are capable of solving a wide range of tasks while being lightweight enough to run on-device. More details in our paper https://arxiv.org/abs/2502.02737 SmolLM2 demonstrates significant advances over its predecessor SmolLM1, particularly in instruction following, knowledge, reasoning. The 135M model was trained on 2 trillion tokens using a diverse dataset combination: FineWeb-Edu, DCLM, The Stack, along with new filtered datasets we curated and will release soon. We developed the instruct version through supervised fine-tuning (SFT) using a combination of public datasets and our own… Open weights apache-2.0 135M parameters 8,192 tokens transformers [View model](https://savrn.com/models/smollm2-135m-instruct) Model · Text generation ### [tuned-SmolLM2-135M-Instruct-test-s](https://savrn.com/models/tuned-smollm2-135m-instruct-test-s) [Zied Ben hadj Amor](https://savrn.com/model-publishers/mrbhazied) This model is a fine-tuned version of HuggingFaceTB/SmolLM2-135M-Instruct on an unknown dataset. The following hyperparameters were used during training: - learningrate: 2e-05 - trainbatchsize: 1 - evalbatchsize: 1 - gradientaccumulationsteps: 4 - totaltrainbatchsize: 4 - lrschedulertype: linear - numepochs: 1 - Transformers 5.17.0 - Pytorch 2.11.0+cu130 - Datasets 4.8.5 - Tokenizers 0.23.2 Open weights apache-2.0 135M parameters 8,192 tokens transformers [View model](https://savrn.com/models/tuned-smollm2-135m-instruct-test-s) Model · Text generation ### [SmolLM2-FT-MyDataset](https://savrn.com/models/smollm2-ft-mydataset) [Roy Mazumder](https://savrn.com/model-publishers/aritroroy0) This model is a fine-tuned version of HuggingFaceTB/SmolLM2-135M on an unknown dataset. It achieves the following results on the evaluation set: The following hyperparameters were used during training: - learningrate: 5e-05 - trainbatchsize: 4 - evalbatchsize: 8 - lrschedulertype: linear - trainingsteps: 1000 - Transformers 4.46.3 - Pytorch 2.11.0+cu130 - Datasets 4.8.5 - Tokenizers 0.20.3 Open weights apache-2.0 135M parameters 8,192 tokens transformers [View model](https://savrn.com/models/smollm2-ft-mydataset) ## OpenAI community [All models and datasets](https://savrn.com/model-publishers/openai-community) ## Versions - [607a30d783df](https://savrn.com/models/gpt2/versions/607a30d783df) · current 2026-10-23 ## Explore More - [All text generation models](https://savrn.com/models/tasks/text-generation) - [All models under mit](https://savrn.com/models/licenses/mit) - [Model comparisons](https://savrn.com/models/comparisons) - [The model directory](https://savrn.com/models) - [Open model prices by host](https://savrn.com/ai-index/pricing/open-models) ## Source - Repository metadata, read 2026-10-19. - [Hugging Face record](https://huggingface.co/openai-community/gpt2) - [How the hub is built](https://savrn.com/model-hub/methodology)