This llama model was trained 2x faster with Unsloth and Huggingface's TRL library.
Smol-GPT-Fp8 is an open-weight model for text generation from Gggff123, released under Apache License 2.0. It has 135M parameters and a 8,192-token context. At 16-bit it needs about 0.3 GB of GPU memory, which fits on 1x MI300X from $1.85 an hour, at the lowest prices in the SAVRN Index.
This llama model was trained 2x faster with Unsloth and Huggingface's TRL library.
Parameters135M
Context8,192
Weights163.2 MB
Licenseapache-2.0
AccessOpen weights
Monthly Downloads—
Runs On
What it takes to serve Smol-GPT-Fp8 (135M parameters): the memory its weights need at each precision, and the cheapest way to rent enough data-center GPUs to hold them.
| Precision | Weights | Memory needed | Cheapest setup | Per hour | Also fits |
|---|---|---|---|---|---|
| 16-bit | 0.3 GB | 0.3 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
| 8-bit | 0.1 GB | 0.2 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
| 4-bit | 0.1 GB | 0.1 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
Memory is the weights at that precision plus 20% for the runtime and a short context; a long context needs more. Prices are the lowest on-demand hourly rates in the SAVRN Index, read Oct 18, 2026.
Smol-GPT-Fp8 on every accelerator the SAVRN Index prices, at every precision
Model Card
By Gggff123, published under apache-2.0, revision 9f2956a09899.
This llama model was trained 2x faster with Unsloth and Huggingface's TRL library.
Read Gggff123's full model card
Uploaded finetuned fp8 model
- Developed by: gggff123
- License: apache-2.0
- Finetuned from model : unsloth/SmolLM2-135M-Instruct
This llama model was trained 2x faster with Unsloth and Huggingface's TRL library.
Configuration
- Architecture
- LlamaForCausalLM
- Context length (tokens)
- 8,192
- Layers
- 30
- Hidden size
- 576
- Feed-forward size
- 1,536
- Attention heads
- 9
- Key/value heads
- 3
- Head dimension
- 64
- Vocabulary size
- 49,153
- Model type
- llama
- Quantization
- compressed-tensors
Identity and Version
- Repository
- gggff123/Smol-GPT-Fp8
- Publisher
- Gggff123
- Task
- Text generation
- Modality
- Text
- Library
- transformers
- Parameters
- 135M parameters
- Languages
- en
- Revision
- 9f2956a098998f1b8df7741cd00a71abf5e9084e
- First published
- 2026-10-12
- Last updated
- 2026-10-12
Files and Weights
9 files, 166.8 MB in total. The weights are 1 file totalling 163.2 MB in safetensors.
Weights1 file · 163.2 MB
Configuration3 files · 2.6 KB
Tokenizer2 files · 3.5 MB
Documentation1 file · 586 B
Other1 file · 368 B
Repository1 file · 1.5 KB
Every file
| File | Type | Size | SHA-256 |
|---|---|---|---|
| model.safetensors | Weights | 163.2 MB | b0c45ec2753e |
| config.json | Configuration | 2.2 KB | — |
| generation_config.json | Configuration | 158 B | — |
| recipe.yaml | Configuration | 237 B | — |
| README.md | Documentation | 586 B | — |
| chat_template.jinja | Other | 368 B | — |
| .gitattributes | Repository | 1.5 KB | — |
| tokenizer.json | Tokenizer | 3.5 MB | — |
| tokenizer_config.json | Tokenizer | 475 B | — |
License and Download
- License
- apache-2.0
- Access
- Open weights, no gate
- Download size
- 163.2 MB
Download from Gggff123
Released by Gggff123 through its official repository on Hugging Face. Read the license.
Built From
- Derived from unsloth/SmolLM2-135M-Instruct
- Quantized from unsloth/SmolLM2-135M-Instruct
Memory Requirements
| Precision | Weights in memory |
|---|---|
| As published | 163.2 MB |
| 16-bit | 0.3 GB |
| 8-bit | 0.1 GB |
| 4-bit | 0.1 GB |
Weights only, from the published parameter count; the key-value cache and runtime add to this.
Questions About Smol-GPT-Fp8
How much GPU memory does Smol-GPT-Fp8 need?
About 0.3 GB at 16-bit and 0.1 GB at 4-bit: the weights (135M parameters) plus a working margin. A long context needs more.
What is the cheapest GPU to run Smol-GPT-Fp8 on?
At 16-bit, 1x MI300X from $1.85 an hour; at 4-bit, 1x MI300X from $1.85 an hour, at the lowest on-demand prices the SAVRN Index lists.
Can I use Smol-GPT-Fp8 commercially?
Yes. Smol-GPT-Fp8 is released under Apache License 2.0. The Apache License 2.0 is a permissive open-source license. It permits commercial use, modification and redistribution. It requires keeping the license and copyright notices and any NOTICE file, stating significant changes, and it includes an express patent grant from contributors.
What is Smol-GPT-Fp8's context length?
8,192 tokens, from the maximum position embeddings in its published configuration.
Similar Models
SmolLM2 is a family of compact language models available in three size: 135M, 360M, and 1.7B parameters. They are capable of solving a wide range of tasks while being lightweight enough to run on-device. More details in our paper: https://arxiv.org/abs/2502.02737 SmolLM2 demonstrates significant advances over its predecessor SmolLM1, particularly in instruction following, knowledge, reasoning. The 135M model was trained on 2 trillion tokens using a diverse dataset combination: FineWeb-Edu, DCLM, The Stack, along with new filtered datasets we curated and will release soon. We developed the instruct version through supervised fine-tuning (SFT) using a combination of public datasets and our…
SmolLM2 is a family of compact language models available in three size: 135M, 360M, and 1.7B parameters. They are capable of solving a wide range of tasks while being lightweight enough to run on-device. More details in our paper https://arxiv.org/abs/2502.02737 SmolLM2 demonstrates significant advances over its predecessor SmolLM1, particularly in instruction following, knowledge, reasoning. The 135M model was trained on 2 trillion tokens using a diverse dataset combination: FineWeb-Edu, DCLM, The Stack, along with new filtered datasets we curated and will release soon. We developed the instruct version through supervised fine-tuning (SFT) using a combination of public datasets and our own…
This model is a fine-tuned version of HuggingFaceTB/SmolLM2-135M-Instruct on an unknown dataset. The following hyperparameters were used during training: - learningrate: 2e-05 - trainbatchsize: 1 - evalbatchsize: 1 - gradientaccumulationsteps: 4 - totaltrainbatchsize: 4 - lrschedulertype: linear - numepochs: 1 - Transformers 5.17.0 - Pytorch 2.11.0+cu130 - Datasets 4.8.5 - Tokenizers 0.23.2
This model is a fine-tuned version of HuggingFaceTB/SmolLM2-135M on an unknown dataset. It achieves the following results on the evaluation set: The following hyperparameters were used during training: - learningrate: 5e-05 - trainbatchsize: 4 - evalbatchsize: 8 - lrschedulertype: linear - trainingsteps: 1000 - Transformers 4.46.3 - Pytorch 2.11.0+cu130 - Datasets 4.8.5 - Tokenizers 0.20.3
Test the whole generation capabilities here: https://transformer.huggingface.co/doc/gpt2-large Pretrained model on English language using a causal language modeling (CLM) objective. It was introduced in and first released at this page. model. Content from this model card has been written by the Hugging Face team to complete the information they provided and give specific examples of bias. GPT-2 is a transformers model pretrained on a very large corpus of English data in a self-supervised fashion. This means it was pretrained on the raw texts only, with no humans labelling them in any way (which is why it can use lots of publicly available data) with an automatic process to generate inputs…