This model is a fine-tuned version of distilbert/distilgpt2 on an unknown dataset. It achieves the following results on the evaluation set: The following hyperparameters were used during training: - learningrate: 2e-05 - trainbatchsize: 8 - evalbatchsize: 8 - lrschedulertype: linear - numepochs: 3.0 - Transformers 5.18.0 - Pytorch 2.11.0+cu130 - Datasets 4.8.5 - Tokenizers 0.23.2
codeparrot-ds is an open-weight model for text generation from Ai Xin, released under Apache License 2.0. It has 82M parameters. At 16-bit it needs about 0.2 GB of GPU memory, which fits on 1x MI300X from $1.85 an hour, at the lowest prices in the SAVRN Index. It draws 19 downloads a month.
This model is a fine-tuned version of distilbert/distilgpt2 on an unknown dataset.
Parameters82M
Context—
Weights327.7 MB
Licenseapache-2.0
AccessOpen weights
Monthly Downloads19
Runs On
What it takes to serve codeparrot-ds (82M parameters): the memory its weights need at each precision, and the cheapest way to rent enough data-center GPUs to hold them.
| Precision | Weights | Memory needed | Cheapest setup | Per hour | Also fits |
|---|---|---|---|---|---|
| 16-bit | 0.2 GB | 0.2 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
| 8-bit | 0.1 GB | 0.1 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
| 4-bit | 0.0 GB | 0.0 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
Memory is the weights at that precision plus 20% for the runtime and a short context; a long context needs more. Prices are the lowest on-demand hourly rates in the SAVRN Index, read Oct 18, 2026.
codeparrot-ds on every accelerator the SAVRN Index prices, at every precision
Model Card
By Ai Xin, published under apache-2.0, revision ae48b5b50ac8.
This model is a fine-tuned version of distilbert/distilgpt2 on an unknown dataset. The following hyperparameters were used during training: - learningrate: 0.0005 - trainbatchsize: 32 - evalbatchsize: 32 - gradientaccumulationsteps: 8 - totaltrainbatchsize: 256 - lrschedulertype: cosine - lrschedulerwarmupsteps: 1000 - numepochs: 1 - mixedprecisiontraining: Native AMP - Transformers 4.57.1 - Pytorch 2.8.0+cu126 - Datasets 4.0.0 - Tokenizers 0.22.1
Read Ai Xin's full model card
This model is a fine-tuned version of distilbert/distilgpt2 on an unknown dataset.
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training: - learning_rate: 0.0005 - train_batch_size: 32 - eval_batch_size: 32 - seed: 42 - gradient_accumulation_steps: 8 - total_train_batch_size: 256 - optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments - lr_scheduler_type: cosine - lr_scheduler_warmup_steps: 1000 - num_epochs: 1 - mixed_precision_training: Native AMP
Training results
Framework versions
- Transformers 4.57.1
- Pytorch 2.8.0+cu126
- Datasets 4.0.0
- Tokenizers 0.22.1
Configuration
- Architecture
- GPT2LMHeadModel
- Vocabulary size
- 50,257
- Model type
- gpt2
Identity and Version
- Repository
- aixin2024/codeparrot-ds
- Publisher
- Ai Xin
- Task
- Text generation
- Modality
- Text
- Library
- transformers
- Parameters
- 82M parameters
- Languages
- Not stated by the source
- Revision
- ae48b5b50ac8e2b1420f701f8955885306058582
- First published
- 2025-09-16
- Last updated
- 2026-10-13
Files and Weights
17 files, 332.5 MB in total. The weights are 2 files totalling 327.7 MB in bin, safetensors.
Weights2 files · 327.7 MB
Configuration3 files · 1.3 KB
Tokenizer4 files · 4.8 MB
Documentation1 file · 1.3 KB
Other6 files · 33.7 KB
Repository1 file · 1.5 KB
Every file
| File | Type | Size | SHA-256 |
|---|---|---|---|
| model.safetensors | Weights | 327.7 MB | d94844276743 |
| training_args.bin | Weights | 5.8 KB | 4d5a5cfc1820 |
| config.json | Configuration | 1.1 KB | — |
| generation_config.json | Configuration | 144 B | — |
| special_tokens_map.json | Configuration | 131 B | — |
| README.md | Documentation | 1.3 KB | — |
| runs/Nov13_02-49-31_8810d63b899f/events.out.tfevents.1763002174.8810d63b899f.251.0 | Other | 5.7 KB | fe6483717be8 |
| runs/Nov14_07-15-40_f4714d55fb18/events.out.tfevents.1763104545.f4714d55fb18.731.0 | Other | 5.7 KB | e1f23af0678f |
| runs/Nov17_07-28-16_4fb57a97247b/events.out.tfevents.1763364532.4fb57a97247b.537.0 | Other | 5.7 KB | dc85e0afaa39 |
| runs/Sep16_03-27-01_6264c73e401e/events.out.tfevents.1757993227.6264c73e401e.178.0 | Other | 5.2 KB | 56517939031d |
| runs/Sep16_03-29-28_6264c73e401e/events.out.tfevents.1757993372.6264c73e401e.178.1 | Other | 5.6 KB | feadcb653c91 |
| runs/Sep17_03-30-07_c1da8971f0d5/events.out.tfevents.1758079814.c1da8971f0d5.1422.0 | Other | 5.7 KB | 8cbd3deb44c0 |
| .gitattributes | Repository | 1.5 KB | — |
| merges.txt | Tokenizer | 456.3 KB | — |
| tokenizer.json | Tokenizer | 3.6 MB | — |
| tokenizer_config.json | Tokenizer | 332 B | — |
| vocab.json | Tokenizer | 798.2 KB | — |
License and Download
- License
- apache-2.0
- Access
- Open weights, no gate
- Download size
- 327.7 MB
Download from Ai Xin
Released by Ai Xin through its official repository on Hugging Face. Read the license.
Built From
- Derived from distilbert/distilgpt2
Memory Requirements
| Precision | Weights in memory |
|---|---|
| As published | 327.7 MB |
| 16-bit | 0.2 GB |
| 8-bit | 0.1 GB |
| 4-bit | 0.0 GB |
Weights only, from the published parameter count; the key-value cache and runtime add to this.
Questions About codeparrot-ds
How much GPU memory does codeparrot-ds need?
About 0.2 GB at 16-bit and 0 GB at 4-bit: the weights (82M parameters) plus a working margin. A long context needs more.
What is the cheapest GPU to run codeparrot-ds on?
At 16-bit, 1x MI300X from $1.85 an hour; at 4-bit, 1x MI300X from $1.85 an hour, at the lowest on-demand prices the SAVRN Index lists.
Can I use codeparrot-ds commercially?
Yes. codeparrot-ds is released under Apache License 2.0. The Apache License 2.0 is a permissive open-source license. It permits commercial use, modification and redistribution. It requires keeping the license and copyright notices and any NOTICE file, stating significant changes, and it includes an express patent grant from contributors.
Similar Models
This model is a fine-tuned version of distilbert/distilgpt2 on the None dataset. It achieves the following results on the evaluation set: The following hyperparameters were used during training: - learningrate: 5e-05 - trainbatchsize: 4 - evalbatchsize: 4 - gradientaccumulationsteps: 4 - totaltrainbatchsize: 16 - lrschedulertype: linear - numepochs: 1 - mixedprecisiontraining: Native AMP - Transformers 5.17.0 - Pytorch 2.11.0+cu128 - Datasets 5.0.1 - Tokenizers 0.23.1
This model is a fine-tuned version of EleutherAI/pythia-160m on an unknown dataset. The following hyperparameters were used during training: - learningrate: 0.001 - trainbatchsize: 16 - evalbatchsize: 16 - gradientaccumulationsteps: 2 - totaltrainbatchsize: 32 - lrschedulertype: cosinewithminlr - lrschedulerwarmupsteps: 13 - trainingsteps: 250 - Transformers 5.4.0 - Pytorch 2.8.0+cu128 - Datasets 3.2.0 - Tokenizers 0.22.1
This model is a fine-tuned version of EleutherAI/pythia-160m on an unknown dataset. The following hyperparameters were used during training: - learningrate: 0.001 - trainbatchsize: 16 - evalbatchsize: 16 - gradientaccumulationsteps: 2 - totaltrainbatchsize: 32 - lrschedulertype: cosinewithminlr - lrschedulerwarmupsteps: 13 - trainingsteps: 250 - Transformers 5.4.0 - Pytorch 2.8.0+cu128 - Datasets 3.2.0 - Tokenizers 0.22.1
This model is a fine-tuned version of EleutherAI/pythia-160m on an unknown dataset. The following hyperparameters were used during training: - learningrate: 0.001 - trainbatchsize: 16 - evalbatchsize: 16 - gradientaccumulationsteps: 2 - totaltrainbatchsize: 32 - lrschedulertype: cosinewithminlr - lrschedulerwarmupsteps: 13 - trainingsteps: 250 - Transformers 5.4.0 - Pytorch 2.8.0+cu128 - Datasets 3.2.0 - Tokenizers 0.22.1
Inferno-Exp-1.0 is a 76.2M-parameter decoder-only base language model trained from scratch by AlgoDriveAI. It is a math-first model: its training data centers on arithmetic and math word problems, and it is geared toward the AxiomicLabs small-language-model benchmarks used in SLM challenges, ArithMark 3.0 and ArithMark 2.0 (results below). Future versions will build on this math foundation with improvements aimed at broader language understanding and logical reasoning. This is a base model (no instruction tuning). The model code ships in the repo, so load it with trustremotecode=True. All scores are zero-shot on the checkpoint in this repo. The ArithMark benchmarks were scored with…