SAVRN
Search Contact SAVRN

Open-weight model · Text generation

ppt-pythia-160m-uniform250-previous_mse_delta_shuffle2-seed324-stage2

by Qing Yao qing-yao/ppt-pythia-160m-uniform250-previous_mse_delta_shuffle2-seed324-stage2

ppt-pythia-160m-uniform250-previous_mse_delta_shuffle2-seed324-stage2 is an open-weight model for text generation from Qing Yao, released under Apache License 2.0. It has 162M parameters and a 2,048-token context. At 16-bit it needs about 0.4 GB of GPU memory, which fits on 1x MI300X from $1.85 an hour, at the lowest prices in the SAVRN Index.

This model is a fine-tuned version of qing-yao/ppt-pythia-160m-uniform250-previousmse-seed324-stage1 on the None dataset.

Parameters162M
Context2,048
Weights5.5 GB
Licenseapache-2.0
AccessOpen weights
Monthly Downloads—

Runs On

What it takes to serve ppt-pythia-160m-uniform250-previous_mse_delta_shuffle2-seed324-stage2 (162M parameters): the memory its weights need at each precision, and the cheapest way to rent enough data-center GPUs to hold them.

PrecisionWeightsMemory neededCheapest setupPer hourAlso fits
16-bit 0.3 GB 0.4 GB 1x MI300X (192 GB)
Vultr		 $1.85 1x H100 $1.99 · 1x MI325X $2.00
8-bit 0.2 GB 0.2 GB 1x MI300X (192 GB)
Vultr		 $1.85 1x H100 $1.99 · 1x MI325X $2.00
4-bit 0.1 GB 0.1 GB 1x MI300X (192 GB)
Vultr		 $1.85 1x H100 $1.99 · 1x MI325X $2.00

Memory is the weights at that precision plus 20% for the runtime and a short context; a long context needs more. Prices are the lowest on-demand hourly rates in the SAVRN Index, read Oct 7, 2026.

ppt-pythia-160m-uniform250-previous_mse_delta_shuffle2-seed324-stage2 on every accelerator the SAVRN Index prices, at every precision

Model Card

By Qing Yao, published under apache-2.0, revision 08460d2f78cf.

This model is a fine-tuned version of qing-yao/ppt-pythia-160m-uniform250-previousmse-seed324-stage1 on the None dataset. It achieves the following results on the evaluation set: The following hyperparameters were used during training: - learningrate: 0.001 - trainbatchsize: 16 - evalbatchsize: 16 - gradientaccumulationsteps: 2 - totaltrainbatchsize: 32 - lrschedulertype: cosinewithminlr - lrschedulerwarmupsteps: 500 - trainingsteps: 10000 - Transformers 5.4.0 - Pytorch 2.8.0+cu128 - Datasets 3.2.0 - Tokenizers 0.22.1

Read Qing Yao's full model card

This model is a fine-tuned version of qing-yao/ppt-pythia-160m-uniform250-previous_mse-seed324-stage1 on the None dataset. It achieves the following results on the evaluation set: - Loss: 3.8171

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training: - learning_rate: 0.001 - train_batch_size: 16 - eval_batch_size: 16 - seed: 324 - gradient_accumulation_steps: 2 - total_train_batch_size: 32 - optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments - lr_scheduler_type: cosine_with_min_lr - lr_scheduler_warmup_steps: 500 - training_steps: 10000

Training results

Training Loss Epoch Step Validation Loss
9.5865 0.005 50 8.1455
7.3765 0.01 100 7.0049
6.7283 0.015 150 6.5829
6.4370 0.02 200 6.3365
6.1904 0.025 250 6.1228
6.0573 0.03 300 6.1086
5.9210 0.035 350 5.8420
5.7340 0.04 400 5.6960
5.6474 0.045 450 5.5913
5.5772 0.05 500 5.5127
5.7459 0.055 550 8.1085
6.5197 0.06 600 5.8621
5.6604 0.065 650 5.5560
5.4758 0.07 700 5.4072
5.3584 0.075 750 5.3305
5.2942 0.08 800 5.2596
5.2382 0.085 850 5.2164
5.1961 0.09 900 5.2125
5.1578 0.095 950 5.1240
5.1514 0.1 1000 5.0813
5.0779 0.105 1050 5.0262
4.9820 0.11 1100 4.9507
4.9086 0.115 1150 4.9021
4.8718 0.12 1200 4.8517
4.8345 0.125 1250 4.8339
4.8015 0.13 1300 4.7665
4.7929 0.135 1350 4.7488
4.7247 0.14 1400 4.6973
4.7010 0.145 1450 4.6959
4.6841 0.15 1500 4.6283
4.6531 0.155 1550 4.6071
4.5922 0.16 1600 4.5890
4.6071 0.165 1650 4.5639
4.5784 0.17 1700 4.5306
4.5483 0.175 1750 4.5433
4.5381 0.18 1800 4.5229
4.5006 0.185 1850 4.5100
4.4911 0.19 1900 4.4655
4.4722 0.195 1950 4.4337
4.4441 0.2 2000 4.4134
4.4314 0.205 2050 4.3873
4.4174 0.21 2100 4.3662
4.4000 0.215 2150 4.3620
4.3919 0.22 2200 4.3632
4.3736 0.225 2250 4.3368
4.3667 0.23 2300 4.3192
4.3575 0.235 2350 4.3043
4.3599 0.24 2400 4.3039
4.3131 0.245 2450 4.2974
4.3520 0.25 2500 4.3056
4.3296 0.255 2550 4.2822
4.2822 0.26 2600 4.2595
4.2810 0.265 2650 4.2496
4.2736 0.27 2700 4.2396
4.2587 0.275 2750 4.2294
4.2563 0.28 2800 4.2284
4.2339 0.285 2850 4.2171
4.2316 0.29 2900 4.2104
4.1980 0.295 2950 4.2042
4.2380 0.3 3000 4.1880
4.1993 0.305 3050 4.2170
4.2239 0.31 3100 4.2077
4.2238 0.315 3150 4.1736
4.1933 0.32 3200 4.1461
4.1748 0.325 3250 4.1664
4.2004 0.33 3300 4.1359
4.1664 0.335 3350 4.1366
4.1853 0.34 3400 4.1139
4.1603 0.345 3450 4.1043
4.1464 0.35 3500 4.0947
4.1264 0.355 3550 4.0997
4.1267 0.36 3600 4.1105
4.1472 0.365 3650 4.1049
4.1207 0.37 3700 4.0824
4.1182 0.375 3750 4.1013
4.1842 0.38 3800 4.1200
4.1261 0.385 3850 4.1048
4.1228 0.39 3900 4.0828
4.1447 0.395 3950 4.0844
4.0855 0.4 4000 4.0642
4.0874 0.405 4050 4.0524
4.0917 0.41 4100 4.0420
4.0515 0.415 4150 4.0374
4.0575 0.42 4200 4.0443
4.0622 0.425 4250 4.0320
4.0444 0.43 4300 4.0225
4.0443 0.435 4350 4.0212
4.0480 0.44 4400 4.0481
4.0593 0.445 4450 4.0172
4.0532 0.45 4500 4.0124
4.0110 0.455 4550 4.0151
4.0156 0.46 4600 4.0035
4.0334 0.465 4650 3.9923
4.0288 0.47 4700 3.9943
4.0126 0.475 4750 3.9811
4.0061 0.48 4800 3.9761
4.0094 0.485 4850 3.9737
4.0078 0.49 4900 3.9692
3.9900 0.495 4950 3.9631
3.9917 0.5 5000 4.0633
4.0421 0.505 5050 3.9672
4.0135 0.51 5100 3.9580
4.0009 0.515 5150 3.9701
3.9920 0.52 5200 3.9559
3.9779 0.525 5250 3.9472
3.9668 0.53 5300 3.9416
3.9644 0.535 5350 3.9410
3.9600 0.54 5400 3.9363
3.9584 0.545 5450 3.9344
3.9686 0.55 5500 3.9314
3.9607 0.555 5550 3.9261
3.9667 0.56 5600 3.9289
3.9690 0.565 5650 3.9213
3.9493 0.57 5700 3.9152
3.9220 0.575 5750 3.9151
3.9558 0.58 5800 3.9164
3.9347 0.585 5850 3.9105
3.9501 0.59 5900 3.9131
3.9466 0.595 5950 3.9101
3.9338 0.6 6000 3.9002
3.9463 0.605 6050 3.8949
3.9407 0.61 6100 3.8941
3.9280 0.615 6150 3.9093
3.9401 0.62 6200 3.9264
3.9342 0.625 6250 3.9006
3.9476 0.63 6300 3.8955
3.9213 0.635 6350 3.8883
3.9069 0.64 6400 3.8874
3.9218 0.645 6450 3.8800
3.8988 0.65 6500 3.8758
3.9081 0.655 6550 3.8745
3.8983 0.66 6600 3.8713
3.8855 0.665 6650 3.8714
3.8850 0.67 6700 3.8686
3.9026 0.675 6750 3.8841
3.9018 0.68 6800 3.8708
3.8853 0.685 6850 3.8650
3.8869 0.69 6900 3.8616
3.8872 0.695 6950 3.8608
3.8818 0.7 7000 3.8644
3.8804 0.705 7050 3.8576
3.9320 0.71 7100 3.8549
3.8944 0.715 7150 3.8531
3.9111 0.72 7200 3.8503
3.9037 0.725 7250 3.8501
3.9053 0.73 7300 3.8476
3.9204 0.735 7350 3.8463
3.8743 0.74 7400 3.8453
3.8790 0.745 7450 3.8425
3.8819 0.75 7500 3.8415
3.8690 0.755 7550 3.8402
3.8620 0.76 7600 3.8393
3.8679 0.765 7650 3.8375
3.8752 0.77 7700 3.8365
3.8801 0.775 7750 3.8359
3.8988 0.78 7800 3.8372
3.8531 0.785 7850 3.8365
3.8870 0.79 7900 3.8341
3.8855 0.795 7950 3.8328
3.8472 0.8 8000 3.8357
3.8581 0.805 8050 3.8313
3.8521 0.81 8100 3.8300
3.8481 0.815 8150 3.8291
3.8474 0.82 8200 3.8287
3.8607 0.825 8250 3.8283
3.8660 0.83 8300 3.8268
3.8560 0.835 8350 3.8267
3.8534 0.84 8400 3.8279
3.8509 0.845 8450 3.8264
3.8621 0.85 8500 3.8259
3.8774 0.855 8550 3.8251
3.8485 0.86 8600 3.8245
3.8578 0.865 8650 3.8238
3.8438 0.87 8700 3.8253
3.8695 0.875 8750 3.8233
3.8230 0.88 8800 3.8259
3.8620 0.885 8850 3.8237
3.8660 0.89 8900 3.8228
3.8742 0.895 8950 3.8219
3.8683 0.9 9000 3.8213
3.8443 0.905 9050 3.8234
3.8420 0.91 9100 3.8218
3.8515 0.915 9150 3.8213
3.8538 0.92 9200 3.8208
3.8426 0.925 9250 3.8202
3.8588 0.93 9300 3.8200
3.8638 0.935 9350 3.8197
3.8587 0.94 9400 3.8193
3.8491 0.945 9450 3.8192
3.8529 0.95 9500 3.8189
3.8438 0.955 9550 3.8187
3.8790 0.96 9600 3.8183
3.8525 0.965 9650 3.8185
3.8477 0.97 9700 3.8180
3.8266 0.975 9750 3.8177
3.8477 0.98 9800 3.8176
3.8438 0.985 9850 3.8173
3.8882 0.99 9900 3.8172
3.8177 0.995 9950 3.8175
3.8620 1.0 10000 3.8171

Framework versions

  • Transformers 5.4.0
  • Pytorch 2.8.0+cu128
  • Datasets 3.2.0
  • Tokenizers 0.22.1

Configuration

Architecture
GPTNeoXForCausalLM
Context length (tokens)
2,048
Layers
12
Hidden size
768
Feed-forward size
3,072
Attention heads
12
Vocabulary size
50,304
Model type
gpt_neox

Identity and Version

Repository
qing-yao/ppt-pythia-160m-uniform250-previous_mse_delta_shuffle2-seed324-stage2
Publisher
Qing Yao
Task
Text generation
Modality
Text
Library
transformers
Parameters
162M parameters
Languages
Not stated by the source
Revision
08460d2f78cf4da50deb9d1e6096c5c17ea66a2a
First published
2026-09-26
Last updated
2026-09-26

Files and Weights

66 files, 5.5 GB in total. The weights are 29 files totalling 5.5 GB in bin, pt, pth, safetensors.

Weights29 files · 5.5 GB
Configuration21 files · 347.3 KB
Tokenizer14 files · 25.0 MB
Documentation1 file · 12.1 KB
Repository1 file · 1.5 KB
Every file
FileTypeSizeSHA-256
checkpoint-10000/model.safetensorsWeights324.7 MB edceb0b64f8b
checkpoint-10000/optimizer.ptWeights649.4 MB 15a900a2f745
checkpoint-10000/rng_state.pthWeights14.6 KB d686d3f57bd1
checkpoint-10000/scheduler.ptWeights1.5 KB c101a5f2562c
checkpoint-10000/training_args.binWeights5.4 KB 58d130b1831f
checkpoint-2000/model.safetensorsWeights324.7 MB 09331f95322b
checkpoint-2000/optimizer.ptWeights649.4 MB f9d91a8412a1
checkpoint-2000/rng_state.pthWeights14.6 KB 4c07f248a4ec
checkpoint-2000/scheduler.ptWeights1.5 KB 70a8cd55d7a2
checkpoint-2000/training_args.binWeights5.4 KB 58d130b1831f
checkpoint-4000/model.safetensorsWeights324.7 MB f80997c54839
checkpoint-4000/optimizer.ptWeights649.4 MB 166a501b83a9
checkpoint-4000/rng_state.pthWeights14.6 KB f7ad907e449a
checkpoint-4000/scheduler.ptWeights1.5 KB 59828c1e7072
checkpoint-4000/training_args.binWeights5.4 KB 58d130b1831f
checkpoint-6000/model.safetensorsWeights324.7 MB 24c9d10bc63d
checkpoint-6000/optimizer.ptWeights649.4 MB c554074a8de7
checkpoint-6000/rng_state.pthWeights14.6 KB d9ff15a0556e
checkpoint-6000/scheduler.ptWeights1.5 KB 5359576a40af
checkpoint-6000/training_args.binWeights5.4 KB 58d130b1831f
checkpoint-8000/model.safetensorsWeights324.7 MB 6e44118ffe69
checkpoint-8000/optimizer.ptWeights649.4 MB 85fd594c0bd7
checkpoint-8000/rng_state.pthWeights14.6 KB a28df9a13af3
checkpoint-8000/scheduler.ptWeights1.5 KB 79dc8472cd40
checkpoint-8000/training_args.binWeights5.4 KB 58d130b1831f
final/model.safetensorsWeights324.7 MB edceb0b64f8b
final/training_args.binWeights5.4 KB 58d130b1831f
model.safetensorsWeights324.7 MB edceb0b64f8b
training_args.binWeights5.4 KB 58d130b1831f
checkpoint-10000/config.jsonConfiguration781 B —
checkpoint-10000/generation_config.jsonConfiguration225 B —
checkpoint-10000/trainer_state.jsonConfiguration74.2 KB —
checkpoint-2000/config.jsonConfiguration781 B —
checkpoint-2000/generation_config.jsonConfiguration225 B —
checkpoint-2000/trainer_state.jsonConfiguration15.4 KB —
checkpoint-4000/config.jsonConfiguration781 B —
checkpoint-4000/generation_config.jsonConfiguration225 B —
checkpoint-4000/trainer_state.jsonConfiguration30.1 KB —
checkpoint-6000/config.jsonConfiguration781 B —
checkpoint-6000/generation_config.jsonConfiguration225 B —
checkpoint-6000/trainer_state.jsonConfiguration44.8 KB —
checkpoint-8000/config.jsonConfiguration781 B —
checkpoint-8000/generation_config.jsonConfiguration225 B —
checkpoint-8000/trainer_state.jsonConfiguration59.5 KB —
config.jsonConfiguration781 B —
experiment.jsonConfiguration41.7 KB —
final/config.jsonConfiguration781 B —
final/generation_config.jsonConfiguration225 B —
generation_config.jsonConfiguration225 B —
trainer_state.jsonConfiguration74.5 KB —
README.mdDocumentation12.1 KB —
.gitattributesRepository1.5 KB —
checkpoint-10000/tokenizer.jsonTokenizer3.6 MB —
checkpoint-10000/tokenizer_config.jsonTokenizer349 B —
checkpoint-2000/tokenizer.jsonTokenizer3.6 MB —
checkpoint-2000/tokenizer_config.jsonTokenizer349 B —
checkpoint-4000/tokenizer.jsonTokenizer3.6 MB —
checkpoint-4000/tokenizer_config.jsonTokenizer349 B —
checkpoint-6000/tokenizer.jsonTokenizer3.6 MB —
checkpoint-6000/tokenizer_config.jsonTokenizer349 B —
checkpoint-8000/tokenizer.jsonTokenizer3.6 MB —
checkpoint-8000/tokenizer_config.jsonTokenizer349 B —
final/tokenizer.jsonTokenizer3.6 MB —
final/tokenizer_config.jsonTokenizer349 B —
tokenizer.jsonTokenizer3.6 MB —
tokenizer_config.jsonTokenizer349 B —

License and Download

License
apache-2.0
Access
Open weights, no gate
Download size
5.5 GB
Download from Qing Yao

Released by Qing Yao through its official repository on Hugging Face. Read the license.

Built From

Memory Requirements

PrecisionWeights in memory
As published5.5 GB
16-bit0.3 GB
8-bit0.2 GB
4-bit0.1 GB

Weights only, from the published parameter count; the key-value cache and runtime add to this.

Questions About ppt-pythia-160m-uniform250-previous_mse_delta_shuffle2-seed324-stage2

How much GPU memory does ppt-pythia-160m-uniform250-previous_mse_delta_shuffle2-seed324-stage2 need?

About 0.4 GB at 16-bit and 0.1 GB at 4-bit: the weights (162M parameters) plus a working margin. A long context needs more.

What is the cheapest GPU to run ppt-pythia-160m-uniform250-previous_mse_delta_shuffle2-seed324-stage2 on?

At 16-bit, 1x MI300X from $1.85 an hour; at 4-bit, 1x MI300X from $1.85 an hour, at the lowest on-demand prices the SAVRN Index lists.

Can I use ppt-pythia-160m-uniform250-previous_mse_delta_shuffle2-seed324-stage2 commercially?

Yes. ppt-pythia-160m-uniform250-previous_mse_delta_shuffle2-seed324-stage2 is released under Apache License 2.0. The Apache License 2.0 is a permissive open-source license. It permits commercial use, modification and redistribution. It requires keeping the license and copyright notices and any NOTICE file, stating significant changes, and it includes an express patent grant from contributors.

What is ppt-pythia-160m-uniform250-previous_mse_delta_shuffle2-seed324-stage2's context length?

2,048 tokens, from the maximum position embeddings in its published configuration.

Similar Models

This model is a fine-tuned version of qing-yao/ppt-pythia-160m-uniform250-previousmse-seed324-stage1 on the None dataset. It achieves the following results on the evaluation set: The following hyperparameters were used during training: - learningrate: 0.001 - trainbatchsize: 16 - evalbatchsize: 16 - gradientaccumulationsteps: 2 - totaltrainbatchsize: 32 - lrschedulertype: cosinewithminlr - lrschedulerwarmupsteps: 500 - trainingsteps: 10000 - Transformers 5.4.0 - Pytorch 2.8.0+cu128 - Datasets 3.2.0 - Tokenizers 0.22.1

Open weights apache-2.0 162M parameters 2,048 tokens transformers
View model

Model · Text generation

astrallm

Pihu Pandey

AstralLM is a compact, efficient causal language model built for Hugging Face Transformers. It features a clean decoder-only transformer architecture with Grouped Query Attention (GQA), SwiGLU feed-forward networks, RoPE positional embeddings, and per-head QK-norm — delivering strong performance at a small parameter count (~138M). AstralLM is a decoder-only transformer with the following design choices: The model uses 8 query heads and 4 key/value heads (2:1 ratio), halving the KV cache memory footprint during inference without measurable quality degradation. Each head operates over a head dimension of 80. Per-head RMSNorm is applied to both query and key projections before the rotary…

Open weights apache-2.0 160M parameters 3,072 tokens transformers
View model

Model · Text generation

Ru-Small-Instruct

LongTime

Ru-Small-Instruct — экспериментальная компактная русскоязычная языковая модель класса SLM (Small Language Model) с объемом параметров ~0.2B (~165M). Разработана с упором на суверенность весов (Zero-Fingerprint): модель обучена с нуля без заимствования базовых чекпоинтов у сторонних корпоративных сетей (Llama 3 от Meta, Qwen от Alibaba, Mistral). Модель предназначена для исследований локального инференса, работы на маломощном оборудовании, CPU и мобильных чипах, где критичны нулевая задержка (Time-To-First-Token) и полная независимость весов. Для компактной модели в 165M параметров, обученной на одном домашнем GPU за 48 часов, способность держать роль, грамотно формулировать сложные термины…

Open weights mit 165M parameters 512 tokens transformers
View model

Model · Text generation

slm-125m-dpo

Tijani Ohiokpehai

tohio/slm-125m-dpo is a small language model (125M parameters) built from scratch and aligned end-to-end using the slm-gpt engine. - Decoder-Only Transformer with Rotary Position Embeddings (RoPE) The base model was pre-trained across an interleaved multi-source domain mixture composed of: - FineWeb-Edu - DCLM-Edu - The Stack-Edu - NuminaMath-CoT - OpenMathReasoning - SLM-Synthetic-Pretrain 1. Pre-training: Multi-GPU Distributed Data Parallel (DDP) over rank-disjoint memory-mapped token shards. 2. Supervised Fine-Tuning (SFT): Full parameter instruction-tuning on ChatML formatted dialogues with prompt loss masking (ignoreindex=-100). 3. Direct Preference Optimization (DPO): Single-stage…

Open weights mit 165M parameters 1,024 tokens transformers
View model

Model · Text generation

slm-125m-sft

Tijani Ohiokpehai

tohio/slm-125m-sft is a small language model (125M parameters) built from scratch and aligned end-to-end using the slm-gpt engine. - Decoder-Only Transformer with Rotary Position Embeddings (RoPE) The base model was pre-trained across an interleaved multi-source domain mixture composed of: - FineWeb-Edu - DCLM-Edu - The Stack-Edu - NuminaMath-CoT - OpenMathReasoning - SLM-Synthetic-Pretrain 1. Pre-training: Multi-GPU Distributed Data Parallel (DDP) over rank-disjoint memory-mapped token shards. 2. Supervised Fine-Tuning (SFT): Full parameter instruction-tuning on ChatML formatted dialogues with prompt loss masking (ignoreindex=-100). 3. Direct Preference Optimization (DPO): Single-stage…

Open weights mit 165M parameters 1,024 tokens transformers
View model

Model · Text generation

slm-125m-base

Tijani Ohiokpehai

tohio/slm-125m-base is a small language model (125M parameters) built from scratch and aligned end-to-end using the slm-gpt engine. - Decoder-Only Transformer with Rotary Position Embeddings (RoPE) The base model was pre-trained across an interleaved multi-source domain mixture composed of: - FineWeb-Edu - DCLM-Edu - The Stack-Edu - NuminaMath-CoT - OpenMathReasoning - SLM-Synthetic-Pretrain 1. Pre-training: Multi-GPU Distributed Data Parallel (DDP) over rank-disjoint memory-mapped token shards. 2. Supervised Fine-Tuning (SFT): Full parameter instruction-tuning on ChatML formatted dialogues with prompt loss masking (ignoreindex=-100). 3. Direct Preference Optimization (DPO): Single-stage…

Open weights mit 165M parameters 2,048 tokens transformers
View model