SmolLM2-FT-MyDataset

Model · Text generation

SmolLM2-135M

Hugging Face Smol Models Research

SmolLM2 is a family of compact language models available in three size: 135M, 360M, and 1.7B parameters. They are capable of solving a wide range of tasks while being lightweight enough to run on-device. More details in our paper: https://arxiv.org/abs/2502.02737 SmolLM2 demonstrates significant advances over its predecessor SmolLM1, particularly in instruction following, knowledge, reasoning. The 135M model was trained on 2 trillion tokens using a diverse dataset combination: FineWeb-Edu, DCLM, The Stack, along with new filtered datasets we curated and will release soon. We developed the instruct version through supervised fine-tuning (SFT) using a combination of public datasets and our…

Model · Text generation

SmolLM2-135M-Instruct

Hugging Face Smol Models Research

SmolLM2 is a family of compact language models available in three size: 135M, 360M, and 1.7B parameters. They are capable of solving a wide range of tasks while being lightweight enough to run on-device. More details in our paper https://arxiv.org/abs/2502.02737 SmolLM2 demonstrates significant advances over its predecessor SmolLM1, particularly in instruction following, knowledge, reasoning. The 135M model was trained on 2 trillion tokens using a diverse dataset combination: FineWeb-Edu, DCLM, The Stack, along with new filtered datasets we curated and will release soon. We developed the instruct version through supervised fine-tuning (SFT) using a combination of public datasets and our own…

This model is a fine-tuned version of HuggingFaceTB/SmolLM2-135M-Instruct on an unknown dataset. The following hyperparameters were used during training: - learningrate: 2e-05 - trainbatchsize: 1 - evalbatchsize: 1 - gradientaccumulationsteps: 4 - totaltrainbatchsize: 4 - lrschedulertype: linear - numepochs: 1 - Transformers 5.17.0 - Pytorch 2.11.0+cu130 - Datasets 4.8.5 - Tokenizers 0.23.2

This model is a fine-tuned version of HuggingFaceTB/SmolLM2-135M on an unknown dataset. It achieves the following results on the evaluation set: The following hyperparameters were used during training: - learningrate: 5e-05 - trainbatchsize: 4 - evalbatchsize: 8 - lrschedulertype: linear - trainingsteps: 1000 - Transformers 4.46.3 - Pytorch 2.11.0+cu130 - Datasets 4.8.5 - Tokenizers 0.20.3

Model · Text generation

Smol-GPT-Fp8

Gggff123

This llama model was trained 2x faster with Unsloth and Huggingface's TRL library.

Model · Text generation

Smol-GPT-RAW

Gggff123

This llama model was trained 2x faster with Unsloth and Huggingface's TRL library.