SAVRN
Search Contact SAVRN

Open-weight model · Text generation

SmolLM2-135M

by Hugging Face Smol Models Research HuggingFaceTB/SmolLM2-135M

SmolLM2 is a family of compact language models available in three size: 135M, 360M, and 1.7B parameters. They are capable of solving a wide range of tasks while being lightweight enough to run on-device.

Parameters135M
Context8,192
Weights269.1 MB
Licenseapache-2.0
AccessOpen weights
Monthly Downloads2.2M

Runs On

What it takes to serve SmolLM2-135M (135M parameters): the memory its weights need at each precision, and the cheapest way to rent enough data-center GPUs to hold them.

PrecisionWeightsMemory neededCheapest setupPer hourAlso fits
16-bit 0.3 GB 0.3 GB 1x MI300X (192 GB)
Vultr
$1.85 1x H100 $1.99 · 1x MI325X $2.00
8-bit 0.1 GB 0.2 GB 1x MI300X (192 GB)
Vultr
$1.85 1x H100 $1.99 · 1x MI325X $2.00
4-bit 0.1 GB 0.1 GB 1x MI300X (192 GB)
Vultr
$1.85 1x H100 $1.99 · 1x MI325X $2.00

Memory is the weights at that precision plus 20% for the runtime and a short context; a long context needs more. Prices are the lowest on-demand hourly rates in the SAVRN Index, read Sep 18, 2026.

SAVRN's Notes on SmolLM2-135M

Two trillion training tokens went into a model that fits in 0.3 GB. Hugging Face Smol Models Research trained this 135M-parameter SmolLM2, smallest of three sizes beside 360M and 1.7B, on FineWeb-Edu, DCLM and The Stack to run on-device. The cheapest priced card we list is one MI300X at $1.85 an hour, and 0.3 GB needed at 16-bit is why we would never dedicate one; here it shares a card, or goes to the device it was built for.

The only evaluation in our file is a third-party MMLU-Pro result of 10.85 from EvalEval, so run your own task first, and plan around an 8,192-token context; the training recipe is in arXiv:2502.02737. Apache 2.0 permits commercial use, modification and redistribution, requires keeping the license, copyright notices and any NOTICE file and stating significant changes, and carries an express patent grant from contributors, which matters if you fine-tune and ship.

Model Card

By Hugging Face Smol Models Research, published under apache-2.0, revision 93efa2f097d5.

SmolLM2

Table of Contents

  1. Model Summary
  2. Limitations
  3. Training
  4. License
  5. Citation

Model Summary

SmolLM2 is a family of compact language models available in three size: 135M, 360M, and 1.7B parameters. They are capable of solving a wide range of tasks while being lightweight enough to run on-device. More details in our paper: https://arxiv.org/abs/2502.02737

SmolLM2 demonstrates significant advances over its predecessor SmolLM1, particularly in instruction following, knowledge, reasoning. The 135M model was trained on 2 trillion tokens using a diverse dataset combination: FineWeb-Edu, DCLM, The Stack, along with new filtered datasets we curated and will release soon. We developed the instruct version through supervised fine-tuning (SFT) using a combination of public datasets and our own curated datasets. We then applied Direct Preference Optimization (DPO) using UltraFeedback.

Read the full model card (554 words)

Configuration

Architecture
LlamaForCausalLM
Context length (tokens)
8,192
Layers
30
Hidden size
576
Feed-forward size
1,536
Attention heads
9
Key/value heads
3
Vocabulary size
49,152
RoPE base
100,000
Stored precision
bfloat16
Model type
llama

Identity and Version

Repository
HuggingFaceTB/SmolLM2-135M
Publisher
Hugging Face Smol Models Research
Task
Text generation
Modality
Text
Library
transformers
Parameters
135M parameters
Languages
en
Revision
93efa2f097d58c2a74874c7e644dbc9b0cee75a2
First published
2024-10-31
Last updated
2025-02-06

Files and Weights

10 files, 272.4 MB in total. The weights are 1 file totalling 269.1 MB in safetensors.

Weights1 file · 269.1 MB
Configuration3 files · 1.6 KB
Tokenizer4 files · 3.4 MB
Documentation1 file · 6.3 KB
Repository1 file · 1.5 KB
Every file
FileTypeSizeSHA-256
model.safetensorsWeights269.1 MB 80521b40281d
config.jsonConfiguration704 B
generation_config.jsonConfiguration111 B
special_tokens_map.jsonConfiguration831 B
README.mdDocumentation6.3 KB
.gitattributesRepository1.5 KB
merges.txtTokenizer466.4 KB
tokenizer.jsonTokenizer2.1 MB
tokenizer_config.jsonTokenizer3.7 KB
vocab.jsonTokenizer800.7 KB

License and Download

License
apache-2.0
Access
Open weights, no gate
Download size
269.1 MB
Download from Hugging Face Smol Models Research

Released by Hugging Face Smol Models Research through its official repository on Hugging Face. Read the license.

Built From

Evaluations

Each result is shown as reported, with the conditions its reporter stated. None is a SAVRN measurement. A comparison lines two results up only when their configuration, unit and setup are all stated and identical.

BenchmarkConditionsResultReported byRevisionDate
TIGER-Lab/MMLU-Pro Task mmlu_proMetric mmlu_proComparison conditions not established 10.85 EvalEval
Reported by a third party
Evaluated revision not stated 2026-06-30

Memory Requirements

PrecisionWeights in memory
As published269.1 MB
16-bit0.3 GB
8-bit0.1 GB
4-bit0.1 GB

Weights only, from the published parameter count; the key-value cache and runtime add to this.

Built on This Model

Compare SmolLM2-135M

Questions About SmolLM2-135M

How much GPU memory does SmolLM2-135M need?

About 0.3 GB at 16-bit and 0.1 GB at 4-bit: the weights (135M parameters) plus a working margin. A long context needs more.

What is the cheapest GPU to run SmolLM2-135M on?

At 16-bit, 1x MI300X from $1.85 an hour; at 4-bit, 1x MI300X from $1.85 an hour, at the lowest on-demand prices the SAVRN Index lists.

Can I use SmolLM2-135M commercially?

Yes. SmolLM2-135M is released under Apache License 2.0. The Apache License 2.0 is a permissive open-source license. It permits commercial use, modification and redistribution. It requires keeping the license and copyright notices and any NOTICE file, stating significant changes, and it includes an express patent grant from contributors.

What is SmolLM2-135M's context length?

8,192 tokens, from the maximum position embeddings in its published configuration.

Similar Models

SmolLM2 is a family of compact language models available in three size: 135M, 360M, and 1.7B parameters. They are capable of solving a wide range of tasks while being lightweight enough to run on-device. More details in our paper https://arxiv.org/abs/2502.02737 SmolLM2 demonstrates significant advances over its predecessor SmolLM1, particularly in instruction following, knowledge, reasoning. The 135M model was trained on 2 trillion tokens using a diverse dataset combination: FineWeb-Edu, DCLM, The Stack, along with new filtered datasets we curated and will release soon. We developed the instruct version through supervised fine-tuning (SFT) using a combination of public datasets and our own…

Open weights apache-2.0 135M parameters 8,192 tokens transformers

Model · Text generation

gpt2

OpenAI community

Test the whole generation capabilities here: https://transformer.huggingface.co/doc/gpt2-large Pretrained model on English language using a causal language modeling (CLM) objective. It was introduced in and first released at this page. model. Content from this model card has been written by the Hugging Face team to complete the information they provided and give specific examples of bias. GPT-2 is a transformers model pretrained on a very large corpus of English data in a self-supervised fashion. This means it was pretrained on the raw texts only, with no humans labelling them in any way (which is why it can use lots of publicly available data) with an automatic process to generate inputs…

Open weights mit 137M parameters transformers

This model is a custom-code derivative of AxiomicLabs/GPT-X2-125M, adapted for experimental long-context causal language modeling and architecture research. The repository includes a Hugging Face Transformers-compatible GPT-X2 implementation with optional Symplectic Metric-RoPE Governor support and training utilities built around CIxOpt, a heterogeneous optimizer developed for efficient parameter routing across large projection matrices, sensitive normalization parameters, and optional governor modules. The model is intended as a research checkpoint for compact long-context generation, positional encoding experiments, optimizer testing, and continued fine-tuning. This implementation uses a…

Open weights apache-2.0 126M parameters 32,768 tokens transformers

Model · Text generation

Haidass-Translate-143M

DALab

English | 中文 A 143M-parameter bidirectional Chinese↔English translation model, instruction-tuned on the Haidass1.5-143M base — the strongest zh⇄en translator at this scale among general chat-architecture models. Drafter-143M: a control model with identical configuration, data and training recipe, except that it starts from random initialization instead of the pretrained base — used to quantify the contribution of base-model pretraining. OPUS-MT models are single-directional — one independent 78M model per direction; "-" marks directions a model does not serve. The same models re-evaluated on FLORES+ devtest (released 2026; zero overlap with dev): devtest sentences do not overlap with dev.…

Open weights apache-2.0 143M parameters 4,096 tokens

English | 中文 The instruction-tolerant sibling of DALabCommunity/Haidass-Translate-143M: same 143M zh⇄en translation training, plus 9.1% cleaned general-domain data (STEPFUN ShareGPT) mixed in. Translation scores are within 0.3 BLEU of the pure-translation version, and the model retains limited general instruction-following ability that the pure-translation version does not have. Drafter-143M: a control model with identical configuration, data and training recipe, except that it starts from random initialization instead of the pretrained base — used to quantify the contribution of base-model pretraining. OPUS-MT models are single-directional — one independent 78M model per direction; "-"…

Open weights apache-2.0 143M parameters 4,096 tokens