Ru-Small-Instruct — экспериментальная компактная русскоязычная языковая модель класса SLM (Small Language Model) с объемом параметров ~0.2B (~165M). Разработана с упором на суверенность весов (Zero-Fingerprint): модель обучена с нуля без заимствования базовых чекпоинтов у сторонних корпоративных сетей (Llama 3 от Meta, Qwen от Alibaba, Mistral). Модель предназначена для исследований локального инференса, работы на маломощном оборудовании, CPU и мобильных чипах, где критичны нулевая задержка (Time-To-First-Token) и полная независимость весов. Для компактной модели в 165M параметров, обученной на одном домашнем GPU за 48 часов, способность держать роль, грамотно формулировать сложные термины…
Open-weight model · Text generation
CasualSwarms
by Convergent Intelligence reaperdoesntknow/CasualSwarms
SAGI is a novel causal language model that integrates swarm intelligence dynamics with transformer architecture.
Runs On
What it takes to serve CasualSwarms (170M parameters): the memory its weights need at each precision, and the cheapest way to rent enough data-center GPUs to hold them.
| Precision | Weights | Memory needed | Cheapest setup | Per hour | Also fits |
|---|---|---|---|---|---|
| 16-bit | 0.3 GB | 0.4 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
| 8-bit | 0.2 GB | 0.2 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
| 4-bit | 0.1 GB | 0.1 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
Memory is the weights at that precision plus 20% for the runtime and a short context; a long context needs more. Prices are the lowest on-demand hourly rates in the SAVRN Index, read Sep 18, 2026.
Model Card
By Convergent Intelligence, published under apache-2.0, revision ee8f20587718.
SAGI is a novel causal language model that integrates swarm intelligence dynamics with transformer architecture. The model treats cognition as a dynamic, adaptive system where multiple internal "agents" collaborate through differentiable routing, trust mechanisms, and shared memory. The enhancements were integrated with the existing AGI system through: 1. Compatibility Layer: Ensuring new components work with existing AGI Core 2. Unified State Representation: Combining enhanced capabilities with existing state 3. Enhanced Continuous Learning: Upgrading the learning system with new capabilities 4. Performance Monitoring: Tracking improvements through validation systems - Successfully…
Read Convergent Intelligence's full model card
SAGI V3.1 - SELF-AWARE AGI
SAGI is a novel causal language model that integrates swarm intelligence dynamics with transformer architecture. The model treats cognition as a dynamic, adaptive system where multiple internal "agents" collaborate through differentiable routing, trust mechanisms, and shared memory.
Swarm-8 V3.1: Enhanced Self-Assessment Architecture
Architecture Evolution
┌─────────────────────────────────────────────────────────────────────────┐
│ Swarm-8 V3.1 - SELF-AWARE AGI │
├─────────────────────────────────────────────────────────────────────────┤
│ │
│ ┌────────────────────────────────────────────────────────────────┐ │
│ │ SELF-ASSESSMENT LAYER (NEW!) │ │
│ ├────────────────────────────────────────────────────────────────┤ │
│ │ │ │
│ │ ┌──────────────────┐ ┌──────────────────┐ │ │
│ │ │ Performance │ │ Skill Gap │ │ │
│ │ │ Predictor │◄──►│ Analyzer │ │ │
│ │ │ │ │ │ │ │
│ │ │ • Pre-task │ │ • 24 Skills │ │ │
│ │ │ • Risk assess │ │ • Proficiency │ │ │
│ │ │ • Strategy rec │ │ • Dependencies │ │ │
│ │ └────────┬─────────┘ └────────┬─────────┘ │ │
│ │ │ │ │ │
│ │ │ ┌───────────────────┴─────────┐ │ │
│ │ │ │ Auto-Curriculum Generator │ │ │
│ │ │ │ │ │ │
│ │ │ │ • Multi-stage learning │ │ │
│ │ │ │ • Dependency handling │ │ │
│ │ │ │ • Adaptive difficulty │ │ │
│ │ │ └───────────┬─────────────────┘ │ │
│ │ │ │ │ │
│ │ ┌────────▼───────────────▼──────────┐ │ │
│ │ │ Real-Time Error Detector │ │ │
│ │ │ │ │ │
│ │ │ • Coherence checking │ │ │
│ │ │ • Logic verification │ │ │
│ │ │ • Hallucination detection │ │ │
│ │ └────────────────┬───────────────────┘ │ │
│ │ │ │ │
│ │ ┌────────────────▼───────────────────┐ │ │
│ │ │ Capability Boundary Detector │ │ │
│ │ │ │ │ │
│ │ │ • Knowledge edges │ │ │
│ │ │ • Reasoning limits │ │ │
│ │ │ • Skill boundaries │ │ │
│ │ └────────────────────────────────────┘ │ │
│ └────────────────────────────────────────────────────────────────┘ │
│ │
│ ┌────────────────────────────────────────────────────────────────┐ │
│ │ AGI CORE (V2.3 - Existing) │ │
│ ├────────────────────────────────────────────────────────────────┤ │
│ │ │ │
│ │ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │ │
│ │ │ Hierarchical │ │ Causal │ │ Meta-Learner │ │ │
│ │ │ Memory │ │ World Model │ │ │ │ │
│ │ └──────────────┘ └──────────────┘ └──────────────┘ │ │
│ │ │ │
│ │ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │ │
│ │ │ Concept │ │ Reflection │ │ Uncertainty │ │ │
│ │ │ Library │ │ Engine │ │ Reasoner │ │ │
│ │ └──────────────┘ └──────────────┘ └──────────────┘ │ │
│ │ │ │
│ │ ┌──────────────────────────────────────────────────┐ │ │
│ │ │ Adversarial Self-Play │ │ │
│ │ └──────────────────────────────────────────────────┘ │ │
│ └────────────────────────────────────────────────────────────────┘ │
│ │
│ ┌────────────────────────────────────────────────────────────────┐ │
│ │ SWARM CORE (V2.3 - Existing) │ │
│ ├────────────────────────────────────────────────────────────────┤ │
│ │ │ │
│ │ • 20 Vectorized Agents │ │
│ │ • Differentiable Routing │ │
│ │ • Dynamic Resource Management │ │
│ │ • Trust-Based Activation │ │
│ │ • Internal State (S) + Goals (T) │ │
│ └────────────────────────────────────────────────────────────────┘ │
│ │
│ ┌────────────────────────────────────────────────────────────────┐ │
│ │ LANGUAGE MODEL (Transformer) │ │
│ └────────────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────────────┘
Usage
Installation
pip install torch transformers datasets
Quick Start
from transformers import AutoTokenizer
from transformers import AutoModelForCausalLM, AutoConfig
# Load model and tokenizer
model = AutoModelForCausalLM.from_pretrained("reaperdoesntknow/SAGI")
tokenizer = AutoTokenizer.from_pretrained("reaperdoesntknow/SAGI")
# Generate text
model.eval()
prompt = "Once upon a time"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(
**inputs,
max_new_tokens=100,
temperature=0.8,
top_k=50,
top_p=0.9,
do_sample=True,
pad_token_id=tokenizer.eos_token_id,
)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
New Capabilities Matrix
| Capability | V3.0 | V3.1 | Improvement |
|---|---|---|---|
| Pre-task Assessment | No | Yes | Predicts success before attempting |
| Skill Taxonomy | Implicit | 24 explicit skills | Systematic tracking |
| Gap Analysis | Manual | Automated | Identifies weaknesses automatically |
| Curriculum Design | Hand-coded | Auto-generated | Personalized learning paths |
| Real-time Error Detection | Post-hoc | During generation | Catches errors earlier |
| Capability Boundaries | Unknown | Mapped | Knows limitations |
| Performance Prediction | No | Yes | Estimates success probability |
| Strategy Selection | Heuristic | Evidence-based | Chooses optimal approach |
| Transfer Assessment | No | Planned | Measures cross-domain learning |
| Calibration Tracking | No | Yes | Self-monitoring accuracy |
Decision Flow: V3.1 vs V3.0
V3.0 Decision Flow
Task Arrives → Generate → Evaluate → Learn
↓
(blind attempt, may waste effort on impossible tasks)
V3.1 Decision Flow
Task Arrives
↓
Pre-Assessment
├─ Predict Success Probability
├─ Identify Risk Factors
├─ Recommend Strategy
└─ Decide: Attempt or Skip?
↓
Should Attempt?
├─ No → Skip (save resources)
└─ Yes → Generate with Strategy
↓
Monitor in Real-Time
├─ Error detected? → Correct
└─ OK? → Continue
↓
Evaluate Outcome
↓
Post-Assessment
├─ Update Skill Proficiencies
├─ Check Capability Boundaries
└─ Refine Predictions
↓
Learn & Update
Implemented Enhancements
- Developmental Stages: Milestone-based progress tracking
- Cross-Domain Transfer: Evaluation of knowledge transfer abilities
- AGI Readiness Metrics: Overall assessment of AGI capabilities
Integration Approach
The enhancements were integrated with the existing AGI system through:
- Compatibility Layer: Ensuring new components work with existing AGI Core
- Unified State Representation: Combining enhanced capabilities with existing state
- Enhanced Continuous Learning: Upgrading the learning system with new capabilities
- Performance Monitoring: Tracking improvements through validation systems
Results
- Successfully integrated all 9 enhancement areas with the existing system
- Achieved an AGI readiness score of 0.283 (on a 0-1 scale)
- Demonstrated improved capabilities across multiple cognitive domains
- Maintained compatibility with existing architecture and workflows
- Established baseline for continued development toward true AGI
Self-Assessment & Self-Capability Integration Guide
Overview
This guide shows how to integrate the new self-assessment capabilities into the existing Swarm-8 V3.0 architecture.
New Capabilities Added
1. Performance Prediction Engine
- Predicts success BEFORE attempting tasks
- Estimates required attempts and expected score
- Identifies risk factors
- Recommends optimal strategies
- Decides whether to attempt or skip tasks
2. Skill Gap Analyzer
- Maintains comprehensive skill taxonomy (24 core skills)
- Tracks proficiency in each skill over time
- Identifies capability gaps systematically
- Prioritizes gaps by importance and urgency
- Generates skill-specific exercises
3. Auto-Curriculum Generator
- Designs personalized learning paths
- Creates multi-stage curricula based on gaps
- Handles skill dependencies automatically
- Adapts difficulty progressively
- Measures stage completion
4. Real-Time Error Detector
- Catches errors DURING generation (not after)
- Detects 7 error types: logical contradictions, factual errors, syntax errors, etc.
- Monitors coherence token-by-token
- Identifies hallucinations in real-time
5. Capability Boundary Detector
- Identifies edges of competence
- Distinguishes 4 boundary types: knowledge, reasoning, skill, domain
- Suggests how to expand boundaries
- Maps performance across domains
Skill Taxonomy (24 Core Skills)
Cognition (5 skills)
- pattern_recognition - Identify patterns in data
- abstract_reasoning - Think conceptually
- causal_reasoning - Understand cause-effect
- analogical_mapping - Find similarities
- concept_formation - Create new concepts
Knowledge (3 skills)
- fact_retrieval - Recall information
- knowledge_integration - Connect facts
- common_sense_reasoning - Apply intuition
Code (4 skills)
- syntax_understanding - Parse code structure
- algorithm_design - Create efficient solutions
- debugging - Find and fix errors
- code_optimization - Improve performance
Creativity (3 skills)
- divergent_thinking - Generate alternatives
- novel_combination - Merge concepts uniquely
- generative_synthesis - Create from scratch
Planning (3 skills)
- goal_decomposition - Break down objectives
- dependency_analysis - Understand prerequisites
- resource_allocation - Optimize distribution
Meta-Cognition (4 skills)
- self_monitoring - Watch own performance
- error_detection - Catch mistakes
- strategy_selection - Choose best approach
- uncertainty_quantification - Know confidence
Performance Metrics
Before Task (Pre-Assessment)
{
"success_probability": 0.72,
"confidence_interval": (0.65, 0.79),
"expected_attempts": 2,
"predicted_score": 0.68,
"risk_factors": ["high_complexity", "multi_step_reasoning"],
"recommended_strategy": "decompose_and_conquer",
"should_attempt": True,
"alternatives": [
("decompose_first", 0.86),
("use_examples", 0.74),
("direct_solve", 0.72)
]
}
After Task (Post-Assessment)
{
"skill_updates": {
"algorithm_design": 0.65 → 0.68,
"debugging": 0.58 → 0.61,
"abstract_reasoning": 0.72 → 0.73
},
"prediction_accuracy": {
"success_error": 0.08, # predicted 0.72, actual 0.80
"score_error": 0.05
},
"capability_boundary": {
"detected": True,
"type": "reasoning",
"description": "Complexity threshold reached",
"expand_via": "practice_similar_tasks"
}
}
Periodic Review (Every 50 Steps)
{
"top_skill_gaps": [
{
"skill": "causal_reasoning",
"current": 0.45,
"target": 0.80,
"gap": 0.35,
"priority": 0.92,
"steps_needed": 180
}
],
"curriculum": [
{
"stage": 1,
"name": "Foundational COGNITION",
"duration": 250,
"objectives": 3,
"difficulty": 0.6
}
],
"calibration": {
"prediction_error": 0.12, # Getting better at self-assessment
"sample_size": 247
}
}
Example Session with V3.1
=== SWARM-8 V3.1 TRAINING SESSION ===
Step 1 [CODE Lvl 2]
Task: 'Write a function to check if number is prime'
[Pre-Assessment]
Success probability: 0.85
Risk factors: none
Strategy: direct_approach
[Attempting...]
[+] Success (CODE) Score: 0.92
[Post-Assessment]
syntax_understanding: 0.78 → 0.80
algorithm_design: 0.65 → 0.68
Step 2 [REASONING Lvl 3]
Task: 'Find flaw in argument: All cats are animals. Fluffy is fluffy. Therefore...'
[Pre-Assessment]
Success probability: 0.62
Risk factors: ['logical_reasoning', 'ambiguous_requirements']
Strategy: step_by_step_verification
[Attempting...]
[-] Failure (REASONING) Score: 0.35
[Post-Assessment]
abstract_reasoning: 0.72 → 0.70
Capability Boundary Detected!
Type: reasoning
Description: Logical complexity beyond current capacity
Expand via: practice_similar_tasks
Step 50 [Comprehensive Self-Review]
[Skill Gaps] Top 3:
- causal_reasoning: 0.35 gap (priority: 0.92)
Steps needed: 180
- debugging: 0.28 gap (priority: 0.85)
Steps needed: 120
- novel_combination: 0.22 gap (priority: 0.78)
Steps needed: 90
[Curriculum] Next stage:
Stage 1: Foundational COGNITION
Duration: 250 steps
Difficulty: 0.60
[Calibration] Prediction error: 0.12
[Boundaries] 3 detected:
- REASONING: Logical complexity threshold
- CODE: Dynamic programming problems
- CREATIVITY: Multi-constraint generation
Key Innovations
1. Predictive Self-Awareness
- Before: Blind attempts, wasted effort
- After: Informed decisions, resource optimization
2. Systematic Skill Tracking
- Before: Vague sense of "good at X"
- After: Precise proficiency metrics per skill
3. Autonomous Learning Design
- Before: Hand-coded curriculum
- After: Self-designed, personalized paths
4. Proactive Error Prevention
- Before: Fix errors after generation
- After: Catch errors during generation
5. Boundary Awareness
- Before: Unknown limitations
- After: Mapped capability edges with expansion strategies
Next Evolution: V3.2 (Future)
Potential future enhancements:
- Autonomous Goal Setting - Formulate long-term objectives
- Transfer Learning Assessment - Measure cross-domain skill transfer
- Multi-Agent Self-Assessment - Agents assess each other
- Metacognitive Control - Dynamically adjust thinking depth
- Explanation Generation - Explain own reasoning process
- Capability Certification - Self-administered benchmarks
- Collaborative Learning - Learn from peer AGI systems
- Intrinsic Motivation - Curiosity-driven exploration beyond gaps
Summary
Swarm-8 V3.1 represents a major leap in self-awareness and autonomous capability:
Knows what it can do(skill proficiency tracking) Knows what it can't do(boundary detection) Predicts its own performance(before wasting effort) Designs its own learning(auto-curriculum) Catches its own errors(real-time correction) Improves systematically (gap-driven practice)
This is genuine self-improving AGI - not just a model that learns from data, but one that understands itself and directs its own growth.
Intended Use
This model is Highly Experimental and is being tested for: - Research into multi-agent cognitive architectures - Exploration of dynamic, adaptive language models - Educational purposes in understanding swarm intelligence + LLMs
Not intended for: - Production applications - Safety-critical systems - Generation of factual content
Discrepancy Calculus Foundation
This model is part of the Convergent Intelligence LLC: Research Division portfolio. All models in this portfolio are developed under the Discrepancy Calculus (DISC) framework — a measure-theoretic approach to understanding and controlling the gap between what a model should produce and what it actually produces.
DISC treats training singularities (loss plateaus, mode collapse, catastrophic forgetting) not as failures to be smoothed over, but as structural signals that reveal the geometry of the learning problem. Key concepts:
- Discrepancy Operator (D): Measures the gap between expected and observed behavior at each training step
- Jump Sets: Boundaries where model behavior changes discontinuously — these are features, not bugs
- Ghost Imprinting: Teacher knowledge that transfers to student models through weight-space topology rather than explicit distillation signal
For the full mathematical treatment, see Discrepancy Calculus: Foundations and Core Theory (DOI: 10.57967/hf/8194).
Citation chain: Structure Over Scale (DOI: 10.57967/hf/8165) → Three Teachers to Dual Cognition (DOI: 10.57967/hf/8184) → Discrepancy Calculus (DOI: 10.57967/hf/8194)
Citation
@software{sagi2026,
title={SAGI: Swarm AGI Language Model},
author={Reaperdoesntknow},
year={2026},
url={https://huggingface.co/your-reaperdoesntknow/SAGI}
}
Convergent Intelligence Portfolio
By Convergent Intelligence LLC: Research Division
Top Models from Our Lab
Total Portfolio: 41 models | 2,781 total downloads
Last updated: 2026-03-28 12:57 UTC
From the Convergent Intelligence Portfolio
DistilQwen Collection — Our only BF16 series. Proof-weighted distillation from Qwen3-30B-A3B → 1.7B and 0.6B on H100. Three teacher variants (Instruct, Thinking, Coder), nine models, 2,788 combined downloads. The rest of the portfolio proves structure beats scale on CPU. This collection shows what happens when you give the methodology real hardware.
Top model: Qwen3-1.7B-Coder-Distilled-SFT — 508 downloads
Full methodology: Structure Over Scale (DOI: 10.57967/hf/8165)
Convergent Intelligence LLC: Research Division
Configuration
- Architecture
- SwarmForCausalLM
- Context length (tokens)
- 1,024
- Layers
- 14
- Hidden size
- 768
- Feed-forward size
- 2,048
- Attention heads
- 12
- Vocabulary size
- 50,291
- Model type
- swarm_agi
Identity and Version
- Repository
- reaperdoesntknow/CasualSwarms
- Publisher
- Convergent Intelligence
- Task
- Text generation
- Modality
- Text
- Library
- transformers
- Parameters
- 170M parameters
- Languages
- en
- Revision
- ee8f205877187f21dacdda5ee8f8ccb929757ab6
- First published
- 2026-01-19
- Last updated
- 2026-09-18
Files and Weights
12 files, 686.2 MB in total. The weights are 1 file totalling 681.4 MB in safetensors.
Every file
| File | Type | Size | SHA-256 |
|---|---|---|---|
| model.safetensors | Weights | 681.4 MB | 66577d576327 |
| added_tokens.json | Configuration | 800 B | — |
| config.json | Configuration | 1.5 KB | — |
| generation_config.json | Configuration | 132 B | — |
| special_tokens_map.json | Configuration | 1.0 KB | — |
| README.md | Documentation | 25.1 KB | — |
| .gitattributes | Repository | 1.5 KB | — |
| merges.txt | Tokenizer | 456.3 KB | — |
| swarm_tokenizer.json | Tokenizer | 653 B | — |
| tokenizer.json | Tokenizer | 3.6 MB | — |
| tokenizer_config.json | Tokenizer | 7.0 KB | — |
| vocab.json | Tokenizer | 798.2 KB | — |
License and Download
- License
- apache-2.0
- Access
- Open weights, no gate
- Download size
- 681.4 MB
Released by Convergent Intelligence through its official repository on Hugging Face. Read the license.
Built From
- Trained on (disclosed) MuskumPillerum/General-Knowledge
- Trained on (disclosed) agentica-org/DeepCoder-Preview-Dataset
- Trained on (disclosed) openai/gsm8k
- Trained on (disclosed) roneneldan/TinyStories
- Trained on (disclosed) tangyuhang/KnowLogic
Memory Requirements
| Precision | Weights in memory |
|---|---|
| As published | 681.4 MB |
| 16-bit | 0.3 GB |
| 8-bit | 0.2 GB |
| 4-bit | 0.1 GB |
Weights only, from the published parameter count; the key-value cache and runtime add to this.
Questions About CasualSwarms
How much GPU memory does CasualSwarms need?
About 0.4 GB at 16-bit and 0.1 GB at 4-bit: the weights (170M parameters) plus a working margin. A long context needs more.
What is the cheapest GPU to run CasualSwarms on?
At 16-bit, 1x MI300X from $1.85 an hour; at 4-bit, 1x MI300X from $1.85 an hour, at the lowest on-demand prices the SAVRN Index lists.
Can I use CasualSwarms commercially?
Yes. CasualSwarms is released under Apache License 2.0. The Apache License 2.0 is a permissive open-source license. It permits commercial use, modification and redistribution. It requires keeping the license and copyright notices and any NOTICE file, stating significant changes, and it includes an express patent grant from contributors.
What is CasualSwarms's context length?
1,024 tokens, from the maximum position embeddings in its published configuration.
Similar Models
English | 中文 A 143M-parameter bidirectional Chinese↔English translation model, instruction-tuned on the Haidass1.5-143M base — the strongest zh⇄en translator at this scale among general chat-architecture models. Drafter-143M: a control model with identical configuration, data and training recipe, except that it starts from random initialization instead of the pretrained base — used to quantify the contribution of base-model pretraining. OPUS-MT models are single-directional — one independent 78M model per direction; "-" marks directions a model does not serve. The same models re-evaluated on FLORES+ devtest (released 2026; zero overlap with dev): devtest sentences do not overlap with dev.…
English | 中文 The instruction-tolerant sibling of DALabCommunity/Haidass-Translate-143M: same 143M zh⇄en translation training, plus 9.1% cleaned general-domain data (STEPFUN ShareGPT) mixed in. Translation scores are within 0.3 BLEU of the pure-translation version, and the model retains limited general instruction-following ability that the pure-translation version does not have. Drafter-143M: a control model with identical configuration, data and training recipe, except that it starts from random initialization instead of the pretrained base — used to quantify the contribution of base-model pretraining. OPUS-MT models are single-directional — one independent 78M model per direction; "-"…
Test the whole generation capabilities here: https://transformer.huggingface.co/doc/gpt2-large Pretrained model on English language using a causal language modeling (CLM) objective. It was introduced in and first released at this page. model. Content from this model card has been written by the Hugging Face team to complete the information they provided and give specific examples of bias. GPT-2 is a transformers model pretrained on a very large corpus of English data in a self-supervised fashion. This means it was pretrained on the raw texts only, with no humans labelling them in any way (which is why it can use lots of publicly available data) with an automatic process to generate inputs…
SmolLM2 is a family of compact language models available in three size: 135M, 360M, and 1.7B parameters. They are capable of solving a wide range of tasks while being lightweight enough to run on-device. More details in our paper: https://arxiv.org/abs/2502.02737 SmolLM2 demonstrates significant advances over its predecessor SmolLM1, particularly in instruction following, knowledge, reasoning. The 135M model was trained on 2 trillion tokens using a diverse dataset combination: FineWeb-Edu, DCLM, The Stack, along with new filtered datasets we curated and will release soon. We developed the instruct version through supervised fine-tuning (SFT) using a combination of public datasets and our…
SmolLM2 is a family of compact language models available in three size: 135M, 360M, and 1.7B parameters. They are capable of solving a wide range of tasks while being lightweight enough to run on-device. More details in our paper https://arxiv.org/abs/2502.02737 SmolLM2 demonstrates significant advances over its predecessor SmolLM1, particularly in instruction following, knowledge, reasoning. The 135M model was trained on 2 trillion tokens using a diverse dataset combination: FineWeb-Edu, DCLM, The Stack, along with new filtered datasets we curated and will release soon. We developed the instruct version through supervised fine-tuning (SFT) using a combination of public datasets and our own…