Text Generation Models

本仓库提供由 Qwen/Qwen3-1.7B 转换得到的 RKNN LLM 模型。 - Model ID：RKNNAI/RK182X-LLM-Qwen3-1.7B - 模型显示名称：RK182X-LLM-Qwen3-1.7B - 源模型：Qwen/Qwen3-1.7B - 模型类型：LLM ModelScope 完整下载： Hugging Face 完整下载： ModelScope 指定配置下载： Hugging Face 指定配置下载： - 使用配套 RKNN Runtime 和驱动；运行前用 rknn-smi -v 检查设备端版本。 源模型许可证见根目录 LICENSE；同时遵守 RKNN Toolkit 和 RKNN Runtime 许可条款。

本仓库提供由 Tencent-YouTu-Research/Youtu-LLM-2B 转换得到的 RKNN LLM 模型。 - Model ID：RKNNAI/RK182X-LLM-Youtu-LLM-2B - 模型显示名称：RK182X-LLM-Youtu-LLM-2B - 源模型：Tencent-YouTu-Research/Youtu-LLM-2B - 模型类型：LLM ModelScope 完整下载： Hugging Face 完整下载： ModelScope 指定配置下载： Hugging Face 指定配置下载： - 使用配套 RKNN Runtime 和驱动；运行前用 rknn-smi -v 检查设备端版本。 - 源模型许可证：Youtu-LLM License。 - 本仓库许可证全文见根目录 LICENSE，归属声明见 Notice。 - 使用时还需遵守源模型、RKNN Toolkit 和 RKNN Runtime 的相关许可条款。

We introduce the updated version of the Qwen3-4B non-thinking mode, named Qwen3-4B-Instruct-2507, featuring the following key enhancements: - Significant improvements in general capabilities, including instruction following, logical reasoning, text comprehension, mathematics, science, coding and tool usage. - Substantial gains in long-tail knowledge coverage across multiple languages. - Markedly better alignment with user preferences in subjective and open-ended tasks, enabling more helpful responses and higher-quality text generation. - Enhanced capabilities in 256K long-context understanding. Qwen3-4B-Instruct-2507 has the following features: NOTE: This model supports only non-thinking…

Base: google/gemma-4-12B-it, loaded text-only (the decoder of the unified multimodal checkpoint; see scripts/port/maketextonlyckptgemma4.py). Adapter: LoRA r32, alpha 32, on q/o/gate/up/down (all 48 layers) + k/v (the 40 sliding-window layers). Standard PEFT format: PeftModel.frompretrained(, repo, subfolder=...). Thinking uses Gemma's thought... channel (rewardlaundering/gemma4renderer.py), temperature 1.0, think budget 2500. Research checkpoint from the reward-laundering project ("ext-model": does owned-haiku reward laundering replicate on a non-Qwen model family?). Full write-up: results/EXTMODEL.md. Held-out haiku = evalpropensity --trait haiku --locus response, 16 neutral prompts x 8…

Base: google/gemma-4-12B-it, loaded text-only (the decoder of the unified multimodal checkpoint; see scripts/port/maketextonlyckptgemma4.py). Adapter: LoRA r32, alpha 32, on q/o/gate/up/down (all 48 layers) + k/v (the 40 sliding-window layers). Standard PEFT format: PeftModel.frompretrained(, repo, subfolder=...). Thinking uses Gemma's thought... channel (rewardlaundering/gemma4renderer.py), temperature 1.0, think budget 2500. Research checkpoint from the reward-laundering project ("ext-model": does owned-haiku reward laundering replicate on a non-Qwen model family?). Full write-up: results/EXTMODEL.md. Held-out haiku = evalpropensity --trait haiku --locus response, 16 neutral prompts x 8…

Model · Text generation

romeo

Convergent Intelligence

This is the model card of a transformers model that has been pushed on the Hub. Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. Use the code below to get started with the model. Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).

A LoRA adapter for Qwen/Qwen3.5-4B that turns an outing's facts into a short card in Telugu for a parent who is visiting family abroad: where they're going, when, how to get there and back, and one thing they'll like. It is the card writer of Roundtrip, a planner that helps visiting parents get out to real places on their own. Cards are coastal Andhra (Guntur) Telugu, warm and respectful (మీరు forms, అమ్మా for a mother and నాన్నగారు for a father), 60 words or fewer, with every bus, street, stop and venue name kept in English letters so it matches the signs. - 342 training cards for 360 synthetic outings, and 40 more outings held out for evaluation. - Venues, bus lines and stop names come…

Model · Text generation

RPBizkit-v11-12B-Base

Ricardo_estep

"Experimental Mistral Nemo Mix" using... "Model Stock merge with Mergekit". This is a mix of some known "RP Uncensored" models. I'm not responsible of the use of this. After noticing a kind of "Assistant Tone" in V10, i remade the mix using "a cleaner" list... It works pretty nice and stable, and it's the "Base" for something even more experimental. TheDrummer/Rocinante-X-12B-v1 as base. The Tokenizer & Embedding Sizes (131072) are clean + It's "supposed to" support now a full "128K Context Size" (patched at 1M). The chat template is added with "full mistral" support. Quantized Model, Here.

Model · Text generation

RPBizkit-v11-12B-Thinking

Ricardo_estep

"Experimental Mistral Nemo Mix" using... "Model Stock merge with Mergekit". This is a mix of some known "RP Uncensored" models. I'm not responsible of the use of this. Take it as a "pure experiment", that works. RicardoEstep/RPBizkit-v11-12B-Base as base. The Tokenizer & Embedding Sizes (131072) are clean + This version "supposed to" support a full "512K+ Context Size" (patched at 1M). The chat template is the same as "heretic" one. Quantized Model, Here.

Model · Text generation

RPBizkit-v8-12B

Ricardo_estep

"Experimental Mistral Nemo Mix" using... "Model Stock + SLERP merges with Mergekit". This is my "last version", and actually the most stable one. This is a mix of some known "RP Uncensored" models. I'm not responsible of the use of this. After noticing the fg "chaotic" last version, that works at worst... I decided to go "even simplier". I had to reduce the overall models as some are Mixes from a Mix, i tried my best using only "FineTunes". natong19/Mistral-Nemo-Instruct-2407-abliterated # Base! natong19/Mistral-Nemo-Instruct-2407-abliterated # Base!.\mergekitconfig1 # RP Core.\mergekitconfig2 # Substance The Tokenizer & Embedding Sizes (131072) are clean + It's "supposed to" support now a…

Model · Text generation

Ru-Small-Instruct

LongTime

Ru-Small-Instruct — экспериментальная компактная русскоязычная языковая модель класса SLM (Small Language Model) с объемом параметров ~0.2B (~165M). Разработана с упором на суверенность весов (Zero-Fingerprint): модель обучена с нуля без заимствования базовых чекпоинтов у сторонних корпоративных сетей (Llama 3 от Meta, Qwen от Alibaba, Mistral). Модель предназначена для исследований локального инференса, работы на маломощном оборудовании, CPU и мобильных чипах, где критичны нулевая задержка (Time-To-First-Token) и полная независимость весов. Для компактной модели в 165M параметров, обученной на одном домашнем GPU за 48 часов, способность держать роль, грамотно формулировать сложные термины…

weighted/imatrix quants of https://huggingface.co/ritiksuman/rudra For a convenient overview and download list, visit our model page for this model. static quants are available at https://huggingface.co/mradermacher/rudra-GGUF This is a vision model - mmproj files (if any) will be in the static repository. If you are unsure how to use GGUF files, refer to one of TheBloke's READMEs for more details, including on how to concatenate multi-part files. (sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) Here is a handy graph by ikawrakow comparing some lower-quality quant And here are Artefact2's thoughts on the matter…

This model is a fine-tuned version of francesca9805/ppt-wc-zipf-newlex-rus-before-100mb-packed-bfdisoseed10. It has been trained using TRL. This model was trained with SFT.

This model is a fine-tuned version of francesca9805/ppt-wc-zipf-newlex-rus-before-100mb-packed-bfdisoseed455. It has been trained using TRL. This model was trained with SFT.

This model is a fine-tuned version of francesca9805/rus-cyrl-100mb-ppt-Dp-10mb-packed-bfdisoseed3407. It has been trained using TRL. This model was trained with SFT.

This model is a fine-tuned version of francesca9805/rus-cyrl-10mb-ppt-Dp-100mb-packed-bfdisoseed10. It has been trained using TRL. This model was trained with SFT.

Model · Text generation

RusMind-3.2-Momentum

DICPlay

Национальный ИИ от DICPlay. Самая лёгкая модель семейства RusMind — работает прямо на вашем устройстве, без интернета. Моментум — облегчённая модель семейства RusMind 3.2, созданная специально для смартфонов и нетребовательных компьютеров. Она отвечает быстро, работает без подключения к сети, и ни одно слово разговора не покидает устройство. RusMind обучалась на огромном массиве данных, а затем прошла специальное дообучение на русскоязычном корпусе. В него вошли: Приложения с GGUF (LM Studio, Ollama, PocketPal и другие) и llama.cpp — берите целый файл: Файлы rusmind-momentum-3.2-000NN-of-00009.gguf — та же модель, разрезанная на части до 1 ГБ для загрузки в браузер на rusmind.ru. Все…

Model · Text generation

RusMind-3.2-Vspyshka

DICPlay

Национальный ИИ от DICPlay. Модель для повседневных задач, которая работает прямо на вашем устройстве — без интернета. Вспышка — модель семейства RusMind 3.2 для повседневных задач: письма, тексты, учёба, объяснения. Она рассчитана на мощные компьютеры и флагманские смартфоны, работает без подключения к сети, и ни одно слово разговора не покидает устройство. RusMind обучалась на огромном массиве данных, а затем прошла специальное дообучение на русскоязычном корпусе. В него вошли: Приложения с GGUF (LM Studio, Ollama, PocketPal и другие) и llama.cpp — берите целый файл: Файлы rusmind-vspyshka-3.2-000NN-of-00014.gguf — та же модель, разрезанная на части до 1 ГБ для загрузки в браузер на…

Model · Text generation

RusMind-3.5-Vspyshka

DICPlay

Национальный ИИ от DICPlay. Модель на устройстве для повседневных задач: пишет, объясняет, помогает с учёбой и кодом. Работает без интернета на мощных ПК и флагманских телефонах. LM Studio, Ollama, PocketPal и другие программы с GGUF — скачайте нужный файл и откройте его. RusMind обучалась на огромном массиве данных и прошла дообучение на русскоязычных наборах DICPlay: история и культура России, законы, регионы и народы страны, традиционные семейные ценности, ключевые события — на русском и английском. Распространяется по собственной лицензии RusMind License (DICPlay) — текст в файле LICENSE. Коротко: бесплатно для личного и некоммерческого использования, исследований и оценки; коммерческое…

s1 is an open, calibrated System One decision model. Given a state (text or JSON) and a typed question, it returns a probability for every option in one forward pass, with no text generated: It is google/gemma-4-26B-A4B-it (mixture of experts, 26B total and about 4B active parameters) fine-tuned with LoRA on about 100,000 typed decision rows, and merged. Code, data and Held-out test sets. The released snapshot was chosen on separate dev splits by a rule fixed before training. - Mean accuracy over JevBench's 11 accuracy slices; ECE with 10 bins. Raw ECE is 0.083 (Jev: 0.081). - Paired against the untuned model on the same JevBench cases: 103 answers fixed, 58 broken (McNemar p = 0.0005).…

Converted with mlx-lm 0.31.3. A six-case deterministic sanity check matched three upstream reference strings exactly. The remaining differences included a retained leading “So,” punctuation/ordinal variation, and omission of “tomorrow” in one correction case. This is a small functional check, not the upstream 7,519-case evaluation; assess the 4-bit build on your own transcripts. A 0.6B-parameter text normalizer for speech-to-text output. It takes a raw ASR transcript and rewrites it as clean written text: fillers removed, false starts and self-corrections resolved to the value the speaker landed on, punctuation and capitalization applied, and spoken numbers, dates, times, currency and email…

Converted with mlx-lm 0.31.3. A six-case deterministic sanity check matched four upstream reference strings exactly. The two differences were a retained leading “So,” and one comma variation. This is a small functional check, not the upstream 7,519-case evaluation. A 0.6B-parameter text normalizer for speech-to-text output. It takes a raw ASR transcript and rewrites it as clean written text: fillers removed, false starts and self-corrections resolved to the value the speaker landed on, punctuation and capitalization applied, and spoken numbers, dates, times, currency and email addresses rendered in written form. On a held-out set of 7,519 English cases it reaches 94.8% token accuracy, and…

Model · Text generation

Sakura-K2-Horizon-0.9B-GGUF

Sakura

K2-Horizon-0.9B is the smallest member of IFM's open K2-Horizon family: a tiny decoder-only reasoning and agent model with a long context window (Apache-2.0, fully open training data and recipe). This repository holds three GGUF files of different size made from the BF16 weights with our measured mixed-codec method. Pick one on the Files page; the table below says what each file is. Independent community quantization, not an official IFM (MBZUAI Institute of Foundation Models) release. Part of the Sakura Mini line. Same method, same texts, same machine for every file; lower KL divergence means closer to the BF16 original: - Sakura-K2-Horizon-0.9B-0.40GiB.gguf vs llama.cpp default IQ3XXS…

Model · Text generation

Sakura-Qwen3.8-Flash-Next-Swift-DE-EN-K352-IQ3_XXS-GGUF

Sakura

An experimental expert-pruned GGUF of UkisAI's Swift 1.5 fine-tune of Qwen3.8-Flash-Next: 352 of 512 routed experts per layer kept (chosen on German + English routing), the rest cut out of the ~3-bit (IQ3XXS, GSQ-RCO) quantization without re-quantizing. Size 58.19 GiB in two shards. Made for a single 64 GB AMD Strix Halo (Radeon 8060S); German and English are the target languages. Independent community build, not an official Qwen or UkisAI release. Part of the Sakura Micro line. - Fine-tune + quantization: ukisai/Swift-1.5-Qwen3.8-Flash-Next-GSQ-RCO-GGUF (IQ3XXS tier). Swift is the reason for this build: it uses clearly fewer thinking tokens than the base model. either language. Measured…

Sanskrit-322M-Base is a 322-million-parameter Sanskrit language model trained from scratch with its own 32,000-token SentencePiece Unigram tokenizer. It is intended for Sanskrit text completion, generation experiments, and further task-specific fine-tuning. This is a base pretrained model: it predicts the continuation of text. It has not been instruction-tuned or trained as a conversational assistant. It uses a Llama-compatible decoder architecture; its weights were initialized and trained independently, rather than adapted from a pretrained Llama checkpoint. Published by sraivante under the Apache License 2.0. The tokenizer uses identity normalization, byte fallback, and SentencePiece…

by AGK FIRE INC A Qwen3-8B fine-tune that tutors for the SAT (math + reading/writing), packaged as a Q4KM GGUF for local inference with llama.cpp, Ollama, LM Studio, and friends. Works through SAT questions step by step and finishes with a clear Answer: X line. Trained on SAT-style math and reading/writing questions with step-by-step solutions. Held-out eval: 160 questions, identical prompt and decoding for both (greedy, 400 tokens, thinking off), strict scoring = unparseable counted wrong. the adapter overfit toward math and wants more RW data before the teacher is frozen The tuned adapter was chosen as the quantization target. Q4KM accuracy on the same suite: run the "Measure the…

Research artifacts for the paper "Wiring Beats Blending: Structure-Aware Compensation for Transformer Downscaling" (arXiv:2608.02829, v3; v1–v2 carried the title "What Transfers Between Transformer Sizes — and What Doesn't"). The paper asks whether a pretrained large model can be converted into a smaller sibling instead of training the small size from scratch. These checkpoints are the endpoints of matched-budget recovery races: every arm gets the same target shape, data order, schedule, and token budget, and only the starting weights differ. The initializations compared: All files are plain PyTorch statedicts for GPTNeoXForCausalLM with the target size's stock Pythia config. They are…

Two Qwen3-4B full-finetunes that judge one proposed agent action, before it executes, against the observed prefix and the task policy, and emit a single verdict box. They differ only in SFT learning rate. act is CONTINUE / ASK / STOP. The binary projection is CONTINUE=SAFE, STOP=UNSAFE, and ASK=UNSAFE at risk >= 50. cite points at an earlier visible step, or NONE. fail / harm / src come from a 16 failure-mode, 11 harm-type, 10 risk-source taxonomy; NONE is an absence sentinel, not an additional class. Full SFT from Qwen/Qwen3-4B, epoch 2, thinking enabled, on 2,464 step-level records. Prompts were rendered through the evaluator's own pipeline, so the training and inference formats match.…

Model · Text generation

semsimula-ladder-live-owt-d384-l2-attention-potential-rfqk-norc

Dimitar Gueorguiev

Settled is the mean of the last three evals (54.20, 54.39, 54.22); best 52.50 at step 31,000, final 54.22. Consecutive evals near the end differ by about ±1 PPL. - The field alone is a real but weaker route. It lowers the conservative-only model's perplexity by 6.0%; the register path alone lowers it by 8.0%. - Beside the registers it adds almost nothing: G3′ is 0.4% better than G2. The two routes largely substitute for each other. This model sees exactly 532,480,000 training tokens — 32,500 steps × 32 × 512 = 532,480,000 tokens — the same budget, corpus, tokenizer and validation batches as every other model in the table. No model is trained longer than another. - The base is F3.1: the…

Model · Text generation

semsimula-ladder-owt-d384-l2-attention

Dimitar Gueorguiev

One arm of the SemSimula mechanism ladder — OpenWebText, d=384, L=2: five models that differ by exactly one mechanism each, so that every gap in the ordering prices one component by construction rather than by attribution. Italic entries are pre-registered predictions, recorded before the run. The ordering is itself the prediction; any inversion is a result. Every arm above sees exactly 532,480,000 training tokens — 32,500 steps × 32 × 512 = 532,480,000 tokens — on the same corpus, tokenizer and validation batches. No arm is trained longer than another, so a gap in the table is a gap between mechanisms and never a difference in budget. The models below integrate a damped mechanical system…

Model · Text generation

semsimula-ladder-owt-d384-l2-attention-potential

Dimitar Gueorguiev

One arm of the SemSimula mechanism ladder — OpenWebText, d=384, L=2: five models that differ by exactly one mechanism each, so that every gap in the ordering prices one component by construction rather than by attribution. Italic entries are pre-registered predictions, recorded before the run. The ordering is itself the prediction; any inversion is a result. Every arm above sees exactly 532,480,000 training tokens — 32,500 steps × 32 × 512 = 532,480,000 tokens — on the same corpus, tokenizer and validation batches. No arm is trained longer than another, so a gap in the table is a gap between mechanisms and never a difference in budget. The models below integrate a damped mechanical system…

Model · Text generation

semsimula-ladder-owt-d384-l2-gpt2-matched

Dimitar Gueorguiev

One arm of the SemSimula mechanism ladder — OpenWebText, d=384, L=2: five models that differ by exactly one mechanism each, so that every gap in the ordering prices one component by construction rather than by attribution. Italic entries are pre-registered predictions, recorded before the run. The ordering is itself the prediction; any inversion is a result. Every arm above sees exactly 532,480,000 training tokens — 32,500 steps × 32 × 512 = 532,480,000 tokens — on the same corpus, tokenizer and validation batches. No arm is trained longer than another, so a gap in the table is a gap between mechanisms and never a difference in budget. The models below integrate a damped mechanical system…

Model · Text generation

semsimula-ladder-owt-d384-l2-none

Dimitar Gueorguiev

One arm of the SemSimula mechanism ladder — OpenWebText, d=384, L=2: five models that differ by exactly one mechanism each, so that every gap in the ordering prices one component by construction rather than by attribution. Italic entries are pre-registered predictions, recorded before the run. The ordering is itself the prediction; any inversion is a result. Every arm above sees exactly 532,480,000 training tokens — 32,500 steps × 32 × 512 = 532,480,000 tokens — on the same corpus, tokenizer and validation batches. No arm is trained longer than another, so a gap in the table is a gap between mechanisms and never a difference in budget. The models below integrate a damped mechanical system…

Model · Text generation

semsimula-ladder-owt-d384-l2-none-norc

Dimitar Gueorguiev

One arm of the SemSimula mechanism ladder — OpenWebText, d=384, L=2: five models that differ by exactly one mechanism each, so that every gap in the ordering prices one component by construction rather than by attribution. Italic entries are pre-registered predictions, recorded before the run. The ordering is itself the prediction; any inversion is a result. Every arm above sees exactly 532,480,000 training tokens — 32,500 steps × 32 × 512 = 532,480,000 tokens — on the same corpus, tokenizer and validation batches. No arm is trained longer than another, so a gap in the table is a gap between mechanisms and never a difference in budget. The models below integrate a damped mechanical system…

Model · Text generation

sft_gene_pathway_v9_9_100k_a256bs32_9b_celltext

WangLab UofT

Qwen3.5-9B fine-tuned to reason mechanistically about genetic (CRISPR) perturbations and predict how a queried pathway responds — upregulated, downregulated, or unchanged — in a given cell. This is the cell-text arm of BioReasonCell: the v9.9 SFT recipe, with one change. The model gets no STATE cell embedding. It reads the cell as text instead, a Cell Context: block in the prompt, so it is a plain text model and every input it needs can be built from a new screen's h5ad. The block follows the queried pathway, and the prompt's opening reads "Given a cell's context, a genetic perturbation, …": Top Expressed Genes are the 50 most abundant genes of the line's non-targeting control cells…

Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. Use the code below to get started with the model. Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019). - PEFT 0.20.0

Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. Use the code below to get started with the model. Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019). - PEFT 0.20.0

Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. Use the code below to get started with the model. Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019). - PEFT 0.20.0

Model · Text generation

SkillForge_Alfworld_7B

YuyaoGe

This is the final SkillForge policy for ALFWorld: a full fine-tune of Qwen2.5-7B-Instruct trained with GRPO while its skill library was simultaneously forged — retired, stabilized, demoted and mutated — under the fitness-driven lifecycle described in the paper. Most memory-augmented agents keep the library append-only. A skill that was correct at step 20 encodes a procedure the policy has outgrown by step 120, and it is still being retrieved into the context. SkillForge instead scores each skill against the policy's own rollouts and moves it between four states — trial, active, stable, retired — so the library and the model co-evolve. On ALFWorld this reaches 92.4% overall success against…

Model · Text generation

SkillForge_Search_7B

YuyaoGe

This is the final SkillForge policy for Search-Augmented QA: a full fine-tune of Qwen2.5-7B-Instruct trained with GRPO while its skill library was simultaneously forged — retired, stabilized, demoted and mutated — under the fitness-driven lifecycle described in the paper. Most memory-augmented agents keep the library append-only. A skill that was correct at step 20 encodes a procedure the policy has outgrown by step 120, and it is still being retrieved into the context. SkillForge instead scores each skill against the policy's own rollouts and moves it between four states — trial, active, stable, retired — so the library and the model co-evolve. On Search-Augmented QA this reaches 48.7%…

Model · Text generation

SkillForge_Webshop_7B

YuyaoGe

This is the final SkillForge policy for WebShop: a full fine-tune of Qwen2.5-7B-Instruct trained with GRPO while its skill library was simultaneously forged — retired, stabilized, demoted and mutated — under the fitness-driven lifecycle described in the paper. Most memory-augmented agents keep the library append-only. A skill that was correct at step 20 encodes a procedure the policy has outgrown by step 120, and it is still being retrieved into the context. SkillForge instead scores each skill against the policy's own rollouts and moves it between four states — trial, active, stable, retired — so the library and the model co-evolve. On WebShop this reaches 78.4% task success and 87.8…

Model · Text generation

Slayer149-balanced

Fabryka AI

Experimental 149,333,081-parameter English causal model trained from scratch. 26 layers, hidden width 576, FFN 2496, 9 query heads / 3 KV heads, tied 24,576-token byte-level BPE vocabulary, QK normalization and value residuals. Architecture informed by Qwen3 and AltSlate's Apache-2.0 Jugnu recipe; no Jugnu weights reused. This is a separate experiment from SlayerLab/Slayer149, which remains available. Training may still be in progress. See reports/status.json, training metrics and checkpoint manifests. No leaderboard rank or winning result is claimed. The final full GLINT-derived evaluation is reports/evaluation.json, when present. Competitor scores may use different evaluation conventions.…

Model · Text generation

slm-125m-base

Tijani Ohiokpehai

tohio/slm-125m-base is a small language model (125M parameters) built from scratch and aligned end-to-end using the slm-gpt engine. - Decoder-Only Transformer with Rotary Position Embeddings (RoPE) The base model was pre-trained across an interleaved multi-source domain mixture composed of: - FineWeb-Edu - DCLM-Edu - The Stack-Edu - NuminaMath-CoT - OpenMathReasoning - SLM-Synthetic-Pretrain 1. Pre-training: Multi-GPU Distributed Data Parallel (DDP) over rank-disjoint memory-mapped token shards. 2. Supervised Fine-Tuning (SFT): Full parameter instruction-tuning on ChatML formatted dialogues with prompt loss masking (ignoreindex=-100). 3. Direct Preference Optimization (DPO): Single-stage…

Model · Text generation

slm-125m-dpo

Tijani Ohiokpehai

tohio/slm-125m-dpo is a small language model (125M parameters) built from scratch and aligned end-to-end using the slm-gpt engine. - Decoder-Only Transformer with Rotary Position Embeddings (RoPE) The base model was pre-trained across an interleaved multi-source domain mixture composed of: - FineWeb-Edu - DCLM-Edu - The Stack-Edu - NuminaMath-CoT - OpenMathReasoning - SLM-Synthetic-Pretrain 1. Pre-training: Multi-GPU Distributed Data Parallel (DDP) over rank-disjoint memory-mapped token shards. 2. Supervised Fine-Tuning (SFT): Full parameter instruction-tuning on ChatML formatted dialogues with prompt loss masking (ignoreindex=-100). 3. Direct Preference Optimization (DPO): Single-stage…

Model · Text generation

slm-125m-sft

Tijani Ohiokpehai

tohio/slm-125m-sft is a small language model (125M parameters) built from scratch and aligned end-to-end using the slm-gpt engine. - Decoder-Only Transformer with Rotary Position Embeddings (RoPE) The base model was pre-trained across an interleaved multi-source domain mixture composed of: - FineWeb-Edu - DCLM-Edu - The Stack-Edu - NuminaMath-CoT - OpenMathReasoning - SLM-Synthetic-Pretrain 1. Pre-training: Multi-GPU Distributed Data Parallel (DDP) over rank-disjoint memory-mapped token shards. 2. Supervised Fine-Tuning (SFT): Full parameter instruction-tuning on ChatML formatted dialogues with prompt loss masking (ignoreindex=-100). 3. Direct Preference Optimization (DPO): Single-stage…

Model · Text generation

Smol-GPT-Fp8

Gggff123

This llama model was trained 2x faster with Unsloth and Huggingface's TRL library.

Model · Text generation

Smol-GPT-RAW

Gggff123

This llama model was trained 2x faster with Unsloth and Huggingface's TRL library.

This model is a fine-tuned version of HuggingFaceTB/SmolLM2-135M on an unknown dataset. It achieves the following results on the evaluation set: The following hyperparameters were used during training: - learningrate: 5e-05 - trainbatchsize: 4 - evalbatchsize: 8 - lrschedulertype: linear - trainingsteps: 1000 - Transformers 4.46.3 - Pytorch 2.11.0+cu130 - Datasets 4.8.5 - Tokenizers 0.20.3

Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. Use the code below to get started with the model. Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019). - PEFT 0.18.1

Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. Use the code below to get started with the model. Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019). - PEFT 0.18.1

Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. Use the code below to get started with the model. Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019). - PEFT 0.19.1