deepseek-v4.1-flash-1017

GPT-Neo storyteller (nominal 33M; actual 68,514,048 parameters including the 50,257-token embedding) trained on TeenyTinyStories (ableian/TeenyTinyStories) batch 1280 x 512, constant LR 5e-4, weight decay 0.1, betas (0.9, 0.95), clip 1.0. - TS-valid capped to stories of at most 400 tokens (20,711 stories): 1.539 / ppl 4.66 Research on small-scale synthetic pretraining data. Generates simple children's stories; sample at temperature 0.8-1.0. See the TeenyTinyStories paper for the corpus recipe and judge evaluations. Apache-2.0.

Model · Text generation

DiscoverLM-70M

Convergent Intelligence

A 69M parameter causal language model built on the Mixture-of-Attentions (MoA) architecture — distance-based metric attention that respects the triangle inequality by construction, not approximation. Every attention head operates in a proper metric space. The geometry is enforced, not hoped for. Standard transformers compute attention as a dot product: Q·Kᵀ. This has no geometric meaning — it's a bilinear form, not a distance. Two tokens can be "close" by dot product while violating basic metric properties. MoA replaces this with negative squared distance under a learned diagonal Mahalanobis metric, then enforces the triangle inequality through a regularizer over random triples sampled…

Model · Text generation

Discovery

Convergent Intelligence

A 70M parameter causal language model built on the Mixture-of-Attentions (MoA) architecture — distance-based metric attention that respects the triangle inequality by construction, not approximation. Every attention head operates in a proper metric space. The geometry is enforced, not hoped for. Standard transformers compute attention as a dot product: Q·Kᵀ. This has no geometric meaning — it's a bilinear form, not a distance. Two tokens can be "close" by dot product while violating basic metric properties. MoA replaces this with negative squared distance under a learned diagonal Mahalanobis metric, then enforces the triangle inequality through a regularizer over random triples sampled…

Model · Text generation

lightning-60m

AobanZ

Lightning is a small, autoregressive transformer which utilizes FlashAttention and MHA. This model is trained on a variety of books from a dataset(300 MB). This is the expanded version of Lightning-30m. Lightning utilizes FlashAttention and AdamW for performance and capability. Lightning is designed to provide quick, coherent outputs, improved with a larger size and weight. Lightning is intended to be used for research, analysis and fine-tuning, stories and other. It is not intended to be used for professional advice, real writing or any kind of heavy work as generated outputs may be incorrect. Lightning can be used directly for text generation, experimentation, and conversational…

Model · Text generation

minimind-math-v3

Jiwubu

minimind-math-v3 是一个基于 MiniMind 项目从零训练的小型中文数学语言模型，权重已转换为 Qwen3 兼容格式，可直接使用 HuggingFace transformers 加载推理。 - 架构：Dense Transformer（Decoder-only），未使用 MoE - 数据：以数学/推理语料为主的预训练数据（pretraint2tmini.jsonl），并经过数学 SFT 训练 - Tokenizer：基于 MiniMind 自训 tokenizer 修改的 patched tokenizer，详见下方说明 本模型的预训练与 SFT 均基于 jiwubu/minimind 代码仓库完成。相对上游 jingyaogong/minimind 的主要修改是新增 math/ 目录，用于生成数学 SFT 用的加减乘除算术数据（genmathdataaddsub.py、genmathdatamuldiv.py 等），并提供配套评测脚本（evalmath.py）。 最简单的方式是用仓库自带的交互式验证脚本（chat template + 贪心解码 + 流式输出）： 或直接用 transformers： - 预训练 2 个 epoch，batch size 32，最大序列长度 340 - 优化器 AdamW，初始学习率 5e-4（带 warmup + 余弦衰减） - 混合精度：bfloat16 本模型使用基于 MiniMind 原版 tokenizer 修改的 patched tokenizer（Sequence[Digits, ByteLevel]）：…