tinybrainbot-360m-a75m-v4.1-moe-instruct

This model is a fine-tuned version of None. It has been trained using TRL. This model was trained with SFT.

RoPE-free long-range attention, trained at 4k with zero-shot length extrapolation. FS-B searches full-prefix and norm-filtered forests of chunk mean keys, reranks their candidates, and reads four chunks. Its attention is superlinear and subquadratic. This 341M research base model was trained on FineWeb-Edu at a 4,096-token context. The short retrieval example asks for the hidden number 5437923. Start with CUDA-enabled PyTorch; Usage gives the tested installation and speed-replication command. FreeSolo learns simple chunk mean keys that support search, norm pruning, and hierarchical summarization. RoPE stays in the local window; long-range attention reads selected original tokens without…

The three-language base with a 2,048-token context: AnuLM-Base-400M continued for 10,000 steps at block 2,048 with YaRN, on 46M tokens of the same Hindi / English / Python proportions it was originally trained on. Five hours on one RTX 5070 Ti. Full log and the honest reading of what it bought: docs/RESULTS.md §28, with the zero-shot measurement it is compared against Hindi and English Wikipedia are CC BY-SA, C4 is ODC-BY, and the Python slice is codeparrot-clean, de-duplicated GitHub Python with mixed licences. Not affiliated with Sarvam AI, AI4Bharat, BharatGen or the Government of India. Every other checkpoint in this project is trained at 512 tokens. This one answers what happens if you…

Model · Text generation

SMOLM2Prover

Convergent Intelligence

SmolLM2Prover is a specialized, fine-tuned version of prithivMLmods/SmolLM2-CoT-360M. While retaining the strong conversational abilities of its base model, this version has been specifically enhanced to excel at deep thinking, logical reasoning, and higher-level mathematics, with a focus on generating step-by-step proofs and explanations (Chain-of-Thought). The model was fine-tuned using multiple rounds of Supervised Fine-Tuning (SFT) with the TRL library on a curated dataset, enhancing its ability to follow complex instructions and reason through problems. This model is intended to be used for text generation tasks that require logical reasoning or advanced conversation. The easiest way…

This is the model card of a transformers model that has been pushed on the Hub. Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. Use the code below to get started with the model. Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).