SAVRN
Search Contact SAVRN

Independent publisher

Alby13

alby13

AI & ML interests

Models in Library1
Datasets in Library0
Models on Hugging Face1
Followers12

Models

Model · Text generation

LibraMindMini

Alby13

LibraMind is a 565M-parameter chat model pretrained from scratch by alby13 on a single consumer GPU (NVIDIA RTX 4090). It is a hybrid architecture that interleaves 18 Gated DeltaNet linear-recurrent layers with 6 Gated Attention layers (a repeating DDDA × 6 pattern), with Block Attention Residuals and logit soft-capping, trained with the Muon optimizer. It went through a full five-stage pipeline: pretraining, midtraining, supervised fine-tuning, preference tuning, and RL with verifiable rewards. - Research on small hybrid recurrent/attention language models - Studying low-resource, single-GPU training pipelines (pretraining through RL) and the Muon optimizer at small scale - Experiments…

Open weights apache-2.0 565M parameters pytorch