SAVRN
Search Contact SAVRN

Organization

FWKV Project

FWKV

Making LM & RNNs even faster

Models in Library1
Datasets in Library0
Models on Hugging Face9
Followers21

Models

Model · Text generation

Myosotis-1-base

FWKV Project

Myosotis-1-base is the first flagship release from us, introducing a 100-million parameter recurrent language model built on the FWKV architecture. Myosotis-1 is engineered to never truly forget—using a mathematically clamped exponential decay that guarantees an infinite effective context window while maintaining blazing-fast inference on consumer hardware. Instead of pairwise attention, Myosotis uses a fixed-size state vector updated via a gated linear recurrence: $$ St = S{t-1} \odot W + kt \odot vt $$ - \\( W = \text{clamp}(\sigma(w), 0.1) \\) is a learned, constant-per-channel decay. - By clamping the minimum decay to 0.1, the model guarantees that past information decays exponentially…

Open weights apache-2.0 102M parameters transformers