A modern 150M-parameter decoder-only language model. Engineered by Sadat Sami · Released under the Apache License 2.0 808 AI is an open-source language model designed and engineered by Sadat Sami as part of his research into efficient, modern language model architectures. It is a compact decoder-only transformer built for fine-tuning on conversational and instruction-following tasks, and can be trained end-to-end on a single workstation GPU. - Grouped Query Attention (GQA) — 12 query heads, 4 key/value heads. - Query-Key Normalization (QK-Norm) — RMSNorm on q/k heads before RoPE. - Layer-wise Sliding Window Attention — alternating local/global pattern. - Rotary Position Embeddings (RoPE)…
Independent publisher
Sadatsami
Sadatsami
LLM Fine-tuning, Open-Source Datasets, Small Language Models (SLMs), NLP Research
Models in Library1
Datasets in Library0
Models on Hugging Face4
Followers—
Compare the published evidence
This profile connects the 1 model and dataset record shown below to their upstream cards and sources. The library counts describe SAVRN's coverage, rather than this publisher's entire output.
Compare model specifications, source dates, and hardware estimates on the individual records. Publisher identity verification is separate from a model's quality, deployment readiness, or suitability for your work.