CrowdGPT's first community-distributed language model architecture. Crowd-v1 is the first official model architecture released for CrowdGPT, a community-driven distributed AI project. Unlike a conventional pretrained model release, Crowd-v1 is distributed with randomly initialized weights. The purpose of this release is to provide a common model definition and weight format that CrowdGPT clients can download and collectively train. The model is designed to be consumed by the CrowdGPT distributed training infrastructure, where individual participants contribute compute toward training a shared model. Crowd-v1 contains approximately 1 billion parameters. Grouped-Query Attention (GQA) Crowd-v1…
Independent publisher
Vxtzq
Vxtzq
LLMs, Computer-vision
Models in Library1
Datasets in Library0
Models on Hugging Face2
Followers—