A 27B dense model compressed to 3.66 effective bits, built from a measurement rather than a convention - and the build that answered a question this project had been avoiding. Most low-bit quants decide which layers to protect by rule of thumb. This one was built from a probe: every depth band of this specific model was pushed to 2-bit one at a time and scored on held-out text, and the protection went where the damage actually was (the last quarter of the network, blocks 51-63). Every equal-bytes win behind this method had been measured on mixture-of-experts models. MoE has a structural feature a dense model lacks - most of each layer's weight sits in routed experts that move between memory…
Independent publisher
Federico Sciuca
FedericoSciuca
Models in Library1
Datasets in Library0
Models on Hugging Face2
Followers—
Compare the published evidence
This profile connects the 1 model and dataset record shown below to their upstream cards and sources. The library counts describe SAVRN's coverage, rather than this publisher's entire output.
Published tasks: Text generation.
Compare model specifications, source dates, and hardware estimates on the individual records. Publisher identity verification is separate from a model's quality, deployment readiness, or suitability for your work.