Using llama.cpp release b3825 for quantization. Original model: https://huggingface.co/ArliAI/Mistral-Small-22B-ArliAI-RPMax-v1.1 All quants made using imatrix option with dataset from here Run them in LM Studio Some of these quants (Q3KXL, Q4KL etc) are the standard quantization method with the embeddings and output weights quantized to Q80 instead of what they would normally default to. Some say that this improves the quality, others don't notice any difference. If you use these models PLEASE COMMENT with your findings. I would like feedback that these are actually used and useful so I don't keep uploading quants no one is using. Thanks! First, make sure you have hugginface-cli installed…
Independent publisher
Dan
dan517
Models in Library1
Datasets in Library0
Models on Hugging Face1
Followers—
Compare the published evidence
This profile connects the 1 model and dataset record shown below to their upstream cards and sources. The library counts describe SAVRN's coverage, rather than this publisher's entire output.
Published tasks: Text generation.
Reported licenses: other. Open a record to review its specific license and conditions.
Compare model specifications, source dates, and hardware estimates on the individual records. Publisher identity verification is separate from a model's quality, deployment readiness, or suitability for your work.