Using llama.cpp release b10262 for quantization. Don't know which to choose? Grab Q4KM (21.86GB) - usually a good mix of size and performance. Download instructions available here First, make sure you have the Hugging Face CLI installed: The files marked true in the Split column above are stored as multiple parts in a folder. To download all the parts to a local folder, run: You can either specify a new local-dir (endless-frontierBigBang-v1-bf16) or download them all in place (./) These quants run with llama.cpp - installable in one line via llama.app: llama-server includes a built-in chat web UI, served at http://localhost:8080 by default. These quants were made with llama.cpp release…
Open weights
apache-2.0
Using llama.cpp release b10142 for quantization. All quants made using imatrix option with dataset from here Run them in your choice of tools: Note: if it's a newly supported model, you may need to wait for an update from the developers. Some of these quants (Q3KXL, Q4KL etc) are the standard quantization method with the embeddings and output weights quantized to Q80 instead of what they would normally default to. First, make sure you have huggingface-cli installed: Then, you can target the specific file you want: If the model is bigger than 50GB, it will have been split into multiple files. In order to download them all to a local folder, run: You can either specify a new local-dir…
Open weights
apache-2.0
Using llama.cpp release b10068 for quantization. All quants made using imatrix option with dataset from here Run them in your choice of tools: Note: if it's a newly supported model, you may need to wait for an update from the developers. Some of these quants (Q3KXL, Q4KL etc) are the standard quantization method with the embeddings and output weights quantized to Q80 instead of what they would normally default to. First, make sure you have huggingface-cli installed: Then, you can target the specific file you want: If the model is bigger than 50GB, it will have been split into multiple files. In order to download them all to a local folder, run: You can either specify a new local-dir…
Open weights
other
transformers
Using llama.cpp release b10630 for quantization. Don't know which to choose? Grab Q4KM (17.04GB) - usually a good mix of size and performance. Download instructions available here First, make sure you have the Hugging Face CLI installed: These quants run with llama.cpp - installable in one line via llama.app: llama-server includes a built-in chat web UI, served at http://localhost:8080 by default. These quants were made with llama.cpp release b10630 - if this model's architecture is newly supported, you'll need that release or newer to run them. They also work in: model supports image and audio input. Alongside the quants, this repo includes the multimodal projector files…
Open weights
Using llama.cpp release b10262 for quantization. Don't know which to choose? Grab Q4KM (19.60GB) - usually a good mix of size and performance. Download instructions available here First, make sure you have the Hugging Face CLI installed: The files marked true in the Split column above are stored as multiple parts in a folder. To download all the parts to a local folder, run: You can either specify a new local-dir (TheDrummerArtemis-31B-v1.1-bf16) or download them all in place (./) These quants run with llama.cpp - installable in one line via llama.app: llama-server includes a built-in chat web UI, served at http://localhost:8080 by default. These quants were made with llama.cpp release…
Open weights
Using llama.cpp release b10068 for quantization. All quants made using imatrix option with dataset from here Run them in your choice of tools: Note: if it's a newly supported model, you may need to wait for an update from the developers. Some of these quants (Q3KXL, Q4KL etc) are the standard quantization method with the embeddings and output weights quantized to Q80 instead of what they would normally default to. First, make sure you have huggingface-cli installed: Then, you can target the specific file you want: If the model is bigger than 50GB, it will have been split into multiple files. In order to download them all to a local folder, run: You can either specify a new local-dir…
Open weights
apache-2.0
