Llama.cpp imatrix quantization of THUDM/glm-4-9b-chat If you do not have hugginface-cli installed: Download the specific file you want: If the model file is big, it has been split into multiple files. In order to download them all to a local folder, run: According to this investigation, it appears that lower quantizations are the only ones that benefit from the imatrix input (as per hellaswag results). 1. Make sure you have gguf-split available - To get hold of gguf-split, navigate to https://github.com/ggerganov/llama.cpp/releases - Download the appropriate zip for your system from the latest release - Unzip the archive and you should be able to find gguf-split 2. Locate your GGUF chunks…