GGUF quantizations of simpledirect/Vinci-Bozza-1.0 for local inference with Ollama, LM Studio, and llama.cpp.
Text-only. The source model is multimodal (image-text-to-text); these GGUF conversions include the language model only — no mmproj / vision projector. For image input, use the safetensors build: simpledirect/Vinci-Bozza-1.0.
These files are not the measured artifact
No GGUF file in this repository has been evaluated. The benchmark figures on the full-precision card were measured on the unquantized bf16 weights, not on any file here. Quantization shifts scores, and with no per-tier measurement this card cannot say in which direction or by how much for any tier.
This card therefore publishes no per-tier score, no quality ranking between the tiers, and no recommended tier. The file sizes and memory figures below are measured, and describe storage and memory cost only. A file's presence in this repository is not a performance claim about that tier, and not a claim of parity with the full-precision weights.
Variants
| File |
Size |
Notes |
Vinci-Bozza-1.0-Q6_K.gguf |
~6.8 GB |
6-bit K-quant |
Vinci-Bozza-1.0-Q5_K_M.gguf |
~6.0 GB |
5-bit K-quant, medium mixture |
Vinci-Bozza-1.0-Q4_K_M.gguf |
~5.2 GB |
4-bit K-quant, medium mixture |
Quick start
# Ollama — default quant auto-selected
ollama run hf.co/simpledirect/Vinci-Bozza-1.0-GGUF
# Specific quant
ollama run hf.co/simpledirect/Vinci-Bozza-1.0-GGUF:Q5_K_M
llama.cpp:
llama-cli -m Vinci-Bozza-1.0-Q5_K_M.gguf --chat-template chatml -p "Hello, who are you?"
Hardware requirements
| Variant |
GPU VRAM |
System RAM (CPU-only) |
| Q6_K |
10 GB |
16 GB |
| Q5_K_M |
8 GB |
14 GB |
| Q4_K_M |
7 GB |
12 GB |
Mac M-series: Q5_K_M runs comfortably on 16 GB unified memory; Q6_K needs 24 GB.
Prompt format
Uses Qwen / ChatML chat template. Pass enable_thinking=False (or equivalent) to suppress <think> blocks for normal chat use.
Links
- Full model (safetensors, BF16): simpledirect/Vinci-Bozza-1.0
- GitHub: https://github.com/getsimpledirect
- The Constitution: https://guide.getsimpledirect.com/constitution