Qwen3.5-0.8B Cyber — GGUF
A compact Qwen3.5 0.8B repository with a practical GGUF quantization ladder for local inference.
This card describes what is present in the repository. The public files do not document the fine-tuning dataset or provide evaluation results, so the Cyber label should be read as the repository variant name—not as a verified capability claim.
Repository contents
| Variant |
Approximate size |
Practical use |
BF16 |
1.52 GB |
Highest-fidelity reference build |
Q8_0 |
812 MB |
High-quality local inference |
Q6_K |
630 MB |
Smaller high-quality option |
Q4_K_M |
529 MB |
Recommended size/quality starting point |
Q2_K_L |
484 MB |
Minimum footprint; expect greater quality loss |
BF16-mmproj |
207 MB |
Multimodal projector artifact; see caveat below |
Run locally
With a recent llama.cpp build:
llama-cli \
-hf reaperdoesntknow/Qwen3.5-0.8-Cyber-GGUF:Q4_K_M \
--jinja
With Ollama:
ollama run hf.co/reaperdoesntknow/Qwen3.5-0.8-Cyber-GGUF:Q4_K_M
Multimodal note
The repository includes a BF16 mmproj file and its configuration includes vision components. That establishes that a projector artifact is present; it does not establish that the end-to-end multimodal path was validated for this release. Verify image input locally before depending on it.
Intended use
- Local experimentation with a small Qwen3.5 model.
- CPU- and memory-constrained prototyping.
- Reproducible comparison of quantization levels.
Do not treat the model as an authoritative source for security, legal, medical, or operational decisions.
Evaluation status and limitations
- No benchmark results or comparison baselines are published in this repository.
- The fine-tuning dataset and training recipe are not documented in the public files reviewed for this card.
- Quantization reduces memory use and can change output quality.
- Model outputs may be incorrect, incomplete, biased, or unsafe.
- The multimodal path requires independent verification.
Lineage and license
- Base model: Qwen/Qwen3.5-0.8B
- Format: GGUF
- Conversion tooling recorded by the repository: Unsloth / llama.cpp
- License: Apache-2.0
Part of the CIx cybersecurity model collection.
