I decided to use jiwo-0.8b because it had a high score on https://huggingface.co/spaces/multimodalart/jev-decision-index compared to its parameter count.
This GGUF conversion / quantization was generated by a local agent powered by Qwen3.8 27B. See agent-trajectory-log.md for exhaustive details.
Run with https://llama.app
git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
cmake -B build -DGGML_VULKAN=ON
cmake --build build --config Release
./build/bin/llama-server -hf breaksoftware/jiwo-0.8b-GGUF
This is a decision model, to be used via /v1/systemone API. See https://github.com/ggml-org/llama.cpp/pull/29818
Source models
- https://huggingface.co/eljiwo/jiwo-0.8b
Usage example
curl -s http://localhost:9931/v1/systemone \
-H "Content-Type: application/json" \
-d '{
"state": "Paris is the capital of France.",
"questions": {
"c": {
"type": "choice",
"instructions": "Which country is Paris the capital of?",
"criteria": { "France": null, "Japan": null, "Brazil": null }
}
}
}'
how to upload to huggingface:
nerdctl build --progress plain -t hfcli .
nerdctl run -e HF_TOKEN=<token that has write permissions> -v './:/mymodel' --entrypoint /bin/bash hfcli -c 'cd /mymodel && whoami && ls -lah && /root/.local/bin/hf upload breaksoftware/jiwo-0.8b-GGUF .'