GGUF files of commreteris/ancient-words-o1
for llama.cpp. The model reads a scanned book page and returns its regions and every word, each
with a polygon, as one JSON object. The prompt, output format, training and evaluation are
described in that repository's model card.
| File |
Contents |
Size |
ancient-words-o1-Q8_0.gguf |
Language model, Q8_0 |
812 MB |
mmproj-ancient-words-o1-F16.gguf |
Vision encoder and projector, F16 |
205 MB |
Both files are needed. They were converted from the float32 weights with llama.cpp 0c1e570
(convert_hf_to_gguf.py --outtype q8_0 --no-mtp, and --mmproj --outtype f16). Qwen3.5
support needs a llama.cpp build from that commit or later. The fine-tuned checkpoint has no
multi-token-prediction head, so these files cannot be used for MTP speculative decoding.
Usage
llama.cpp needs a Hugging Face token with read access in
HF_TOKEN.
llama-server -hf commreteris/ancient-words-o1-gguf:Q8_0 --jinja -c 65536 \
--image-min-tokens 1024 --image-max-tokens 16384
The model was trained on pages at their scanned resolution. llama.cpp shrinks large images to at
most --image-max-tokens image tokens (one token per 32×32 pixel patch), so raise it for large
scans, and keep --image-min-tokens at 1024 or more so small images are not read at too low a
resolution.
Send one user message per page with the image first and then this prompt text. The chat template
disables thinking by default, which is how the model was trained.
Extract every document region and orphan word in source order. Return each region's source type and full polygon, and each word as its ordered text/polygon parts. Coordinates are page-relative 0..10000 integer values; preserve coordinates outside that range, arbitrary vertex counts, empty records, multipart words, and text exactly.
The answer is one line of JSON. Depending on the client and llama.cpp's reasoning settings, it
can be preceded by an empty <think></think> block, which must be removed before parsing. Coordinates are integers on a 0..10000 grid relative to the page's
width and height. Dense pages can need more than 32,000 output tokens.
