Diplodocus preview 4, the Osaurus assistant for finance, documents, spreadsheets and tool-assisted work, built on IFM/K2-Horizon-7B and trained on Osaurus' own workspace tools (workbooks and documents edited in place, the built-in calculate tool, reasoning on and off). This is the GGUF build for llama.cpp: one calibrated Q6_K file, 7.76 GB.
Using Osaurus on a Mac? Use the MLX bundle OsaurusAI/Diplodocus-preview-JANG_6M instead. Same weights, same template, same reasoning toggle.
Requires llama.cpp with K2 Horizon support (ggml-org/llama.cpp#29535, merged 2026-10-06). Older builds, and apps that bundle an older llama.cpp, will not load the k2-horizon architecture.
What preview 4 adds over preview 3
Trained rounds R8–R11 on top of preview 3: the Osaurus calculate tool (when to call it, chained money arithmetic, recovery from engine errors), workbooks and documents through Osaurus' in-place file_edit operations (bounded file_read windows, mode:"structure", set_cells with live formulas, never regenerating an existing workbook), PDF forms and signed-PDF handling, and a loop fix for reasoning-off answers. The source's own held-out evaluation moved end-to-end workbook/document tasks from 59/51 (on/off, of 100) to 70/70 and calculator-executed long-document money answers from 31 to 43 of 60 with reasoning on.
Quantization
| Component |
Format |
| Attention projections (q, k, v, o; all 36 layers) |
Q8_0 |
| MLP gate / up / down projections (all 36 layers) |
Q6_K, importance-weighted |
| Embeddings and untied output head |
Q6_K, importance-weighted |
| Norms |
F32 |
File: Diplodocus-preview-Q6_K.gguf, 7,760,248,672 bytes, sha256 c81c80cc2f3d5f9fec0eb06d9a8732404ac37516fde2d8692eb49f6e1d39cfd9.
The importance matrix (Diplodocus-preview.imatrix.gguf, included) was computed on the BF16 model over 1,653,067 tokens of Osaurus tool conversations, reasoning-on/off traces and calculator chains, with chat and tool special tokens parsed as the model sees them.
The layout was chosen by measurement, not by default. Every candidate was scored against the BF16 source on a held-out set disjoint from calibration: 64 rows, 206k tokens, reasoning on and off, 41 with tool calls. Lower KL divergence is better; "top-1 same" is how often the quant picks the same next token as BF16.
| Candidate |
Size |
Mean KLD |
Median KLD |
Top-1 same |
| Q8_0 (reference) |
9.57 GB |
0.0285 |
0.0019 |
95.5% |
| Q6_K, standard |
7.39 GB |
0.1325 |
0.0109 |
89.7% |
| Q6_K + importance matrix |
7.39 GB |
0.1020 |
0.0076 |
91.3% |
| Q6_K + importance matrix + Q8_0 attention (shipped) |
7.76 GB |
0.0608 |
0.0044 |
93.1% |
| shipped + Q8_0 output head |
8.01 GB |
0.0603 |
0.0039 |
93.5% |
Moving the attention projections to 8-bit more than halves the standard Q6_K's divergence for +0.37 GB. This matches the JANG bundle's finding that preview 4's fine-tuning lives mostly in attention. An 8-bit output head bought almost nothing for another 0.25 GB, so it stays at Q6_K. For scale, this model is noise-sensitive: the JANG card measured BF16 with one-ULP embedding noise at 96.2% top-1 on its set. These are quantization diagnostics on training-distribution data, not an independent task-accuracy benchmark.
Reasoning, tools and sampling
- High: reasoning on (
reasoning_effort: "high", the default). Low: reasoning off, using the embedded template's empty prefilled reasoning block. Do not send medium.
- The embedded chat template is the same template shipped with the JANG bundle. Run
llama-server with --jinja so it is used.
- Default sampling is temperature 1.0, top_p 0.95, stamped in the GGUF metadata.
- Tool calls use the native
k2_horizon envelope. Upstream llama-server --jinja parses them into OpenAI tool_calls and puts reasoning in reasoning_content.
- For money and exact arithmetic, use reasoning on and let the model drive the
calculate tool.
llama-server -hf OsaurusAI/Diplodocus-preview-GGUF:Q6_K --jinja -ngl 99 -c 32768
curl http://127.0.0.1:8080/v1/chat/completions -H 'Content-Type: application/json' -d '{
"messages": [{"role": "user", "content": "What is 1234.56 * 0.0825, rounded to cents?"}],
"tools": [{"type": "function", "function": {"name": "calculate", "description": "Evaluate an arithmetic expression exactly.",
"parameters": {"type": "object", "properties": {"expression": {"type": "string"}}, "required": ["expression"]}}}],
"chat_template_kwargs": {"reasoning_effort": "high"}
}'
Send "chat_template_kwargs": {"reasoning_effort": "low"} to turn reasoning off.
Validation
Checked in upstream llama-server (llama.cpp a0e6dc738) on an NVIDIA GB10:
- Template and tokens: the rendered prompt carries exactly one BOS; reasoning-off prompts end in the empty prefilled block. Generation stops at
<|ifm|im_end|>; <|ifm|endoftext|> also stops.
- Reasoning tags: reasoning-off outputs contain no reasoning tags; reasoning-on outputs close with exactly one
</ifm|think>. This held at greedy and at temperature 1.0.
- OpenAI API: reasoning on returns
reasoning_content plus clean content; reasoning off returns clean content with no reasoning. A calculate request returns finish_reason: tool_calls with parsed arguments, and after the tool result the final answer is the correct $101.85.
- Tool use under the default sampling (temperature 1.0, top_p 0.95, reasoning on, 40 runs over four money prompts): the Q6_K called the calculator in 38/40 runs vs 39/40 for BF16, with think-tag integrity 40/40 for both.
Known limits
From the source's own evaluation: new workbooks with live formulas sometimes end up with text formulas; change requests on signed PDFs with reasoning on are handled in 3 of 10 cases; money arithmetic with reasoning off is unreliable, so use reasoning on (in our checks, reasoning-off answered a calculator question from memory, and both BF16 and Q6_K got it wrong). The model may identify itself as K2 and may attempt a tool from older conversation history. Use current tool definitions and validate consequential outputs.
한국어 안내
Diplodocus 프리뷰 4의 llama.cpp용 GGUF 모델입니다. 파일 하나(Q6_K, 7.76 GB)로 제공되며, 어텐션은 Q8_0, MLP·임베딩·출력 헤드는 중요도 행렬로 보정한 Q6_K입니다. K2 Horizon을 지원하는 llama.cpp(#29535 이후)가 필요하며 --jinja로 실행하세요. 추론은 reasoning_effort를 high로 켜거나 low로 끌 수 있고, 기본 샘플링은 temperature 1.0, top_p 0.95입니다. 금액 계산에는 추론을 켜고 calculate 도구를 사용하세요. Mac의 Osaurus에서는 Diplodocus-preview-JANG_6M을 사용하세요.
Quantized and validated by Jinho Jang — [email protected]
