MERNIK

MERNIK — measure-first quantization protocol

MERNIK ("the one who measures") is the evolution of the ASHQ1 battlefield zoo: fewer utility duels, more verdicts. The method (priority queue) is built; MERNIK is how we prove anything about it. Think of it as a finetune of ASHQ1: same base weights (queue, pins, tied groups), retrained objective (measure-first protocol, three-column verdicts) — and far more capabilities on top (slow capability ring, KLD-aware teachers, relief ceilings, the zoo bench, norms shields). But the legend is not forgotten: every scar in the ledger traces back to it.

The two rings

Allocator signals never certify. Capability scores never steer. Mixing them is how the PPL disease happened.

Columns (every claim carries all three or stays home)

• PPL = canary. Cheap, lies by sharpening. Demoted everywhere.
• KLD-vs-ref = rank column for allocator decisions (Soulfate24 battery: fixed span, fixed chunks, Flash-Attention). Triage, not certificate: blind to long-ctx, instruction drift, tool format, rare modes.
• Tasks = verdict. Slow ring. Only column that can crown or kill a build.

House rules: metrics diverge → KLD wins the argument; metrics agree → trust the pair. Every retraction keeps its scar in the ledger (README-ZOO.md).

Silence is signal (audit 2026-09-13). Empty completions scale with weakness across all clean runs: Ox 4–16 → Neo 20–27 → Q2K 63 → S2 115. Weak models go silent rather than babble — incapacity, not harness bug (server deaths look different: connection errors, audited per-run). S2's 39 passes stand taller for the 115 silences beside them. Strength reads as decisiveness: the distillate answers almost always (Ox empties 4–5) and is right almost always. Whether a quant goes silent or babbles is set by the finetune, not the size: same Qwen3.5 bones, Neo hesitates (20–27), OxCoder commits (4–5). Compare finetunes, not just budgets.

Slow-ring protocol: HumanEval (NeoHorse-1-9B reference implementation)

Server (one model at a time, GPU is a strict queue):

llama-server -m MODEL.gguf --port 28082 -ngl 99 -c 8192 --jinja --log-disable

Sampling (NeoHorse/Ornith thinking models; peg-native template):

temperature 1.0, top_p 0.95, top_k 20, min_p 0.0,
presence_penalty 0.0, repetition_penalty 1.0, max_tokens 2048

presence_penalty 1.5 (vendor-reported recipe) breaks thinking templates (500 peg-native format); 0.0 verified. Preflight /health before every battery — never score 164 empties against a dead server (cost us one battery once; lesson kept).

Runner: scripts/run_humaneval.py (env overrides HE_*, HUMANEVAL_OUT, HUMANEVAL_SERVER). Eval: human_eval.evaluation.evaluate_functional_correctness, pass@1, k=[1]. Chain: serve_and_run per model (kill server, serve, preflight, run, log exit=), results to eval_results/humaneval_<tag>.jsonl{,_results.jsonl}.

Saturation rule: base at ~98% can't resolve 97 vs 96 (floor-effect noise, the mirror of the PPL disease). Capability duels need bases at 60–85%, or they only answer the collapse question (does the small build fall off the cliff?).

Reference results (NeoHorse-1-9B, official BF16 98.17% @ undisclosed ctx)

Cross-KLD between the quants: D(MERNIK‖Q6) = 0.320, D(Q6‖MERNIK) = 0.318 — symmetric, and large: the pie deviates structurally, as pies do.

Recipe (repeatable one-to-one): llama-perplexity -m MODEL -f wiki.test.raw -ngl 99 -c 1024 -n 64 -b 512 --seed 7, KLD via --kl-divergence --kl-divergence-base <REF.dat>, ref logits from --save-all-logits on the Q8 build (no BF16 runs — the Q6 duel stands without it). HE per slow-ring protocol above. Corpus: wikitext-2-raw wiki.test.raw.

The bet, resolved as designed: KLD ranks Q6_K higher (4× closer to Q8), tasks crown MERNIK (+3.7pp at −0.5 GB). Two readings stay on the card: (a) hybrids deviate from any flat reference by construction, so KLD punishes the pie for being a pie — expected, not a refutation of the rank column inside its jurisdiction (allocator decisions); (b) on the verdict column the rank column does not transfer — capability and distribution-fidelity diverge here, and any claim that KLD picks the better model answers to this row.

Reference results II — OxCoder-9B (finetune duel, same Qwen3.5 bones)

Full card: wepiqx/OxCoder-9B-MERNIK-GGUF.

RINIQ series (2026-09-16/17): monster standings

Ox base + per-block donors (weight/imatrix compasses, no labels), SMAPE-5100 MERNIK quants unless noted. Full saga: FUSION.md. Files (as verdicts complete): wepiqx/RINIQ-MERNIK-GGUF.

Q38 (own-qwen38 single lens) took the crown losing PPL+GPQA and winning the verdict — capability has its own address. 6500 hunt closed: MSE-6500 ties 5100 exactly (ceiling, not precision). M4a/b/c quants running — "who is weaker" finale (M4b = one-block A/B on Orn-25).

How the monsters are assembled (all 427-trunk, MTP head excluded)

Donors (same Qwen3.5 skeleton, 0 shape mismatches): - Ox = OxCoder-9B.BF16 (agentic code distillate, HE 90.24%) — base + globals - Neo = NeoHorse-1-9B-BF16 (thinking model) — donor of blk 31 - Orn = Ornith-1.5-9B-MTP-BF16 trunk (GDN hybrid) — donor of mid/late blocks

Tool: scripts/fuse_layers.py --a OX --b ORN --c NEO --map "15:b,...". M1 = Ox + Orn blks 15,19,23,27 (weight-compass FFN peak) + Neo blk 31. M2 = Ox + Orn blks 24,25,26 (imatrix pick) + Neo blk 31. M3 = M1 + blks 0–8 weight-averaged Ox+Neo (--soup "0-8" --soup-from "a,c"). M4a = M2 + blk 16←Orn (Orn-center). M4b = M2 with blk 25→Ox (drop-25 A/B). M4c = M2 + blk 30←Orn (seam with Neo-31).

Series name: RINIQ, quants as RINIQ-M<i>-MERNIK-<size>[-Q38]. M4 verdicts close the series — 93% ends it with fanfare either way.

Engine note: every monster quant is built by the MERNIK engine itself (main.py — SMAPE/MSE, dual-wiki or own-qwen38 lens) — same protocol, same pins, so duels are one-to-one. Quants ship as RINIQ-M<i>-MERNIK-<size>[-Q38].gguf to wepiqx/RINIQ-MERNIK-GGUF. Verdicts and recipes live in FUSION.md.

Fast ring II: GPQA-recognition (non-standard, read this)

How we score (NOT the vendor method): no generation, no CoT. Each Diamond question + shuffled choices goes in one prompt; we read P(letter) off the first generated token's top-30 distribution and pick the max. 198 questions, ~3 min/model. Script: scripts/gpqa_duel.py.

What it measures: recognition decisiveness, not reasoning. Proof: OxCoder-6500 scores below NeoHorse-6500 here (49.5 vs 51.5) while beating it on HE (90.2 vs 86.0) — the thinker spreads first-token mass (it wants to reason first), the decisive model stabs the letter. Rank uses: cheap allocator signal only. Never compare these numbers to vendor generation+CoT scores (their 86.9 lives in another universe).

Canon: MSE spreads, SMAPE sacrifices

• MSE (default): relative-blind absolute gain — tiers spread evenly across layers (Q5/Q6-heavy middle). Best PPL and KLD on Ornith-class models (Ornith-1.5 @6500: MSE 8.6341 vs SMAPE 8.7845; NeoHorse: 7.7695 vs Q6 7.9419, KLD 0.0453 vs pair 0.32).
• SMAPE: relative lens — junk layers dumped to the Q4 floor, kings pushed to Q8 penthouses (barbell). Loses distribution columns on dense 9B — and on code at big budgets (82.32 vs 85.98 @6500), but owns small budgets (82.93 vs 79.88 @5100, see budget law). The lens, not the size.

Distributions @6500 (dry-run, 427 tensors incl. 177 F16 norms/1D):

Same budget, opposite philosophies: MSE fills the middle (Q5+Q6 = 94 tensors, 4107 MiB), SMAPE hollows it (Q5+Q6 = 10) to double the floor and buy 41 more Q8 penthouses. Top-importance ffn_down@31: Q6_K under MSE, Q8_0 under SMAPE — the kings do fly higher.

Top-down twins: TD-MSE moved ~30 small tensors (norms!) from F16 down to Q4 — PPL didn't blink (7.7701 vs 7.7695), HE bled −3.7pp. Norms are free real estate for perplexity and load-bearing walls for code. TD-SMAPE kept a 949 MiB F16 core of pure attention (qkv layers 20–30 + 3) and drowned everything else (236 at Q4) — the attention-vs-FFN experiment, HE pending. Both statements are true; the column decides which one you see.

Verdict @6500: SMAPE HE 82.32% = stock Q6K to the digit, MSE holds 85.98%. The barbell taxes code exactly where it taxes distribution: 91 tensors on the Q4 floor cost −3.7pp of capability. Kings flying higher (149×Q8) did not pay for the drowned middle. (Bet log: predicted 81–83 — closed.)

The budget law (confirmed 2026-09-14 after clean reruns)

Correction arc (scar kept). The 18.3% / 11.59% below were DEAD servers scored silently (121/136 errors, OOM kills). Fixed with a mid-run health watchdog (aborts LOUDLY). Clean reruns: MSE-5100 79.88%, Q4_K_M 76.83%, 0 errors. The "5 GB cliff" never existed — the law holds narrower and honest.

Same PPL both times (~tie). Symmetric crossover between 5100 and 6500 on 9B dense: SMAPE owns small budgets, MSE owns big ones. PPL never saw any of this.

MTP transplant (world-first OxCoder+MTP, 2026-09-15)

Donor MTP head (15 tensors, 464 MB) grafted onto OxCoder-9B via scripts/graft_mtp3.py (structs from gguf-py, data streamed — 15 GB box survived). Full saga: SAGA.md. Status: draft engages (acceptance 5–23%, donor mismatch), plain load healthy. Economics negative (draft slows: 31 → 28 → 20 t/s) — mechanics proven, speed pending a matched head.

Method in brief (the queue being measured)

Every tensor starts at a floor tier by class; a global max-heap drains the budget best-first by Σ(importance) × ΔMSE / MiB. Tied groups (identical imatrix energy) upgrade as one unit with summed importance. Structural pins — MTP head → Q8_0, output/token_embd → Q5_K, MoE routers → F16 — sit outside the budget. One knob: --size in MiB. --top-down mirrors it: everything from F16, downgrade cheapest-loss-first, slack refilled by the same metric.

pip install -r requirements.txt
python main.py --model M.gguf --imatrix M.imatrix.gguf --size 6800 --run

Needs a stock llama.cpp build (llama-quantize, llama-perplexity, llama-server; override via LLAMA_QUANTIZE / LLAMA_PPL / LLAMA_SERVER). Code map: main.py (CLI), classifier.py (queue), constants.py (tiers, pins, toxicity, arch presets), experimental.py (retired zoo utilities), scripts/group_damage_sweep.py (teacher labels), scripts/run_humaneval.py (slow ring), scripts/audit_tiers.py (per-layer tier map of any quant file), scripts/build_normstd.py (top-down with native norms shield).

Flags (full usage)

# classify + dry-run (~1 sec) — always preview before burning 10 min
python main.py --model M.gguf --imatrix M.imatrix.gguf --size 6800

# actual quant
python main.py --model M.gguf --imatrix M.imatrix.gguf --size 6800 --run

# print the --tensor-type config only / show hard floors
python main.py --model M.gguf --imatrix M.imatrix.gguf --size 6800 --show-config
python main.py --show-floors

Fails fast: if even base floors exceed --size, the run aborts instead of silently producing an oversized file. Estimator accuracy ±1–3 MiB vs llama-quantize dry-run.

# audit any quant file: histogram, biggest tensors, per-layer map
python scripts/audit_tiers.py --model M-6500.gguf
python scripts/audit_tiers.py --model M-6500.gguf --big 10
python scripts/audit_tiers.py --model M-6500.gguf --layer 31

Supported architectures

Detection prefers general.architecture metadata, falls back to tensor-name heuristics. New archs plug in via ARCH_FEATURES in constants.py.

Scales down (not just 9B flagships)

The same queue, same --size knob, no retuning — from pocket budgets to flagships (all PPL canary, ctx 1024, seed 7):

Small dense, MoE (Ling-3.0-tiny 8B), GDN hybrid, MTP heads — allocation holds across families. Per-family quirks (Spark pipeline bubble, MoE padding) are logged in the scar ledger, not hidden.

Quant repos (verdicts live here)

• NeoHorse-1-9B-MERNIK-GGUF — HE triple above
• Ornith-1.5-9B-MTP-ASHQ1-GGUF — KLD-validated GDN hybrid
• Spark-X2.5-1.7B-ASHQ1-GGUF — zoo stand + pin-validation duels

Incoming (logged when landed, win or lose)

• Q4_K_M stock duel (PPL → HE running): does our 5100 pair beat stock?
• Quality-36 vs MERNIK-6500 on HE — the fair fight, file requested
• MiniCPM teacher DONE (168 labels, Gini 0.498, max damage 0.173 — late units carry the signal) → MLP un-paused, Gini answer to Soulfate24
• Relief ceilings on 9B (NeoHorse damage sweep → --relief duel vs MSE)
• smape_frag for NeoHorse, gated on SMAPE-5100 beating stock + community demand
• KLD columns for the Ox trio (same Q8-proxy recipe)
• Gini v3 (Q5→Q2 drops): Q5→Q3 labels sit at noise floor (max 0.06) — bigger drops for a signal with teeth. Queued after the current tail.

Lineage

MERNIK is the ASHQ1 battlefield zoo grown up: same queue, plus a protocol. Most features migrated from the original ASHQ1 and the ZOO — priority-queue drain, tied groups with summed importance, MTP-head handling, embd/router pins, top-down mode with slack refilling, sub-4-bit toxicity, MoE padding — and there is more here: KLD-aware teacher sweeps, relief ceilings, the experimental utility bench, the slow capability ring. Fully open code, Apache-2.0 — take it, fork it, beat us with it (the ledger will record that too). Built in the open with Soulfate24, whose KLD battery ended our PPL theater — the scar ledger (README-ZOO.md) records every round. Steel sharpens steel.