Occamy-1.0-oQ6e-mtp

[!CAUTION] This model was quantized using oQ (oMLX v0.7) mixed-precision quantization.

中文摘要

• 本仓库是什么：把官方 occamy-1.0（35B-A3B MoE + 视觉编码器）与官方 occamy-1.0-MTP 草稿头合并进同一份权重（mtp_num_hidden_layers: 1），再用 oMLX 的 oQ6e 量化导出，专供 Apple Silicon 上的 oMLX 推理。非官方发布，未经 Accio-Lab 验证。

• 实测（本机真实使用日志，113 次请求，Mac Studio M4 Max 64GB）：草稿接受率 77.95%（48,501/62,218），分深度 d1 79.4% / d2 76.3% / d3 77.3%；中位 2.83 tokens/cycle；中位解码 110.5 tok/s（p10 78.0 / p90 146.9）。工具调用类结构化输出接受率 85.7%，自由文本 75.8%。

• 量化配方：oQ6e = oMLX 通用动态量化，6-bit 基准（专家层 6-bit/group 64；语言部分 ≈6.6 bpw，整文件含视觉塔 ≈6.7 bpw）；嵌入、注意力/GDN 投影、共享专家提升到 8-bit；全模型 norms、MoE 路由、MTP 融合层与整个视觉塔保持未量化；imatrix 校准（oqe_code_multilingual，128×512，专家覆盖 100%）。
• 体积：30.28 GB（28.20 GiB），6 个分片；64GB 统一内存可运行（实测占用约 29 GB）。

1. What this repository is

Upstream ships the MTP draft head as a separate artifact that must be assembled into the base model and served through a patched SGLang runtime. This repository takes the same two official artifacts, merges the head into the model directory the way the upstream assemble_head.py helper does (register the 19 mtp.* tensors in the weight index, set text_config.mtp_num_hidden_layers from 0 to 1), and then quantizes the merged model with oMLX oQ6e so it runs natively in oMLX on Apple Silicon with multi-token prediction enabled.

Inherited model characteristics (from the upstream checkpoint, unchanged by the merge): 35B total / 3B active MoE, 40 layers + 1 MTP layer, 256 experts (8 routed + 1 shared per token), hybrid linear-attention (GDN) + full-attention layers (attention every 4th layer), 262,144-token context, and a 27-layer vision encoder that is preserved here at full precision.

2. Quickstart (oMLX)

Tested with oMLX 0.7.0 on a Mac Studio (M4 Max, 64 GB). Requirements: macOS 15+, Apple Silicon, ~30 GB of free unified memory for the weights plus KV cache / context.

Install oMLX from omlx.ai, place this repository folder inside an oMLX model directory, and start the server (omlx serve --model-dir /path/to/models --port 8000).

1. In the oMLX model settings for this model, enable MTP. The measurements below used adaptive draft depth up to 3 (mtp_adaptive_max_depth: 3); leave vision-MTP off (default — see boundaries).
2. Use the OpenAI-compatible endpoint at http://127.0.0.1:8000/v1 (or the oMLX app UI / your agent tooling).

Verify MTP is active. The oMLX log prints, per request:

MTP path activated for uid=N (model has mtp_forward, batch=1, primed=...)
MTP[N] finish=stop tokens=... cycles=... tok/cycle=... accept=.../... (..%) depth[d1=../..,d2=../..,d3=../..] ...

accept is accepted drafted tokens over proposed drafted tokens; tok/cycle is generated tokens per verification cycle (plain non-MTP decoding is 1 token per forward pass). If acceptance degrades mid-request, oMLX may park MTP and re-probe later (visible as finish=parked); this was observed in our logs and is normal runtime behavior.

Sampling: the sessions below ran at oMLX's temperature 0.7 setting. Upstream recommends temperature 1.0, top_p 0.95, top_k 20, presence_penalty 1.5 with thinking enabled for general agentic use — sampling behavior of this head is not separately validated here or upstream.

3. Measured MTP behavior

These numbers come from real usage logs on a single machine (interactive agent/tool sessions plus a few long-form generations), not from a controlled benchmark.

Draft acceptance

Efficiency and speed

Observed maximum decode throughput: 165.7 tok/s. Long single generations included 5,675 tokens @ 128.2 tok/s and 5,463 tokens @ 123.8 tok/s.

By output type (acceptance depends heavily on content predictability)

No speedup multiplier is claimed. These sessions did not include an MTP-disabled baseline run, so no MTP-on vs MTP-off comparison exists for this artifact on oMLX. The mechanism is captured by tok/cycle: 2.83 generated tokens per verification cycle versus 1 for non-MTP decoding, at the cost of verifying drafted tokens in the same forward pass. Treat all figures above as single-machine observations, bounded to the conditions listed.

4. Boundaries — what was not validated

• Unofficial status. Accio-Lab has not reviewed, validated or endorsed this merge or this quantization. Upstream's own validation (single-request greedy MTP1 parity at 87.60% heldout acceptance in BF16 on patched SGLang; native 3-step MTP parity checks on H200) applies to the official artifacts and runtimes, not to this repository.
• No exact-parity claim. Quantization changes numerics; token-level and logprob-level parity with the BF16 base + head is not expected and was not measured. Upstream notes the head is sensitive to the runtime path — different builds and settings require revalidation.
• No MTP on/off speedup measurement (see §3). Also no measurement under concurrency: all logged requests ran with a single in-flight request.
• Image input was not exercised with MTP. The vision tower is preserved at full precision and loads as a VLM in oMLX, but multimodal MTP is unvalidated here and upstream as well; keep vision-MTP disabled.
• Sampling. Measured sessions used temperature 0.7; upstream validated the head under greedy decoding only. Acceptance under different samplers is unknown.
• Single machine, single runtime. One M4 Max / 64 GB / oMLX 0.7.0 configuration; no other oQ levels, no other Apple Silicon tiers, and no llama.cpp/GGUF variant of this merge are published here.
• Content-dependent acceptance. Structured/tool-call outputs accept noticeably better than free-form text (85.7% vs 75.8%) and long prose can show lower acceptance; oMLX may park MTP mid-request when acceptance degrades. Expect throughput to vary with workload.

5. What is "oQ"?

See "oQ: oMLX Universal Dynamic Quantization" for details.

6. License and attribution

Released under Apache 2.0, following both upstream artifacts (occamy-1.0 and occamy-1.0-MTP, Apache 2.0).

Credits: Occamy-1.0 and the MTP head are by Accio-Lab (built on Qwen3.6-35B-A3B; head adapted on Accio-Lab/occamy-data-1.0). Quantization tooling and the MTP runtime used here are by oMLX.

If you use the underlying model, please cite the upstream technical report and model pages: arXiv:2609.11977.