Qwen3.8-27B-nvfp4full-dflash2-NInfer-v3

Qwen3.8-27B nvfp4full + DFlash2 for NInfer — v3 container

The same bytes you know from kaushikvira/Qwen3.8-27B-nvfp4full-dflash2-NInfer (v2), migrated to the NInfer v3 artifact container (upstream f76e19c0, model/weight decoupling). Weight tensors are bit-preserved — this is a packaging upgrade, not a re-quantization.

No engine patch needed anymore

The v2 artifact required a forked NInfer engine: the qwen3.8-27b/nvfp4full weights profile had to be registered in C++ (4 files: profile enum, resolver, binder, workspace cases). NInfer v3's loader is data-driven — every weight binds from the artifact's own logical bindings and per-tensor formats — so this artifact needs no model-support patch of any kind. Any NInfer engine at upstream v3 (Neroued/ninfer master ≥ f76e19c0, 2026-09-13) loads and serves it. Our previous fork's C++ registration commits were deleted during the v3 rebase and the model still passes the full quality gate (see below), which is the proof: the engine path in our build is upstream's.

The only fork-local piece left is one allow-list line in the offline upgrade tool (to produce v3 files like this one) — not in the engine, not needed to serve.

Artifact

Verify after download:

printf '%s  %s\n' \
  '74d2fc17742537defa6897583caaabcc9adfc99bdfbea402ec5e8facd0d5fdfc' \
  'qwen3_8_27b_nvfp4full-dflash2.v3.ninfer' | sha256sum --check

Lineage / provenance (complete chain)

Qwen/Qwen3.8-27B  (base model)
  └─ unsloth/Qwen3.8-27B-NVFP4  (quantized source, per cometkim's conversion.json)
      └─ cometkim/Qwen3.8-27B-nvfp4full-NInfer  (v1, v2-container, 1,259 objects)
          └─ graft: z-lab/Qwen3.8-27B-DFlash2  (+66 W8G32/BF16 objects → 1,325)
              = kaushikvira/Qwen3.8-27B-nvfp4full-dflash2-NInfer  (v2)
                  └─ tools/upgrade_ninfer_v2_to_v3.py  (bytes-preserved container upgrade)
                      = this repository  (v3)

• L1→L2: base tensors copied byte-for-byte; only the 66 DFlash2 objects (21 × W8G32_F16S matrices, 45 × BF16 norms/conv bases/codebooks) produced from z-lab/Qwen3.8-27B-DFlash2 @ 50307d4c4cde6860d4eee73e2547cd786fe8e8a4 via tools/artifact/graft_dflash2_w8.py (recipe qwen3_8_27b_nvfp4full-dflash2-w8, report graft.json included; MTP module retained, validate-only).
• L2→L3: upstream's tools/upgrade_ninfer_v2_to_v3.py, run 2026-09-18 on the exact L2 file above. Per the tool's contract: "Stored weight values and formats are preserved" — the +289 KB delta is the v3 directory (JSON logical bindings), format table entries, framing padding, and the maintained qwen3_8.jinja chat template. Weights, formats and object set are unchanged.
• The tool allow-lists the seven official NInfer v2 inputs; our nvfp4full identity required adding one KNOWN_COUNTS entry (("qwen3.8-27b","nvfp4full"): (1259, 1325)) in kaushikvira/ninfer@3369b385. Everything else validated unchanged — every format/layout we carry already exists upstream.

Reproduce the L2→L3 step yourself

git clone https://github.com/kaushikvira/ninfer && cd ninfer
# tool from our main (allow-list extended; engine not required for the tool)
python3 tools/upgrade_ninfer_v2_to_v3.py \
  /path/to/qwen3_8_27b_nvfp4full-dflash2.ninfer \
  /path/to/qwen3_8_27b_nvfp4full-dflash2.v3.ninfer
# run from the repo root (it installs chat_templates/qwen3_8.jinja from there)

Input must match SHA-256 373dab08…937de1; output must match 74d2fc17742537defa6897583caaabcc9adfc99bdfbea402ec5e8facd0d5fdfc.

Requirements

• NInfer v3 — upstream Neroued/ninfer master at/after the v3 loader switch (4cde7ad0, 2026-09-14; any current master qualifies). No forks, no registrations, no build flags.
• v3 is one-way in this respect: a v3 enginerejects v2 artifacts (explicit error), and pre-v3 engines reject v3. This file only runs on v3.
• 64-bit Linux; NVIDIA RTX 5090 (sm_120a); CUDA Toolkit 13.1+. (Upstream v3 targets the Qwen3.5/3.6/3.8 family on Blackwell; other devices are upstream's concern, our validation is on a 5090.)

Download and run

hf download kaushikvira/Qwen3.8-27B-nvfp4full-dflash2-NInfer-v3 \
  qwen3_8_27b_nvfp4full-dflash2.v3.ninfer --local-dir models

# CLI smoke
apps/ninfer models/qwen3_8_27b_nvfp4full-dflash2.v3.ninfer \
  --prompt "Explain prefill and decode in three sentences." \
  --max-context 32768 --max-new 8192 --kv-dtype k8v4 \
  --spec dflash2 --draft-tokens 7

# server
apps/ninfer-serve models/qwen3_8_27b_nvfp4full-dflash2.v3.ninfer \
  --host 127.0.0.1 --port 8080 \
  --max-context 262144 --kv-capacity auto --max-concurrency 4 \
  --kv-dtype k8v4 --spec dflash2 --draft-tokens 7 --preserve-thinking

Sanitized full serving profile (same flags, no secrets): examples/ninfer-nvfp4full-grafted-dflash2.cfg.example.

Validation (gate run on the cutover box, 2026-09-18)

Engine: upstream f76e19c0 + serving conveniences only (no engine-model code). One RTX 5090, 32 GB, 450 W SM-capped, k8v4, 262,144 context, conc 4, DFlash2 K=7, vision on. Probes: the repo's own gate.sh (needle 12 / probes 4 / tool 10 / streaming) + perf.py:

v3's ops reorg (partial-M-tile TMA route 5f5fccab, activation-scale tiling 1d8587bc) is on top — the +1.7% decode and small prefill shifts are our single-rig numbers, not vendor benchmarks.

Notes / limits

• Chat template changed at L3: the upgrade installs upstream's maintained qwen3_8.jinja in place of cometkim's embedded copy. All quality probes pass; if a client is picky about thinking-tag formatting, diff it against the v2 artifact's behavior.
• Same limits as v2: single device, startup-bounded concurrency 1–8, NInfer does not execute generated tool calls.
• License: Apache-2.0; base, quantized source, and DFlash2 drafter likewise Apache-2.0.