Qwen3.8-27B-NVFP4-Tachyon

Qwen3.8-27B NVFP4 for Tachyon

This model card is the version-controlled source for pinvou/Qwen3.8-27B-NVFP4-Tachyon.

The repository contains a mixed NVFP4/FP8 representation of Qwen3.8-27B. It combines the official BF16 checkpoint with the fixed packed Text weights from unsloth/Qwen3.8-27B-NVFP4 in the native Tachyon .tachyon artifact format. The artifact is intended only for Tachyon; it is not a Transformers checkpoint, Safetensors distribution, or GGUF file.

The artifact uses the Qwen3.5 Dense architecture. All 64 Text layers use NVFP4 MLP weights and an NVFP4 output head; the token embedding, attention input/output projections, and GDN Q/K/V/Z and output projections use row-scaled FP8. Control weights use BF16, with separate MTP, Vision and DFlash2 weights.

Artifact

The file contains Text, Vision, MTP, DFlash2, the optimized proposal head and frontend resources. Vision and speculative weights are loaded only when selected at startup. Source-derived NVFP4 and FP8 words are preserved without decode and requantization; only the official BF16 token embedding is encoded locally as row-scaled FP8.

Verify a downloaded file with:

printf '%s  %s\n' \
  '05c74c488b56a30097b25973888b31a16ddcfa1dec90556a2ae5e0c3201e50bb' \
  'qwen3_8_27b_nvfp4.tachyon' | sha256sum --check

This release includes the complete DFlash2 companion weights from z-lab/Qwen3.8-27B-DFlash2 at revision 50307d4c4cde6860d4eee73e2547cd786fe8e8a4. Select --spec dflash2 --draft-tokens 7 --lm-head-draft; draft counts 1..15 are supported. DFlash2 requires the runtime revision listed below. Existing performance and evaluation tables retain their stated MTP configurations and revisions.

Requirements

• the current shared Tachyon checkout, built from source; its artifact/component support must include upstream revision 7fca766 or later;
• 64-bit Linux;
• a GPU supported by the selected Tachyon build: GB10 / DGX Spark (sm_121a, current project target) or RTX 5090 (sm_120a, explicit build selection);
• a CUDA toolkit capable of compiling the selected architecture.

The published artifact measurements below used RTX 5090 and CUDA 13.1. Engine support does not establish GB10 validation for every component, context length, or weight profile in this card. Use the GB10 run records for their measured scope, and the selected checkout's build guide for toolchain requirements.

Already have the official v2 file? Upgrade it locally without downloading the weights again.

Tachyon does not provide an install target or packaged binary. See the repository README for source-build dependencies.

Download and run a CLI example

hf download pinvou/Qwen3.8-27B-NVFP4-Tachyon \
  qwen3_8_27b_nvfp4.tachyon \
  --local-dir models

./build/apps/tachyon models/qwen3_8_27b_nvfp4.tachyon \
  --prompt "Explain prefill and decode in three sentences." \
  --max-context 32768 \
  --max-new 8192 \
  --kv-dtype fp8 \
  --spec dflash2 --draft-tokens 7 \
  --lm-head-draft \
  --no-thinking

For images, videos, and structured chat history, see the CLI guide.

Start a local server

./build/apps/tachyon-serve models/qwen3_8_27b_nvfp4.tachyon \
  --host 127.0.0.1 \
  --port 8080 \
  --max-context 240000 \
  --kv-capacity 240000 \
  --max-concurrency 2 \
  --kv-dtype fp8 \
  --device-state-slots 2 \
  --host-state-slots 8 \
  --host-kv-mib 8192 \
  --spec dflash2 --draft-tokens 7 \
  --lm-head-draft \
  --no-thinking

Each request has a 240,000-token logical ceiling. The shared 240,000-token Device KV pool admits two active requests when their combined completion reservations fit; either request may use the full pool while running alone. Two extra Device checkpoint slots, eight pinned Host State slots, and 8 GiB of pinned Host KV retain reusable continuations under resource pressure.

See the HTTP serving guide for the API surface and the resource scheduling reference for cache and admission semantics.

Supported use

The artifact supports:

• text generation in thinking and non-thinking modes;
• image, multi-image, video, and mixed multimodal messages;
• MTP speculative decoding with draft windows from one to five;
• DFlash2 with draft windows from one to fifteen using the included DFlash2 companion weights (--spec dflash2 --draft-tokens 7, optionally --lm-head-draft);
• BF16, INT8, FP8, NVFP4, and K8V4 KV cache;
• CUDA Graph decode and compatible-prefix reuse;
• startup-bounded small-scale concurrent serving with true batched decode;
• the Tachyon CLI;
• OpenAI Responses Core, OpenAI Chat Completions, and Anthropic Messages serving.

Performance

GB10 / MegaCube serving (October 2026, DFlash2 K=7)

Measured on one GB10 (MegaCube) with Tachyon revision 7fca766, CUDA 13.0, FP8 KV, thinking disabled, and the exact serve command above. Sampling matches the public SGLang NEXTN baseline on the same hardware (temperature 1.0, top-p 0.95, top-k 20).

Single-request decode by content class (stochastic; greedy where higher):

Concurrency, mixed content (aggregate, client-side, includes admission ramp):

Steady-state 8-way structured counting reaches 405-461 tok/s aggregate (95%+ acceptance, server metric; ~51 tok/s per lane at 90% acceptance). Hot-turn TTFT on a 16.7K-token session with prefix reuse: 223-227 ms. Speculative decoding never changes the target distribution; greedy output differs from non-speculative decode only in near-tie floating-point orderings.

The tables below are historical RTX 5090 measurements retained for reference.

The MTP0 measurements below were collected at Tachyon revision f08597d, and the MTP3 measurements at revision 32c9881. Both campaigns used one NVIDIA GeForce RTX 5090, CUDA 13.1 compile/runtime, CUDA driver API 13.3, stochastic sampling, INT8 group-64 KV, CUDA Graphs, a 1,024-token prefill chunk, and prefix reuse disabled. MTP0 used no speculative backend and a 262,144-token context limit; MTP3 used a 131,072-token per-request context limit and three draft tokens.

Concurrent MTP=3 corpus makespan

The fixed corpus contains three long-reasoning and twelve cross-scenario fixtures with five seeds each, for 75 requests. Every concurrency point starts a fresh server and uses the same shuffle seed and ordered HTTP send sequence. C=1 is the serial single-request corpus. Makespan includes prefill, decode, workload transitions, and final drain.

All 300 requests completed without a request, CUDA, or out-of-memory failure. C=4 gives the shortest complete-corpus makespan. C=8 is limited by memory pressure, which makes its complete-corpus result slower than C=4. Sampling is stochastic, so the fixed prompts and seeds do not imply token-identical continuations across concurrency-specific numerical routes; exact decode-token totals are shown above.

Long-context baseline (MTP disabled)

Each value is the arithmetic mean ± sample standard deviation over five fixed seeds after server warm-up.

MTP=3 single-request long-reasoning decode

The C=1 point supplies five samples for each fixture. Values are arithmetic mean ± sample standard deviation from server phase timings and speculative counters.

MTP=3 single-request cross-scenario decode

Each category contains three fixtures and five seeds per fixture, for 15 samples.

See the full methodology and results for metric definitions and the exact reproduction command.

Evaluation

The artifact was evaluated through Tachyon's OpenAI-compatible serving route with thinking enabled, MTP=3, and INT8 group-64 KV. EvalScope 1.9.0 used 0-shot prompts, rule-based scoring, and one sample per problem with temperature 1.0, top-p 0.95, top-k 20, presence penalty 0.0, and seed 42. The text suite ran at a 252,928-token context limit; the multimodal suite ran with --vision at a 81,920-token limit.

All 1,723 configured samples completed and were scored. IFBench additionally reports 80.50% instruction-level strict, 80.33% prompt-level loose, and 83.50% instruction-level loose. These are single-sample results, not pass@k.

The official Qwen3.8-27B BF16 figures come from the upstream model card; its sampling settings and IFBench metric level are not stated there, so the last column is not a same-protocol comparison. The NVFP4 deltas stay within ±2.5 points on the four overlapping benchmarks, and the upstream card reports no AIME results.

Limits

• Tachyon executes on one GPU and one CUDA device, with a startup-fixed capacity of 1–8 active requests per Engine.
• It does not provide large-scale or preemptive continuous batching, priority/QoS scheduling, multi-GPU execution, CPU/GPU offload, or distributed serving.
• Context allocation is subject to GPU memory and the selected KV-cache type.
• Tachyon does not execute generated tool calls.

Provenance

The artifact identity, summarized object inventory, and conversion provenance are published in artifact-manifest.json. The exact storage contract is maintained in the v3 container reference.

License

This Tachyon artifact is distributed under the Apache License 2.0. The Qwen3.8-27B base repository and the quantized source repository are also licensed under Apache-2.0. Users remain responsible for complying with the license and applicable laws.