apex-flash-1-EXL3-4bit-DGX-Spark

Apex Flash 1 EXL3 for two DGX Sparks

This is a quantized version of Cantina Apex Flash 1 for two DGX Spark GB10 systems. It contains 175.6 GB of tensors in 23 shards.

The routed experts use 4-bit EXL3 MCG encoding. Attention, shared experts, routers, embeddings, the output head and vision tensors retain source precision. 4-bit describes the experts, not the average size of every weight. This is not a SAGE quantization and does not fit the tested runtime on one Spark.

Upload status: the model files are not available yet. This page describes the prepared release.

Running it

Use the TensorFold image pinned in serve-rank.sh on both machines. Each machine needs the complete model directory. The launcher uses Podman and keeps the model read-only.

podman pull ghcr.io/miaai-lab/glm-5.3-flash-exl3-2x-dgx-sparks-tensorfold@sha256:22789f0cb3dc308f0b2ce52a33961b88bd624af1725e91e8aba0a74a671bb969

# Set RANK=0 on the first machine and RANK=1 on the second.
export MODEL_DIR=/path/to/model
export RANK=0
export MASTER_ADDR=<first-machine-address>
# Set NCCL_SOCKET_IFNAME, NCCL_IB_HCA and NCCL_IB_GID_INDEX for your network.
bash serve-rank.sh

Both ranks must use the same context, concurrency and draft settings. The API listens on the first machine at 127.0.0.1:8000 by default. Keep the default BF16 dense path. Other dense encodings were not tested.

The vision option applies a small adapter to the pinned runtime because Apex already has native image markers in its chat template. prepare-vision-adapter.py checks the runtime file before making the adapter. It does not change the model weights or chat template. Video has not been tested.

Results and limitations

Text checks covered arithmetic, inventory, scheduling, deduplication, reasoning, French, Python types, tool calls and retrieval. Retrieval returned the correct answers at 7,727, 30,885 and 64,445 prompt tokens. One plain-text response added unwanted explanation.

Four simultaneous text decoders returned correct JSON and matched serial responses with DFlash2 enabled and disabled. Two concurrently submitted retrieval requests of about 31,000 prompt tokens also passed. Four-request serving at 32K and concurrent serving at 64K were not tested.

Three simple image checks passed at both 8K and 32K settings. Six additional composed images returned correct counts and text. Only 2/6 followed a strict JSON-only request without JSON mode; all 6/6 passed with response_format: {"type": "json_object"}. These were synthetic images. Photos, scanned documents, charts, video and simultaneous image requests remain untested.

Six additional text checks had correct content in 5/6 cases. Only 1/6 followed the strict JSON contract without JSON mode. A JSON-mode retest passed 5/6. The remaining Python example returned the correct nested values but used result instead of the requested results key. Validate required keys as well as JSON syntax. The retest used a clearer multiline prompt, so it is not a direct comparison with the first run. These failures have not been compared with BF16 task responses, so they cannot be attributed specifically to quantization.

A small comparison against the unquantized source used eight separate 128-token text windows in the same ExLlamaV3 engine. Top-token agreement was 92.22%, mean KL divergence was 0.05935 nats, and perplexity was 4.7390 versus 4.6869, about 1.1% higher. Eight cross-reader checks also matched the next token between ExLlamaV3 and TensorFold. These results do not establish accuracy across all tasks.

Native MTP matched serial tokens on three greedy checks and improved decode speed by 1.7 to 2.3 times. DFlash2 matched those tokens and improved decode speed by 1.8 to 3.0 times. Three seeded sampling checks also matched serial tokens at temperature 0.7. One reasoning run stopped at a 512-token limit, so that result covers a matching prefix rather than a complete answer. Speed will vary with the request.

Optional DFlash2

The separately downloaded incoai/GLM-5.3-Flash-DFlash2 drafter was tested at revision bf582e4eacc1810f76656d1811693ff6c6737d2a.

Its license is CC BY-NC-ND 4.0 with separate commercial terms. It is not included here and is not covered by this model's MIT license. Native MTP is available without that separate drafter.

Build details

• Source revision: 28c647a6bd0444973a7ef3d940c24c67e64f52fe.
• ExLlamaV3 v1.5.3: d3739fd393337b1ff4d6c2a342b12f0c87a9592f, with the integer K patch recorded in the recipe.
• Recipe adapted from WamboDNS/apex-flash-1-EXL3-4bpw at 099b04bf3cd669ebff1e1ba6527b27aa5600dcb8.
• Calibration: the pinned ExLlamaV3 corpus, 250 rows of 2,048 tokens. Comparison inputs were separate.
• Tensor checks: 150,226 tensors, 37,152 routed expert projections including MTP, and 1,618 native tensors restored from the source.

manifest.json and SHA256SUMS record file sizes and hashes. The build recipe, restoration scripts, package versions and test summaries are included.

The encoder used a cached base image identified in encoder-base-provenance.json. An exact public registry reference for that base has not been verified. Encoder.Containerfile therefore requires ENCODER_BASE_IMAGE; a different base needs a new trial conversion before a full build.

The source MIT license and Z.AI copyright are preserved in LICENSE.