INC Optimized Models 4

This checkpoint is an AutoRound model-free MXFP8 RTN quantization of exported in llmcompressor / compressed-tensors format. Routed experts and the self-attention projections present in the source are stored as F8E4M3; sensitive/shared and multimodal weights remain BF16. Static FP8 KV scales were calibrated with AutoRound using the text dataset NeelNanda/pile-10k; the vision tower was not quantized. On the paired repository lmeval protocol, the four primary metrics were non-decreasing relative to the BF16 baseline using the same vLLM FP8 KV cache. The AQA gate was GO. This is a result for those tasks and settings only, not a claim of lossless quantization or general quality improvement. The…

This repository contains an MXFP8-weight checkpoint derived from exported in compressed-tensors format. The checkpoint retains the source multimodal components, but the evaluation reported here covers text tasks only. The exported checkpoint contains 11,635 F8E4M3 weight tensors, 838 BF16 weight tensors, 11,635 U8 block-scale tensors, and 171 FP32 scale/metadata tensors. Routed-expert weights and source-present text self-attention projections are MXFP8; the router, shared MLP, vision tower, and embeddings remain BF16. Measured on 2026-09-30 with lm-eval 0.4.13 and vLLM 0.29.0. The tested settings were TRITONATTN, tensor parallelism 2, pipeline parallelism 1, batch size 64, maxnumseqs=64…