This model was quantized using oQ (oMLX v0.7.1) mixed-precision quantization with Improved oQe Quantization (jundot/omlx#4385).
A mixed-precision MLX checkpoint of vllm-sr/d3, the 27B multimodal decision model of Decision 3.0 from vLLM Semantic Router, built on Qwen/Qwen3.8-27B. It reads text or JSON with optional images and videos and returns a probability for every option of choice, yes/no and score questions without generating text. It includes the language backbone, the vision encoder and the code readout.
The code readout (readout.safetensors, FP32) and decision_config.json are copied unchanged from the source. oMLX serves this checkpoint through POST /v1/systemone; Decision 3.0 support (jundot/omlx#4470) is newer than oMLX v0.7.1.dev1.
The effective average is 4.880 bits/weight for the complete checkpoint.
Quantization and Bit Distribution
oQ4e allocates bits by measured layer sensitivity, then rounds each group with an importance matrix (128 samples x 512 tokens) and a weighted least-squares refit of its scale and bias.
| Storage format |
Logical weights |
Share of total |
Tensor storage |
Effective bits/weight |
| Affine 4-bit, group 64 |
21.075B |
80.790% |
11.041 GiB |
4.50 |
| Affine 5-bit, group 64 |
4.547B |
17.429% |
2.911 GiB |
5.50 |
| BF16 |
0.463B |
1.776% |
0.863 GiB |
16.00 |
| FP32 |
0.001B |
0.005% |
0.005 GiB |
32.00 |
| Total |
26.087B |
100% |
14.820 GiB |
4.880 |
Storage Breakdown
| Component |
Logical weights |
Storage |
| Language backbone |
25.625B |
14.986 GB / 13.957 GiB |
| Vision encoder |
0.461B |
0.921 GB / 0.858 GiB |
| Code readout |
0.001B |
0.005 GB / 0.005 GiB |
| Total |
26.087B |
15.913 GB / 14.820 GiB |
License
Apache-2.0, following the source model. See LICENSE and NOTICE.