Qwen3.8-27B-NVFP4-RTX5090

quantized to OCP MXFP4 4-bit weights in the same Quark checkpoint container as The weights are plain RTN, not AWQ: see the note below. The 15 MTP tensors are kept in BF16; the radiance runtime loads them with RADIANCEQUARKBF16MTP=1. Why RTN instead of AWQ. The first build of this checkpoint used AWQ with the same smoothing recipe as AMD's release. The AWQ fold itself was mathematically consistent, but on this model a few layers converged to weights to 32-element MXFP4 blocks then destroyed those layers on real, outlier-carrying inputs (layer 7 output relative error ~324), and repairing the worst layers individually was not enough; the remaining smoothed layers still accumulated too much…

Papers: https://arxiv.org/abs/2609.19745 (Vision-RL²) · https://arxiv.org/abs/2509.16944 (SD-RPN) SD-RPN stage-1 checkpoint: a self-distilled RoI predictor twig (K = 27, T = 3) trained on a frozen, encoder-free google/gemma-4-12B-it. This is the initialisation of the Vision-RL² RL run The backbone weights are unchanged from the base model; only the three attached twig blocks are trained, from self-distilled attention pseudo-labels (no human RoI annotation). The Gemma stage-1 driver trains the delta and its assemble stage turns it into the full directory; both forms are published so you can start from either. These weights need the modeling code in YuHengsss/VisionRL2. They are not loadable…

GLM-5.3-Flash with its routed experts quantized to NestQuant, a nested two-level format. Each expert has a 1.5-bit base plus a residual plane that lifts it to 4 bits. At serving time an expert moves from 1.5 bit to 4 bit by loading its residual plane on top of the base, and the base bytes stay the same. This build targets a 128 GB Mac with 96 GB usable for weights, KV cache and runtime. The native vision tower is included, so the model still takes images. Status: TODO. Encoding in progress. This repo cannot be loaded with stock transformers, vLLM or MLX. The serving kernel and loader will be published separately. - 67 of 288 experts per layer (23%) run at 4 bit. The rest run at 1.5 bit.…

Model · Image and text to text

Jev-Omni-MLX-4bit

Leo Zhang

An Apple Silicon MLX conversion of akhilaaa3/Jev-Omni for local inference on a Mac mini with 16GB unified memory. This repository keeps the upstream Jev-Omni unified multimodal checkpoint and its trained 256-way decision head, then converts the language-model weights to 4-bit affine quantization with group size 64. The vision weights remain BF16 and the decision head remains FP32. It is an independent conversion. It is not an official TypeSafe Jev release and does not claim to reproduce TypeSafe's proprietary system. It is also not a new fine-tune. - Mac mini, Apple M4, 10 CPU cores, 16GB unified memory - macOS 26.5.1 - Python 3.13.12 - MLX 0.32.2 - MLX-VLM 0.7.1 - single request, batch…

Model · Image and text to text

Clemma-12B

Andrew DeLisa

This gemma4unified model was trained 2x faster with Unsloth and Huggingface's TRL library.

Model · Image and text to text

Qwen3.8-27B-NVFP4

Inferact Inc.

Quantized version of https://huggingface.co/Qwen/Qwen3.8-27B