Gemma-4-Writers-31B-V2-W8A16-FP8

Model · Image and text to text

gemma-4-31B-it

Google

Gemma is a family of open models built by Google DeepMind. Gemma 4 models are multimodal, handling text and image input (with audio supported on E2B, E4B, and 12B) and generating text output. This release includes open-weights models in both pre-trained and instruction-tuned variants. Gemma 4 features a context window of up to 256K tokens and maintains multilingual support in over 140 languages. Featuring both Dense and Mixture-of-Experts (MoE) architectures, Gemma 4 is well-suited for tasks like text generation, coding, and reasoning. The models are available in five distinct sizes: E2B, E4B, 12B, 26B A4B, and 31B. Their diverse sizes make them deployable in environments ranging from…

Gemma is a family of open models built by Google DeepMind. Gemma 4 models are multimodal, handling text and image input (with audio supported on E2B, E4B, and 12B) and generating text output. This release includes open-weights models in both pre-trained and instruction-tuned variants. Gemma 4 features a context window of up to 256K tokens and maintains multilingual support in over 140 languages. Featuring both Dense and Mixture-of-Experts (MoE) architectures, Gemma 4 is well-suited for tasks like text generation, coding, and reasoning. The models are available in five distinct sizes: E2B, E4B, 12B, 26B A4B, and 31B. Their diverse sizes make them deployable in environments ranging from…

Model · Image and text to text

Gemma-4-Writers-31B-W8A16-FP8

Hoborific

Quantized version of Ateron/Gemma-4-Writers-31B. Offline-quantized W8A16 FP8 in the float-quantized format: weights in float8e4m3fn with per-output-channel symmetric scales, activations kept in bf16/fp16. For each linear layer, every output row gets its own scale starting from amax / 448, refined by an MSE clip search over ~9 clip fractions (0.8–1.0× amax) picking the lowest-error scale per row. Weights are then quantized q = e4m3(w / scale) with round-to-nearest and saturation. This per-channel + clipping scheme gives better SNR than vLLM's online per-tensor --quantization fp8 path. Only 2D linear projection weights are quantized (attention q/k/v/o, MLP gate/up/down). Embeddings, norms…

Model · Image and text to text

Swift-1.5-Qwen3.8-Flash-Next-EXL3-3.75bpw-96GB

Jorge Ramírez Espejo

EXL3 quantization (3.75 bits per weight on average) of UkisAI's Swift 1.5 Qwen3.8-Flash-Next (MoE, 125B total / 6B active, plus a 51B n-gram "PLE" table, MTP head and vision), made to serve the whole model on one 96 GB GPU with vLLM (continuous batching): 262,144 tokens per session without YaRN, 4 concurrent sessions in a 1.1M-token KV cache with 15 GB still free, vision and MTP. Validated on a real NVIDIA RTX PRO 6000 Blackwell Server Edition (96 GB, power-limited to 300 W). It is ≈3.3× closer to the BF16 model than our 2.25 bpw build (KLD 0.013 vs 0.044). All Swift 1.5 Qwen3.8-Flash-Next (125B-A6B) variants we publish, measured on 2026-10-14 under the same conditions (see Family…

Model · Image and text to text

Muse-Glimmer-30B

Ares Realm Studios

Muse Glimmer is a 30-billion-parameter causal language model with a dedicated perception encoder, distilled from Muse Spark and purpose-built for autonomous agentic tasks on consumer hardware. The model integrates multi-step reasoning, reliable tool use, multimodal understanding, and failure recovery into a single model that runs locally without requiring cloud infrastructure or network access. Building effective agents requires key capabilities working together to achieve the user’s goals. Muse Glimmer is trained and evaluated on these capabilities: End-to-end Agentic Task Completion. Muse Glimmer achieves strong success rates on full-task benchmarks including DeepSearch QA, MCP-Atlas…

Model · Image and text to text

Qwen3.6-27B-AWQ-INT4

Cyankiwi

Following the February release of the Qwen3.5 series, we're pleased to share the first open-weight variant of Qwen3.6. Built on direct feedback from the community, Qwen3.6 prioritizes stability and real-world utility, offering developers a more intuitive, responsive, and genuinely productive coding experience. This release delivers substantial upgrades, particularly in For more details, please refer to our blog post Qwen3.6-27B. Empty cells (--) indicate scores not yet available or not applicable. For streamlined integration, we recommend using Qwen3.6 via APIs. Below is a guide to use Qwen3.6 via OpenAI-compatible API. Qwen3.6 can be served via APIs with popular inference frameworks. In…