Open-Weight Models

Model · Image and text to text

Qwen3.8-27B-NVFP4-QAD

Local Inference Lab

WORK IN PROGRESS A mixed NVFP4/MXFP8 quantization-aware distillation of Qwen3.8-27B, trained for one epoch. The student learns from the original BF16 teacher while its MLP weights are quantized in the forward pass. Distillation updates the MLP weights and text normalization weights to account for quantization error. This is a trained distillation checkpoint, not a post-training conversion of the original weights. Attention/GDN projections and the LM head were frozen in their MXFP8 representations during distillation. Packed NVFP4 and MXFP8 weights reconstruct to the same BF16 weight values used by the student during training. The tokenizer, chat template, generation configuration and…

This model was quantized using oQ (oMLX v0.7.0.dev2) mixed-precision quantization.

This model was quantized using oQ (oMLX v0.7.0.dev2) mixed-precision quantization.

This model was quantized using oQ (oMLX v0.7.0.dev2) mixed-precision quantization.

Prism ML's ternary Ternary-Bonsai-2-27B build of Qwen/Qwen3.8-27B, repacked for chad, a Claude-Code-style local coding agent for Apple Silicon, with its speculative decoder bundled in. This is chad's default model. Created using Bonsai by Prism ML. with, already quantized. Nothing is built on first run. Every projection of Qwen3.8-27B (a dense qwen35 hybrid: 64 layers, 48 GatedDeltaNet + 16 full attention) is stored in a Hadamard-rotated basis: multiplied by a fixed sign vector and put through a blockwise Walsh-Hadamard transform offline, then quantized to 2-bit affine group-128 whose three levels reproduce the ternary set {−s, 0, +s}. The rotation costs no extra bits and no extra weight…

GGUF layer package for running Qwen3.8-27B-UD-Q8KXL across a local Mesh LLM cluster. This package is derived from unsloth/Qwen3.8-27B-GGUF and keeps the original GGUF distribution split into per-layer artifacts for distributed inference. - Local and private inference with Mesh LLM. - Multi-machine serving when the full GGUF is too large for one host. - OpenAI-compatible chat/completions workflows through Mesh LLM's local API. For upstream architecture details, chat template guidance, sampling recommendations, license terms, and benchmark notes, see the source model card: unsloth/Qwen3.8-27B-GGUF. Generated by the Mesh LLM HF Jobs splitter from mesh-llm ref main. Each artifact is checksummed…

Qwen3.8-27B uncensored by HauhauCS 0/465 Refusals. This is the Aggressive variant: direct answers, no refusal behavior, and minimal preamble on hard prompts. Every text GGUF preserves Qwen3.8's native NextN head, and this release adds HauhauCS FastMTP: a specific acceleration sidecar qualified across the complete quant lineup at maximum native context. Vision is included through the separate BF16 projector. No changes to datasets or intended capabilities. This release preserves Qwen3.8-27B's text, reasoning, agentic, image, and video capabilities while applying the HauhauCS Aggressive uncensoring profile. Pick Aggressive when you specifically want the model to get to the answer without…

The same bytes you know from (v2), migrated to the NInfer v3 artifact container (upstream f76e19c0, model/weight decoupling). Weight tensors are bit-preserved — this is a packaging upgrade, not a re-quantization. - L1→L2: base tensors copied byte-for-byte; only the 66 DFlash2 objects (21 × W8G32F16S matrices, 45 × BF16 norms/conv bases/codebooks) produced from z-lab/Qwen3.8-27B-DFlash2 @ 50307d4c4cde6860d4eee73e2547cd786fe8e8a4 via tools/artifact/graftdflash2w8.py (recipe qwen3827bnvfp4full-dflash2-w8, report graft.json included; MTP module retained, validate-only). - L2→L3: upstream's tools/upgradeninferv2tov3.py, run 2026-09-18 on the exact L2 file above. Per the tool's contract: "Stored…

As the frontier of foundation models pushes toward ever-larger parameter counts and ever-longer context windows, the question is no longer just how much we can scale, but how efficiently we can do so. Sustainable progress toward artificial general intelligence (AGI) that benefits everyone demands architectural innovation. Today, we are sharing a concrete step in that direction: Qwen3.8-Flash-Next. This experimental preview of the architecture that will underpin Qwen4 is built around a fundamental rethinking of how the core components of modern large language models (LLMs) interact at scale. The first open-weight release under this architecture is Qwen3.8-Flash-Next, which introduces: For…

Model · Text generation

Qwen3.8-Flash-Next-Abliterated-EXL3-2.50bpw

Ghost

A refusal-removed (abliterated) build of Qwen's Qwen3.8-Flash-Next, quantized to EXL3 2.50 bpw so the full model runs on a single 24 GB card (RTX 3090 / 4090) using MoE CPU-offload — with the vision tower, MTP head, native 262,144-token context, and the PLE n-gram table all intact. Requires the same MoE CPU-offload setup as the stock 2.50bpw pack. Needs ~59 GB host RAM for the CPU expert tail and a fast NVMe for the streamed n-gram table. Expected on an RTX 3090: ~38 tok/s decode with MTP on (~28 without), ~20 tok/s at 175K depth, ~664 tok/s prefill. See the upstream repo for the full measured ledger; this quant uses the identical flags and layout, so numbers should track closely. Fired on…

Model · Text generation

Qwen3.8-Flash-Next-CNQ4.5-M

Robin

Qwen3.8-Flash-Next quantized to CNQ4.5-M: one NVFP4 container at 4.5 bpw with a BF16 keep set and MSE sub-block scales (--scales mse), weights for the crow-nest engine on Linux and Windows. - Verify the download with the first command below, in the package directory; it must report 6 of 6 OK and covers the container, the sidecar, the hot-set manifest and the three self-test files. The second command checks the self-test files alone, which costs no read of the 105 GB container. - The container is the CNQ v1 format written by the converter in the crow-nest repository (converter/src/main.rs, module comment of record main.rs:1-59). That converter reproduces this package from the original…

A personal, measured-on-one-machine deployment recipe for serving Qwen/Qwen3.8-Flash-Next on a single NVIDIA DGX Spark (GB10) with SGLang. It is not a benchmark leaderboard claim and not "the fastest possible"; it is what is verifiably running on one GB10, with every patch, script, and check needed to reproduce it from upstream weights. Verified: 2026-09-13 · SGLang 00143e9c23aee2dead5e6fe217bda4fa8739cb92 (nightly-dev-cu13-20260911) · runtime image qwen38-flashnext-sm121-hybrid-sharp:00143e9c-hc2, built by bounded-ple/nightly/01c-build-sglang-nightly.sh. The v4 lineage (d91c3682-hc1) is still in the repo as the stable fallback. Key runtime facts (all asserted at deploy time by the 05…

Model · Image and text to text

Qwen3.8-Flash-Next-MLX-oQ3-MTP

Robot Haus

A sensitivity-guided, mixed-precision MLX conversion of Qwen/Qwen3.8-Flash-Next, rebuilt directly from the official BF16 checkpoint with the model's matching native MTP block preserved. oQ3 uses a 3-bit affine base and spends additional precision on sensitive modules. Layer sensitivity was measured with a validated quantized calibration proxy, while every released weight was quantized from the official BF16 checkpoint. The result is a compact model with 746 higher-precision module overrides rather than a uniform 3-bit layout. The upstream tokenizer, current chat template, vision processor, generation configuration, licence, and native MTP configuration are retained. In a compatible oMLX…

Model · Image and text to text

Qwen3.8-Flash-Next-MLX-oQ3-MTP-fp16

Robot Haus

An FP16 conversion of Vontra/Qwen3.8-Flash-Next-MLX-oQ3-MTP, reprocessed with omlx-fp16-clone. Weights removed after benchmarking showed no performance benefit on M1 Ultra. FP16 was not faster than BF16 on M1 Ultra. The results show a clear split: FP16 prefill is significantly faster, but FP16 decode is slower and memory usage is higher. The omlx-fp16-clone script must promote all vision/audio passthrough tensors from BF16 to FP32 (an oQ invariant — oMLX enforces this and it cannot be skipped). This produces a checkpoint with three simultaneous dtypes: integer (quantized weights), FP16 (scales and biases), and FP32 (vision/audio tensors). The mixed-dtype layout appears to prevent oMLX's…

Model · Text generation

Qwen3.8-Flash-Next-ik-llama-GGUF_v2

Joel Farthing

HuggingFace's download widget does not recognize KT GGUFs. Download links are in the table below. Text-only GGUF conversion of Qwen3.8-Flash-Next, quantized with an importance matrix calibrated for agentic coding. This version uses IQ4KT for the 160-column PLE table and IQ3KT for the expert-down and shared-expert-down tensors. The previous release used IQ4NL for tensors whose row widths were not multiples of 256. The new file is 80.72 GiB, compared with 88.42 GiB for the previous IQ4KT. This file requires the IQ3KT/IQ4KT 32-element row-tail changes. Runtime validation used candidate 0b7b979. The source branch and PR are pending publication. This upload is the test artifact for that…

Model

Qwen3.8-Flash-Next-oQ4e-mtp

Robot Haus

This model was quantized using oQ (oMLX v0.6.3rc3) mixed-precision quantization.

Artifacts of the RACE2 (transition-aware fine-tuning of pi0.5) experiments on the VLABench The original training set (5,000 episodes, PELT labels) is SeonghoonYu/vlabenchprimitiveftlerobot224. Each run folder carries its assets/ (norm stats), weights, and where available README.md, trainingconfig.json, provenance.json and logs/.

We introduce the updated version of the Qwen3-4B non-thinking mode, named Qwen3-4B-Instruct-2507, featuring the following key enhancements: - Significant improvements in general capabilities, including instruction following, logical reasoning, text comprehension, mathematics, science, coding and tool usage. - Substantial gains in long-tail knowledge coverage across multiple languages. - Markedly better alignment with user preferences in subjective and open-ended tasks, enabling more helpful responses and higher-quality text generation. - Enhanced capabilities in 256K long-context understanding. Qwen3-4B-Instruct-2507 has the following features: NOTE: This model supports only non-thinking…

Rainbow-AI-Specialist is a custom multi-domain fine-tuned AI model based on google/gemma-2-2b-it, trained using QLoRA via Hugging Face TRL (SFTTrainer). You can load and test this adapter easily in Python using PEFT and Transformers: import torch from transformers import AutoModelForCausalLM, AutoTokenizer from peft import PeftModel modelid = "google/gemma-2-2b-it" tokenizer = AutoTokenizer.frompretrained(modelid) basemodel = AutoModelForCausalLM.frompretrained(modelid, torchdtype=torch.float16, devicemap="auto") model = PeftModel.frompretrained(basemodel, "RainbowLord/Rainbow-AI-Specialist") inputs = tokenizer("### User: Write a Python script for a simple trading bot.\n### Rainbow:"…