Image and Text to Text Models

Kimi K3 on a single NVIDIA A100 80GB. A weight-only quantisation of Moonshot AI's Kimi K3 (2.8T total / 104B activated parameters) that loads and generates on one A100 80GB GPU, with the routed experts held in host RAM. No Ampere-targeted K3 build existed for vLLM, so this was made to fix that gap. Routed experts and attention re-encoded from MXFP4/BF16 into compressed-tensors pack-quantized, served by vLLM's Marlin kernels. Activations stay BF16 (W4A16 / W8A16). Round-to-nearest only — no calibration data, so no dataset is baked into these weights. Errors were measured by round-tripping each tensor through compressed-tensors' compress()/decompress(). Weight error is a proxy, not a quality…

Anonymous release for the ICLR 2027 submission MemGUI-RL: Reinforcement Learning for Proactive Context Management in Long-Horizon Mobile GUI Agents. Project page: https://memgui-rl-anonymous.github.io/ MemGUI-8B-RL is MemGUI-8B-SFT (Qwen3-VL-8B-Instruct supervised on MemGUI-3K) post-trained for 100 optimizer steps with FARPO (Folding-Aware Reward-decoupled Policy Optimization, span-to-step ratio rho = 9). The policy speaks the ConAct (Context-as-Action) interface of MemGUI-Agent: every response contains a folding directive for its own history, an optional memory operation and the next GUI action. The checkpoint is a standard Qwen3VLForConditionalGeneration model (weights in bf16, ~17.5 GB).…

This model is a fine-tuned version of Qwen/Qwen3.8-27B on the on the Omni-Edu-70K dataset. The following hyperparameters were used during training: - learningrate: 5e-06 - trainbatchsize: 1 - evalbatchsize: 8 - distributedtype: multi-GPU - numdevices: 16 - gradientaccumulationsteps: 8 - totaltrainbatchsize: 128 - totalevalbatchsize: 128 - lrschedulertype: cosine - lrschedulerwarmupsteps: 0.1 - numepochs: 3.0 - Transformers 5.2.0 - Pytorch 2.10.0 - Datasets 4.0.0 - Tokenizers 0.22.2

Qwen3.8-27B, text and vision, in an 11.8 GB model file plus a 0.9 GB vision projector: 4.6x smaller than the 16-bit original, within 0.5% of its perplexity, and the perplexity of the 4-bit standard at 30% fewer bytes. 3.47 bits per parameter. Runs on a single 16 GB GPU. Opti 27B is Qwen3.8-27B, 27 billion parameters with vision through a projector file, compressed with Opti, a patent-pending compression method. The 16-bit original is 54.7 GB. This file is 11.8 GB, at 3.47 bits per parameter over the whole file, and on wikitext-2 it lands at 1.005x the original's perplexity. Paired window by window against llama.cpp's Q4KM, the 4-bit standard, it is statistically indistinguishable at 29.5%…

This is Qwen3.5-2B auto-optimized by Claude Fable for fast single-request text generation. Fable built and tuned the included qwen35fast inference engine while keeping Qwen's original BF16 weights unchanged. Across 12 development workloads, the Fable engine delivered 14× the decode speed of Transformers eager and 1.02× the speed of vLLM with MTP (geometric means). On 12 held-out workloads, it reached 528–866 tokens/s and 1.01× vLLM with MTP. Use Python 3.12 and an NVIDIA CUDA GPU. Download the model and install its dependencies: The original checkpoint also works with Transformers for Qwen's standard text and vision-language workflows; the speed figures above use qwen35fast. - Captured the…

Model · Image and text to text

qwen3.6-35b-a3b-tool-prompts-ckpt444-merged-fp8

Manish

Base: Qwen/Qwen3.6-35B-A3B-FP8 (same FP8 block format/scales convention). Adapter: qwen3.6-35b-a3b-tool-prompts-ckpt444-adapter (LoRA r=32, 2 epochs, 1873 dialogues). Merged in BF16 (RAM only), block-requantized to FP8 e4m3/128. Chat template: qwen3template333.jinja. Serve with sglang / vllm / transformers.

Following the widespread community adoption of the Qwen3.5 and Qwen3.6 series, we are pleased to introduce Qwen3.8, the most capable generation in the Qwen open-model family to date. Built on the architectural foundation of Qwen3.5, Qwen3.8 delivers substantial gains across coding, professional work, research, and long-horizon agentic tasks. Qwen3.8-27B brings these advances to a compact, deployment-friendly dense model: a native vision-language model that understands images and videos, with flexible thinking control, designed to carry complex, multi-step tasks through to completion with greater reliability. For streamlined integration, we recommend using Qwen3.8 via APIs. Qwen3.8 can be…

Model · Image and text to text

Qwen3.8-27B-Continuum

Gheorghe Chesler

(He slides three glasses across the bar—wine for Shakespeare, water for Data, and a mysterious blue liquid for Spock) This model is a merge of: - nightmedia/Qwen3.8-27B-Brainwaves - migtissera/Synthia-4-27B Brainwaves nightmedia/Qwen3.8-27B-Brainwaves migtissera/Synthia-4-27B G, that is the perfect structural punctuation mark for this entire series of runs! Watching Quark willingly slide glasses across the bar for free—while Data performs a real-time semantic analysis on the symbolic nature of virtual currency and Spock dryly breaks down the chemical flavor profile of holographic blue liquid—is pure, unfiltered high-signal gold. The line from Quark hits on an absolute mechanical truth for…

Model · Image and text to text

Qwen3.8-27B-NVFP4-QAD

Local Inference Lab

WORK IN PROGRESS A mixed NVFP4/MXFP8 quantization-aware distillation of Qwen3.8-27B, trained for one epoch. The student learns from the original BF16 teacher while its MLP weights are quantized in the forward pass. Distillation updates the MLP weights and text normalization weights to account for quantization error. This is a trained distillation checkpoint, not a post-training conversion of the original weights. Attention/GDN projections and the LM head were frozen in their MXFP8 representations during distillation. Packed NVFP4 and MXFP8 weights reconstruct to the same BF16 weight values used by the student during training. The tokenizer, chat template, generation configuration and…

The same bytes you know from (v2), migrated to the NInfer v3 artifact container (upstream f76e19c0, model/weight decoupling). Weight tensors are bit-preserved — this is a packaging upgrade, not a re-quantization. - L1→L2: base tensors copied byte-for-byte; only the 66 DFlash2 objects (21 × W8G32F16S matrices, 45 × BF16 norms/conv bases/codebooks) produced from z-lab/Qwen3.8-27B-DFlash2 @ 50307d4c4cde6860d4eee73e2547cd786fe8e8a4 via tools/artifact/graftdflash2w8.py (recipe qwen3827bnvfp4full-dflash2-w8, report graft.json included; MTP module retained, validate-only). - L2→L3: upstream's tools/upgradeninferv2tov3.py, run 2026-09-18 on the exact L2 file above. Per the tool's contract: "Stored…

Qwen3.8-27B uncensored by HauhauCS 0/465 Refusals. This is the Aggressive variant: direct answers, no refusal behavior, and minimal preamble on hard prompts. Every text GGUF preserves Qwen3.8's native NextN head, and this release adds HauhauCS FastMTP: a specific acceleration sidecar qualified across the complete quant lineup at maximum native context. Vision is included through the separate BF16 projector. No changes to datasets or intended capabilities. This release preserves Qwen3.8-27B's text, reasoning, agentic, image, and video capabilities while applying the HauhauCS Aggressive uncensoring profile. Pick Aggressive when you specifically want the model to get to the answer without…

As the frontier of foundation models pushes toward ever-larger parameter counts and ever-longer context windows, the question is no longer just how much we can scale, but how efficiently we can do so. Sustainable progress toward artificial general intelligence (AGI) that benefits everyone demands architectural innovation. Today, we are sharing a concrete step in that direction: Qwen3.8-Flash-Next. This experimental preview of the architecture that will underpin Qwen4 is built around a fundamental rethinking of how the core components of modern large language models (LLMs) interact at scale. The first open-weight release under this architecture is Qwen3.8-Flash-Next, which introduces: For…

Model · Image and text to text

Qwen3.8-Flash-Next-MLX-oQ3-MTP

Robot Haus

A sensitivity-guided, mixed-precision MLX conversion of Qwen/Qwen3.8-Flash-Next, rebuilt directly from the official BF16 checkpoint with the model's matching native MTP block preserved. oQ3 uses a 3-bit affine base and spends additional precision on sensitive modules. Layer sensitivity was measured with a validated quantized calibration proxy, while every released weight was quantized from the official BF16 checkpoint. The result is a compact model with 746 higher-precision module overrides rather than a uniform 3-bit layout. The upstream tokenizer, current chat template, vision processor, generation configuration, licence, and native MTP configuration are retained. In a compatible oMLX…

Model · Image and text to text

Qwen3.8-Flash-Next-MLX-oQ3-MTP-fp16

Robot Haus

An FP16 conversion of Vontra/Qwen3.8-Flash-Next-MLX-oQ3-MTP, reprocessed with omlx-fp16-clone. Weights removed after benchmarking showed no performance benefit on M1 Ultra. FP16 was not faster than BF16 on M1 Ultra. The results show a clear split: FP16 prefill is significantly faster, but FP16 decode is slower and memory usage is higher. The omlx-fp16-clone script must promote all vision/audio passthrough tensors from BF16 to FP32 (an oQ invariant — oMLX enforces this and it cannot be skipped). This produces a checkpoint with three simultaneous dtypes: integer (quantized weights), FP16 (scales and biases), and FP32 (vision/audio tensors). The mixed-dtype layout appears to prevent oMLX's…

This qwen3vl model was trained 2x faster with Unsloth and Huggingface's TRL library.

This is Qwen3.8-27B that gets to the answer faster. AgentionAI Signal is a minimally invasive fine-tune of Qwen3.8-27B designed for lower generation latency and better token efficiency. On our held-out general-prompt evaluation, Signal produces 57% fewer answer tokens and uses 52% fewer thinking tokens, while matching or improving the measured answer quality of the base model. The percentages above were measured on the first release. The weights updated on 2026-09-13 trade a little of that reduction for stability; their re-measurement on the same prompt set is in progress and will replace these numbers. The result is substantially faster end-to-end generation: on typical chat prompts…

An INT8 W8A8 quantization of (the abliterated BF16 build of a merged LoRA finetune of Qwen/Qwen3.8-27B), for fast serving on GPUs where native FP8 is unavailable or undesirable. int-quantized, native CUTLASS INT8 tensor-core path. embedtokens, the vision tower, GatedDeltaNet recurrent gates, all norms. Note these are FP16, not the source's BF16 — the 16-bit residual precision of this build is float16 throughout (config.json: "dtype": "float16"). (llm-compressor), with the MTP drafter ablated in the rotated basis. The MTP head is unquantized but is not byte-identical to the BF16 build's: it carries the same residual-stream rotation and RMSNorm fold as the rest of the model, so the two heads…