Lite-OmniJev

This gemma4 model was trained 2x faster with Unsloth and Huggingface's TRL library.

Model · Image and text to text

Swift-1.5-Qwen3.8-27B-EXL3-3.85bpw-24GB

Jorge Ramírez Espejo

An unofficial community EXL3 quantization of ukisai/Swift-1.5-Qwen3.8-27b, UkisAI's reasoning-efficient derivative of Qwen3.8-27B, made with exllamav3's per-tensor bit-allocation ("SC") pipeline with our own calibration, YAQA Hessians, a 5-bit lmhead, a 6-bit vision tower, a 4-bit MTP head and INT8 token embeddings. It is served with vLLM (continuous batching) through the OrcaSAQ2 plugin and runs a 215k-token context with vision and MTP speculative decoding on a single 24 GB RTX 3090, with a KL divergence of 0.009 against BF16 (≈5× lower than our W4A16 GPTQ version). The 3.70bpw branch trades a little fidelity for a 226k context. Highlights (one RTX 3090 24 GB, measured on 2026-10-14…

Model · Image and text to text

Qwopus3.8-27B-Flash-V2-EXL3-3.85bpw-24GB

Jorge Ramírez Espejo

An EXL3 quantization of Jackrong/Qwopus3.8-27B-Flash-V2 made with exllamav3's per-tensor bit-allocation ("SC") pipeline with our own calibration, a 5-bit lmhead, a 6-bit vision tower, a 4-bit MTP head and INT8 token embeddings. It is served with vLLM (continuous batching) through the OrcaSAQ2 plugin, with the base Qwen3.8 chat template for reliable tool calls. KL divergence against BF16: 0.0060 (≈6× lower than our W4A16 GPTQ version). It runs a 215k-token context with vision and MTP speculative decoding on a single 24 GB RTX 3090; the 3.70bpw branch (since 2026-10-14) runs 226k with vision and 4 sessions (see Variants). Highlights (one RTX 3090 24 GB, measured on 2026-10-14; details in…

Model · Image and text to text

Qwen3.8-27B-EXL3-3.85bpw-24GB

Jorge Ramírez Espejo

An EXL3 quantization of Qwen/Qwen3.8-27B made with exllamav3's per-tensor bit-allocation ("SC") pipeline with our own calibration, a 5-bit lmhead, a 6-bit vision tower, a 4-bit MTP head and INT8 token embeddings. It is served with vLLM (continuous batching) through the OrcaSAQ2 plugin and runs a 215k-token context with vision and MTP speculative decoding on a single 24 GB RTX 3090, with a KL divergence of 0.0073 against BF16 (≈5× lower than our W4A16 GPTQ version, and lower than turboderp's 4.00 bpw at a smaller size). The 3.70bpw branch trades a little fidelity (KLD 0.0089) for a 226k context. Highlights (one RTX 3090 24 GB, measured on 2026-10-14; details in Family comparison): - KLD…

Model · Image and text to text

Qwen3.8-27B-pi-EXL3-3.85bpw-24GB

Jorge Ramírez Espejo

An EXL3 quantization of bytkim/Qwen3.8-27B-pi (bytkim's agent/coding fine-tune of Qwen3.8-27B for the Pi harness), made with exllamav3's per-tensor bit-allocation ("SC") pipeline calibrated on a trace generated by the pi model itself, with a 5-bit lmhead, a 6-bit vision tower, a 4-bit MTP head and INT8 token embeddings. It is served with vLLM (continuous batching) through the OrcaSAQ2 plugin and runs a 215k-token context with vision and MTP speculative decoding on a single 24 GB RTX 3090, with a KL divergence of 0.0074 against BF16 (0.0070 on turboderp's public trace). The 3.70bpw branch trades a little fidelity (KLD 0.0093) for a 226k context. Same recipe, plugins and serving command as…

A multi-image chain-of-thought fine-tune of OpenGVLab/InternVL3-8B, merged into standalone weights so it loads with trustremotecode=True in transformers or vLLM without the adapter. final model. - LoRA, r=64, alpha=128, dropout=0.05, on q/k/v/oproj and gate/up/downproj - modulestosave: ["visionmodel", "mlp1"] — the vision tower and the projector were trained in full rather than low-rank adapted, which is why the adapter was 1.3 GB and why merging is required for serving Accuracy against the base model. The prompt is stated per row, because it matters more than the fine-tune does: rows are only comparable to other rows evaluated the same way. MIRB and MuirBench/BLINK use different image…