Open-Weight Models Under Apache License 2.0

Model

jebadiah-4b-v2-GGUF

Frontier Infra

GGUF builds of Jebadiah 4B v2 for llama.cpp, which runs on NVIDIA, AMD and Apple GPUs and on plain CPUs. Jebadiah answers a typed question (choice, noul or score) with a probability for every option, read from one forward pass. Nothing is generated. Code, trainer and evals: Results and docs - Project site, with every result and how to run the models: jebadiah.ai. Which one should I use? For local use, start with Jebadiah 9B v2 GGUF. On Apple silicon, use an MLX build: 27B, 9B v2 or 4B v2. For vLLM or fine-tuning, use the full weights: 27B, 9B v2 or 4B v2. Setup for every runtime, and for JDE: Run Jeb locally. Every file was checked on the 260 held-out questions the merged weights were…

static quants of https://huggingface.co/sophosympatheia/Glistening-Gem-31B-v1.0 For a convenient overview and download list, visit our model page for this model. weighted/imatrix quants are available at https://huggingface.co/mradermacher/Glimmer-31B-v1.0-i1-GGUF If you are unsure how to use GGUF files, refer to one of TheBloke's READMEs for more details, including on how to concatenate multi-part files. (sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) Here is a handy graph by ikawrakow comparing some lower-quality quant And here are Artefact2's thoughts on the matter…

Vivy-R-Finance-3B (R-FINANCE-3B) is the quantitative market reasoning and autonomous execution model of the Vivy.v6 R-Series ecosystem. Designed strictly around the Vytrading Core Design Rules: 1. The Brain (100% Local AI Autonomy): - A complete 3,085,938,688-parameter (3.09B) 36-layer Transformer model (vivy-r-finance-3b-q4km.gguf, ~2.10 GB, 435 tensors) fine-tuned via 36-layer BitFit (attnq.bias, attnk.bias, attnv.bias) and RMSNorm-IA³ (attnnorm.weight, ffnnorm.weight, outputnorm.weight) alongside a multi-head dmodel = 2048 Quantitative Semantic Compiler (.safetensors) and the native Rust coupled-decision solver (vivycore.dll). - The AI is 100% responsible for regime analysis, order-flow…

Model · Text generation

XcaliburBiz-1.8B-report

Crissy Deutsch

A lightweight report writer. A compact data summary goes in and a professional report comes out. This model is a derivative work based on Qwen1.5-1.8B-Chat by the Qwen Team, modified via Supervised Fine-Tuning and Direct Preference Optimization (DPO) by XcaliburMoon. - Supervised fine-tuning of a low rank adapter over section pairs built from the house reports. - Direct preference optimization from the supervised adapter. - Merging the aligned adapter into the base weights at full precision. - Quantization of the merged weights into the GGUF forms and one ONNX export. Apache License 2.0 grants no right to the trade names of the licensor beyond the use that describes the origin of the work.…

Instruction-tuned checkpoint of Han2Han (han2han-ul2-base-1-it, step 43153). Han2Han is a 169M-parameter encoder-decoder model that learns script-invariant representations of Korean text: a document written in Hanja and its Hangul transcription land at the same point in embedding space. The recipe (jamo and character-level embedding fusion, morpheme-aware denoising, bidirectional Hanja-Hangul transcription) is described in the paper, accepted to Findings of EMNLP 2026. This repo holds the PyTorch weights, the SentencePiece tokenizer, and the modeling code needed to load them through the transformers Auto classes with trustremotecode=True. Training code, the Flax model, the Flax-to-PyTorch…

Model · Text generation

Vinci-Cyber-8B-1.0-GGUF

SimpleDirect

GGUF distributions of SimpleDirect’s Canadian-developed, open-weight infrastructure-security model. This repository provides GGUF conversions of Vinci-Cyber-8B-1.0, the defensive infrastructure-as-code adaptation developed by SimpleDirect, a Canadian AI lab, from IBM Granite 4.1 8B. The intended application is human-reviewed security work on infrastructure you own or are authorized to assess. The full-weight model’s training centres on repair, no-change, and recovery examples. These are intended behaviours, not guarantees of correct or safe operation. The immediate source is Vinci-Cyber-8B-1.0. Its underlying parent is IBM Granite 4.1 8B, pinned to revision…

static quants of https://huggingface.co/Carchofa/gemma-3-12b-voice-assistant For a convenient overview and download list, visit our model page for this model. weighted/imatrix quants seem not to be available (by me) at this time. If they do not show up a week or so after the static ones, I have probably not planned for them. Feel free to request them by opening a Community Discussion. If you are unsure how to use GGUF files, refer to one of TheBloke's READMEs for more details, including on how to concatenate multi-part files. (sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) Here is a handy graph by ikawrakow comparing some…

Model · Text generation

Qwen3-1.7B-RKLLM

Hanzo Huang

Text generation on Rockchip RK3576 and RK3588, converted from Qwen/Qwen3-1.7B. Use the files for your board’s exact SoC. w4a16 uses 4-bit weights and 16-bit activations; w4a16g128 adds a weight group size of 128. w8a8 uses 8-bit weights and activations. File size below is download size, not total runtime RAM. With Docker installed on a matching Rockchip board, this example starts the RK3576 w4a16g128 model and its API on port 8001: The image includes its model file. See rkllm-docker for API requests, other board variants, and configuration. To download the files separately instead: These are native RKLLM/RKNN artifacts. Use a compatible RKLLM runtime and the model’s correct chat template…

Model

LuminaV-Optimizer-Paper

Lumina Moon

This repository (cloverx-id/LuminaV-Optimizer-Paper) is the official standalone and living development repository for the LuminaV optimizer family. While LuminaV was originally conceived and validated as the core engine for the XoneLM-1.0 language model series, all subsequent optimizer upgrades, low-precision Triton kernels, PyTorch standards compliance, and bug fixes are actively maintained and released directly in this repository. The v1.3.3 and v1.3.2 releases bring substantial compiler compatibility enhancements, distributed pretraining synchronization, zero-allocation memory stabilization, and mathematical precision refinements: - Validated JIT Execution on NVIDIA Architectures…

Model

TestSubject-2

Mr. Black Beard

Rose X1.5 SLM (98.8M params).

Fine-tune of unsloth/Qwen2.5-Coder-1.5B-Instruct that flags AI slop (over-defensive code, redundant comments, filler docstrings, emoji logs, placeholder TODOs…) in Python pull-request hunks and returns JSON findings. Built with tidypr. Use it in a PR: LoRA fine-tune (Unsloth, QLoRA) on a mix of: - inline review comments from humans on AI-agent pull requests (AIDev), and review comments from CodeReviewer, turned into labels by a Qwen2.5-Coder-7B teacher - hunks reviewers accepted without comment (CodeReviewer) - synthetic slop injected into human-written Python (CPython standard library) Weights: Apache-2.0, same as the base model unsloth/Qwen2.5-Coder-1.5B-Instruct. Code: MIT (tidypr). The…

Model · Text generation

tinybrainbot-350mV3-math

Nk Bass

Try it in your browser: the new tinybrainbot-350mV3-math-Q80.gguf (371 MB) runs locally on your device with WebGPU, and nothing you type is sent anywhere. A 348M-parameter decoder-only model fine-tuned to do column arithmetic with a written-out scratchpad. Successor to the TinyBrainBot 320M V2 Math. TL;DR: 99.4% average across the nine GPT-3 arithmetic sub-tasks, beating the 320M V2 Math (91.5%). Adds cleanly to 14 digits, does 3-digit multipliers (98.0%), and handles negative results (85% overall, but only 58% at 5 digits - see below). It is a calculator with a scratchpad, not a word-problem solver — see the honest limits below. Use applychattemplate rather than hand-building the prompt…

Model

mini-Jev

Samat Zharassov

mini-Jev is a 0.6B agentic decision model specialized for finite-choice tool and action selection. Give it a state, a question, and candidate actions; it returns a probability for each candidate. It uses a Qwen3-0.6B backbone with an adapter and a compact decision head. Candidate scoring is permutation equivariant, and the model returns finite-choice probabilities rather than generated text or tool arguments. This is an early 10,626-update checkpoint. Training is planned through 50,000 updates; more training and broader decision supervision are coming. This interim release is not the final checkpoint. The current step-010626 checkpoint loads its adapter from adapter/, its decision-head…

Release v2-20261020. The Scylla's Band runtime supports this release. Tag v2-20261020-apple adds Core ML and Core AI bundles of the same model; the runtime downloads from that tag. The model exposes energy, tension, valence and assertiveness on 0–4 scales with neutral at 2. Combining the controls changes the generated delivery; a request is not a guarantee of a particular perceived emotion or exact external scorer value. Install the runtime (pip install -e. from a checkout), then: The default download is the Core ML bundle on an Apple silicon Mac and the LiteRT bundle elsewhere, installed under scyllasband/models/. A Mac that can build apps for iOS, iPadOS or visionOS 27 is also offered the…

Model · Text generation

Hunter-1-4B

Lalakai Labs

Hunter 4B is a 4-billion-parameter conversational reasoning model from Lalakai Labs, built for local inference on consumer hardware. It works through problems step by step inside... traces before giving its final answer, and is tuned for reasoning, agentic coding, tool use, and cybersecurity analysis. Weights are distributed in GGUF format for llama.cpp, Ollama, LM Studio, and other GGUF-compatible runtimes. GGUF stores tensors only and does not use pickled code. - 4B parameters, optimized for reasoning, coding, and security tasks - Visible reasoning in traces for math, code, and multi-step problems - Q4KM quantization runs in roughly 3.5 GB of working memory at a 4K context - Apache-2.0…

static quants of https://huggingface.co/nectec/Pathumma-ThaiLLM-qwen3-8b-think-2.0.0 For a convenient overview and download list, visit our model page for this model. weighted/imatrix quants seem not to be available (by me) at this time. If they do not show up a week or so after the static ones, I have probably not planned for them. Feel free to request them by opening a Community Discussion. If you are unsure how to use GGUF files, refer to one of TheBloke's READMEs for more details, including on how to concatenate multi-part files. (sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) Here is a handy graph by ikawrakow comparing some…

Model · Text generation

Vinci-Piccolo-1.0-GGUF

SimpleDirect

GGUF (quantized) builds of Vinci Piccolo, for local inference with Ollama, LM Studio, and llama.cpp. For the full-precision weights, evals, and details, see simpledirect/Vinci-Piccolo-1.0. No GGUF file in this repository has been evaluated. The benchmark figures on the full-precision card were measured on the unquantized bf16 weights, not on any file here. Quantization shifts scores, and with no per-tier measurement this card cannot say in which direction or by how much for any tier. This card therefore publishes no per-tier score, no quality ranking between the tiers, and no recommended tier. The file sizes and memory figures below are measured, and describe storage and memory cost only. A…

static quants of https://huggingface.co/AscendKernelGen/KernelGen-LM-8B For a convenient overview and download list, visit our model page for this model. weighted/imatrix quants seem not to be available (by me) at this time. If they do not show up a week or so after the static ones, I have probably not planned for them. Feel free to request them by opening a Community Discussion. If you are unsure how to use GGUF files, refer to one of TheBloke's READMEs for more details, including on how to concatenate multi-part files. (sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) Here is a handy graph by ikawrakow comparing some lower-quality…

Model

Qwen3.8-27b-Terse-Coder-GGUF

Jad El-Khatib

GGUF builds of (round 8, final) — a fine-tune of Qwen/Qwen3.8-27B with ~1/10 the chain-of-thought reasoning tokens on coding tasks and correctness preserved. Quant spread (Q4KM smoke-tested on llama.cpp — correct code generation, reasoning parsing, ~22 tok/s on 2× RTX 5060 Ti): The MTP draft layer is included in the GGUF but unused by llama.cpp (harmless warnings about unused blk.N.nextn. tensors are expected); speculative decoding is a vLLM-side feature. Behavior/quality reference: HE+ 91.5 / MBPP+ 79.4 / GSM8K 98.5 / GPQA 79.8 on the source repo (vLLM measurements; GGUF Q4KM verified by smoke test only — validate quality on your own hardware before relying on it). vLLM deployment build at…

Laura is the tutor in an offline German course app for adult beginners (A1–A2). This model talks to the learner in English and teaches German with German examples. It runs on the phone, CPU only. This is a modified version of Qwen/Qwen3-4B-Instruct-2507. Changes made: a LoRA adapter (rank 16, all linear layers, 2 epochs) was trained on ~2,500–3,300 short tutoring conversations in English, merged into the weights, and the result converted to GGUF and quantised to Q4KM with llama.cpp. Distributed under the base model's Apache-2.0 licence. The app does not trust the model to judge German on its own. It checks the learner's sentence with its own dictionary first and adds the verdict to the…

Model · Image and text to text

road-model-public

Dante

This repository contains a LoRA adapter fine-tuned for the Zindi R.O.A.D. Barbados Historic Handwriting Challenge. The adapter was trained to transcribe cropped images of historical handwritten Barbados archival records into text. The repository includes the fine-tuned LoRA adapter, processor files, and run artifacts such as submission.csv, validation metrics, validation predictions, and the training config. The original challenge images and CSV data are not uploaded.

Model

Vinci-Prova-7B-1.0-GGUF

SimpleDirect

Quantised GGUF builds of Verify what you downloaded against these hashes. Equal file size is not identity. No GGUF file in this repository has been evaluated. Every published measurement was taken on the unquantised weights in the main repo, not on any file here. Quantisation changes model behaviour, and with no per-tier measurement this card cannot say in which direction or by how much for any tier. This card therefore publishes no per-tier score, no quality ranking between the tiers, and no recommended tier. A file's presence in this repository is not a performance claim about that tier, and not a claim of parity with the unquantised weights. All evaluation results, limitations, intended…

Model · Text generation

SmolVLM-500M-Instruct-Q4_K_M-GGUF

Shayon Sarker

Downloads: Hugging Face — ShayonSarker/SmolVLM-500M-Instruct-Q4KM-GGUF · Source & build recipes: GitHub — model-SmolVLM-500M-Instruct-gguf · Base model: HuggingFaceTB/SmolVLM-500M-Instruct · Failed vision attempt: MM-GGUF The GGUF binaries are hosted on Hugging Face; this repository holds the reproducible conversion pipeline, calibration data, and checksums. Base: HuggingFaceTB/SmolVLM-500M-Instruct (Apache-2.0) · Quant: llama.cpp 0.4.1-dev · imatrix on 3000-line wikitext-2 (seed 42, 625 chunks, 4 threads) Scope correction: this repository holds the text tower only (291 tensors, llama arch — verified: image input is rejected). The SigLIP vision encoder and mmproj are NOT included, so this…

Model · Text generation

BlazerApex-2B

Davi Rediske de Oliveira

BlazerApex-2B é um modelo de linguagem ultracompacto (2.5B parâmetros), altamente otimizado e fine-tuned com foco total em Português do Brasil (PT-BR), Programação, Lógica e Matemática. Projetado especificamente para ser o SOTA (State-of-the-Art) de bolso, este modelo roda 100% offline, de forma rápida e fluida em smartphones intermediários (como o Motorola Moto G34 5G com Snapdragon 695), sem precisar de internet ou servidores externos. Este repositório contém a versão quantizada IQ4XS (~1.4 GB), ideal para dispositivos com 4GB a 8GB de RAM. 1. Baixe o arquivo BlazerApex-2B-IQ4XS.gguf direto no seu celular. 2. Instale o aplicativo Maid ou Layla na Play Store. 3. Abra o app, vá em "Models"…

Overall Accuracy: 15.80%; Text Accuracy: 15.32%; Icon Accuracy: 16.43%; Mean Pixel Error: 347.5 px.

Model · Text generation

Huihui-Kolibri-1-abliterated-GGUF

Huihui.ai

This is an uncensored version of Aleph-Alpha/Kolibri-1 created with abliteration. This ablation was performed entirely using llama.cpp's GGUF model, without relying on Transformers, For the specific ablation method, please refer to cvector-generator. GGUFs come from Hob-forge/Kolibri-1-GGUF. The ablation did not modify all expert modules. Stock llama.cpp does not support the kolibri1 architecture yet. Apply kolibri1-llama.cpp.patch (included in this repo) to llama.cpp at upstream commit 836d571, then build: - Risk of Sensitive or Controversial Outputs: This model’s safety filtering has been significantly reduced, potentially generating sensitive, controversial, or inappropriate content.…

Part of the Qwen3-VL-8B Hebrew / Cairo Genizah fine-tune series (https://github.com/AIStream-Peelout/historical-document-analysis). Flagship: v21b step 1200. Trained 2026-09-06/09 (steps 100–200, then 300–2000); the only run in the series that completed its 2,000 steps. Warm start v19a step 1300; data identical to v20a, so it is a clean A/B. merger is the lever for better boxes. Eval loss 0.750 → 0.668. Finding: merger LoRA made the box geometry slightly worse and the transcription no better; the lever for accurate boxes is direct grounding data, not merger capacity. Final step 2000 4cae9d52d9cd167cb0d1ddd0248ed728c7308f29. All versions are QLoRA adapters on Qwen/Qwen3-VL-8B-Instruct…

static quants of https://huggingface.co/PersonalAILab/SMTL-30B For a convenient overview and download list, visit our model page for this model. weighted/imatrix quants seem not to be available (by me) at this time. If they do not show up a week or so after the static ones, I have probably not planned for them. Feel free to request them by opening a Community Discussion. If you are unsure how to use GGUF files, refer to one of TheBloke's READMEs for more details, including on how to concatenate multi-part files. (sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) Here is a handy graph by ikawrakow comparing some lower-quality quant And…

Valen-4B combines Qwen3.5-4B with a two-layer MLP-Mixer decision head. It predicts probability distributions for Choice, Noul (yes/no), and Score questions. A shared state can contain text, images, or video and serve multiple questions. Valen-4B 基于 Qwen3.5-4B 和两层 MLP-Mixer 决策头，支持图文、文本、视频，以及共享 state 的多题推理。 These weights include merged LoRA adapters. BF16 rounding can cause small probability differences from the original training checkpoints used for the reported benchmarks. The verified environment uses Python 3.10, PyTorch 2.6.0, Transformers 5.4.0, torchvision 0.21.0, Pillow, and av 16.1.0. Flash Attention 2 requires a compatible flash-attn installation. For a local copy, replace the…

Model · Text generation

ProactiveInquirer-Qwen3-8B-GGUF

Ido Levy

Ido Levy 1,2 · 1,2 GGUF quantizations of the trained questioner from Asking for What Was Never Requested: Horizontal and Vertical Proactivity in Agents, for llama.cpp, Ollama, LM Studio and Jan. They were made from the merged model with llama.cpp (commit 9adc7f4). Before upload, each file ran the adapter card's two-turn example with greedy decoding. Every file asked the same two questions as the full-precision model: "Who directed the film The Great Flamarion?" and, once the evidence named the director, "Who was the spouse of film director Anthony Mann?". The results are the trained questioner's, as the paper reports them: see the adapter card's Results. The paper evaluated the unquantized…

Published by TokForge for on-device inference. Runs on-device in the TokForge app. MNN-format 4-bit quantization of the Heretic-abliterated Qwen3.6-35B-A3B multimodal MoE, packaged for the TokForge Android MNN-fork runtime. - config.json: MNN LLM geometry (modeltype qwen35moe, jinja chat template, MRoPE, vision offsets) - llmconfig.json: backend defaults (cpu, threadnum=4, precision=low, memory=low) - llm.mnn / llm.mnn.weight: quantized MNN graph + external weight blob (Q4, block 64, HQQ) - embeddingsbf16.bin: BF16 embedding table (separated from quantized weights via --seperateembed) - tokenizer.mtok: MNN binary tokenizer - visual.mnn / visual.mnn.weight: vision transformer (DeepStack VLM)…

static quants of https://huggingface.co/Qwen/Qwen3.5-397B-A17B For a convenient overview and download list, visit our model page for this model. weighted/imatrix quants are available at https://huggingface.co/mradermacher/Qwen3.5-397B-A17B-i1-GGUF If you are unsure how to use GGUF files, refer to one of TheBloke's READMEs for more details, including on how to concatenate multi-part files. (sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) Here is a handy graph by ikawrakow comparing some lower-quality quant And here are Artefact2's thoughts on the matter: https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9 See…

static quants of https://huggingface.co/DarkArtsForge/Agares-31B-v1 For a convenient overview and download list, visit our model page for this model. weighted/imatrix quants are available at https://huggingface.co/mradermacher/Agares-31B-v1-i1-GGUF If you are unsure how to use GGUF files, refer to one of TheBloke's READMEs for more details, including on how to concatenate multi-part files. (sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) Here is a handy graph by ikawrakow comparing some lower-quality quant And here are Artefact2's thoughts on the matter: https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9 See…

static quants of https://huggingface.co/trillionlabs/gWorld-8B For a convenient overview and download list, visit our model page for this model. weighted/imatrix quants are available at https://huggingface.co/mradermacher/gWorld-8B-i1-GGUF If you are unsure how to use GGUF files, refer to one of TheBloke's READMEs for more details, including on how to concatenate multi-part files. (sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) Here is a handy graph by ikawrakow comparing some lower-quality quant And here are Artefact2's thoughts on the matter: https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9 See…

static quants of https://huggingface.co/Michael4933/Migician For a convenient overview and download list, visit our model page for this model. weighted/imatrix quants are available at https://huggingface.co/mradermacher/Migician-i1-GGUF If you are unsure how to use GGUF files, refer to one of TheBloke's READMEs for more details, including on how to concatenate multi-part files. (sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) Here is a handy graph by ikawrakow comparing some lower-quality quant And here are Artefact2's thoughts on the matter: https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9 See…

Model · Text classification

Kev-0.8B-LiteRT

LiteRT Community (FKA TFLite)

jaredpalmer/kev-0.8b is a decision model by Jared Palmer. It reads a text, called the state, and typed questions about it: yes/no (noul), multiple choice (choice) and rating (score). For each question it returns an answer with probabilities. It never generates text. A pointer head of two linear layers reads the hidden states of Qwen/Qwen3.5-0.8B-Base, adapted with a rank-16 LoRA, and turns them into those probabilities. Requests and responses follow the /v1/systemone shape of the author's server (github.com/jaredpalmer/kev). This repository holds the backbone as eight LiteRT files, with the LoRA folded into the weights. Six are row graphs, for rows of up to 64, 128, 256, 512, 1,024 and…

Model · Text classification

GLiNER2.5-Decide-LiteRT

LiteRT Community (FKA TFLite)

Run GLiNER2.5-Decide on a phone GPU: pass a text and any set of labels at call time (intent, routing, sentiment, yes/no gates, multi-label tags), get one decision per task from a single forward pass. The files are the official fastino/GLiNER2.5-Decide weights converted with LiteRT Torch. The validated Android configuration is LiteRT 2.2.0 with explicit GPU FP32 computation on a Samsung Galaxy S26 (SM-S942Q, SM8850, Android 16). In the Android sample on that phone, the decisions equal the official gliner2 fp32 result on all 126 tested (request, window) pairs; on desktop LiteRT CPU they equal it on all 361 test requests. Other Android GPU families have not been validated here. The image shows…

static quants of https://huggingface.co/TheZeez/Qwen-3.5-9b-Egyptian-1 For a convenient overview and download list, visit our model page for this model. weighted/imatrix quants seem not to be available (by me) at this time. If they do not show up a week or so after the static ones, I have probably not planned for them. Feel free to request them by opening a Community Discussion. If you are unsure how to use GGUF files, refer to one of TheBloke's READMEs for more details, including on how to concatenate multi-part files. (sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) Here is a handy graph by ikawrakow comparing some lower-quality…

firela-pa is a Sovereign Personal AI — a FIRE financial advisor that lives on your own device: your device decides what ever leaves, its memory of you never leaves home, and the cloud is used only when needed — with the question redacted, the provider swappable, and your identity never on the cloud side. The PC distribution of firela-pa: a task-fine-tuned Qwen3-1.7B privacy router (five intents: ledger query / portfolio / market quote / local chat / redacted-cloud), shipped as GGUF (Q4KM default, Q80 quality tier; 200-question eval G4 98.0) — the same weights family as the on-board W8A8 build. - Five-branch local routing — what leaves the device is decided by a router on your device, not by…

static quants of https://huggingface.co/ArtysicistZ/UI-TARS-MCTS-SFT-7B For a convenient overview and download list, visit our model page for this model. weighted/imatrix quants seem not to be available (by me) at this time. If they do not show up a week or so after the static ones, I have probably not planned for them. Feel free to request them by opening a Community Discussion. If you are unsure how to use GGUF files, refer to one of TheBloke's READMEs for more details, including on how to concatenate multi-part files. (sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) Here is a handy graph by ikawrakow comparing some lower-quality…

Model · Text to speech

nano

Nabad

MOSS-TTS-Nano is an open-source multilingual tiny speech generation model from MOSI.AI and the OpenMOSS team. With only 0.1B parameters, it is designed for realtime speech generation, can run directly on CPU without a GPU, and keeps the deployment stack simple enough for local demos, web serving, and lightweight product integration. 2026.4.10: We release MOSS-TTS-Nano. A demo Space is available at OpenMOSS-Team/MOSS-TTS-Nano. You can also view the demo and more details at openmoss.github.io/MOSS-TTS-Nano-Demo/. MOSS-TTS-Nano focuses on the part of TTS deployment that matters most in practice: small footprint, low latency, good enough quality for realtime products, and simple local setup. It…

Part of the Qwen3-VL-8B Hebrew / Cairo Genizah fine-tune series (https://github.com/AIStream-Peelout/historical-document-analysis). Flagship: v21b step 1200. Status: flagship and model of record (step 1200). It is the reader behind the two-reader consensus pipeline on cairogenizah.ai and the warm start of v22a. Trained 2026-09-11/15: steps 100–1200 in one session (the session died at ~1,240), then a continuation 1300–1700 (died at ~1,780). Warm start v20a step 1800. families), curated documentary pages.165, synthetic.13, Talmud replay.12. (colab/genizahv21bdispatchfix.ipynb). Eval loss 0.697 → 0.655 at step 1200 (best); 0.657 at 1700. Why it is the flagship: the two-column penalty that…

static quants of https://huggingface.co/Nimbz/Gemma-4-Giftige-Blume-31B-v2 For a convenient overview and download list, visit our model page for this model. weighted/imatrix quants are available at https://huggingface.co/mradermacher/Gemma-4-Giftige-Blume-31B-v2-i1-GGUF If you are unsure how to use GGUF files, refer to one of TheBloke's READMEs for more details, including on how to concatenate multi-part files. (sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) Here is a handy graph by ikawrakow comparing some lower-quality quant And here are Artefact2's thoughts on the matter…

Model · Image and text to text

openthai2.0-qwen3.8-27b-MLX-4bit

iApp Technology

MLX 4-bit quantization (mlx-vlm) of for Apple silicon. Vision included. ~16 GB — runs on 24 GB+ unified memory. pip install mlx-vlm python -m mlxvlm generate --model iapp/openthai2.0-qwen3.8-27b-MLX-4bit \ --image document.jpg --prompt "อ่านข้อความในเอกสารนี้ทั้งหมด" --max-tokens 8192 Notes: the MTP draft head is not included (mlx-vlm has no drafter support for this architecture yet). Under TensorFold (below), drafting comes from z-lab's DFlash2 drafter TensorFold serves this checkpoint as published, images included, through an OpenAI-compatible API (/v1/chat/completions, /v1/responses, /v1/messages). It runs on Apple silicon and on NVIDIA GPUs with compute capability 8.9+: RTX 40/50…

Qwen3.5 with a two-layer MLP-Mixer decision head. Supports text, images and videos, and returns Choice, Noul and Score decisions. Multiple questions can share one state encoding with execution="sharedstate". Trained on millions of samples. Install Python 3.10+, PyTorch, Transformers 5.4.0, torchvision, Pillow and av. Flash Attention 2 is optional with a compatible CUDA build. The model contains its tokenizer, processor and custom inference code; a separate base-model download is unnecessary. Use attnimplementation="flashattention2" for Flash Attention, or execution="question" for independent questions. Video defaults to 16 frames. Image/video request examples and training instructions are…

A GGUF quantization of — Microsoft's compact repository-level coding agent, Qwen3.5-4B post-trained with reinforcement learning on ~1,500 synthetic software-engineering tasks (SWE-bench Verified 61.5% Avg@3 in Microsoft's Leaf harness) — with two deliberate improvements over a stock Q4KM: 1. imatrix computed on a broad real corpus (bartowski's calibrationdatav3 — code + prose + facts + multilingual). 2. JANG-mixed precision: full-attention layers, embeddings and the gated-DeltaNet linear-attention layers (most of the network) kept at Q6K, feed-forward at the Q4KM base preset. (Not Q80 on attention — see "Honest notes": Q80 is ~3x slower than Q6K on Vulkan hardware without a hardware int8…

static quants of https://huggingface.co/analist/Qwen2-VL-2B-ChartQA-Foundation For a convenient overview and download list, visit our model page for this model. weighted/imatrix quants seem not to be available (by me) at this time. If they do not show up a week or so after the static ones, I have probably not planned for them. Feel free to request them by opening a Community Discussion. If you are unsure how to use GGUF files, refer to one of TheBloke's READMEs for more details, including on how to concatenate multi-part files. (sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) Here is a handy graph by ikawrakow comparing some…

static quants of https://huggingface.co/3tic/Orion-Qwen3.5-2B-SFT-v2603-v1 For a convenient overview and download list, visit our model page for this model. weighted/imatrix quants are available at https://huggingface.co/mradermacher/Orion-Qwen3.5-2B-SFT-v2603-v1-i1-GGUF If you are unsure how to use GGUF files, refer to one of TheBloke's READMEs for more details, including on how to concatenate multi-part files. (sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) Here is a handy graph by ikawrakow comparing some lower-quality quant And here are Artefact2's thoughts on the matter…

Part of the Qwen3-VL-8B Hebrew / Cairo Genizah fine-tune series (https://github.com/AIStream-Peelout/historical-document-analysis). Flagship: v21b step 1200. Status: pilot, not promoted. Best documentary reader in the series; a step back on religious pages; QA not yet learned. The full v22 run will follow with several times more unique rows. Trained 2026-09-25/26, steps 100–1300 (the Colab session hit its 24-hour cap at ~1,340). Warm start v21b step 1200. training rows, 188 validation rows on pages disjoint from training, 4,598 images): KTIV v4 transcription.30, line-broken PGP editions.25, KTIV v4 grounding.15, documentary grounding.10 (Kraken ink boxes on lines where Kraken and v21b…

Qwen3.5 with a two-layer MLP-Mixer decision head. Supports text, images and videos, and returns Choice, Noul and Score decisions. Multiple questions can share one state encoding with execution="sharedstate". Trained on millions of samples. Install Python 3.10+, PyTorch, Transformers 5.4.0, torchvision, Pillow and av. Flash Attention 2 is optional with a compatible CUDA build. The model contains its tokenizer, processor and custom inference code; a separate base-model download is unnecessary. Use attnimplementation="flashattention2" for Flash Attention, or execution="question" for independent questions. Video defaults to 16 frames. Image/video request examples and training instructions are…

Model · Image and text to text

Qwen3.8-27B-EXL3-3.85bpw-24GB

Jorge Ramírez Espejo

An EXL3 quantization of Qwen/Qwen3.8-27B made with exllamav3's per-tensor bit-allocation ("SC") pipeline with our own calibration, a 5-bit lmhead, a 6-bit vision tower, a 4-bit MTP head and INT8 token embeddings. It is served with vLLM (continuous batching) through the OrcaSAQ2 plugin and runs a 215k-token context with vision and MTP speculative decoding on a single 24 GB RTX 3090, with a KL divergence of 0.0073 against BF16 (≈5× lower than our W4A16 GPTQ version, and lower than turboderp's 4.00 bpw at a smaller size). The 3.70bpw branch trades a little fidelity (KLD 0.0089) for a 226k context. Highlights (one RTX 3090 24 GB, measured on 2026-10-14; details in Family comparison): - KLD…

Model · Image and text to text

Qwen3.8-27B-TTURBO-Cold-Fusion-709-L-Uncensored-mlx-oQ6e-1M

Solstice-AI

Solstice-AI/Qwen3.8-27B-TTURBO-Cold-Fusion-709-L-Uncensored-mlx-oQ6e-1M is the official Apple Silicon MLX release of DavidAU's Qwen3.8-27B-TWIN-TURBO-Fable-Cold-Fusion-709-L-Uncensored. Engineered specifically for Apple Silicon Unified Memory architectures (M2, M3, M4, M5 Max / Ultra), this release couples oQ6e imatrix-guided mixed-precision with Solstice's proprietary 10-Level Cognitive Variable Thinking System and native Hardware Multi-Token Prediction (MTP). This checkpoint embeds the Solstice 10-Level Cognitive Jinja chat template, allowing instant dynamic control over reasoning depth

Model · Image and text to text

Qwen3.8-27B-TTURBO-Cold-Fusion-709-L-Uncensored-mlx-oQ8e

Solstice-AI

Solstice-AI/Qwen3.8-27B-TTURBO-Cold-Fusion-709-L-Uncensored-mlx-oQ8e is the official Apple Silicon MLX release of DavidAU's Qwen3.8-27B-TWIN-TURBO-Fable-Cold-Fusion-709-L-Uncensored. Engineered specifically for Apple Silicon Unified Memory architectures (M2, M3, M4, M5 Max / Ultra), this release couples oQ8e imatrix-guided mixed-precision with Solstice's proprietary 10-Level Cognitive Variable Thinking System and native Hardware Multi-Token Prediction (MTP). This checkpoint embeds the Solstice 10-Level Cognitive Jinja chat template, allowing instant dynamic control over reasoning depth

Model · Text generation

Vinci-Bozza-1.0-GGUF

SimpleDirect

GGUF quantizations of simpledirect/Vinci-Bozza-1.0 for local inference with Ollama, LM Studio, and llama.cpp. Text-only. The source model is multimodal (image-text-to-text); these GGUF conversions include the language model only — no mmproj / vision projector. For image input, use the safetensors build: simpledirect/Vinci-Bozza-1.0. No GGUF file in this repository has been evaluated. The benchmark figures on the full-precision card were measured on the unquantized bf16 weights, not on any file here. Quantization shifts scores, and with no per-tier measurement this card cannot say in which direction or by how much for any tier. This card therefore publishes no per-tier score, no quality…

Model · Image and text to text

MiniCPM-V-4.7-35B-A3B

OpenBMB

MiniCPM-V 4.7 is the latest model in the MiniCPM-V series, available in a 35B-A3B variant, with 35B-A3B being the first MoE model in the MiniCPM-V series. It builds on the family's single-image, multi-image, and video understanding capabilities, combining stronger foundation capabilities with efficient visual encoding and on-device inference. Notable features of MiniCPM-V 4.7 include: - The First Open-Source MoE Model with Lossless, Smooth On-Device Inference. With our Lookahead Expert Scheduling (LES) technique introduced alongside MiniCPM-V 4.7, combined with fine-grained layer-wise quantization and DSpark speculative sampling integration, MiniCPM-V 4.7 35B-A3B becomes the first…

Model · Text generation

Qwen3.8-27B-coreml-agent

岡優志 / Okayuji

Run Qwen3.8 27B locally on an Apple silicon Mac with Core ML. With the companion Swift app it searches the web, reads pages, writes notes and answers with sources. Model inference stays on the Mac; search and page retrieval require network access. Requires: / Swift 6.2 · 96 GB+ RAM · ~30 GB disk. Measurements used a 128 GB Mac; 96 GB is untested. See Requirements. This repository contains weights and configuration: int8 weights, a 16,384-token context and the model's own multi-token-prediction head. The companion runtime supplies the stateful chain and speculation loop; the demo app supplies search, page reading, notes and tool calls. To run the agent: open CoreMLSamples.xcodeproj at the…

Model · Text generation

A11OY-MINI

SZL Holdings

749/14/163 · Λ = Conjecture 1 (advisory) · a-11-oy.com GGUFs are on this repo. GGUF AVAILABLE. Bytes MEASURED (size + sha256 below, ATELIER-verified). Evals none-this-run. Not MEASURED as an eval. publicationeligible: false. Quality ROADMAP in prose only. Lab load forbidden. GPU UNAVAILABLE. SKU of live SZLHOLDINGS/chaski merged shard 1c55df8. Not a new train. Not SZLHOLDINGS/chaski-5050. Not a Hub basemodelrelation: quantized child until a Chaski eval gate passes. A11oy is a Space and a substrate. The mini is the voice you can actually load. Not the organ. Not the mesh. A governed command voice that fits in llama.cpp. Nobody else ships this combination. That is the point of a one-of-one.…

Model · Image and text to text

Qwen3.8-27B-CQ-4bit

ThousandML

Qwen3.8 27B quantized to 4-bit with Coded Quantization (CQ). CQ is a calibration-free 4-bit quantization method designed to preserve the behavior of the original model without requiring a calibration dataset. Because CQ does not optimize the quantization around a fixed calibration corpus, it is designed to generalize beyond calibration-specific data and workloads. This repository contains the 27B-parameter Qwen3.8 model quantized with Coded Quantization (CQ) 4-bit. Lower mean KL divergence indicates closer agreement between the quantized model's output distribution and the original model. Top-1 agreement measures how often the quantized model and original model select the same…