Open-Weight Models Under Apache License 2.0

Model · Image and text to text

flux-3-image-klein-pup-Q8_0-GGUF

KJS

This model was converted to GGUF format from black-forest-labs/flux-3-image-klein-pup using latest llama.cpp Refer to the original model card for more details on the model. Install llama.cpp through brew (works on Mac and Linux) Invoke the llama.cpp server or the CLI. Note: You can also use this checkpoint directly through the usage steps listed in the Llama.cpp repo as well. Step 2: Move into the llama.cpp folder and build it with LLAMACURL=1 flag along with other hardware-specific flags (for ex: LLAMACUDA=1 for Nvidia GPUs on Linux).

FLUX 3 Action (flux3) is Black Forest Labs' video-action model: the FLUX.3 video trunk with an action modality that is denoised jointly with the next video frames, finetuned per robot embodiment with fresh action heads. This policy has been trained and pushed to the Hub using LeRobot. Learn how to train and run it in the LeRobot flux3 guide, or browse the full documentation. The policy consumes these observation features and produces these action features. Inputs Outputs New to LeRobot? These guides cover the full workflow: - Install LeRobot — set up the lerobot package. - Hardware setup — assemble, wire, and calibrate your robot and cameras. - Record data & train a policy — the end-to-end…

FLUX 3 Action (flux3) is Black Forest Labs' video-action model: the FLUX.3 video trunk with an action modality that is denoised jointly with the next video frames, finetuned per robot embodiment with fresh action heads. This policy has been trained and pushed to the Hub using LeRobot. Learn how to train and run it in the LeRobot flux3 guide, or browse the full documentation. The policy consumes these observation features and produces these action features. Inputs Outputs New to LeRobot? These guides cover the full workflow: - Install LeRobot — set up the lerobot package. - Hardware setup — assemble, wire, and calibrate your robot and cameras. - Record data & train a policy — the end-to-end…

Model · Speech recognition

fongbe-asr-omni-llm-1b-v2-fon-v08

Kévyn ODJO

Modèle ASR complet affiné pour transcrire le fongbe à partir de la parole. Base: omniASRLLM1Bv2; tokenizer: Fon v0.4.1. Le bundle préparé contient les poids complets sélectionnés, leur configuration, l’espace de sortie, les artefacts du tokenizer et le runtime autonome. Arrêt anticipé sur la validation après 8 000 updates, au septième epoch. Plafond: 54 016 updates et 20 epochs; horizon calculé sur ce train: 23 607 updates. Le checkpoint retenu est celui du step 4 721, choisi sur le CER exact du dev (WER en cas d’égalité). Split privé JaimeMaLangue/fon-meilleur-asr:split-v2.1, révision e105be46cca8784e3b89142b09fae555d905e5e8. Train: 50 323 énoncés, 106,0355 heures réellement décodées, 612…

This model is a fine-tuned version of distilbert/distilbert-base-uncased on an unknown dataset. It achieves the following results on the evaluation set: The following hyperparameters were used during training: - learningrate: 0.0001 - trainbatchsize: 32 - evalbatchsize: 32 - lrschedulertype: linear - numepochs: 10 - Transformers 5.16.1 - Pytorch 2.11.0+cu128 - Datasets 4.8.5 - Tokenizers 0.23.1

FORM-8B is a QLoRA fine-tune of Qwen3-8B (unsloth/qwen3-8b-unsloth-bnb-4bit) specialised for zero-shot FORM code generation. FORM is the symbolic manipulation language used for large perturbative QFT calculations (Grassmann The model is trained exclusively on execution-verified data: every training example was validated by running the generated FORM program through the FORM binary (deterministic-output match) plus an LLM consistency audit. Training config: QLoRA r=16, alpha=32, lr=2e-4, 5 epochs, 4-bit NF4 quantised base. is available from the author upon request. - Generating complete FORM programs from plain-English instructions - Assisting physicists with symbolic-algebra scripting…

This is a pretrained foveated vision model from the fovi library. - Cortical Magnification Function Hyperparameter (a): 2.78 Apache 2.0

Jianze Li 1, Kai Liu 1, Ziqing Zhang 1, Weihang Zhang 2, Yong Guo 1, Linghe Kong 1, Yulun Zhang 1,† 1. Stage I — Spatial and composition training. Learn spatial localization, composition preferences, and complete reference-frame prediction. 2. Stage II — Quality-Guided Online Supervision (QOS). Generate greedy and sampled answers, evaluate complete-frame quality, and imitate the best answer. 3. Stage III — Greedy-Referenced Optimization (GRO). Refine fresh samples using positive and negative quality differences relative to the current greedy answer. At inference, FrameLite directly predicts one normalized crop through greedy decoding. No candidate search, external scorer, retrieval…

SmolVLA is a compact, efficient vision-language-action model that achieves competitive performance at reduced computational costs and can be deployed on consumer-grade hardware. This policy has been trained and pushed to the Hub using LeRobot. See the full documentation at LeRobot Docs. For a complete walkthrough, see the training guide. Below is the short version on how to train and run inference/eval: Writes checkpoints to outputs/train/ /checkpoints/. Prefix the dataset repo with eval\ and supply --policy.path pointing to a local or hub checkpoint.

Expert-pruned variant of FreeToken-Qwen3.5-35B-A3B-FP8: 46% of routed experts removed (241 of 512 kept per layer, selected by routing-count calibration), with no requantization — surviving FP8 tensors and block scales are byte-identical to the base. Router rows compacted and renumbered; MTP block pruned consistently. Teacher-forced logprob-equivalence (32 samples/domain, mean per-token delta vs base server; negative = worse): HumanEval pass@1 (temp 0, thinking off): 14/24, identical to base, same 14 tasks. For context, deeper tiers degrade measurably: at 63% pruning (192 experts) OOD deltas grow ~4x while in-domain still looks clean (calibration-coverage failure); at 75% pruning every…

This model is a fine-tuned version of bert-base-uncased on an unknown dataset. It achieves the following results on the evaluation set: The following hyperparameters were used during training: - learningrate: 5e-05 - trainbatchsize: 32 - evalbatchsize: 64 - lrschedulertype: linear - numepochs: 2 - Transformers 5.16.1 - Pytorch 2.11.0+cu128 - Datasets 4.8.5 - Tokenizers 0.23.1

This model is a fine-tuned version of facebook/detr-resnet-50 on an unknown dataset. The following hyperparameters were used during training: - learningrate: 1e-05 - trainbatchsize: 8 - evalbatchsize: 8 - lrschedulertype: linear - numepochs: 100 - mixedprecisiontraining: Native AMP - Transformers 4.57.6 - Pytorch 2.11.0+cu130 - Datasets 4.8.5 - Tokenizers 0.22.2

This model is a fine-tuned version of facebook/detr-resnet-50 on an unknown dataset. The following hyperparameters were used during training: - learningrate: 1e-05 - trainbatchsize: 8 - evalbatchsize: 8 - lrschedulertype: linear - numepochs: 100 - mixedprecisiontraining: Native AMP - Transformers 4.57.6 - Pytorch 2.11.0+cu130 - Datasets 4.8.5 - Tokenizers 0.22.2

This model is a fine-tuned version of facebook/detr-resnet-50 on an unknown dataset. The following hyperparameters were used during training: - learningrate: 1e-05 - trainbatchsize: 8 - evalbatchsize: 8 - lrschedulertype: linear - numepochs: 100 - mixedprecisiontraining: Native AMP - Transformers 4.57.6 - Pytorch 2.11.0+cu130 - Datasets 4.8.5 - Tokenizers 0.22.2

static quants of https://huggingface.co/endless-frontier/Fx-Work For a convenient overview and download list, visit our model page for this model. weighted/imatrix quants seem not to be available (by me) at this time. If they do not show up a week or so after the static ones, I have probably not planned for them. Feel free to request them by opening a Community Discussion. If you are unsure how to use GGUF files, refer to one of TheBloke's READMEs for more details, including on how to concatenate multi-part files. (sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) Here is a handy graph by ikawrakow comparing some lower-quality quant…

Model

g1_depth_dodge

Martino Russi

Frozen SONIC whole-body controller + seven rank-16 LoRAs on the decoder, conditioned on a 128-d percept from a head depth camera (Theia-Tiny tokens + temporal adapter). PPO, iteration 4000. Consumed by lerobot's DepthDodgeController for the Unitree G1. Camera contract: 64x96 optical-Z depth in metres, 45 deg vertical FOV, 0 deg pitch, valid 0.2-6 m, 25 Hz. See manifest.json for the full training config.

weighted/imatrix quants of https://huggingface.co/vulture3/G4-WanaMeroQueen-v3-31B-heretic For a convenient overview and download list, visit our model page for this model. static quants are available at https://huggingface.co/mradermacher/G4-WanaMeroQueen-v3-31B-heretic-GGUF This is a vision model - mmproj files (if any) will be in the static repository. If you are unsure how to use GGUF files, refer to one of TheBloke's READMEs for more details, including on how to concatenate multi-part files. (sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) Here is a handy graph by ikawrakow comparing some lower-quality quant And here are…

Gander-Omni/Gander, release Gander-v63-Unit8 on MiniCPM-o 4.5, with refusal behaviour reduced by abliteration of the Thinker's language tower. It is BF16 and keeps upstream's layout, thinker/ and talker/, so it drops into the Gander runtime the same way as the upstream release. An NVFP4 build is at Only the Thinker's language tower (llm., Qwen3-8B shape, 36 layers). The Thinker's vision tower, audio tower, resampler and audio projection are upstream's, unmodified. So are the Talker (talker/) and its token2wav assets. - 72 matrices were changed: selfattn.oproj and mlp.downproj in all 36 layers. - The largest relative change is ‖ΔW‖/‖W‖ = 0.135. - The language tower was extracted as a…

The Thinker of justsumguy/Gander-ablated, quantised with NVIDIA ModelOpt 0.46.1, plus the unmodified BF16 Talker. The Thinker is 7.2 GB and the Talker 0.7 GB. The abliteration details and numbers are on the BF16 page. In short: heretic ARA branch, ARA-LoRA rank 128, trial 4, refusals 98 → 7 of 100, KL 0.128 on recheck, no healing applied. talker/ is BF16 and identical to the BF16 repository's. - llm calibration: 256 samples × 512 tokens of tatsu-lab/alpaca (shuffled, seed 0), run through the extracted text tower. Mean NLL on 8 calibration chats: BF16 4.335, NVFP4 fake-quant 4.408 (+0.073). - vpm calibration: 80 images from johnowhitaker/imagenette2-320. Three quarters were unsliced; one…

Model

GDN_pico_ByT5_tokenizer

Kurogane Himaki

use attahed notebook.

Model · Text generation

Gemma-3-27b-it-abliterated-GGUF

Alan Frederick Benitez

Unlike incomplete GGUF uploads, this repository provides the full 11-quantization spectrum (from high-precision F16 down to lightweight Q2K) of the abliterated Gemma-3-27b-it model. Choose the exact fit for your VRAM/RAM constraints without sacrificing reasoning capabilities. This is the integral and complete collection of quantizations in GGUF format for the gemma-3-27b-it-abliterated model, prepared locally for use with llama.cpp, Ollama, LM Studio, or any application compatible with multimodal models. This model is an uncensored version based on Google's powerful Gemma 3 27B multimodal engine. Being significantly more resistant to neutralization than previous architectures, it was…

This gemma3 model was trained 2x faster with Unsloth

On a laptop running Debian 13 Linux, with an Intel Core Ultra 9 185H, its integrated Arc GPU and 32 GB of memory, Janas-LLM reads a prompt at 65 tok/s and writes at 9.9 tok/s, 14 tok/s with its own assistant in a chat: llama.cpp, on the same laptop, 46 and 7.1. Measured on 9-11 October 2026: llama-bench -p 1024 -n 16 and janas-bench --prompt 1024 --gen 16, both at their best configuration; the drafts with janas-bench on a chat reply, the mean of three rounds (a draft guesses the next tokens and the model checks them: the same text, faster). On other machines the figures will differ. Details: Janas README. - Debian 13 (trixie) Linux, x86-64, kernel 6.12.107 - its integrated Intel Arc GPU…

Model · Text generation

gemma-4-12B-it-Pollard

Pollard Weights

Pollard builds of google/gemma-4-12B-it made with Pollard Weights -- a ladder of measured-allocation quants (bits placed by per-layer sensitivity, not a uniform crush). Standard GGUF -- every file here runs in stock llama.cpp / ikllama.cpp, Ollama, LM Studio. Every rung is the same weights, sized to a different RAM budget by the measured allocation. Pick the largest one that fits your machine with room for context: - ~12 GB RAM / VRAM -> Q6K (9.65 GB). - ~9 GB RAM / VRAM -> IQ4XS (6.78 GB). - ~7 GB RAM / VRAM -> IQ2XXS (4.64 GB). f16 reference PPL 23.041768. These are standard GGUF and run with llama.cpp: They also work in anything built on llama.cpp -- LM Studio, koboldcpp, Jan, ramalama…

Google publishes the QAT Q40 model google/gemma-4-12B-it-qat-q40-gguf as GGUF, but its multi-token-prediction (MTP) assistant only as safetensors There was no GGUF MTP head for the QAT GGUF, so this repository is that assistant converted to GGUF, nothing else: no training or fine-tuning. Use it together with google/gemma-4-12B-it-qat-q40-gguf for speculative decoding in llama.cpp. The Google model card asks for exactly this pairing: "When using multi-token prediction (speculative decoding) with an assistant model alongside a QAT target model, the assistant model must also be a QAT checkpoint with the same precision." llama.cpp at commit 0c1e570 (Gemma4UnifiedAssistantForCausalLM → GGUF…

Model · Text generation

gemma-4-12B-it-qat-q4_0-unquantized-MoE

Aifeifei

Inspired by the big.LITTLE architecture in phone SoCs — a few big cores for heavy lifting, many LITTLE cores for efficient background work, and a scheduler that migrates tasks by demand — Myriad-MoE applies the same idea to LLM adapters. Grafted onto a frozen Gemma-4-12B backbone (48 layers, work, 16 clans × 16 LITTLE micro-experts absorb fine-grained load, and 1 always-on shared core holds global context like a system housekeeping core. Learned routers play scheduler; training uses supervised domain routing, 4-bit QLoRA-style adapters, and a hot-swappable patch workflow. Ships with a FastAPI backend and a pnpm (Vite + React) chat UI with a live routing probe. MoE adapters are grafted onto…

Model · Text generation

gemma-4-26b-a4b-GGUF

Omega Factory

Certified GGUF precision quantization ladder for google/gemma-4-26b-a4b by Omega-Factory. ` ash llama-cli -m gemma-4-26b-a4b.q4km.gguf -p "Explain the difference between dense and sparse mixture-of-experts architectures." -n 256 - Released under Apache-2.0 license.

Model

gemma-4-26b-a4b-gsm8k-synthetic-cot6144-step1-full-state

Yen Ru Chen

Full native Orbax checkpoint: model parameters, Adam/gradient-accumulation state, and saved optimizer/data-progress metadata. This is not a Transformers safetensors export. Training uses camel-ai/gsm8kdistilled, 6,144-token examples, completion-only loss, global batch 32, and seed 42. One optimizer update consumed 32 examples (two source microbatches on 16 devices). See recipe.json and checkpoint-manifest.json for pinned revisions and hashes. Load the native checkpoint root nativecheckpoint at step 1 with the pinned MaxText/Tunix runtime. Restoring onto a different device topology or accumulation schedule requires explicit sharding and data-position validation; that portability has not yet…

On a laptop running Debian 13 Linux, with an Intel Core Ultra 9 185H, its integrated Arc GPU and 32 GB of memory, Janas-LLM reads a prompt at 84 tok/s and writes at 18 tok/s, 26 tok/s with its own assistant in a chat: llama.cpp, on the same laptop, 76 and 12. Measured on 9-11 October 2026: llama-bench -p 1024 -n 16 and janas-bench --prompt 1024 --gen 16, both at their best configuration; the drafts with janas-bench on a chat reply, the mean of three rounds (a draft guesses the next tokens and the model checks them: the same text, faster). On other machines the figures will differ. Details: Janas README. - Debian 13 (trixie) Linux, x86-64, kernel 6.12.107 - its integrated Intel Arc GPU…

An MLX (Apple silicon) build of the abliterated Gemma-4-26B-A4B-it MoE (128 experts, top-8, ~4B active). It uses Google's QAT int4 lattice weights wherever they are exact. The build targets a Mac mini M4 with 24 GB serving 2 concurrent requests at 64k context. It runs through oMLX fully resident at a 21.5 GB Metal limit. At a 20 GB limit, MoE expert offload streams non-resident experts from the NVMe SSD on demand. - Quality vs the exact bf16 model: KL 0.053, top-1 agreement 92.4%. This is the same as the measured noise floor between two exact bf16 implementations (≈0.05 / ≈91%). - Refusal behaviour is identical to the bf16 abliterated model (XSTest safe prompts: 0.8%). The source is…

Google publishes the QAT Q40 model google/gemma-4-26B-A4B-it-qat-q40-gguf as GGUF, but its multi-token-prediction (MTP) assistant only as safetensors There was no GGUF MTP head for the QAT GGUF, so this repository is that assistant converted to GGUF, nothing else: no training or fine-tuning. Use it together with google/gemma-4-26B-A4B-it-qat-q40-gguf for speculative decoding in llama.cpp. The Google model card asks for exactly this pairing: "When using multi-token prediction (speculative decoding) with an assistant model alongside a QAT target model, the assistant model must also be a QAT checkpoint with the same precision." llama.cpp at commit 0c1e570 (Gemma4AssistantForCausalLM → GGUF…

This repository contains an MXFP8-weight checkpoint derived from exported in compressed-tensors format. The checkpoint retains the source multimodal components, but the evaluation reported here covers text tasks only. The exported checkpoint contains 11,635 F8E4M3 weight tensors, 838 BF16 weight tensors, 11,635 U8 block-scale tensors, and 171 FP32 scale/metadata tensors. Routed-expert weights and source-present text self-attention projections are MXFP8; the router, shared MLP, vision tower, and embeddings remain BF16. Measured on 2026-09-30 with lm-eval 0.4.13 and vLLM 0.29.0. The tested settings were TRITONATTN, tensor parallelism 2, pipeline parallelism 1, batch size 64, maxnumseqs=64…

4-bit quantization of google/gemma-4-26B-A4B (commit 24548b6), produced for efficient vLLM serving. Quantized by axteam. - Scheme — W4A16 — FP4 (E2M1) weights group 16 (FP8 E4M3 scale + per-tensor FP32 global scale), BF16 activations, stored as compressed-tensors nvfp4-pack-quantized. - Kept in BF16 — vision tower, vision projection, token embeddings (tied with lmhead), MoE router, and all norms, plus the shared MLP. These are left unquantized to preserve quality (a small fraction of total parameters). - Tokenizer / chattemplate / processorconfig / generationconfig are byte-identical to the original, including Google's July-2026 corrected chattemplate. config.json is the original plus a…

Model · Text generation

gemma-4-31b-GGUF

Omega Factory

Certified GGUF precision quantization ladder for google/gemma-4-31b by Omega-Factory. ` ash llama-cli -m gemma-4-31b.q4km.gguf -p "Write a Python function to compute the eigenvalues of a symmetric matrix." -n 256 - Released under Apache-2.0 license.

Gemma is a family of open models built by Google DeepMind. Gemma 4 models are multimodal, handling text and image input (with audio supported on E2B, E4B, and 12B) and generating text output. This release includes open-weights models in both pre-trained and instruction-tuned variants. Gemma 4 features a context window of up to 256K tokens and maintains multilingual support in over 140 languages. Featuring both Dense and Mixture-of-Experts (MoE) architectures, Gemma 4 is well-suited for tasks like text generation, coding, and reasoning. The models are available in five distinct sizes: E2B, E4B, 12B, 26B A4B, and 31B. Their diverse sizes make them deployable in environments ranging from…

4-bit quantization of google/gemma-4-31B-it (commit 842da37), produced for efficient vLLM serving. Quantized by axteam. - Scheme — W4A16 — FP4 (E2M1) weights group 16 (FP8 E4M3 scale + per-tensor FP32 global scale), BF16 activations, stored as compressed-tensors nvfp4-pack-quantized. - Kept in BF16 — vision tower, vision projection, token embeddings (tied with lmhead), MoE router, and all norms. These are left unquantized to preserve quality (a small fraction of total parameters). - Tokenizer / chattemplate / processorconfig / generationconfig are byte-identical to the original, including Google's July-2026 corrected chattemplate. config.json is the original plus a quantizationconfig block.…

Google publishes the QAT Q40 model google/gemma-4-31B-it-qat-q40-gguf as GGUF, but its multi-token-prediction (MTP) assistant only as safetensors There was no GGUF MTP head for the QAT GGUF, so this repository is that assistant converted to GGUF, nothing else: no training or fine-tuning. Use it together with google/gemma-4-31B-it-qat-q40-gguf for speculative decoding in llama.cpp. The Google model card asks for exactly this pairing: "When using multi-token prediction (speculative decoding) with an assistant model alongside a QAT target model, the assistant model must also be a QAT checkpoint with the same precision." llama.cpp at commit 0c1e570 (Gemma4AssistantForCausalLM → GGUF…

Model

gemma-4-E2B-it-dflash2

Yosef Worku Alemneh

A DFlash2 draft model for speculative decoding with google/gemma-4-E2B-it as the verifier, trained with speculators. The drafter proposes 7 tokens at a time and the verifier checks them in one forward pass, so output is identical to running the verifier alone. This makes it a lossless speedup. Mean acceptance length is 3.09 tokens committed per verification step, up to 5.41 on mathreasoning. that gives a throughput speedup over the verifier alone of 3.57× on mathreasoning and 3.06× on HumanEval. vLLM loads the verifier automatically from the config, so don't pass it separately. Then query the OpenAI-compatible endpoint at http://localhost:8000/v1. Architecture. A Qwen3-style decoder with 5…

On a laptop running Debian 13 Linux, with an Intel Core Ultra 9 185H, its integrated Arc GPU and 32 GB of memory, Janas-LLM reads a prompt at 191 tok/s and writes at 38 tok/s, 52 tok/s with its own assistant in a chat: llama.cpp, on the same laptop, 150 and 19. Measured on 9-11 October 2026: llama-bench -p 1024 -n 16 and janas-bench --prompt 1024 --gen 16, both at their best configuration; the drafts with janas-bench on a chat reply, the mean of three rounds (a draft guesses the next tokens and the model checks them: the same text, faster). On other machines the figures will differ. Details: Janas README. - Debian 13 (trixie) Linux, x86-64, kernel 6.12.107 - its integrated Intel Arc GPU…

This model card provides the Gemma 4 E2B model in a way that is ready for deployment on Android, iOS, Desktop, IoT and Web. Gemma is a family of lightweight, state-of-the-art open models from Google, built from the same research and technology used to create the Gemini models. This particular Gemma 4 model is small so it is ideal for on-device use cases. By running this model on device, users can have private access to Generative AI technology without even requiring an internet connection. These models are provided in the.litertlm format for use with the LiteRT-LM framework. LiteRT-LM is a specialized orchestration layer built directly on top of LiteRT, Google’s high-performance…

Google publishes the QAT Q40 model google/gemma-4-E2B-it-qat-q40-gguf as GGUF, but its multi-token-prediction (MTP) assistant only as safetensors There was no GGUF MTP head for the QAT GGUF, so this repository is that assistant converted to GGUF, nothing else: no training or fine-tuning. Use it together with google/gemma-4-E2B-it-qat-q40-gguf for speculative decoding in llama.cpp. The Google model card asks for exactly this pairing: "When using multi-token prediction (speculative decoding) with an assistant model alongside a QAT target model, the assistant model must also be a QAT checkpoint with the same precision." llama.cpp at commit 0c1e570 (Gemma4AssistantForCausalLM → GGUF…

Model · Any to any

gemma-4-E4B-Chronos-q8-hi-mlx

Gheorghe Chesler

This model is a merge of: - nightmedia/gemma-4-E4B-Fable-Deckard-Expresso-Universe - armand0e/Gemma-4-E4B-it-Fable-Distill - DavidAU/gemma-4-E4B-it-The-DECKARD-Expresso-Universe-HERETIC-UNCENSORED-Thinking - gemma-4-E4B-Fable-Deckard-Expresso-Universe-Polaris-F451-Twain Brainwaves gemma-4-E4B-Fable-Deckard-Expresso-Universe-Polaris-F451 gemma-4-E4B-Fable-Deckard-Expresso-Universe-Polaris gemma-4-E4B-Fable-Deckard-Expresso-Universe E4B-Gemma4-Polar-Rev1 E4B-Gemma4-F451 E4B-Gemma4-Twain-1-2 gemma-4-E4B-it-The-DECKARD-Expresso-Universe-HERETIC-UNCENSORED-Thinking armand0e/Gemma-4-E4B-it-Fable-Distill gemma-4-E4B-it Think mode has been enabled in the jinja template, to allow the model to…

m gemma-4-E4B-it-heretic-QAT-UD-Q4KXL.gguf \ --jinja -ngl 99 -c 4096 \ --port 8001

On a laptop running Debian 13 Linux, with an Intel Core Ultra 9 185H, its integrated Arc GPU and 32 GB of memory, Janas-LLM reads a prompt at 122 tok/s and writes at 20 tok/s, 32 tok/s with its own assistant in a chat: llama.cpp, on the same laptop, 109 and 15. Measured on 9-11 October 2026: llama-bench -p 1024 -n 16 and janas-bench --prompt 1024 --gen 16, both at their best configuration; the drafts with janas-bench on a chat reply, the mean of three rounds (a draft guesses the next tokens and the model checks them: the same text, faster). On other machines the figures will differ. Details: Janas README. - Debian 13 (trixie) Linux, x86-64, kernel 6.12.107 - its integrated Intel Arc GPU…

This is a GGUF LoRA adapter, not a standalone model. It is intended for the anonymization.ner entity-extraction route in Jurilix. Apply it at strength 0.75 for that route and explicitly send strength 0 for other routes. 4b4a2c1d584be7264f87aac328a1bc739ce81b6c, file gemma-4-E4Bq40-it.gguf. The published source recipe pins upstream llama.cpp and a Gemma JSON parser fix. GPU performance or inference quality on other platforms. Start llama-server with --lora ADAPTER.gguf --lora-init-without-apply --cache-ram 0. For entity extraction, include "lora": [{"id": 0, "scale": 0.75}] in each request. For every other route, include "lora": [{"id": 0, "scale": 0}]. Disabling the RAM prompt cache is…

Google publishes the QAT Q40 model google/gemma-4-E4B-it-qat-q40-gguf as GGUF, but its multi-token-prediction (MTP) assistant only as safetensors There was no GGUF MTP head for the QAT GGUF, so this repository is that assistant converted to GGUF, nothing else: no training or fine-tuning. Use it together with google/gemma-4-E4B-it-qat-q40-gguf for speculative decoding in llama.cpp. The Google model card asks for exactly this pairing: "When using multi-token prediction (speculative decoding) with an assistant model alongside a QAT target model, the assistant model must also be a QAT checkpoint with the same precision." llama.cpp at commit 0c1e570 (Gemma4AssistantForCausalLM → GGUF…

INT8 weight-only quantization of google/gemma-4-E4B-it in the compressed-tensors format, ready for vLLM. Text, image and audio inputs are supported as in the original: only the text decoder is quantized. - Per-Layer Embeddings (embedtokensperlayer, perlayerinputgate, perlayerprojection, perlayermodelprojection) - Multimodal projections (embedvision, embedaudio) - Token embeddings and lmhead A large part of the E4B checkpoint is Per-Layer Embeddings, which stay in bf16, so the size reduction is smaller than the "8-bit" label suggests. The full recipe is in recipe.yaml. Perplexity and top-1 next-token agreement against the bf16 original on Wikipedia (wikimedia/wikipedia, dump 20231101)…

Model · Text generation

Gemma-4-Writers-31B

Ateron

"Less Gemma - more human" I had a lot of fresh and creative experience with both ConicCat tunes, but they had some instability issues, that I hopefully fixed there.

weighted/imatrix quants of https://huggingface.co/Ateron/Gemma-4-Writers-31B For a convenient overview and download list, visit our model page for this model. static quants are available at https://huggingface.co/mradermacher/Gemma-4-Writers-31B-GGUF This is a vision model - mmproj files (if any) will be in the static repository. If you are unsure how to use GGUF files, refer to one of TheBloke's READMEs for more details, including on how to concatenate multi-part files. (sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) Here is a handy graph by ikawrakow comparing some lower-quality quant And here are Artefact2's thoughts on the…