Text Generation Models

Stage-1 tokenizers and stage-2 paired-pretraining checkpoints for fMRI-LM, a foundation model that aligns functional MRI with language. Three variants, differing in the objective the stage-1 fMRI tokenizer was trained with. vq-contrastive/ — vector quantization + SigLIP contrastive alignment. vq-domain/ — vector quantization + adversarial domain loss. mae/ — masked autoencoding (mask ratio 0.5) + adversarial domain loss. All three were trained on UK Biobank with robust normalisation and Qwen3-0.6B. Stage-2 files are DeepSpeed checkpoints already merged to a single file. The MAE stage-1 file loads with MaskedAutoencoderViT; the two VQ stage-1 files load with the Tokenizer class. They are not…

FORM-8B is a QLoRA fine-tune of Qwen3-8B (unsloth/qwen3-8b-unsloth-bnb-4bit) specialised for zero-shot FORM code generation. FORM is the symbolic manipulation language used for large perturbative QFT calculations (Grassmann The model is trained exclusively on execution-verified data: every training example was validated by running the generated FORM program through the FORM binary (deterministic-output match) plus an LLM consistency audit. Training config: QLoRA r=16, alpha=32, lr=2e-4, 5 epochs, 4-bit NF4 quantised base. is available from the author upon request. - Generating complete FORM programs from plain-English instructions - Assisting physicists with symbolic-algebra scripting…

RoPE-free long-range attention, trained at 4k with zero-shot length extrapolation. FS scores all completed chunk mean keys and reads four selected chunks per query and head. This 341M research base model was trained on FineWeb-Edu at a 4,096-token context. The short retrieval example asks for the hidden number 5437923. Start with CUDA-enabled PyTorch; Usage gives the tested installation and speed-replication command. FreeSolo learns simple chunk mean keys that support search, norm pruning, and hierarchical summarization. RoPE stays in the local window; long-range attention reads selected original tokens without rotating their queries and keys. 12 tasks at 2k/4k; 13 at 8k/16k, where QA-2 is…

RoPE-free long-range attention, trained at 4k with zero-shot length extrapolation. FS scores all completed chunk mean keys and reads four selected chunks per query and head. This 743M research base model was trained on FineWeb-Edu at a 4,096-token context. The short retrieval example asks for the hidden number 5437923. Start with CUDA-enabled PyTorch; Usage gives the tested installation and speed-replication command. FreeSolo learns simple chunk mean keys that support search, norm pruning, and hierarchical summarization. RoPE stays in the local window; long-range attention reads selected original tokens without rotating their queries and keys. 12 tasks at 2k/4k; 13 at 8k/16k, where QA-2 is…

RoPE-free long-range attention, trained at 4k with zero-shot length extrapolation. FS-B searches full-prefix and norm-filtered forests of chunk mean keys, reranks their candidates, and reads four chunks. Its attention is superlinear and subquadratic. This 341M research base model was trained on FineWeb-Edu at a 4,096-token context. The short retrieval example asks for the hidden number 5437923. Start with CUDA-enabled PyTorch; Usage gives the tested installation and speed-replication command. FreeSolo learns simple chunk mean keys that support search, norm pruning, and hierarchical summarization. RoPE stays in the local window; long-range attention reads selected original tokens without…

RoPE-free long-range attention, trained at 4k with zero-shot length extrapolation. FS-L retains a bounded pool of chunks by key norm and recency, then reads four selected chunks. Its attention scales linearly with sequence length. This 341M research base model was trained on FineWeb-Edu at a 4,096-token context. The short retrieval example asks for the hidden number 5437923. Start with CUDA-enabled PyTorch; Usage gives the tested installation and speed-replication command. FreeSolo learns simple chunk mean keys that support search, norm pruning, and hierarchical summarization. RoPE stays in the local window; long-range attention reads selected original tokens without rotating their queries…

Model · Text generation

FrogNano-4B-2609-MLX-4bit

CapyCTL

A 4-bit MLX affine quantization of microsoft/FrogNano-4B-2609 (revision b90468c1), made so TensorFold can serve it on NVIDIA GPUs. TensorFold's CUDA engine reads quantized weights only; the original checkpoint is BF16. All credit for the model goes to its authors. Read the original model card for intended use, limitations and safety guidance; they apply unchanged. This repository changes only the storage format. The conversion script is in the CapyCTL recipe linked below. --no-drafts is required: there is no drafter for this model, and TensorFold's Qwen dense engine on CUDA needs either a drafter or --no-drafts. --parallel 8 decodes up to eight requests together; without it TensorFold on…

Model · Text generation

Frontis-MA1-30B-SFT

Frontis AI

This is the supervised fine-tuning checkpoint of the Frontis-MA1 family, before OpenMLE reinforcement learning. It is intended for research on machine learning engineering and program transformation. The actual 24,350-row training input is distinct from the 26,259-example public SFT trace collection. Dataset cutoff configuration was 32,768 tokens; this does not establish validated long-context inference performance. No standalone SFT benchmark claim is made by this release. The separately released RL model is FrontisAI/Frontis-MA1-30B. Pin the published commit SHA for reproducibility. This model requires substantial memory; use the dependency versions recorded in the release validation…

Model · Text generation

futolm-cs-base-gguf

Padak padakovic

Separate 29.37M-parameter Czech and Slovak language models for next-word prediction in FUTO Keyboard. All files are F16 FUTO KeyboardLM GGUF exports with the shared SentencePiece tokenizer embedded. For typing, use the chat models: cschat.gguf (Czech) or skchat.gguf (Slovak). They continue the base models on conversational text and predict everyday messages much better. The base models are kept for comparison and further training. The mix is OpenSubtitles v2024 dialogue, discussion-forum pages selected from FineWeb2, and FineWeb2 replay: Czech 50/15/35, Slovak 48/12/40. Peak LR 1.5e-4, 300 warmup steps, cosine decay to 10%. Whole-word next-word prediction, model only (FUTO's dictionary not…

gpt2-large fine-tuned on CommonGen, used as the base language model for the CommonGen experiments in Mitigating Bias in Locally Constrained Decoding via Tractable Proposals (arXiv:2606.01926). This is a plain causal language model: it supplies the base distribution that GCD and P-GCD steer. The tractable proposal it is paired with is the HMM at which shares its 50257-token vocabulary. configs/common-gen.yaml in github.com/MhDang/gelatwo already points at this checkpoint, so the CommonGen runs need no override.

LoRA adapter for google/gemma-3-12b-it trained to answer the Polish history matura (poziom rozszerzony, CKE). Built at the Warsaw Model Trainers Hackathon (team repo: WarsawModelTrainersHackathon, src/posttrain/). SFT (QLoRA r=32 on the mixed matura set, no thinking; repo gemma-3-12b-it-matura-history-sft-qlora-r32), then GRPO with Unsloth + TRL on closed exam items with an exact key (single choice, true/false, matching, ordering). Reward = 0.2format + 0.8exact match, Dr.GRPO loss, beta=0 (no KL). THIS VERSION: GRPO step 25 (interim); the full 300-step run replaces it when it finishes. Closed-question exact match on validation (321 items, no judge): 68.2 overall (SFT alone 66.4, base 45.5).…

LoRA adapter for google/gemma-3-12b-it trained to answer the Polish history matura (poziom rozszerzony, CKE). Built at the Warsaw Model Trainers Hackathon (team repo: WarsawModelTrainersHackathon, src/posttrain/). Supervised fine-tuning (QLoRA) of Gemma 3 12B-it on the same mixed set; Gemma 3 has no thinking mode, so teacher traces contribute their final answers only. LoRA r=32, alpha=64, lr 1e-4 cosine, 1 epoch, batch 16, maxlen 6144, Unsloth + TRL Consolidated training set data/trainingdatasets/sfttrain.jsonl (buildsft.py): real CKE papers 2005–2026 with keys, rephrased papers, synthetic image tasks, synthetic full papers, e-podręczniki textbook questions, synthetic Claude/DeepSeek tasks…

Model · Text generation

Gemma-3-27b-it-abliterated-GGUF

Alan Frederick Benitez

Unlike incomplete GGUF uploads, this repository provides the full 11-quantization spectrum (from high-precision F16 down to lightweight Q2K) of the abliterated Gemma-3-27b-it model. Choose the exact fit for your VRAM/RAM constraints without sacrificing reasoning capabilities. This is the integral and complete collection of quantizations in GGUF format for the gemma-3-27b-it-abliterated model, prepared locally for use with llama.cpp, Ollama, LM Studio, or any application compatible with multimodal models. This model is an uncensored version based on Google's powerful Gemma 3 27B multimodal engine. Being significantly more resistant to neutralization than previous architectures, it was…

On a laptop running Debian 13 Linux, with an Intel Core Ultra 9 185H, its integrated Arc GPU and 32 GB of memory, Janas-LLM reads a prompt at 65 tok/s and writes at 9.9 tok/s, 14 tok/s with its own assistant in a chat: llama.cpp, on the same laptop, 46 and 7.1. Measured on 9-11 October 2026: llama-bench -p 1024 -n 16 and janas-bench --prompt 1024 --gen 16, both at their best configuration; the drafts with janas-bench on a chat reply, the mean of three rounds (a draft guesses the next tokens and the model checks them: the same text, faster). On other machines the figures will differ. Details: Janas README. - Debian 13 (trixie) Linux, x86-64, kernel 6.12.107 - its integrated Intel Arc GPU…

Model · Text generation

gemma-4-12B-it-Pollard

Pollard Weights

Pollard builds of google/gemma-4-12B-it made with Pollard Weights -- a ladder of measured-allocation quants (bits placed by per-layer sensitivity, not a uniform crush). Standard GGUF -- every file here runs in stock llama.cpp / ikllama.cpp, Ollama, LM Studio. Every rung is the same weights, sized to a different RAM budget by the measured allocation. Pick the largest one that fits your machine with room for context: - ~12 GB RAM / VRAM -> Q6K (9.65 GB). - ~9 GB RAM / VRAM -> IQ4XS (6.78 GB). - ~7 GB RAM / VRAM -> IQ2XXS (4.64 GB). f16 reference PPL 23.041768. These are standard GGUF and run with llama.cpp: They also work in anything built on llama.cpp -- LM Studio, koboldcpp, Jan, ramalama…

Google publishes the QAT Q40 model google/gemma-4-12B-it-qat-q40-gguf as GGUF, but its multi-token-prediction (MTP) assistant only as safetensors There was no GGUF MTP head for the QAT GGUF, so this repository is that assistant converted to GGUF, nothing else: no training or fine-tuning. Use it together with google/gemma-4-12B-it-qat-q40-gguf for speculative decoding in llama.cpp. The Google model card asks for exactly this pairing: "When using multi-token prediction (speculative decoding) with an assistant model alongside a QAT target model, the assistant model must also be a QAT checkpoint with the same precision." llama.cpp at commit 0c1e570 (Gemma4UnifiedAssistantForCausalLM → GGUF…

Model · Text generation

gemma-4-12B-it-qat-q4_0-unquantized-MoE

Aifeifei

Inspired by the big.LITTLE architecture in phone SoCs — a few big cores for heavy lifting, many LITTLE cores for efficient background work, and a scheduler that migrates tasks by demand — Myriad-MoE applies the same idea to LLM adapters. Grafted onto a frozen Gemma-4-12B backbone (48 layers, work, 16 clans × 16 LITTLE micro-experts absorb fine-grained load, and 1 always-on shared core holds global context like a system housekeeping core. Learned routers play scheduler; training uses supervised domain routing, 4-bit QLoRA-style adapters, and a hot-swappable patch workflow. Ships with a FastAPI backend and a pnpm (Vite + React) chat UI with a live routing probe. MoE adapters are grafted onto…

基于 google/gemma-4-12B-it 的 LoRA 适配器。使用经预处理和筛选的六个公开数据来源，以 RegMix config-025 配比组成 50,000 条训练数据；不包含 Lusy，也不以 Lusy 为筛选或优化目标。 row% 为样本数占比，target token 为按本次 Gemma tokenizer 和训练监督区间统计的单轮数据集监督 token 总数，tgt% 为其占比；中位数按逐条样本的监督 token 数计算，合计行为全部 50,000 条的中位数。百分比四舍五入。训练共 2 个 epoch；表中不重复计数。不同基座 tokenizer 下的 token 数不能与 v1 直接等同比较。 六个来源分别为 CoSER、XPersona、Aya、Tulu-3-SFT-Mixture、SmolTalk 和 Infinity-Instruct。混合数据包含 50,000 个唯一 ID、50,000 个唯一内容哈希，与留出集的分组重叠数为 0。 训练集 SHA-256：8c626b338270652de2e4d1130b28d8b0fc2c2eca0b90f5dcffa7580a26e0d859。 配比来自 64 个 Gemma-E2B 代理配置及候选确认实验。此版本发布已完成的 config-025 正式训练产物；不将其宣称为 Gemma 12B 上已证明的全局最佳配比。 本仓库包含 LoRA 权重、适配器配置、tokenizer 和 chat template；不包含合并后的基座权重。加载时使用 google/gemma-4-12B-it…

Model · Text generation

gemma-4-26b-a4b-GGUF

Omega Factory

Certified GGUF precision quantization ladder for google/gemma-4-26b-a4b by Omega-Factory. ` ash llama-cli -m gemma-4-26b-a4b.q4km.gguf -p "Explain the difference between dense and sparse mixture-of-experts architectures." -n 256 - Released under Apache-2.0 license.

On a laptop running Debian 13 Linux, with an Intel Core Ultra 9 185H, its integrated Arc GPU and 32 GB of memory, Janas-LLM reads a prompt at 84 tok/s and writes at 18 tok/s, 26 tok/s with its own assistant in a chat: llama.cpp, on the same laptop, 76 and 12. Measured on 9-11 October 2026: llama-bench -p 1024 -n 16 and janas-bench --prompt 1024 --gen 16, both at their best configuration; the drafts with janas-bench on a chat reply, the mean of three rounds (a draft guesses the next tokens and the model checks them: the same text, faster). On other machines the figures will differ. Details: Janas README. - Debian 13 (trixie) Linux, x86-64, kernel 6.12.107 - its integrated Intel Arc GPU…

An MLX (Apple silicon) build of the abliterated Gemma-4-26B-A4B-it MoE (128 experts, top-8, ~4B active). It uses Google's QAT int4 lattice weights wherever they are exact. The build targets a Mac mini M4 with 24 GB serving 2 concurrent requests at 64k context. It runs through oMLX fully resident at a 21.5 GB Metal limit. At a 20 GB limit, MoE expert offload streams non-resident experts from the NVMe SSD on demand. - Quality vs the exact bf16 model: KL 0.053, top-1 agreement 92.4%. This is the same as the measured noise floor between two exact bf16 implementations (≈0.05 / ≈91%). - Refusal behaviour is identical to the bf16 abliterated model (XSTest safe prompts: 0.8%). The source is…

Google publishes the QAT Q40 model google/gemma-4-26B-A4B-it-qat-q40-gguf as GGUF, but its multi-token-prediction (MTP) assistant only as safetensors There was no GGUF MTP head for the QAT GGUF, so this repository is that assistant converted to GGUF, nothing else: no training or fine-tuning. Use it together with google/gemma-4-26B-A4B-it-qat-q40-gguf for speculative decoding in llama.cpp. The Google model card asks for exactly this pairing: "When using multi-token prediction (speculative decoding) with an assistant model alongside a QAT target model, the assistant model must also be a QAT checkpoint with the same precision." llama.cpp at commit 0c1e570 (Gemma4AssistantForCausalLM → GGUF…

This repository contains an MXFP8-weight checkpoint derived from exported in compressed-tensors format. The checkpoint retains the source multimodal components, but the evaluation reported here covers text tasks only. The exported checkpoint contains 11,635 F8E4M3 weight tensors, 838 BF16 weight tensors, 11,635 U8 block-scale tensors, and 171 FP32 scale/metadata tensors. Routed-expert weights and source-present text self-attention projections are MXFP8; the router, shared MLP, vision tower, and embeddings remain BF16. Measured on 2026-09-30 with lm-eval 0.4.13 and vLLM 0.29.0. The tested settings were TRITONATTN, tensor parallelism 2, pipeline parallelism 1, batch size 64, maxnumseqs=64…

Model · Text generation

gemma-4-31b-GGUF

Omega Factory

Certified GGUF precision quantization ladder for google/gemma-4-31b by Omega-Factory. ` ash llama-cli -m gemma-4-31b.q4km.gguf -p "Write a Python function to compute the eigenvalues of a symmetric matrix." -n 256 - Released under Apache-2.0 license.

Google publishes the QAT Q40 model google/gemma-4-31B-it-qat-q40-gguf as GGUF, but its multi-token-prediction (MTP) assistant only as safetensors There was no GGUF MTP head for the QAT GGUF, so this repository is that assistant converted to GGUF, nothing else: no training or fine-tuning. Use it together with google/gemma-4-31B-it-qat-q40-gguf for speculative decoding in llama.cpp. The Google model card asks for exactly this pairing: "When using multi-token prediction (speculative decoding) with an assistant model alongside a QAT target model, the assistant model must also be a QAT checkpoint with the same precision." llama.cpp at commit 0c1e570 (Gemma4AssistantForCausalLM → GGUF…

Model · Text generation

Gemma-4-E2B-Coder-Uncensored

Minas Fostieris

A code-focused finetune of llmfan46/gemma-4-E2B-it-ultra-uncensored-heretic (Gemma 4 E2B, ~4.6B effective) built specifically for front-end development (React, TypeScript, HTML/CSS) and general Python scripting. Built on an uncensored base, so it just answers the question—no lectures attached. It’s a small model, trained on a single consumer GPU. It writes React components, fixes CSS grid layouts, and won't judge your useEffect dependencies. Measured with lm-evaluation-harness (greedy decoding, temp 0.0, seed 0). Same harness, same conditions for both models. Note: The base model scores higher on pure, edge-case Python algorithmic logic. Hiding it would be weird. This model intentionally…

On a laptop running Debian 13 Linux, with an Intel Core Ultra 9 185H, its integrated Arc GPU and 32 GB of memory, Janas-LLM reads a prompt at 191 tok/s and writes at 38 tok/s, 52 tok/s with its own assistant in a chat: llama.cpp, on the same laptop, 150 and 19. Measured on 9-11 October 2026: llama-bench -p 1024 -n 16 and janas-bench --prompt 1024 --gen 16, both at their best configuration; the drafts with janas-bench on a chat reply, the mean of three rounds (a draft guesses the next tokens and the model checks them: the same text, faster). On other machines the figures will differ. Details: Janas README. - Debian 13 (trixie) Linux, x86-64, kernel 6.12.107 - its integrated Intel Arc GPU…

Google publishes the QAT Q40 model google/gemma-4-E2B-it-qat-q40-gguf as GGUF, but its multi-token-prediction (MTP) assistant only as safetensors There was no GGUF MTP head for the QAT GGUF, so this repository is that assistant converted to GGUF, nothing else: no training or fine-tuning. Use it together with google/gemma-4-E2B-it-qat-q40-gguf for speculative decoding in llama.cpp. The Google model card asks for exactly this pairing: "When using multi-token prediction (speculative decoding) with an assistant model alongside a QAT target model, the assistant model must also be a QAT checkpoint with the same precision." llama.cpp at commit 0c1e570 (Gemma4AssistantForCausalLM → GGUF…

Web-optimized gemma-4-E2B-it LiteRT-LM bundles for browser (WebGPU) execution, built with the official conversion pipeline (exportvit, convertermain, litertlmbuildercli). - Multi-signature visionencoder (vision70, vision140, vision280, vision560, vision1120) exported via //thirdparty/odml/generative/modules/gemini/export:exportvit with QAT Int8 weight recovery and query-chunked attention (maxquerychunksize = 630, $k=16$ at vision1120). - Artisan decoder visiontokensnum: 1121 (tokensnum = 1120).

On a laptop running Debian 13 Linux, with an Intel Core Ultra 9 185H, its integrated Arc GPU and 32 GB of memory, Janas-LLM reads a prompt at 122 tok/s and writes at 20 tok/s, 32 tok/s with its own assistant in a chat: llama.cpp, on the same laptop, 109 and 15. Measured on 9-11 October 2026: llama-bench -p 1024 -n 16 and janas-bench --prompt 1024 --gen 16, both at their best configuration; the drafts with janas-bench on a chat reply, the mean of three rounds (a draft guesses the next tokens and the model checks them: the same text, faster). On other machines the figures will differ. Details: Janas README. - Debian 13 (trixie) Linux, x86-64, kernel 6.12.107 - its integrated Intel Arc GPU…

Google publishes the QAT Q40 model google/gemma-4-E4B-it-qat-q40-gguf as GGUF, but its multi-token-prediction (MTP) assistant only as safetensors There was no GGUF MTP head for the QAT GGUF, so this repository is that assistant converted to GGUF, nothing else: no training or fine-tuning. Use it together with google/gemma-4-E4B-it-qat-q40-gguf for speculative decoding in llama.cpp. The Google model card asks for exactly this pairing: "When using multi-token prediction (speculative decoding) with an assistant model alongside a QAT target model, the assistant model must also be a QAT checkpoint with the same precision." llama.cpp at commit 0c1e570 (Gemma4AssistantForCausalLM → GGUF…

Web-optimized gemma-4-E4B-it LiteRT-LM bundles for browser (WebGPU) execution, built with the official conversion pipeline (exportvit, convertermain, litertlmbuildercli). - Multi-signature visionencoder (vision70, vision140, vision280, vision560, vision1120) exported via //thirdparty/odml/generative/modules/gemini/export:exportvit with QAT Int8 weight recovery and query-chunked attention (maxquerychunksize = 630, $k=16$ at vision1120). - Artisan decoder visiontokensnum: 1121 (tokensnum = 1120).

Model · Text generation

Gemma-4-Writers-31B

Ateron

"Less Gemma - more human" I had a lot of fresh and creative experience with both ConicCat tunes, but they had some instability issues, that I hopefully fixed there.

Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. Use the code below to get started with the model. Stage 1 Stage 2 Run04 used the original text-based SFT training configuration, not completion-only SFT. Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019). This model was fine-tuned using Qwen3-4B-Base and the Tulu 3 and GSM8K datasets. Please cite the original works when using or referencing this model. Qwen3 title={Qwen3 Technical Report}, author={Yang, An and others}, year={2025} Tulu 3 title={Tulu 3: Pushing Frontiers in Open Language Model Post-Training}…

Qwen/Qwen3-4B-Base Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. Use the code below to get started with the model. This adapter was originally trained in an earlier run and reused unchanged for Run04. Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019). This model was fine-tuned using Qwen3-4B-Base and the Tulu 3 and GSM8K datasets. Please cite the original works when using or referencing this model. Qwen3 title={Qwen3 Technical Report}, author={Yang, An and others}, year={2025} Tulu 3 title={Tulu 3: Pushing Frontiers in Open Language Model Post-Training}…

This is the model card of a transformers model that has been pushed on the Hub. Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. Use the code below to get started with the model. Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).

GLM-5.3 uses the same base model as GLM-5.2 — every gain comes from post-training. Compared with GLM-5.2, it is much better at complex coding and long-horizon tasks: GLM-5.3 supports deployment with the following frameworks. Feel free to try them out: - SGLang — see cookbook - vLLM — see recipes - TokenSpeed — see here - Transformers — see transformers docs - KTransformers — see tutorial - Unsloth — see guide - For deployment on the Ascend NPU platform, inference frameworks such as vLLM-Ascend, xLLM and SGLang are supported — see here. - GLM-5.3 supports controlling the thinking budget through the reasoningeffort parameter, which accepts three levels: low, high, and max. It defaults to max…

GLM-5.3 uses the same base model as GLM-5.2 — every gain comes from post-training. Compared with GLM-5.2, it is much better at complex coding and long-horizon tasks: GLM-5.3 supports deployment with the following frameworks. Feel free to try them out: - SGLang — see cookbook - vLLM — see recipes - TokenSpeed — see here - Transformers — see transformers docs - KTransformers — see tutorial - Unsloth — see guide - For deployment on the Ascend NPU platform, inference frameworks such as vLLM-Ascend, xLLM and SGLang are supported — see here. - GLM-5.3 supports controlling the thinking budget through the reasoningeffort parameter, which accepts three levels: low, high, and max. It defaults to max…

Model · Text generation

GLM-5.3-DSpark

Alaya NeW

This repository releases a DSpark draft model for GLM-5.3. It is a speculator, not a standalone LLM: pair it with a GLM-5.3 target (BF16 or FP8) in SGLang or vLLM. Requires a recent SGLang or vLLM build with native DSpark support. This draft is for GLM-5.3 only. --speculative-num-draft-tokens is block size + 1. The draft is bf16 dense. Keep the draft KV cache dtype on auto so it does not inherit the target's FP8 MLA KV cache. - Context-length numbers are the mean of 3 runs (50 prompts, SPEED-Bench low-entropy throughput) - Qualitative numbers are one run on SPEED-Bench categories (80 prompts each) Acceptance length is the per-request mean of completion tokens divided by verification steps.…

A weight-space combination of two independent abliterations of the same base model, produced by different teams with different methods. The two edits turn out to lie in nearly orthogonal directions, so applying both removes the refusals that each one leaves behind — the combination is more thoroughly decensored than either parent, with no fine-tuning and no gradient steps. This card documents what was done and why it works at the weight level. It does not report harmful- capability evaluations, and standard capability benchmarks (MMLU, etc.) have not been run — see Why Q80 is the base to requantize from. Both parents are FP8 (e4m3) native. Q80 (int8 + one fp16 scale per 32 weights)…

Model · Text generation

GLM-5.3-Flash-ewaste-edition-GGUF

SixVolts

This is a quantization of GLM-5.3-flash that is optimized to work better on older hardware. This quant was built using i-matrix GGUF quantizations with the same principle as the GLM-5.2 and GLM-5.3 e-waste editions: K-quants for the routed experts, no codebook i-quants, because on pre-AVX-512 CPUs and previous-generation datacenter GPUs (MI100 / gfx908) expert dequantization, not bandwidth, is the decode bottleneck. Format: mainline llama.cpp glm5-next (merged 2026-09-30, ggml-org/llama.cpp#27773). These files do not load in builds that only know the earlier glm5next arch from the unmerged PR #27754 (unsloth's September files are that format). Any llama.cpp from 2026-10 on loads them. The…

Model · Text generation

GLM-5.3-Flash-NVFP4-CSF-QAD

Local Inference Lab

A lossless NVFP4-CSF container of local-inference-lab/GLM-5.3-Flash-NVFP4 at revision 175ae8ce3b5af842b0d0140dbeb43e9cfc557c49, the GLM-5.3-Flash QAD checkpoint (NVFP4 routed experts, BF16 dense layers). CSF stores the routed-expert block-scale planes in a compressed form. Nothing else changes: FP4 weight nibbles, every other tensor, the tensor names, dtypes, shapes and the source metadata are all byte-identical. Decoding is integer arithmetic on bytes, with no requantization or fitting. The original safetensors shards can be restored bit-exactly. - Built with trellis-quant trellisquant.losslessscalecheckpoint (commit 60feca330087, family glm53nvfp4). - verification.json: every original…

Model · Text generation

GLM-5.3-Flash-Uncensored-W4A16

Mike Chambers

This is a quantized build of GLM-5.3-Flash-Uncensored-FP8. Only the routed MoE experts are re-quantized; every other tensor is kept exactly as in the FP8 source (FP8 tensors stay FP8, BF16 tensors stay BF16). The model card below this section is the original card of the FP8 checkpoint. Total ~179 GiB (the FP8 source is 306 GiB). Format. compressed-tensors, mixed-precision with two config groups in config.json: - group0 (pack-quantized, targets re:.mlp\.experts\..): the layout llm-compressor writes, per expert projection weightpacked (int32, eight 4-bit values each), weightscale (BF16, one per 32 consecutive input-channel weights) and weightshape. - group1 (float-quantized, targets the…

Model · Text generation

GLM-5.3-NVFP4-CSF

Local Inference Lab

A lossless NVFP4-CSF container of local-inference-lab/GLM-5.3-NVFP4 at revision b472e4ee53f6a9862da5486c56c6ca21be3dab70, the GLM-5.3 NVFP4 checkpoint (744B MoE; NVFP4 routed experts, BF16 dense layers). CSF stores the routed-expert block-scale planes in a compressed form. Nothing else changes: FP4 weight nibbles, every other tensor, the tensor names, dtypes, shapes and the source metadata are all byte-identical. Decoding is integer arithmetic on bytes, with no requantization or fitting. The original safetensors shards can be restored bit-exactly. - Built with trellis-quant trellisquant.losslessscalecheckpoint (commit 6ecc90456275, family glm53744bnvfp4). - verification.json: every original…

Model · Text generation

glm47-flash-blockwise-fp8

RadixArk

Blockwise FP8 quantization of zai-org/GLM-4.7-Flash: e4m3 weights with one scale per 128×128 block and dynamic activation scaling, about half the size of the BF16 original (30 GiB vs 58 GiB). See the original model card for model details. gsm8k (5-shot, full test set) on SGLang with MTP speculative decoding: 0.809 for FP8 vs 0.819 for BF16; average accept length 2.41 vs 2.43. Attention tensor parallelism must be 1 or 2: at 4, each rank's kvbproj shard (2240 rows) is not a multiple of the 128-row block. For more GPUs, use DP attention with expert parallelism, for example --tp-size 4 --dp-size 4 --enable-dp-attention --ep-size 4 --moe-a2a-backend deepep --cuda-graph-max-bs-decode 128. MIT…

Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. Use the code below to get started with the model. Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019). - PEFT 0.19.1

Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. Use the code below to get started with the model. Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019). - PEFT 0.19.1

Model · Text generation

gollem-v5-ckpts

Fabryka AI

Research checkpoints of sub-100M-parameter English language models, GPT-style decoders (nanoGPT lineage) trained for the The repository holds a controlled scaling study (tokens, width, optimizer) and a set of data-attribution experiments on the 64M model (paired continued-training runs that differ only in data). Every run folder holds plain PyTorch checkpoints (ckpt.pt or ckpt k.pt, where N is the training step in thousands). Folders with a README.md (all runs since 2026-09-25) contain the full specification: data pool and its hash, trainer version, seed, steps and tokens. "Result" is the checkpoint that the reported numbers come from; it is the planned last step (for the final runs, the…

Model · Text generation

gpt-oss-120b-blackhole-fast

Jashan Pratap Singh

gpt-oss-120b served with vLLM on four Blackhole chips (a QuietBox 2, P150x4, tensor parallel 4). A lone request decodes through a fused batch-1 decode: DRAM-streaming expert matmuls that read the 128 BFP4 experts in place (one ND-sharded copy shared with prefill and the batched decode), on-core top-4 routing over the 128 router logits, and all-reduces fused into the producing kernels. Batches of 2-32, prefill, prefix caching and requests that need penalties, log-probs, top-k > 32, logit bias or grammars run the batched paths. Status: round-2 release candidate (stage 14): measured on p150x4 against tt-hous/gpt-oss-120b-p150x4 on the same day: c=1 decode 3.8x, GSM8K-250 and IFEval-100 above…

Model · Text generation

gpt-oss-20b-blackhole-fast

Jashan Pratap Singh

Runs on p150 (one Blackhole P150 chip), p150x2 (the two P150 ASICs of one P300C board) and p150x4 (the four P150 ASICs of two P300C boards, a QuietBox 2). Same model, same serving stack and the same six profiles as tt-hous/gpt-oss-20b-p150, with faster decode: a lone request on P150x4, P150x2, P150 decodes through fused, DRAM-streaming decode ops (from tenstorrent/tt-metal#60161, adapted to this runtime and extended to one and two chips). On P150x4, 2 to 4 decoding requests share one pass of those ops, and 5 to 32 run the base package's batched decode with the routed experts streamed from the fused path's weight copies. Prefill takes the base package's code path with faster expert dispatch…