Text Generation Models

An OpenRLHF GRPO reinforcement-learning checkpoint for Qwen3-4B. - Saved at global step 56 of RL run seededrlbaseramp25stoppengen4kep2ncp5q3cgroot16. - This is the best checkpoint by pass@8 so far in this run. Trained and validated on the cobalt-train ≤2/64 frontier (canonical cleaneval prompts): 1833 train / 112 held-out val problems the base model solved on at most 2 of 64 samples under the iidcanonical@64 hardness scan. Val evals sample at temperature 1.0 (matching the cleaneval frontier eval). Reward signal: binary code-correctness (1.0 if the generated program passes the problem's tests, otherwise 0.0). This checkpoint is the main revision (git branch) of the repo, with the model at…

Model · Text generation

Instella-MoE-16B-A3B-Think-GGUF

Tullus Augustus

A clean F16 GGUF conversion of AMD's Instella-MoE-16B-A3B-Think, using the model's native instella GGUF architecture. This repository is intentionally centered on one artifact: The F16 file is my Gold Standard Reference for subsequent GGUF quantization, runtime, and ROCmFPX experiments. Experimental IQ, Q, and ROCmFP4 derivatives may be produced from this file or from the original AMD checkpoint, but they are considered downstream work and are not the reference release represented by this repository. Strix support will come after I build the ROCmFP4 GGUFs and get all the runtimes sorted out. This GGUF is converted from the official AMD checkpoint: AMD describes Instella-MoE as a fully open…

Files: merged bf16 safetensors + tokenizer + chat template (final LoRA adapter merged into grpo-v2). Load with plain AutoModelForCausalLM; saved with transformers 5.x (dtype key, chattemplate.jinja). Companion repo with the adapter and checkpoints: tbooy/Qwen2.5-3B-Instruct-Sheldon-RLVR-math-v1-LoRA. Stage-3 (RLVR) model of a Harvard CS 2881R project: reinforcement learning with a verifiable math reward on top of tbooy/Qwen2.5-3B-Instruct-Sheldon-RLAIF-grpo-v2 (SFT persona model -> RLAIF GRPO -> this), the persona being Dr. Sheldon Cooper. Why this stage exists: the persona SFT stage cost most of Qwen2.5-3B-Instruct's competition-math ability (MATH-500 greedy 68.0 -> 35.4), and the RLAIF…

An OpenRLHF GRPO reinforcement-learning checkpoint for Qwen3-4B. - Saved at global step 66 of RL run seededrlbaseramp25stoppengen4kep2ncp5baseq3c. - This is the best checkpoint by pass@8 so far in this run. Trained and validated on the cobalt-train ≤2/64 frontier (canonical cleaneval prompts): 1833 train / 112 held-out val problems the base model solved on at most 2 of 64 samples under the iidcanonical@64 hardness scan. Val evals sample at temperature 1.0 (matching the cleaneval frontier eval). Reward signal: binary code-correctness (1.0 if the generated program passes the problem's tests, otherwise 0.0). This checkpoint is the main revision (git branch) of the repo, with the model at the…

Model · Text generation

Qwen-Unitopia-Style

Jann Thomsen

LoRA fine-tunes of Qwen models on the UNItopia LPC mudlib, a German LPMud library (sources and documentation from ftp://unitopia.de). The goal is a model that writes LPC in the UNItopia house style: file headers, inherit lines, the room, item and monster APIs, and the German documentation format. The current release is built on Qwen3.8-27B. Everything is in qwen3.8-27b/: All GGUFs are text-only conversions with llama.cpp (converthftogguf.py); the vision tower of the base checkpoint is not included. The Q80 files were converted directly from bf16 safetensors, the Q4KM with llama-quantize at default settings. qwen3.5-35b-a3b/ holds the same fine-tune applied to Qwen3.5-35B-A3B, a…

Model · Text generation

qwen-metaphor-unke-camada-6-literal-metaforica

Luisa Muniz Stellet

This is the model card of a transformers model that has been pushed on the Hub. Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. Use the code below to get started with the model. Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).

A GGUF quantization of — Microsoft's compact repository-level coding agent, Qwen3.5-4B post-trained with reinforcement learning on ~1,500 synthetic software-engineering tasks (SWE-bench Verified 61.5% Avg@3 in Microsoft's Leaf harness) — with two deliberate improvements over a stock Q4KM: 1. imatrix computed on a broad real corpus (bartowski's calibrationdatav3 — code + prose + facts + multilingual). 2. JANG-mixed precision: full-attention layers, embeddings and the gated-DeltaNet linear-attention layers (most of the network) kept at Q6K, feed-forward at the Q4KM base preset. (Not Q80 on attention — see "Honest notes": Q80 is ~3x slower than Q6K on Vulkan hardware without a hardware int8…

This is the model card of a transformers model that has been pushed on the Hub. Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. Use the code below to get started with the model. Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).

Model · Text generation

Vinci-Bozza-1.0-GGUF

SimpleDirect

GGUF quantizations of simpledirect/Vinci-Bozza-1.0 for local inference with Ollama, LM Studio, and llama.cpp. Text-only. The source model is multimodal (image-text-to-text); these GGUF conversions include the language model only — no mmproj / vision projector. For image input, use the safetensors build: simpledirect/Vinci-Bozza-1.0. No GGUF file in this repository has been evaluated. The benchmark figures on the full-precision card were measured on the unquantized bf16 weights, not on any file here. Quantization shifts scores, and with no per-tier measurement this card cannot say in which direction or by how much for any tier. This card therefore publishes no per-tier score, no quality…

Model · Text generation

Qwen3.8-27B-coreml-agent

岡優志 / Okayuji

Run Qwen3.8 27B locally on an Apple silicon Mac with Core ML. With the companion Swift app it searches the web, reads pages, writes notes and answers with sources. Model inference stays on the Mac; search and page retrieval require network access. Requires: / Swift 6.2 · 96 GB+ RAM · ~30 GB disk. Measurements used a 128 GB Mac; 96 GB is untested. See Requirements. This repository contains weights and configuration: int8 weights, a 16,384-token context and the model's own multi-token-prediction head. The companion runtime supplies the stateful chain and speculation loop; the demo app supplies search, page reading, notes and tool calls. To run the agent: open CoreMLSamples.xcodeproj at the…

Model · Text generation

A11OY-MINI

SZL Holdings

749/14/163 · Λ = Conjecture 1 (advisory) · a-11-oy.com GGUFs are on this repo. GGUF AVAILABLE. Bytes MEASURED (size + sha256 below, ATELIER-verified). Evals none-this-run. Not MEASURED as an eval. publicationeligible: false. Quality ROADMAP in prose only. Lab load forbidden. GPU UNAVAILABLE. SKU of live SZLHOLDINGS/chaski merged shard 1c55df8. Not a new train. Not SZLHOLDINGS/chaski-5050. Not a Hub basemodelrelation: quantized child until a Chaski eval gate passes. A11oy is a Space and a substrate. The mini is the voice you can actually load. Not the organ. Not the mesh. A governed command voice that fits in llama.cpp. Nobody else ships this combination. That is the point of a one-of-one.…

Model · Text generation

Kartik-Kundli-AI-3B-v2.0

Kartik Sharma

India's Own Vedic Astrology AI - Ab aapka Kundli Expert aapke laptop pe! Ye model Vedic Astrology, Kundli, Rashi, Nakshatra, Dasha pe specially fine-tune kiya gaya hai. 3B size ka hai isliye aapke laptop / mobile pe bhi tez chalega. - "Meri Mesh rashi ka aaj ka bhavishya kya hai?" - "Kundli me Mangal Dosh kya hota hai?" - "Mere janm nakshatra ka matlab kya hai?" - "Shani ki sade sati kab khatam hogi?" Ye model Medical, Legal ya Financial advice ke liye nahi hai. Ye sirf educational astrology purpose ke liye hai. from transformers import AutoModelForCausalLM, AutoTokenizer modelid = "UX4567/Kartik-Kundli-AI-3B-v2.0" tokenizer = AutoTokenizer.frompretrained(modelid) model =…

Quantized language models for Tiny Agent — a Windows application that gives you Word, Excel and PowerPoint with an AI assistant that runs on your own CPU, with no GPU, no cloud API and no account. double-click, and it installs like any other Windows program. This repository supplies the model that installer uses — or that you can point any local llama-server at. These files are quantizations of existing Apache-2.0 models, not new models. and limitations are inherited unchanged from the base model. Sizes are the real blob sizes, not estimates. All three are Apache-2.0 and safe to redistribute inside a product. Measured by actually running it — these are observed behaviours, not claims…

Model · Text generation

stark-mcp-scratch

Victor cherif

Modelo de lenguaje entrenado desde cero (pesos aleatorios) por Victor Cherif. Arquitectura tipo Llama de ~110M de parametros, pensado como asistente en espanol orientado a ciencia, programacion, robotica y fisica, con soporte de tool-calling / MCP entrenado contra servidores MCP reales. RMSNorm + SwiGLU - Al ser un modelo pequeno (~110M), el lenguaje puede ser incoherente en respuestas largas, sobre todo pasados los primeros parrafos (tiende a repetir bloques o rellenar con generalidades). - La aritmetica mental sigue sin ser confiable: depende de que el modelo dispare la tool calculate en vez de calcular "de memoria" -- verificado que esto mejoro pero no es perfecto. - El razonamiento ( )…

基于 Qwen3.8-27B（qwen35 架构，稠密 27B，48 线性注意力 + 16 全注意力，带视觉模块）的佛学辞典微调模型， 使用 佛光大辞典 与 中华佛教大百科全书 的语料做 35,841 条中文佛教 SFT 微调， 各目录的 quantizationconfig 不同（quantmethod 各异），加载命令需对应： 未量化的层主要是三类（各版本 config.json 的 ignore 字段可查）： 1. 线性注意力（Gated DeltaNet / linearattn）投影：nvfp4a16 明确忽略了 240 条、autoawq 忽略 inproja / inprojb —— 量化工具链对线性注意力算子支持不完整，硬量化容易掉点； 2. 视觉塔 visual（约 110 个模块）：对精度敏感，且纯文本部署可用 --language-model-only 直接跳过； 3. lmhead 与各 norm / bias / embedding：量化收益小、风险大。 - ① SM75 没有 FP8 硬件能力（FP8 张量核从 Ada / SM89 才有）。所以在 2080Ti 上 FP8 权重只能走社区 fork 的模拟路径（Marlin fallback + 实时反量化），实际行为是"权重 FP8 + 计算 BF16"（weight-only），激活不参与量化 —— 这也是为什么单独做了一份 block 128×128 的 FP8（面向通用 vLLM 的 per-channel 那份在 SM75 上加载不了）。 - ② 覆盖率低 ≠ 残次品：BF16…

Model · Text generation

Olmo-3-7B-Think-heretic-GGUF

Richard Young

GGUF quantizations of richardyoung/Olmo-3-7B-Think-heretic, an abliterated build of allenai/Olmo-3-7B-Think (Ai2's fully open 7B reasoning model (open weights, data and training code) that thinks before answering) made with Heretic. The very low KL divergence (0.026, far below the 0.5 “damage” threshold) means the model retains essentially all of its original capabilities and coherence. Abliteration removes the "refusal direction" from a model's residual stream — a surgical modification that disables safety refusals without retraining. See Refusal in Language Models Is Mediated by a Single Direction. Heretic automates it: it searches ablation parameters with an optimizer (200 trials here)…

The small on-device model inside MindMopAI, a free brain-dump app for Windows and Mac. It does two jobs, and runs entirely on your computer through llama.cpp. Nothing you say is sent anywhere. 1. Sorting. You say whatever is on your mind, and it splits it into separate items and sorts each one: task, bill, exam, idea, note, shopping, read, social or appointment. 2. Mai, the in-app helper. It answers questions about MindMopAI from a help sheet the app provides, reads your own items and settings to answer questions about them, offers actions (for example a theme picker, or "mark this done" for you to confirm), and reviews your task list with practical suggestions. 2,000 helper examples, all…

Model · Text generation

DeepSeek-V4.1-Flash-UNCENSORED-GGUF

Christoffer Adolphsson

Uncensored 552B MoE with 1M-token context, vision, and full tool/reasoning support — converted to GGUF for llama.cpp. Built from dealignai/DeepSeek-V4.1-Flash-UNCENSORED-FP8 (permanent weight-level abliteration of deepseek-ai/DeepSeek-V4.1-Flash), converted to GGUF and requantized. No guardrails: the model follows on prompts the base model refuses (verified, see Benchmarks). All quant files are sharded. The Q2 set is sharded at ≤50 GB (friendlier resumable downloads); the older Q3/Q4/Q8 sets at ≤180 GB. Download every file with the same prefix — llama-server -m accepts any one shard filename and loads all of them automatically. Heads-up on Q80 routed experts: the source checkpoint stores…

This is the model card of a transformers model that has been pushed on the Hub. Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. Use the code below to get started with the model. Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).

Model · Text generation

Qwen3-4B-Q4_K_M-GGUF

Shayon Sarker

Downloads: Hugging Face — ShayonSarker/Qwen3-4B-Q4KM-GGUF · Source & build recipes: GitHub — model-qwen3-4b-q4-k-m-gguf · Base model: Qwen/Qwen3-4B The GGUF binaries are hosted on Hugging Face; this repository holds the reproducible conversion pipeline, calibration data, and checksums. Base: Qwen/Qwen3-4B (Apache-2.0) · Quant: llama.cpp 0.4.1-dev (60081bb) · imatrix on 3000-line wikitext-2 (seed 42, 618 chunks, 4 threads) The F16 row in the eval table below is a measurement reference, not a downloadable file. Gate was <5% — passed. Task evals not run: perplexity only, stated honestly. Files: smol-Q4KM.gguf, smol-Q80.gguf, imatrix.dat, calib.txt, REPRO.json, SHA256.txt. Full repro: see…

This is the vector-configuration finalist from Muta's ADTC fine-tuning campaign. It starts from Qwen/Qwen2.5-1.5B-Instruct, applies BF16 LoRA with rank 16 for 500 steps on a licence-clean multiple-choice math and science mixture, merges the adapter, and exports the result as Q4KM GGUF. Muta-Tutor-Qwen2.5-1.5B-Q4KM.gguf SHA-256: a750d00d458c6ab38925364ea1413db00648449180941e47025736d09922e1eb The candidate and untuned control were evaluated with the same GGUF conversion, quantization, prompt format, and GCP CPU-proxy harness. The repository includes the training manifest, licence-clean dataset manifest, artifact hashes, and the full fine-tuning summary. Performance was measured on a cloud…

Model · Text generation

SRA-RiskGate-4B-Q4_K_M-GGUF

Sriram Ramakrishnan

This repository contains the quantized GGUF weights (Q4KM, ~2.5 GB) for SRA-RiskGate-4B, an autonomous risk scoring, compliance verification, and dispute resolution agent for stablecoin payments (EIP-3009, x402, smart escrow). Deploying with autonomous on-chain agents or payment gateways? Use the companion Python SDK for zero-latency deterministic pre-filtering (sanctions, address validation, depeg protection) and drop-in Coinbase AgentKit firewall support: Note: Always run with greedy decoding (--temp 0.0) to guarantee deterministic JSON output conforming to the benchmark schema. Distributed under the Apache 2.0 License.

Model · Text generation

szl-receiptagent-qwen35-0.8b-v3

SZL Holdings

This is a separate research bf16 LoRA adapter (r=16, alpha=32) on Qwen/Qwen3.5-0.8B, trained with Unsloth on a laptop Blackwell GPU, with a derived merged artifact also present in the repository. "Placeholder" describes release status, not absence of weights. It does not overwrite ReceiptAgent v2 Sizes were read from the Hub listing; no weight file was downloaded or hashed by this pass, and the revision SHA was not captured. The Hub-hosted banner referenced by the szl-forge card (holo-banner.svg) is not on this tree, so it is not embedded here. The training receipt records 180 training rows and train loss 0.0537. Training loss is not evaluation. The receipt describes the local laptop…

Model · Text generation

startup32

Harsh Srivastava

Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. Use the code below to get started with the model. Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019). - PEFT 0.18.1

Model · Text generation

gemma-4-E4B-meeting-agent-zh-LiteRT

Luigi

Gemma-4-E4B fine-tuned to read a live zh-TW meeting transcript on a phone, window by window, and write short cited notes: decisions, actions, proposals, open issues and key figures. The same model turns the notes into a prose summary and a title. It is the E4B sibling of mobile-v1: same prompts, protocol and note format, fewer factual errors, about 3× slower. Latest: mobile-v2/ (2026-10-17), see the mobile-v2 section. The files at the root are mobile-v1. mobile-v2/ holds the E4B mobile-v2 reader. It continues mobile-v1 with two training stages: 1. SFT on reading rows plus dense-list conversion rows, including a third domain: Taiwanese financial TV programs; 2. one round of on-policy…

Shreyansh Singh · Founder & Chief Architect · ExperimentLab.in Test Vidhi AI 7B with Real-Time Statutory RAG in a modern ChatGPT-style interface: Evaluated against 1,000 unseen statutory questions across Indian criminal and corporate jurisprudence: 1. "Explain Section 103 of Bharatiya Nyaya Sanhita (BNS) 2023 regarding punishment for murder and contrast it with IPC Section 302." 2. "What is the mandatory statutory threshold for undertrial bail under Section 479 of BNSS 2023 for a first-time offender?" 3. "What are the mandatory conditions for claiming Input Tax Credit (ITC) under Section 16(2) of the CGST Act?" 4. "Explain the statutory 30-day notice and 15-day cure window under Section 138…

Model · Text generation

hpc-assistant-qwen3-4b

Vaibhav Kailash Mangroliya

A fine-tuned Qwen/Qwen3-4B-Instruct-2507 that answers HPC questions: Slurm job scripts, GPUs, modules, storage, and access to EuroHPC systems and the European AI Factories. It works best with retrieved documentation excerpts in the prompt. This release is a full merged model (bf16 safetensors), built in four stages: 1. Logit distillation (round 12a). The student learns the full next-token distribution of Qwen3-Next-80B-A3B-Instruct (top-20 probabilities at every answer token) on 46,717 verified question–answer pairs. LoRA rank 32. Merged. 2. On-policy distillation from Qwen3-Next (round 12e). The stage-1 model answers the training questions itself; 14,514 of its own answers are scored token…

An OpenRLHF GRPO reinforcement-learning checkpoint for Qwen3-4B. - Saved at global step 72 of RL run seededrlbaseramp25stoppengen4kep2ncp5n3ncvs16s2. - This is the best checkpoint by pass@8 so far in this run. Trained and validated on the cobalt-train ≤2/64 frontier (canonical cleaneval prompts): 1833 train / 112 held-out val problems the base model solved on at most 2 of 64 samples under the iidcanonical@64 hardness scan. Val evals sample at temperature 1.0 (matching the cleaneval frontier eval). Reward signal: binary code-correctness (1.0 if the generated program passes the problem's tests, otherwise 0.0). This checkpoint is the main revision (git branch) of the repo, with the model at…

Fixed memory for long-running generation. This EdgeRock-4B deployment package retains recurrent state, local attention, and Rock Memory Anchors (RMA), keeping persistent history at a fixed capacity as context grows. 固定记忆，长程生成。 本版本保留 EdgeRock-4B 的递推状态、局部注意力和记忆岩锚（RMA），历史内存不随上下文增长。 Rock Matrix Whitepaper / 岩芯矩阵技术白皮书： whitepaper connects EdgeRock's fixed memory, DiffusionRock's parallel generation, and SiliconRock's model-native silicon into the route toward Ultra-Fast Token. 白皮书讲清固定记忆、并行生成、模型固化三条路线，以及它们如何走向更快、更经济的生成。 Public development preview for Google AI Edge Gallery device validation. The container includes the text decoder, tokenizer, ChatML prompt template and fixed-state executor…

Model · Text generation

logi-1.5b-gguf

Alves

Versões quantizadas em GGUF do LOGI 1.5B, um modelo educacional em português brasileiro criado para o jogo Decifra.IA. O LOGI atua como tutor de: - lógica proposicional: AND, OR, NOT, IMPLICA, BICONDICIONAL, tabelas-verdade, equivalências, falácias e regras de inferência; - alfabetização em inteligência artificial: modelos, dados, treinamento, inferência, vieses, alucinação, RAG, privacidade e uso responsável; - segurança digital relacionada à IA: phishing, deepfakes, engenharia social, prompt injection e proteção de dados. Este repositório contém as quantizações da versão v5, treinada com 13.000 exemplos. Ela substitui a versão inicial do LOGI, que utilizava 1.183 exemplos. Para…

A 0.75B-parameter local model that lets a coding agent hand off routine, read-only repository work. Hydra Nano is the model behind Hydra, a CLI that a frontier coding agent (Claude Code, Codex and others) calls instead of running noisy commands itself. Hydra asks the model to choose from a fixed menu of read-only probes, runs those probes in code, and returns the command's own output lines. The model never writes a shell command. One model serves all five Hydra heads: All figures are end-to-end Hydra results: the task counts as solved only when the probe plan is right and the returned evidence contains every expected fact with nothing forbidden. Errors and timeouts count as failures. Every…

Model · Text generation

pulse-local-qwen3-0.6b

Pulse Finance

A 0.6B Qwen3 fine-tuned for one job: being the language layer of Pulse Local, a Mac app where a personal-finance assistant, its ledger and every question stay on the user's own machine. It is published so the app can download it on first launch. Beyond that download, the app's Local Mode makes only an optional check for a newer version, which can be switched off and sends nothing about the user. Its optional Connected Mode adds a bank feed through Plaid; even then the model, the ledger and every question stay on the machine. Read this before you try it as a general assistant: on its own it is not one. On our private 48-question finance benchmark this model alone scores about 55 out of 100.…

This repository contains official GGUF quantizations of the Vigyan AI STEM Model Series engineered for 100% offline, air-gapped on-device deployment across consumer hardware, laptops, and edge devices. These checkpoints represent intermediate research iterations developed by Vigyan AI to evaluate sovereign neuro-symbolic SLM execution combined with deterministic symbolic solvers (SymPy) and embedded C++ GraphRAG.

Model · Text generation

Vinci-Cyber-123B-1.0-GGUF

SimpleDirect

GGUF quantizations of simpledirect/Vinci-Cyber-123B-1.0, for llama.cpp and compatible runtimes. Vinci Cyber 123B is designed for defensive infrastructure review and targeted remediation: propose a focused repair when needed, and leave correct configuration alone. Three formats. Direct task measurements. A clearer deployment choice. All three tiers have now been evaluated on the V2-B neutral-review infrastructure test. Q80 matched the released BF16 model's task outcomes: 18/24 repairs and 24/24 correct configurations unchanged. Q5KM repaired 12/24 and Q4KM 11/24; both preserved 24/24 correct inputs. Every credited repair passed the reported offline provider-schema check. Choose from Q80…

Reformatted Task 4.1 release used for external evaluation: datht/vlegal The paper contains the split sizes, training configuration, evaluation-set sizes, and reported results; they are not repeated here. - adaptermodel.safetensors — final CEFC-RFT LoRA weights; - adapterconfig.json — PEFT adapter configuration; - cefcrunconfig.json — exact controller and final-run configuration; - cefctrainingmetadata.json — metadata for the weighted LoRA continuation run; - selectionstate.json — exported selection state from the CEFC-RFT run; - calibratecefcstagegates.py — validation-only Stage-gate replay script; - chattemplate.jinja, tokenizer.json, tokenizerconfig.json, specialtokensmap.json…

Model · Text generation

Qwen3-4B-RKLLM

Hanzo Huang

Text generation on Rockchip RK3576 and RK3588, converted from Qwen/Qwen3-4B. Use the files for your board’s exact SoC. w4a16 uses 4-bit weights and 16-bit activations; w4a16g128 adds a weight group size of 128. w8a8 uses 8-bit weights and activations. File size below is download size, not total runtime RAM. With Docker installed on a matching Rockchip board, this example starts the RK3576 w4a16g128 model and its API on port 8001: The image includes its model file. See rkllm-docker for API requests, other board variants, and configuration. To download the files separately instead: These are native RKLLM/RKNN artifacts. Use a compatible RKLLM runtime and the model’s correct chat template when…

Model · Text generation

Qwen2-0.5B-Instruct-RKLLM

Hanzo Huang

Text generation on Rockchip RK3576 and RK3588, converted from Qwen/Qwen2-0.5B-Instruct. Use the files for your board’s exact SoC. w4a16 uses 4-bit weights and 16-bit activations; w4a16g128 adds a weight group size of 128. w8a8 uses 8-bit weights and activations. File size below is download size, not total runtime RAM. With Docker installed on a matching Rockchip board, this example starts the RK3576 w4a16g128 model and its API on port 8001: The image includes its model file. See rkllm-docker for API requests, other board variants, and configuration. To download the files separately instead: These are native RKLLM/RKNN artifacts. Use a compatible RKLLM runtime and the model’s correct chat…

Model · Text generation

MiniCPM3-4B-RKLLM

Hanzo Huang

Text generation on Rockchip RK3576 and RK3588, converted from openbmb/MiniCPM3-4B. Use the files for your board’s exact SoC. w4a16 uses 4-bit weights and 16-bit activations; w4a16g128 adds a weight group size of 128. w8a8 uses 8-bit weights and activations. File size below is download size, not total runtime RAM. With Docker installed on a matching Rockchip board, this example starts the RK3576 w4a16g128 model and its API on port 8001: The image includes its model file. See rkllm-docker for API requests, other board variants, and configuration. To download the files separately instead: These are native RKLLM/RKNN artifacts. Use a compatible RKLLM runtime and the model’s correct chat…

Model · Text generation

LLaDA-8B-Instruct-PIG

IFML

LLaDA-8B-Instruct with Parallel Inconsistency Guidance (PIG), which lets the model commit more tokens per forward pass at the same accuracy. This repository is the whole model: the backbone, a 2.2M-parameter PIG adapter and its calibration tables. Paper: PIG: Parallel Inconsistency Guidance for Accelerating Diffusion Language Models, Fruytier, Xue, Shakkottai, Sanghavi (UT Austin), 2026; arXiv:2610.17375. Code: https://github.com/Quentin-Fruytier/PIG-DLM-release Blog: Which tokens can a diffusion language model commit together? Needs transformers 5.x and a GPU with 24 GB. Implementation details, every argument, and how the adapter and tables were fitted: AGENTS.md. MIT. The backbone is…

Model · Text generation

mato-v1.0-sft

German Gimenez

Mato es el asistente de IA entrenado en casa por Napsix — una plataforma B2B agéntica donde los agentes corren solos: investigan, escriben, operan herramientas y producen resultados (documentos, planillas, tableros, formularios, sitios). Este repositorio contiene el adapter LoRA (SFT) sobre Qwen/Qwen3.8-27B. En producción se sirve como LoRA sobre el checkpoint FP8 del base, con MTP speculative decoding y 256k de contexto, en GPUs spot de Daytona. - Tool calling nativo en el formato XML de Qwen (parser qwen3xml de vLLM): llamadas correctas, argumentos válidos y recuperación ante fallos. directo y conciso, sin relleno ni muletillas. - Conocimiento de las superficies de Napsix: XIA, Agentes…

Model · Text generation

MiniCPM4-0.5B-RKLLM

Hanzo Huang

Text generation on Rockchip RK3576 and RK3588, converted from openbmb/MiniCPM4-0.5B. Use the files for your board’s exact SoC. w4a16 uses 4-bit weights and 16-bit activations; w4a16g128 adds a weight group size of 128. w8a8 uses 8-bit weights and activations. File size below is download size, not total runtime RAM. With Docker installed on a matching Rockchip board, this example starts the RK3576 w4a16g128 model and its API on port 8001: The image includes its model file. See rkllm-docker for API requests, other board variants, and configuration. To download the files separately instead: These are native RKLLM/RKNN artifacts. Use a compatible RKLLM runtime and the model’s correct chat…

Model · Text generation

Nemotron-Labs-Diffusion-3B-PIG

IFML

Nemotron-Labs-Diffusion-3B, NVIDIA's instruct model, with Parallel Inconsistency Guidance (PIG), which lets the model commit more tokens per forward pass at the same accuracy. This repository is the whole model: the backbone, a 1.9M-parameter PIG adapter and its calibration tables. Paper: PIG: Parallel Inconsistency Guidance for Accelerating Diffusion Language Models, Fruytier, Xue, Shakkottai, Sanghavi (UT Austin), 2026; arXiv:2610.17375. Code: https://github.com/Quentin-Fruytier/PIG-DLM-release Blog: Which tokens can a diffusion language model commit together? Needs transformers 5.x and a GPU with 8 GB. Implementation details, every argument, and how the adapter and tables were fitted…

Goal: oQ4e speed and memory footprint, oQ5e-or-better quality reaching toward oQ6e. Runs at ~73–75 GB resident with ~82→66 tok/s decode on a 128 GB Apple Silicon host — the speed/RAM economics of a 4-bit build, at an accuracy tier equal to or above uniform oQ5e. How: a mixed-precision MLX build of Qwen3.8-Flash-Next, assembled by component-wise byte-copy from Jundot's native oQ quantizations — zero re-quantization in the entire package. Every tensor is byte-identical to its source, so quality is traceable, not re-derived. Uniform-bit quantization pays accuracy where it isn't needed (middle layers) and loses it where it hurts (first/last layers, n-gram table, attention). A Data-Free…

Model · Text generation

Qwen3-Coder-Next-CQ-4bit

ThousandML

Qwen3-Coder-Next quantized to 4-bit with Coded Quantization (CQ). CQ is a calibration-free 4-bit quantization method designed to preserve the behavior of the original model without requiring a calibration dataset. Because CQ does not optimize the quantization around a fixed calibration corpus, it is designed to generalize beyond calibration-specific data and workloads. This repository contains Qwen3-Coder-Next quantized with Coded Quantization (CQ) 4-bit. Calibration-free. CQ does not require a representative calibration dataset before quantization. This avoids tying the quantization procedure to a particular calibration corpus and is intended to provide better generalization beyond the…

Model · Text generation

Llama-3.1-8B-Instruct-CQ-4bit

ThousandML

Llama 3.1 8B Instruct quantized to 4-bit with Coded Quantization (CQ). CQ is a calibration-free 4-bit quantization method designed to preserve the behavior of the original model without requiring a calibration dataset. Because CQ does not optimize the quantization around a fixed calibration corpus, it is designed to generalize beyond calibration-specific data and workloads. This repository contains the 8B-parameter Llama 3.1 Instruct model quantized with Coded Quantization (CQ) 4-bit. Calibration-free. CQ does not require a representative calibration dataset before quantization. This avoids tying the quantization procedure to a particular calibration corpus and is intended to provide better…

This modelcard aims to be a base template for new models. It has been generated using this raw template. DucGPT v0.1 is an incomplete, continuously developing GPT-style language model created from curiosity and a desire to understand how modern language models work. It is an independent project developed by a 17-year-old student, covering the process from dataset preprocessing and tokenization to Transformer architecture, training, inference, and user accessibility. DucGPT is not intended to represent a finished or production-ready language model. Instead, it serves as an ongoing exploration of language-model engineering, with each version reflecting what I have learned and experimented…

MurinTune — серия русскоязычных диалоговых языковых моделей, дообученных методом QLoRA (PEFT) на базе Gemma-4-E2B-it для моделирования экспрессивного разговорного дискурса, неологизмов и фольклора современного молодежного интернет-сообщества. Модель является юмористической сатирой и художественной AI-пародией (parody / satire). Создана исключительно в образовательных и научно-исследовательских целях в сфере компьютерной лингвистики (NLP). 1. В поисковой строке моделей введите: MurinTune (или mxbtv/MurinTune-0.1-E2B-GGUF) 2. Скачайте файл: MurinTune-0.5-E2B.Q4KM.gguf. Температура (Temperature): 0.65 Штраф за повторы (Repetition Penalty): 1.18 Режим размышлений (Enable Thinking): Включен (ON)…

Built with Qwen. Qwen/Qwen2.5-3B-Instruct cuantizado a INT4 con (2.57x mas chico) perdiendo 2.43% de calidad. Incluye las dos formas de usarlo: qwen3b-int4.gguf para llama.cpp y Android, y la carpeta.tq para PyTorch. Perplejidad en wikitext-2 test, 20 ventanas de 2048 tokens, todo medido dentro de llama.cpp con llama-perplexity para que la comparacion sea de tu a tu. Este modelo pierde 2.6 veces menos calidad que llama.cpp Q4KM, el formato mas usado para correr modelos en local, a cambio de 0.47 GB mas de archivo. La comparativa de los tres tamaños esta en En PyTorch el modelo cuantizado genera mas lento que el original: Octuma desempaqueta los 4 bits en cada multiplicacion sin un kernel…

Model · Text generation

SmolLM3-3B-Q4_K_M-GGUF

Shayon Sarker

Downloads: Hugging Face — ShayonSarker/SmolLM3-3B-Q4KM-GGUF · Source & build recipes: GitHub — edge-smollm3-gguf · Base model: HuggingFaceTB/SmolLM3-3B-Base The GGUF binaries are hosted on Hugging Face; this repository holds the reproducible conversion pipeline, calibration data, and checksums. Base: HuggingFaceTB/SmolLM3-3B-Base (SmolLM3ForCausalLM, Apache-2.0) · Quant: llama.cpp 0.4.1-dev (c77ae69) · imatrix on 4000-line wikitext-2 (seed 42, 798 chunks, 4 threads) The F16 row in the eval table below is a measurement reference, not a downloadable file. Gate was <5% — passed. Task evals (ARC/HellaSwag) not run: perplexity only, stated honestly. Files: smol-Q4KM.gguf, smol-Q80.gguf…

Model · Text generation

LFM2.5-1.2B-Instruct-heretic-ExecuTorch

Experimental Machines

ExecuTorch exports of experimentalmachines/LFM2.5-1.2B-Instruct-heretic (revisions 300a0717dd75, 435177079cc7 (see each variant's sourcerevision)) for on-device inference with the openweights Android app or any ExecuTorch 1.4.0 runtime. Windows in this repo: XNNPACK (CPU) at 2k to 32k; MediaTek NeuroPilot MT6991 (Dimensity 9400) at 512 to 16k. The window is fixed inside the file: the runtime allocates the whole KV cache at load, so pick the largest window the device can hold (fitsphonebudget in each folder's config.json is the estimate against a 5 GB budget). MediaTek folders also hold the token embedding table the NeuroPilot runner reads from disk, shared by every window…

一个在 Apple M3 Max / 128GB 统一内存上手工量化出的 Qwen3.8-Flash-Next（qwen4exp 架构，180B 级 MoE：125B 主干 + 51B n-gram 表 + 4B MTP 投机头）的 mlx-serve 混合位宽包：常驻约 60GB，无审查源，带视觉塔。 - 基座模型：Qwen 官方 Qwen3.8-Flash-Next；原始权重：社区团队 orcarouter 的无审查 BF16 包（335.3 GiB / 131 分片，逐文件 sha256 验收 132/132 通过） - 量化工具链：mlx-serve 仓库自带产线（imatrix 校准采集 → 贪心位宽分配 → 混合宽度转换 → 留出窗口打分），感谢上游作者 - 本包作者做的四件事：用自己的 AI 会话日志做校准语料（三个 agent 工具的真实流量 + 官方通用语料，按内容 sha1 去重 26.11%，密钥/凭证脱敏）、逐层贪定位宽配方、跑完整产线、做质量与业务验收 - 卡片样式沿用 mlx-serve 生态包卡片惯例（模板参考 ddalcu 上游卡片），读数全部替换为 M3 Max 128GB 本机实测 整库约 92GB：权重与视觉塔 59.83GB（索引载荷口径，100 个编号分片 + model-vision.safetensors）+ 独立 n-gram 表 32,000,153,976 字节（ngramtable.bin）+ 9 个配置/文档文件。 生产运行口径（日常 DSH agent 会话，2026-10-06 17:24 起 51 小时）：…

MLX 4-bit weights with group size 32; a 2.875 GB weight package. Fixed memory for long-running generation. This EdgeRock-4B deployment package retains recurrent state, local attention, and Rock Memory Anchors (RMA), keeping persistent history at a fixed capacity as context grows. 固定记忆，长程生成。 本版本保留 EdgeRock-4B 的递推状态、局部注意力和记忆岩锚（RMA），历史内存不随上下文增长。 Rock Matrix Whitepaper / 岩芯矩阵技术白皮书： whitepaper connects EdgeRock's fixed memory, DiffusionRock's parallel generation, and SiliconRock's model-native silicon into the route toward Ultra-Fast Token. 白皮书讲清固定记忆、并行生成、模型固化三条路线，以及它们如何走向更快、更经济的生成。 The instructions and validation below apply to this package. Browse all EdgeRock models. Original-model benchmarks…

MLX 8-bit weights with group size 64; a 4.874 GB weight package. Fixed memory for long-running generation. This EdgeRock-4B deployment package retains recurrent state, local attention, and Rock Memory Anchors (RMA), keeping persistent history at a fixed capacity as context grows. 固定记忆，长程生成。 本版本保留 EdgeRock-4B 的递推状态、局部注意力和记忆岩锚（RMA），历史内存不随上下文增长。 Rock Matrix Whitepaper / 岩芯矩阵技术白皮书： whitepaper connects EdgeRock's fixed memory, DiffusionRock's parallel generation, and SiliconRock's model-native silicon into the route toward Ultra-Fast Token. 白皮书讲清固定记忆、并行生成、模型固化三条路线，以及它们如何走向更快、更经济的生成。 The instructions and validation below apply to this package. Browse all EdgeRock models. Original-model benchmarks…

Model · Text generation

Vinci-Cyber-30B-1.0-GGUF

SimpleDirect

Quantized GGUF conversions of simpledirect/Vinci-Cyber-30B-1.0, a narrow, verifier-grounded Infrastructure-as-Code security specialist developed by SimpleDirect, a Canadian AI Lab, from IBM Granite 4.1 30B. See the main repository for what the model does, how it was trained, and its evaluation status. What this repository establishes, in one paragraph. All four tiers convert completely (578/578 tensors), carry faithful architecture metadata (14/14 fields), load and generate on CUDA with zero NaN and zero errors, and have had their distributional distortion against the BF16 quant master measured per token and reported below. What it does not establish is capability: no tier was benchmarked…

uLM-28M is a 28.8M-parameter decoder-only causal language model released with the paper Micro Language Models Enable Instant Responses. It is designed for instant local response initiation: the model generates a short, contextually grounded opening of approximately 4–8 words on a resource-constrained device, and a larger cloud model continues the response. For representative behavior, use the tokenizer's chat template, sampling, and short-prefix generation as shown below. The model was trained from scratch on chat-style data. Further training and evaluation details are provided in the accompanying paper. uLM-28M is intended to generate only the first few words of a response: The exact…

GGUF files of jialinyyzz/humanizer (v2), a 12B model that rewrites AI-written drafts in English and Chinese so they read like a person wrote them, while keeping every number, date, name and quote. This page is about the quantized files only: what each one costs in quality, and how they were made. For the model itself (training, full results, examples), see the main model card and GitHub. The files here are byte-for-byte the same as the GGUF files in the main repository; download from either. promptformat.json (tiny) holds the instruction and separator verbatim; you need it unless you use the chat template (see Usage). ¹ Peak resident memory of llama-server (llama.cpp, Metal) on an Apple M5…

Model · Text generation

parallax-8b-kappa

Chutes

These are research checkpoints from kappa, a training run of Parallax that is still in progress. Parallax is a mixture-of-experts language model trained on consumer GPUs spread over several countries. The hosts have no direct connections to each other and synchronize over the public internet. - Base model. It is pretrained on a mixture of web, document, math, code, knowledge-focused and book text. It is not instruction-tuned, chat-tuned or safety-tuned, and it will continue text rather than follow instructions. - Mid-training. Each export is a snapshot of a run that has not finished. Later exports are expected to differ, and quality between exports is not monotonic. - Research artifact. It…

MLX mixed precision: 4-bit MLPs and 8-bit other eligible matrices, with group size 64; 3.741 GB of weights. Fixed memory for long-running generation. This EdgeRock-4B deployment package retains recurrent state, local attention, and Rock Memory Anchors (RMA), keeping persistent history at a fixed capacity as context grows. 固定记忆，长程生成。 本版本保留 EdgeRock-4B 的递推状态、局部注意力和记忆岩锚（RMA），历史内存不随上下文增长。 Rock Matrix Whitepaper / 岩芯矩阵技术白皮书： whitepaper connects EdgeRock's fixed memory, DiffusionRock's parallel generation, and SiliconRock's model-native silicon into the route toward Ultra-Fast Token. 白皮书讲清固定记忆、并行生成、模型固化三条路线，以及它们如何走向更快、更经济的生成。 The instructions and validation below apply to this package. Browse all…

Model · Text generation

Flex-Model-LoRA

Frox Studio

Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. Use the code below to get started with the model. Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019). - PEFT 0.21.0

Hades 8B is an open-source Small Language Model built on Meta's Llama 3.1 8B architecture. Post-trained across 50,000 steps on C-DAC Airawat supercomputing infrastructure using the mlabonne/FineTome-100k dataset, it is optimized for high-reasoning instruction following, structured output, and fast local execution. Evaluated using lm-evaluation-harness (lm-eval) in Zero-Shot Chain-of-Thought mode: ollama run hf.co/ProjectMosiacAI/Hades-8B