4-bit, group-size-128 AutoRound quantization of Altworld/Hemmingway-1, a 27B-parameter fine-tune of Qwen3.8-27B. This repository is an independent quantization, not the original model release. See the original model card for training, intended use, and benchmark information. Quantization: AutoRound 0.15.1, bits=4, groupsize=128, packingformat=autoround:autoawq, seqlen=1024, nsamples=64, iters=50. Certain layers remain in higher precision, as specified in quantizationconfig.json. The included six-shard weight index and auxiliary modelextratensors.safetensors are needed together. Approximate download size: 18.65 GB (17.37 GiB). This is a quantization of an existing fine-tuned model, not a new…
Open weights
apache-2.0
5.8B parameters
262,144 tokens
transformers
I wanted a smaller GGUF I could run, and I wanted to know more than just whether it loaded. This is where that experiment landed. How we got here has the story, including the first Q2 export, the H64-S4 experiment, the interrupted run, and what I could and couldn't verify. You need a runtime that supports the matching Hadamard transform. A runtime that ignores the rotation metadata can compute the wrong result. The tested llama.cpp changes, source patch, and build instructions are in runtime/. The export has nine GGUF shards, about 68.9 GB in total. It is based on Qwen/Qwen3.8-Flash-Next, revision de4b8e4d43b917e7706784d8bb445c9af86a3540. I used mixed precision to spend the bits where the…
Open weights
other
gguf
This is an MLX model for Apple Silicon. It is not directly compatible with the PyTorch CUDA runner. Converted from mlx-community/MOSS-TTS-8B-8bit at revision b9262754ad5245942472c7b4ff2b53cec3a07ec0: Q6 backbone and Q8 speech path, affine quantization with group size 64. Structurally validated on Linux; Apple Silicon runtime and speech-quality validation are pending. See RESULT.md, recipe.json, and the audit reports for details. The upstream model card follows. This MLX implementation brings MOSS-TTS to Apple Silicon through the MLX runtime, adapting the TTS backbone, multi-head audio token generation, and MOSS Audio Tokenizer workflow to run inside the MLX-Audio stack. As part of…
Open weights
apache-2.0
40,960 tokens
Open weights
apache-2.0
9.7B parameters
gptqmodel