# QTIP: Quantization with Trellises and Incoherence Processing
Source: https://savrn.com/papers/qtip-quantization-with-trellises-and-incoherence-processing
Markdown alternate of the page above; the site index is https://savrn.com/llms.txt

---

## Abstract

Post-training quantization (PTQ) reduces the memory footprint of LLMs by quantizing weights to low-precision datatypes. Since LLM inference is usually memory-bound, PTQ methods can improve inference throughput. Recent state-of-the-art PTQ approaches use vector quantization (VQ) to quantize multiple weights at once, which improves information utilization through better shaping. However, VQ requires a codebook with size exponential in the dimension. This limits current VQ-based PTQ works to low VQ dimensions (le 8) that in turn limit quantization quality. Here, we introduce QTIP, which instead uses trellis coded quantization (TCQ) to achieve ultra-high-dimensional quantization. TCQ uses a stateful decoder that separates the codebook size from the bitrate and effective dimension. QTIP introduces a spectrum of lookup-only to computed lookup-free trellis codes designed for a hardware-efficient "bitshift" trellis structure; these codes achieve state-of-the-art results in both quantization quality and inference speed.

[Full paper on arXiv](https://arxiv.org/abs/2406.11235) · [Code](https://github.com/https://github.com/Cornell-RelaxML/qtip)

## Details

arXiv identifier

2406.11235

Published

2024-06-17

Authors

Albert Tseng, Qingyao Sun, David Hou, Christopher De Sa

## Open Models Built on This Paper

Every model in the SAVRN Model Hub whose card cites this paper, most downloaded first, with what it takes to run each one.

| Model | Task | Size | License | Monthly downloads | Cheapest setup at 16-bit |
| --- | --- | --- | --- | --- | --- |
| [Swift-1.5-Qwen3.8-27B-EXL3-3.85bpw-24GB](https://savrn.com/models/swift-1-5-qwen3-8-27b-exl3-3-85bpw-24gb) Jorge Ramírez Espejo | [Image and text to text](https://savrn.com/models/tasks/image-and-text-to-text) | 7.9B | other | 312 | 1x [MI300X](https://savrn.com/ai-index/pricing/gpus/mi300x) $1.85/hr |
| [Ornith-1.5-9B-EXL3-16GB-12GB](https://savrn.com/models/ornith-1-5-9b-exl3-16gb-12gb) Jorge Ramírez Espejo | [Image and text to text](https://savrn.com/models/tasks/image-and-text-to-text) | 4.4B | mit | 192 | 1x [MI300X](https://savrn.com/ai-index/pricing/gpus/mi300x) $1.85/hr |
| [Qwen3.8-27B-EXL3-3.85bpw-24GB](https://savrn.com/models/qwen3-8-27b-exl3-3-85bpw-24gb) Jorge Ramírez Espejo | [Image and text to text](https://savrn.com/models/tasks/image-and-text-to-text) | 7.9B | apache-2.0 | 173 | 1x [MI300X](https://savrn.com/ai-index/pricing/gpus/mi300x) $1.85/hr |
| [Qwopus3.8-27B-Flash-V2-EXL3-3.85bpw-24GB](https://savrn.com/models/qwopus3-8-27b-flash-v2-exl3-3-85bpw-24gb) Jorge Ramírez Espejo | [Image and text to text](https://savrn.com/models/tasks/image-and-text-to-text) | 7.9B | apache-2.0 | 153 | 1x [MI300X](https://savrn.com/ai-index/pricing/gpus/mi300x) $1.85/hr |
| [Qwen3.8-27B-pi-EXL3-3.85bpw-24GB](https://savrn.com/models/qwen3-8-27b-pi-exl3-3-85bpw-24gb) Jorge Ramírez Espejo | [Image and text to text](https://savrn.com/models/tasks/image-and-text-to-text) | 7.9B | apache-2.0 | 130 | 1x [MI300X](https://savrn.com/ai-index/pricing/gpus/mi300x) $1.85/hr |
| [Ostrich-27B-261008-Qwen3.8-Abliterated-EXL3](https://savrn.com/models/ostrich-27b-261008-qwen3-8-abliterated-exl3) Emin Temiz | — | — | apache-2.0 | — | — |
| [Ostrich-27B-261008-Qwen3.8-Abliterated](https://savrn.com/models/ostrich-27b-261008-qwen3-8-abliterated) Emin Temiz | — | 27.8B | apache-2.0 | — | 1x [MI300X](https://savrn.com/ai-index/pricing/gpus/mi300x) $1.85/hr |

## Explore More

- [All research papers](https://savrn.com/papers)
- [The model directory](https://savrn.com/models)
- [Datasets](https://savrn.com/datasets)

## Source

- arXiv metadata, read 2026-10-16.
- [How the hub is built](https://savrn.com/model-hub/methodology)