Skip to page content
SAVRN
Search Plan a project

Open-weight model · Text generation

Llama-3.3-70B-Instruct-CQ-4bit

by ThousandML thousandml/Llama-3.3-70B-Instruct-CQ-4bit

Llama-3.3-70B-Instruct-CQ-4bit is an open-weight model for text generation from ThousandML, released under llama3.3. It has 4.2B parameters and a 131,072-token context. At 16-bit it needs about 10.2 GB of GPU memory, which fits on 1x MI300X from $1.85 an hour, at the lowest prices in the SAVRN Index. It draws 21 downloads a month.

Llama 3.3 70B Instruct quantized to 4-bit with Coded Quantization (CQ). CQ is a calibration-free 4-bit quantization method designed to preserve the behavior of the original model without requiring a calibration dataset.

Parameters4.2B
Context131,072
Weights42.1 GB
Licensellama3.3
AccessOpen weights
Monthly Downloads21

Runs On

What it takes to serve Llama-3.3-70B-Instruct-CQ-4bit (4.2B parameters): the memory its weights need at each precision, and the cheapest way to rent enough data-center GPUs to hold them.

PrecisionWeightsMemory neededCheapest setupPer hourAlso fits
16-bit 8.5 GB 10.2 GB 1x MI300X (192 GB)
Vultr
$1.85 1x H100 $1.99 · 1x MI325X $2.00
8-bit 4.2 GB 5.1 GB 1x MI300X (192 GB)
Vultr
$1.85 1x H100 $1.99 · 1x MI325X $2.00
4-bit 2.1 GB 2.5 GB 1x MI300X (192 GB)
Vultr
$1.85 1x H100 $1.99 · 1x MI325X $2.00

Memory is the weights at that precision plus 20% for the runtime and a short context; a long context needs more. Prices are the lowest on-demand hourly rates in the SAVRN Index, read Oct 16, 2026.

Llama-3.3-70B-Instruct-CQ-4bit on every accelerator the SAVRN Index prices, at every precision

Model Card

Llama 3.3 70B Instruct quantized to 4-bit with Coded Quantization (CQ). CQ is a calibration-free 4-bit quantization method designed to preserve the behavior of the original model without requiring a calibration dataset. Because CQ does not optimize the quantization around a fixed calibration corpus, it is designed to generalize beyond calibration-specific data and workloads. This repository contains the 70B-parameter Llama 3.3 Instruct model quantized with Coded Quantization (CQ) 4-bit. Calibration-free. CQ does not require a representative calibration dataset before quantization. This avoids tying the quantization procedure to a particular calibration corpus and is intended to provide…

Excerpt from the card by ThousandML, licensed llama3.3.

Configuration

Architecture
LlamaForCausalLM
Context length (tokens)
131,072
Layers
80
Hidden size
8,192
Feed-forward size
28,672
Attention heads
64
Key/value heads
8
Head dimension
128
Vocabulary size
128,256
Model type
llama
Quantization
cq

Identity and Version

Repository
thousandml/Llama-3.3-70B-Instruct-CQ-4bit
Publisher
ThousandML
Task
Text generation
Modality
Text
Library
Not stated by the source
Parameters
4.2B parameters
Languages
cq
Revision
c50d49c89f82bab3f05debf6d25985f5d1c31cc5
First published
2026-08-17
Last updated
2026-10-16

Files and Weights

46 files, 42.2 GB in total. The weights are 34 files totalling 42.1 GB in safetensors.

Weights34 files · 42.1 GB
Configuration6 files · 366.4 KB
Tokenizer2 files · 17.3 MB
Documentation3 files · 19.0 KB
Repository1 file · 1.6 KB
Every file
FileTypeSizeSHA-256
model-00001-of-00030.safetensorsWeights3.4 GB 7e574b499875
model-00002-of-00030.safetensorsWeights167.9 MB c3b09d0df1d9
model-00003-of-00030.safetensorsWeights939.6 MB 91eac63cddad
model-00004-of-00030.safetensorsWeights939.6 MB 793473c12712
model-00005-of-00030.safetensorsWeights67.9 KB 6b14fe1434b1
model-00006-of-00030.safetensorsWeights100.9 KB 9f2b198b8034
model-00007-of-00030.safetensorsWeights100.9 KB 75632cb22a63
model-00008-of-00030.safetensorsWeights100.9 KB af83907dfdbc
model-00009-of-00030.safetensorsWeights100.7 KB 75079e68ab13
model-00010-of-00030.safetensorsWeights67.9 KB 787e27bf561d
model-00011-of-00030.safetensorsWeights100.9 KB 83ddd4b15f48
model-00012-of-00030.safetensorsWeights100.9 KB ce887852f0cf
model-00013-of-00030.safetensorsWeights939.6 MB af4448890019
model-00014-of-00030.safetensorsWeights100.7 KB f5f477bbba5a
model-00015-of-00030.safetensorsWeights67.9 KB 3ee33530fefc
model-00016-of-00030.safetensorsWeights100.9 KB 9a5e57d5cd98
model-00017-of-00030.safetensorsWeights100.9 KB 8aa1bd3dc37c
model-00018-of-00030.safetensorsWeights100.9 KB 8eb37a4ab85d
model-00019-of-00030.safetensorsWeights100.7 KB 3e4c1e53e7ae
model-00020-of-00030.safetensorsWeights67.9 KB 271d3e082660
model-00021-of-00030.safetensorsWeights100.9 KB 30b3d16a30c4
model-00022-of-00030.safetensorsWeights100.9 KB 1683deeebf23
model-00023-of-00030.safetensorsWeights100.9 KB 9d1d92d09648
model-00024-of-00030.safetensorsWeights100.7 KB 33e34d00af93
model-00025-of-00030.safetensorsWeights67.9 KB fa1df16a4366
model-00026-of-00030.safetensorsWeights100.9 KB ee1b4d343a43
model-00027-of-00030.safetensorsWeights100.9 KB c024e90c2b1a
model-00028-of-00030.safetensorsWeights100.9 KB 7f909e5d68fa
model-00029-of-00030.safetensorsWeights117.2 KB 67206ff8b23b
model-00030-of-00030.safetensorsWeights2.1 GB b0436c98c9b4
quantization_data/cq-00001-of-00004.safetensorsWeights10.6 GB 410941286da4
quantization_data/cq-00002-of-00004.safetensorsWeights10.8 GB 272570286aec
quantization_data/cq-00003-of-00004.safetensorsWeights10.7 GB c0cf58dfdb43
quantization_data/cq-00004-of-00004.safetensorsWeights1.5 GB d38369620ddf
config.jsonConfiguration1.3 KB —
generation_config.jsonConfiguration189 B —
model.safetensors.index.jsonConfiguration59.6 KB —
quantization_data/_layers.jsonConfiguration11.1 KB —
quantization_data/cq_shards.jsonConfiguration294.1 KB —
special_tokens_map.jsonConfiguration68 B —
LICENSEDocumentation7.9 KB —
README.mdDocumentation5.0 KB —
USE_POLICY.mdDocumentation6.0 KB —
.gitattributesRepository1.6 KB —
tokenizer.jsonTokenizer17.2 MB 6b9e4e7fb171
tokenizer_config.jsonTokenizer55.4 KB —

License and Download

License
llama3.3
Access
Open weights, no gate
Download size
42.1 GB
Download from ThousandML

Released by ThousandML through its official repository on Hugging Face.

Built From

Memory Requirements

PrecisionWeights in memory
As published42.1 GB
16-bit8.5 GB
8-bit4.2 GB
4-bit2.1 GB

Weights only, from the published parameter count; the key-value cache and runtime add to this.

Questions About Llama-3.3-70B-Instruct-CQ-4bit

How much GPU memory does Llama-3.3-70B-Instruct-CQ-4bit need?

About 10.2 GB at 16-bit and 2.5 GB at 4-bit: the weights (4.2B parameters) plus a working margin. A long context needs more.

What is the cheapest GPU to run Llama-3.3-70B-Instruct-CQ-4bit on?

At 16-bit, 1x MI300X from $1.85 an hour; at 4-bit, 1x MI300X from $1.85 an hour, at the lowest on-demand prices the SAVRN Index lists.

What license is Llama-3.3-70B-Instruct-CQ-4bit released under?

llama3.3, as its publisher declares it. Read the license text before commercial use.

What is Llama-3.3-70B-Instruct-CQ-4bit's context length?

131,072 tokens, from the maximum position embeddings in its published configuration.

Similar Models

Model · Text generation

dQwen3.5-4B-Base

IFML

A masked diffusion language model adapted from Qwen3.5-4B. The backbone is hybrid: only its attention layers are made bidirectional, and the Gated DeltaNet layers stay causal. This is a base model, with no instruction tuning. Paper: dQwen3.5: Hybrid-Attention Diffusion Language Models. Code: https://github.com/AntonXue/dQwen Needs a CUDA GPU and transformers>=5.13 (tested with torch 2.7.1+cu128, flash-linear-attention 0.5.1). generate decodes the whole canvas at once, committing positions above a confidence threshold (tau=0.9); pass blocklength=32 for left-to-right block decoding, or tau=None, stepsperblock=k for a fixed budget. The 50B-token checkpoint from the paper is…

Open weights apache-2.0 4.2B parameters 262,144 tokens transformers

Model · Text generation

Huihui-NeoHorse-1-4B-abliterated

Huihui.ai

This is an uncensored version of TokenRhythm/NeoHorse-1-4B created with abliteration (see remove-refusals-with-transformers to know more about it). This is a crude, proof-of-concept implementation to remove refusals from an LLM model without using TransformerLens. Layers 5-17 are being ablated (0-based indexing). The MTP and Visual components were extracted from the original Qwen/Qwen3.5-4B and can provide excellent support. If needed, you only need to copy the contents of MTP-Visual to overwrite the model directory. You can use this model in your applications by loading it with Hugging Face's transformers library: - Risk of Sensitive or Controversial Outputs: This model’s safety filtering…

Open weights apache-2.0 4.2B parameters 262,144 tokens transformers

Model · Text generation

JevK5-FP8

Liodon AI

FP8 quantization of alibiserikbay/JevK5, published by Liodon AI. Quantized with llm-compressor using the FP8DYNAMIC scheme: weights are cast to FP8 (E4M3) per-channel ahead of time, activations are quantized to FP8 dynamically per-token at inference time. No calibration dataset is needed for this scheme, so the quantized weights are numerically just a direct cast of the original — no calibration-set bias to worry about. lmhead is left unquantized (standard practice — negligible size, disproportionate quality impact if quantized). vLLM Text Generation Inference (TGI) SGLang FP8 execution requires an NVIDIA GPU with compute capability ≥ 8.9 (Ada/Hopper/Blackwell — RTX 40-series, L4/L40S…

Open weights other 4.2B parameters 262,144 tokens transformers

Model · Text generation

WikiQwen-4B

Devon Yanitski

Ask anything, get a how-to article. WikiQwen-4B is Qwen3.5-4B fine-tuned to answer every message the same way: as a tidy, step-by-step how-to article in markdown, with a picture caption above each step. Hand those captions to WikiQwen-Illustrator and they become illustrations. Send it a question ("how do I keep basil alive?"), a problem ("my bike chain keeps falling off") or just "hi", and it sends back a guide. Every reply has the same shape: - a # How to … title and a short intro - one or more ## Method N: or ## Part N: sections, or a single ## Steps - an [IMAGE: caption] line above each step, written for the Illustrator to draw - steps written as N. Bold summary. Details…, with bullets…

Open weights cc-by-nc-sa-4.0 4.2B parameters 262,144 tokens transformers

This is a text-only, 4-bit affine/group-64 MLX derivative for Minstrel's optional Advanced local language engine. It is based on the pinned mlx-community/Qwen3.5-4B-4bit snapshot 0e7ffd5c629ef7719d4cbc04069232580bfa9d9c. The deriveqwen35text.py tool removed vision tensors and capped the packaged context at 8,192 tokens. The signed Minstrel manifest and provenance file are included. The manifest is for Minstrel's detailed-local slot, artifact version 0.1.0; it is not a general quality certification. The reference upstream is Qwen/Qwen3.5-4B at 851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a. The source conversion does not establish the exact Qwen revision it used. The Apache-2.0 license file is…

Open weights apache-2.0 4.2B parameters 8,192 tokens mlx

Qwen3-8B after full-parameter supervised fine-tuning for mathematical keyword and meaning generation. This is the exact saved epoch 4 checkpoint at optimizer step 2976. The repository contains the model weights, configuration, and tokenizer files for inference. Training dataset: devpotatopotato/math-keyword-training, keyword-261001-bigmath-gpt-6-sol.jsonl. Use the tokenizer chat template with enablethinking=False. Pass the following template as a user message and replace {problem} with the problem text

Open weights apache-2.0 4.1B parameters 40,960 tokens transformers