answerai-colbert-small-v1 is a new, proof-of-concept model by Answer.AI, showing the strong performance multi-vector models with the new JaColBERTv2.5 training recipe and some extra tweaks can reach, even with just 33 million parameters.
Runs On
What it takes to serve answerai-colbert-small-v1 (33M parameters): the memory its weights need at each precision, and the cheapest way to rent enough data-center GPUs to hold them.
| Precision | Weights | Memory needed | Cheapest setup | Per hour | Also fits |
|---|---|---|---|---|---|
| 16-bit | 0.1 GB | 0.1 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
| 8-bit | 0.0 GB | 0.0 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
| 4-bit | 0.0 GB | 0.0 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
Memory is the weights at that precision plus 20% for the runtime and a short context; a long context needs more. Prices are the lowest on-demand hourly rates in the SAVRN Index, read Sep 18, 2026.
SAVRN's Notes on answerai-colbert-small-v1
We keep answerai-colbert-small-v1 on the list for one reason: it is the retrieval piece that picks which passages a larger model reads, and it costs almost nothing to host. Thirty-three million parameters stored as float32 come to 894.9 MB on disk and 0.1 GB of memory at 16-bit, so it shares a card with whatever generator it feeds. The cheapest single-card setup in our figures is one MI300X with 192 GB at $1.85 an hour, which this model barely touches. The 512-token context is the constraint: passages get chunked to fit, so the indexing pipeline matters more than the GPU.
Because the license is Apache 2.0, with an express patent grant on top of commercial use and redistribution, it is the kind of model we are comfortable embedding in a product. Two things to check: Answer.AI calls this a proof of concept, and it ships as safetensors and ONNX, so match the format to your serving stack.
Model Card
By Answer.AI, published under apache-2.0, revision 934fa8bb4ce2.
answerai-colbert-small-v1 is a new, proof-of-concept model by Answer.AI, showing the strong performance multi-vector models with the new JaColBERTv2.5 training recipe and some extra tweaks can reach, even with just 33 million parameters.
While being MiniLM-sized, it outperforms all previous similarly-sized models on common benchmarks, and even outperforms much larger popular models such as e5-large-v2 or bge-base-en-v1.5.
For more information about this model or how it was trained, head over to the announcement blogpost.
Usage
This model was designed with the upcoming RAGatouille overhaul in mind. However, it's compatible with all recent ColBERT implementations!
Sentence Transformers
This model can be used with Sentence Transformers as a multi-vector (ColBERT-style late interaction) retriever via the MultiVectorEncoder:
pip install "sentence-transformers>=6.0.0"
Configuration
- Architecture
- HF_ColBERT
- Context length (tokens)
- 512
- Layers
- 12
- Hidden size
- 384
- Feed-forward size
- 1,536
- Attention heads
- 12
- Vocabulary size
- 30,522
- Stored precision
- float32
- Model type
- bert
Identity and Version
- Repository
- answerdotai/answerai-colbert-small-v1
- Publisher
- Answer.AI
- Task
- Not stated by the source
- Modality
- Other
- Library
- sentence-transformers
- Parameters
- 33M parameters
- Languages
- en
- Revision
- 934fa8bb4ce2284f4c2baa232d81aca4d076fa5e
- First published
- 2024-08-12
- Last updated
- 2026-08-17
Files and Weights
21 files, 894.9 MB in total. The weights are 12 files totalling 893.9 MB in onnx, safetensors.
Every file
| File | Type | Size | SHA-256 |
|---|---|---|---|
| model.onnx | Weights | 133.3 MB | bd4bee8b055e |
| model.safetensors | Weights | 133.6 MB | 35bb6ef696cc |
| model_int8.onnx | Weights | 33.9 MB | de45baeaa7a4 |
| onnx/model.onnx | Weights | 133.2 MB | b078342c7af0 |
| onnx/model_bnb4.onnx | Weights | 60.2 MB | 9b664122c834 |
| onnx/model_fp16.onnx | Weights | 66.7 MB | 2165cdb0057d |
| onnx/model_int8.onnx | Weights | 34.0 MB | 250889c55509 |
| onnx/model_q4.onnx | Weights | 61.5 MB | 9e8a2fcc60df |
| onnx/model_q4f16.onnx | Weights | 36.2 MB | a5286e7efbc2 |
| onnx/model_quantized.onnx | Weights | 34.0 MB | 250889c55509 |
| onnx/model_uint8.onnx | Weights | 34.0 MB | fb1967fe95a4 |
| vespa_colbert.onnx | Weights | 133.3 MB | 9161e64cab96 |
| config.json | Configuration | 702 B | — |
| onnx_config.json | Configuration | 767 B | — |
| special_tokens_map.json | Configuration | 695 B | — |
| README.md | Documentation | 7.8 KB | — |
| artifact.metadata | Other | 2.2 KB | — |
| .gitattributes | Repository | 1.5 KB | — |
| tokenizer.json | Tokenizer | 711.4 KB | — |
| tokenizer_config.json | Tokenizer | 1.2 KB | — |
| vocab.txt | Tokenizer | 231.5 KB | — |
License and Download
- License
- apache-2.0
- Access
- Open weights, no gate
- Download size
- 893.9 MB
Released by Answer.AI through its official repository on Hugging Face. Read the license.
Built From
- Described by arXiv:2407.20750
Memory Requirements
| Precision | Weights in memory |
|---|---|
| As published | 893.9 MB |
| 16-bit | 0.1 GB |
| 8-bit | 0.0 GB |
| 4-bit | 0.0 GB |
Weights only, from the published parameter count; the key-value cache and runtime add to this.
Questions About answerai-colbert-small-v1
How much GPU memory does answerai-colbert-small-v1 need?
About 0.1 GB at 16-bit and 0 GB at 4-bit: the weights (33M parameters) plus a working margin. A long context needs more.
What is the cheapest GPU to run answerai-colbert-small-v1 on?
At 16-bit, 1x MI300X from $1.85 an hour; at 4-bit, 1x MI300X from $1.85 an hour, at the lowest on-demand prices the SAVRN Index lists.
Can I use answerai-colbert-small-v1 commercially?
Yes. answerai-colbert-small-v1 is released under Apache License 2.0. The Apache License 2.0 is a permissive open-source license. It permits commercial use, modification and redistribution. It requires keeping the license and copyright notices and any NOTICE file, stating significant changes, and it includes an express patent grant from contributors.
What is answerai-colbert-small-v1's context length?
512 tokens, from the maximum position embeddings in its published configuration.