SAVRN
Search Contact SAVRN

Open-weight model · Feature extraction

meta-encoder

by AI at Meta facebook/meta-encoder

meta-encoder is a model for feature extraction from AI at Meta, released under Apache License 2.0 (access requested at publisher). It has 29.8B parameters. At 16-bit it needs about 71.5 GB of GPU memory, which fits on 1x MI300X from $1.85 an hour, at the lowest prices in the SAVRN Index. It draws 22 downloads a month.

Multimodal System One Encoder with a natural language interface for instruction-following encoding: give it a task and a list of candidates, and it scores the candidates by task specification.

Parameters29.8B
Context—
Weights59.6 GB
Licenseapache-2.0
AccessAccess requested at publisher
Monthly Downloads22

Runs On

What it takes to serve meta-encoder (29.8B parameters): the memory its weights need at each precision, and the cheapest way to rent enough data-center GPUs to hold them.

PrecisionWeightsMemory neededCheapest setupPer hourAlso fits
16-bit 59.6 GB 71.5 GB 1x MI300X (192 GB)
Vultr
$1.85 1x H100 $1.99 · 1x MI325X $2.00
8-bit 29.8 GB 35.7 GB 1x MI300X (192 GB)
Vultr
$1.85 1x H100 $1.99 · 1x MI325X $2.00
4-bit 14.9 GB 17.9 GB 1x MI300X (192 GB)
Vultr
$1.85 1x H100 $1.99 · 1x MI325X $2.00

Memory is the weights at that precision plus 20% for the runtime and a short context; a long context needs more. Prices are the lowest on-demand hourly rates in the SAVRN Index, read Oct 9, 2026.

meta-encoder on every accelerator the SAVRN Index prices, at every precision

Model Card

By AI at Meta, published under apache-2.0, revision 3d0df704aa5b.

MetaEncoder-30B

Multimodal System One Encoder with a natural language interface for instruction-following encoding: give it a task and a list of candidates, and it scores the candidates by task specification. Both a task and each candidate are expressed in natural language — instructions, queries, questions, state descriptions, criteria — with image and video as side information.

Key features:

  • Natural language interface. No rigid schemas. Describe the task and the candidates in free-form text.
  • Highly-efficient cacheable representations. Candidates and task are encoded separately by prompt instructions that ground each other.
  • Scales to arbitrarily large candidate sets. Matching is an inner product over representations, so an ANN index can serve millions of candidates without re-running the model.

The model is built by contrastively fine-tuning meta-models/Muse-Glimmer-30B. It is described in the paper MetaEncoder: Exploring the Limit of Bi-Encoders for Multimodal System One Decision Making with Natural Language Interface.

Results

Read the full model card (1,050 words)

Identity and Version

Repository
facebook/meta-encoder
Publisher
AI at Meta
Task
Feature extraction
Modality
Text
Library
transformers
Parameters
29.8B parameters
Languages
Not stated by the source
Revision
3d0df704aa5bf66d9c49da1393181f33e893e5f7
First published
2026-09-30
Last updated
2026-10-09

Files and Weights

20 files, 59.6 GB in total. The weights are 2 files totalling 59.6 GB in safetensors.

Weights2 files · 59.6 GB
Configuration6 files · 158.5 KB
Tokenizer2 files · 28.2 MB
Documentation5 files · 33.9 KB
Other4 files · 25.9 KB
Repository1 file · 143 B
Every file
FileTypeSizeSHA-256
model-00001-of-00002.safetensorsWeights49.9 GB —
model-00002-of-00002.safetensorsWeights9.7 GB —
config.jsonConfiguration5.2 KB —
generation_config.jsonConfiguration238 B —
model.safetensors.index.jsonConfiguration132.7 KB —
modeling_metaencoder.pyConfiguration14.8 KB —
processor_config.jsonConfiguration1.1 KB —
vllm_metaencoder.pyConfiguration4.6 KB —
LICENSEDocumentation11.4 KB —
NOTICEDocumentation2.2 KB —
README.mdDocumentation9.5 KB —
USAGE.mdDocumentation5.6 KB —
USAGE_POLICY.mdDocumentation5.2 KB —
README.md.bak.2111Other7.7 KB —
benchmark.pngOther7.7 KB —
chat_template.jinjaOther10.0 KB —
requirements.txtOther576 B —
.gitattributesRepository143 B —
tokenizer.jsonTokenizer28.1 MB —
tokenizer_config.jsonTokenizer80.0 KB —

License and Download

License
apache-2.0
Access
Access requested at publisher
Download size
59.6 GB
Request access from AI at Meta

AI at Meta grants access through its official repository on Hugging Face. Read the license.

Built From

  • Derived from meta-models/Muse-Glimmer-30B
  • Described by arXiv:2610.11316

Memory Requirements

PrecisionWeights in memory
As published59.6 GB
16-bit59.6 GB
8-bit29.8 GB
4-bit14.9 GB

Weights only, from the published parameter count; the key-value cache and runtime add to this.

Questions About meta-encoder

How much GPU memory does meta-encoder need?

About 71.5 GB at 16-bit and 17.9 GB at 4-bit: the weights (29.8B parameters) plus a working margin. A long context needs more.

What is the cheapest GPU to run meta-encoder on?

At 16-bit, 1x MI300X from $1.85 an hour; at 4-bit, 1x MI300X from $1.85 an hour, at the lowest on-demand prices the SAVRN Index lists.

Can I use meta-encoder commercially?

Yes. meta-encoder is released under Apache License 2.0. The Apache License 2.0 is a permissive open-source license. It permits commercial use, modification and redistribution. It requires keeping the license and copyright notices and any NOTICE file, stating significant changes, and it includes an express patent grant from contributors.

Similar Models

Model · Feature extraction

matilda-jev-v1

Maincode

Matilda-Jev is Maincode's one-pass decision model. It scores the options supplied in a choice, noul (yes/no), or ordered score question. It accepts text or JSON state and optional images. This is a decision checkpoint with a 255-option readout, not a text-generation checkpoint. This configuration edition uses MatildaJevModel, MatildaJevConfig, and MATILDA tokenizer/processor classes. Use the bundled runtime below, or load the custom AutoClasses with trustremotecode=True. The package includes the backbone weights, decision readout, tokenizer, preprocessing configuration and calibrated temperature. On AMD MI355X with Python 3.12, Transformers 5.17.0 and PyTorch 2.14.0: - 25/25 smoke-test…

Open weights apache-2.0 26.1B parameters 262,144 tokens transformers

Model · Feature extraction

webshop_8b_b8_1node_step150

Longyy

This model is a fine-tuned version of JetLM/SDAR-8B-Chat-b8 on the sdarwebshopbs4eighthreason dataset. The following hyperparameters were used during training: - learningrate: 1e-05 - trainbatchsize: 1 - evalbatchsize: 8 - distributedtype: multi-GPU - numdevices: 4 - totaltrainbatchsize: 4 - totalevalbatchsize: 32 - lrschedulertype: constantwithwarmup - lrschedulerwarmupratio: 0.03 - numepochs: 1.0 - Transformers 4.52.4 - Pytorch 2.9.1+cu129 - Datasets 3.6.0 - Tokenizers 0.21.1

Open weights other 8.2B parameters 32,768 tokens transformers

Model · Feature extraction

CLM-v0.1-8B-MLX-8bit

Zhu Lin

This is the 8bit variant. Also available: czl/CLM-v0.1-8B-MLX-4bit, czl/CLM-v0.1-8B-MLX-6bit, and the bf16 reference. The encoder half of Contrastive-LM/CLM-v0.1-8B, quantised for MLX on Apple Silicon. mlxlm is generate-only — it has no embeddings entrypoint, and mlxlm.server routes just /v1/completions, /v1/chat/completions, /v1/models and /health. So clmmlx supplies the pooling pass over mlxlm internals: Qwen3Model.call already returns self.norm(h), the post-final-RMSNorm hidden states, which is what vLLM's pooling runner returns in last-token mode. Agreement against a bf16 MLX reference of the same encoder in the same runtime, over 23,926 scored System One questions, through the real…

Open weights apache-2.0 8.2B parameters 40,960 tokens mlx

Model · Feature extraction

CLM-v0.1-8B-MLX

Zhu Lin

The encoder half of Contrastive-LM/CLM-v0.1-8B, quantised for MLX on Apple Silicon. One repository per bit width, matching mlx-community. Each has the weights at the repo root, so the Hub file browser lists every file with its size and mlxlm.load(" ") works. Agreement against a bf16 MLX reference of the same encoder in the same runtime, over 23,926 scored System One questions, through the real head stack (argmax(scale · cos), scale = 100.0 — cosine error is amplified 100×). Pass line. top-1 >= 1.0000 — the measured bf16-vs-bf16 noise floor of this corpus in this runtime — and top-1 (decisive) >= 0.995, where decisive means the reference's own top-1 led by more than 1 nat. A third condition…

Open weights apache-2.0 8.2B parameters 40,960 tokens mlx

Model · Feature extraction

CLM-v0.1-8B-MLX-6bit

Zhu Lin

This is the 6bit variant. Also available: czl/CLM-v0.1-8B-MLX-4bit, czl/CLM-v0.1-8B-MLX-8bit, and the bf16 reference. The encoder half of Contrastive-LM/CLM-v0.1-8B, quantised for MLX on Apple Silicon. mlxlm is generate-only — it has no embeddings entrypoint, and mlxlm.server routes just /v1/completions, /v1/chat/completions, /v1/models and /health. So clmmlx supplies the pooling pass over mlxlm internals: Qwen3Model.call already returns self.norm(h), the post-final-RMSNorm hidden states, which is what vLLM's pooling runner returns in last-token mode. Agreement against a bf16 MLX reference of the same encoder in the same runtime, over 23,926 scored System One questions, through the real…

Open weights apache-2.0 8.2B parameters 40,960 tokens mlx

Model · Feature extraction

Qwen3-Embedding-8B

Qwen

The Qwen3 Embedding model series is the latest proprietary model of the Qwen family, specifically designed for text embedding and ranking tasks. Building upon the dense foundational models of the Qwen3 series, it provides a comprehensive range of text embeddings and reranking models in various sizes (0.6B, 4B, and 8B). This series inherits the exceptional multilingual capabilities, long-text understanding, and reasoning skills of its foundational model. The Qwen3 Embedding series represents significant advancements in multiple text embedding and ranking tasks, including text retrieval, code retrieval, text classification, text clustering, and bitext mining. Exceptional Versatility: The…

Open weights apache-2.0 7.6B parameters 40,960 tokens sentence-transformers