# meta-encoder by AI at Meta: Open-Weight Model Source: https://savrn.com/models/meta-encoder Markdown alternate of the page above; the site index is https://savrn.com/llms.txt --- ## Runs On What it takes to serve meta-encoder (29.8B parameters): the memory its weights need at each precision, and the cheapest way to rent enough data-center GPUs to hold them. | Precision | Weights | Memory needed | Cheapest setup | Per hour | Also fits | | --- | --- | --- | --- | --- | --- | | 16-bit | 59.6 GB | 71.5 GB | 1x [MI300X](https://savrn.com/ai-index/pricing/gpus/mi300x) (192 GB) Vultr | $1.85 | [1x H100](https://savrn.com/ai-index/pricing/gpus/h100) $1.99 · [1x MI325X](https://savrn.com/ai-index/pricing/gpus/mi325x) $2.00 | | 8-bit | 29.8 GB | 35.7 GB | 1x [MI300X](https://savrn.com/ai-index/pricing/gpus/mi300x) (192 GB) Vultr | $1.85 | [1x H100](https://savrn.com/ai-index/pricing/gpus/h100) $1.99 · [1x MI325X](https://savrn.com/ai-index/pricing/gpus/mi325x) $2.00 | | 4-bit | 14.9 GB | 17.9 GB | 1x [MI300X](https://savrn.com/ai-index/pricing/gpus/mi300x) (192 GB) Vultr | $1.85 | [1x H100](https://savrn.com/ai-index/pricing/gpus/h100) $1.99 · [1x MI325X](https://savrn.com/ai-index/pricing/gpus/mi325x) $2.00 | Memory is the weights at that precision plus 20% for the runtime and a short context; a long context needs more. Prices are the lowest on-demand hourly rates in the [SAVRN Index](https://savrn.com/ai-index/pricing/gpus), read Oct 9, 2026. [meta-encoder on every accelerator the SAVRN Index prices, at every precision](https://savrn.com/models/meta-encoder/gpus) ## Model Card By AI at Meta, published under apache-2.0, revision 3d0df704aa5b. ### MetaEncoder-30B Multimodal System One Encoder with a natural language interface for instruction-following encoding: give it a task and a list of candidates, and it scores the candidates by task specification. Both a task and each candidate are expressed in natural language — instructions, queries, questions, state descriptions, criteria — with image and video as side information. Key features: - Natural language interface. No rigid schemas. Describe the task and the candidates in free-form text. - Highly-efficient cacheable representations. Candidates and task are encoded separately by prompt instructions that ground each other. - Scales to arbitrarily large candidate sets. Matching is an inner product over representations, so an ANN index can serve millions of candidates without re-running the model. The model is built by contrastively fine-tuning [meta-models/Muse-Glimmer-30B](https://huggingface.co/meta-models/Muse-Glimmer-30B). It is described in the paper [MetaEncoder: Exploring the Limit of Bi-Encoders for Multimodal System One Decision Making with Natural Language Interface](https://arxiv.org/abs/2610.11316). ### Results [Read the full model card (1,050 words)](https://savrn.com/models/meta-encoder/card) ## Identity and Version Repository facebook/meta-encoder Publisher AI at Meta Task Feature extraction Modality Text Library transformers Parameters 29.8B parameters Languages Not stated by the source Revision 3d0df704aa5bf66d9c49da1393181f33e893e5f7 First published 2026-09-30 Last updated 2026-10-09 ## Files and Weights 20 files, 59.6 GB in total. The weights are 2 files totalling 59.6 GB in safetensors. Weights2 files · 59.6 GB Configuration6 files · 158.5 KB Tokenizer2 files · 28.2 MB Documentation5 files · 33.9 KB Other4 files · 25.9 KB Repository1 file · 143 B Every file | File | Type | Size | SHA-256 | | --- | --- | --- | --- | | model-00001-of-00002.safetensors | Weights | 49.9 GB | — | | model-00002-of-00002.safetensors | Weights | 9.7 GB | — | | config.json | Configuration | 5.2 KB | — | | generation_config.json | Configuration | 238 B | — | | model.safetensors.index.json | Configuration | 132.7 KB | — | | modeling_metaencoder.py | Configuration | 14.8 KB | — | | processor_config.json | Configuration | 1.1 KB | — | | vllm_metaencoder.py | Configuration | 4.6 KB | — | | LICENSE | Documentation | 11.4 KB | — | | NOTICE | Documentation | 2.2 KB | — | | README.md | Documentation | 9.5 KB | — | | USAGE.md | Documentation | 5.6 KB | — | | USAGE_POLICY.md | Documentation | 5.2 KB | — | | README.md.bak.2111 | Other | 7.7 KB | — | | benchmark.png | Other | 7.7 KB | — | | chat_template.jinja | Other | 10.0 KB | — | | requirements.txt | Other | 576 B | — | | .gitattributes | Repository | 143 B | — | | tokenizer.json | Tokenizer | 28.1 MB | — | | tokenizer_config.json | Tokenizer | 80.0 KB | — | ## License and Download License apache-2.0 Access Access requested at publisher Download size 59.6 GB [Request access from AI at Meta](https://huggingface.co/facebook/meta-encoder) AI at Meta grants access through its official repository on Hugging Face. [Read the license](https://www.apache.org/licenses/LICENSE-2.0). ## Built From - Derived from meta-models/Muse-Glimmer-30B - Described by arXiv:2610.11316 ## Memory Requirements | Precision | Weights in memory | | --- | --- | | As published | 59.6 GB | | 16-bit | 59.6 GB | | 8-bit | 29.8 GB | | 4-bit | 14.9 GB | Weights only, from the published parameter count; the key-value cache and runtime add to this. ## Questions About meta-encoder ### How much GPU memory does meta-encoder need? About 71.5 GB at 16-bit and 17.9 GB at 4-bit: the weights (29.8B parameters) plus a working margin. A long context needs more. ### What is the cheapest GPU to run meta-encoder on? At 16-bit, 1x MI300X from $1.85 an hour; at 4-bit, 1x MI300X from $1.85 an hour, at the lowest on-demand prices the SAVRN Index lists. ### Can I use meta-encoder commercially? Yes. meta-encoder is released under Apache License 2.0. The Apache License 2.0 is a permissive open-source license. It permits commercial use, modification and redistribution. It requires keeping the license and copyright notices and any NOTICE file, stating significant changes, and it includes an express patent grant from contributors. ## Similar Models Model · Feature extraction ### [matilda-jev-v1](https://savrn.com/models/matilda-jev-v1) [Maincode](https://savrn.com/model-publishers/maincode) Matilda-Jev is Maincode's one-pass decision model. It scores the options supplied in a choice, noul (yes/no), or ordered score question. It accepts text or JSON state and optional images. This is a decision checkpoint with a 255-option readout, not a text-generation checkpoint. This configuration edition uses MatildaJevModel, MatildaJevConfig, and MATILDA tokenizer/processor classes. Use the bundled runtime below, or load the custom AutoClasses with trustremotecode=True. The package includes the backbone weights, decision readout, tokenizer, preprocessing configuration and calibrated temperature. On AMD MI355X with Python 3.12, Transformers 5.17.0 and PyTorch 2.14.0: - 25/25 smoke-test… Open weights apache-2.0 26.1B parameters 262,144 tokens transformers [View model](https://savrn.com/models/matilda-jev-v1) Model · Feature extraction ### [webshop_8b_b8_1node_step150](https://savrn.com/models/webshop-8b-b8-1node-step150) [Longyy](https://savrn.com/model-publishers/loongyy) This model is a fine-tuned version of JetLM/SDAR-8B-Chat-b8 on the sdarwebshopbs4eighthreason dataset. The following hyperparameters were used during training: - learningrate: 1e-05 - trainbatchsize: 1 - evalbatchsize: 8 - distributedtype: multi-GPU - numdevices: 4 - totaltrainbatchsize: 4 - totalevalbatchsize: 32 - lrschedulertype: constantwithwarmup - lrschedulerwarmupratio: 0.03 - numepochs: 1.0 - Transformers 4.52.4 - Pytorch 2.9.1+cu129 - Datasets 3.6.0 - Tokenizers 0.21.1 Open weights other 8.2B parameters 32,768 tokens transformers [View model](https://savrn.com/models/webshop-8b-b8-1node-step150) Model · Feature extraction ### [CLM-v0.1-8B-MLX-8bit](https://savrn.com/models/clm-v0-1-8b-mlx-8bit) [Zhu Lin](https://savrn.com/model-publishers/czl) This is the 8bit variant. Also available: czl/CLM-v0.1-8B-MLX-4bit, czl/CLM-v0.1-8B-MLX-6bit, and the bf16 reference. The encoder half of Contrastive-LM/CLM-v0.1-8B, quantised for MLX on Apple Silicon. mlxlm is generate-only — it has no embeddings entrypoint, and mlxlm.server routes just /v1/completions, /v1/chat/completions, /v1/models and /health. So clmmlx supplies the pooling pass over mlxlm internals: Qwen3Model.call already returns self.norm(h), the post-final-RMSNorm hidden states, which is what vLLM's pooling runner returns in last-token mode. Agreement against a bf16 MLX reference of the same encoder in the same runtime, over 23,926 scored System One questions, through the real… Open weights apache-2.0 8.2B parameters 40,960 tokens mlx [View model](https://savrn.com/models/clm-v0-1-8b-mlx-8bit) Model · Feature extraction ### [CLM-v0.1-8B-MLX](https://savrn.com/models/clm-v0-1-8b-mlx) [Zhu Lin](https://savrn.com/model-publishers/czl) The encoder half of Contrastive-LM/CLM-v0.1-8B, quantised for MLX on Apple Silicon. One repository per bit width, matching mlx-community. Each has the weights at the repo root, so the Hub file browser lists every file with its size and mlxlm.load(" ") works. Agreement against a bf16 MLX reference of the same encoder in the same runtime, over 23,926 scored System One questions, through the real head stack (argmax(scale · cos), scale = 100.0 — cosine error is amplified 100×). Pass line. top-1 >= 1.0000 — the measured bf16-vs-bf16 noise floor of this corpus in this runtime — and top-1 (decisive) >= 0.995, where decisive means the reference's own top-1 led by more than 1 nat. A third condition… Open weights apache-2.0 8.2B parameters 40,960 tokens mlx [View model](https://savrn.com/models/clm-v0-1-8b-mlx) Model · Feature extraction ### [CLM-v0.1-8B-MLX-6bit](https://savrn.com/models/clm-v0-1-8b-mlx-6bit) [Zhu Lin](https://savrn.com/model-publishers/czl) This is the 6bit variant. Also available: czl/CLM-v0.1-8B-MLX-4bit, czl/CLM-v0.1-8B-MLX-8bit, and the bf16 reference. The encoder half of Contrastive-LM/CLM-v0.1-8B, quantised for MLX on Apple Silicon. mlxlm is generate-only — it has no embeddings entrypoint, and mlxlm.server routes just /v1/completions, /v1/chat/completions, /v1/models and /health. So clmmlx supplies the pooling pass over mlxlm internals: Qwen3Model.call already returns self.norm(h), the post-final-RMSNorm hidden states, which is what vLLM's pooling runner returns in last-token mode. Agreement against a bf16 MLX reference of the same encoder in the same runtime, over 23,926 scored System One questions, through the real… Open weights apache-2.0 8.2B parameters 40,960 tokens mlx [View model](https://savrn.com/models/clm-v0-1-8b-mlx-6bit) Model · Feature extraction ### [Qwen3-Embedding-8B](https://savrn.com/models/qwen3-embedding-8b) [Qwen](https://savrn.com/model-publishers/qwen) The Qwen3 Embedding model series is the latest proprietary model of the Qwen family, specifically designed for text embedding and ranking tasks. Building upon the dense foundational models of the Qwen3 series, it provides a comprehensive range of text embeddings and reranking models in various sizes (0.6B, 4B, and 8B). This series inherits the exceptional multilingual capabilities, long-text understanding, and reasoning skills of its foundational model. The Qwen3 Embedding series represents significant advancements in multiple text embedding and ranking tasks, including text retrieval, code retrieval, text classification, text clustering, and bitext mining. Exceptional Versatility: The… Open weights apache-2.0 7.6B parameters 40,960 tokens sentence-transformers [View model](https://savrn.com/models/qwen3-embedding-8b) ## AI at Meta [All models and datasets](https://savrn.com/model-publishers/facebook) ## Versions - [3d0df704aa5b](https://savrn.com/models/meta-encoder/versions/3d0df704aa5b) · current 2026-10-09 ## Explore More - [All feature extraction models](https://savrn.com/models/tasks/feature-extraction) - [All models under apache-2.0](https://savrn.com/models/licenses/apache-2-0) - [Model comparisons](https://savrn.com/models/comparisons) - [The model directory](https://savrn.com/models) - [Open model prices by host](https://savrn.com/ai-index/pricing/open-models) ## Source - Repository metadata, read 2026-10-09. - [AI at Meta website](https://ai.meta.com/resources/models-and-libraries/) - [Hugging Face record](https://huggingface.co/facebook/meta-encoder) - [How the hub is built](https://savrn.com/model-hub/methodology)