Native MLX conversion of ewin-reg/WeMM-Embedding-2B-Quantized — the hybrid FP8-protected attention + group-16 INT4 checkpoint — that preserves its mixed-precision allocation instead of flattening everything to one precision. The source checkpoint is curvature-guided: it deliberately keeps the precision-sensitive path at ~8-bit (FP8 E4M3) and compresses only the quantization-robust masses to INT4. Uniform 4-bit quantization destroys that — the source README itself documents softmax attention collapse (DKL >= 7.50) when Q/K go to 4 bits. This build mirrors the Two deviations from the source, both improvements or forced by MLX: - 8-bit group-64 is finer than the source's per-tensor-scalar FP8…
Open-weight model · Feature extraction
MOSS-Audio-Tokenizer-v2
by Le Jin Jinstudio/MOSS-Audio-Tokenizer-v2
MOSS-Audio-Tokenizer-v2 is an open-weight model for feature extraction from Le Jin, released under Apache License 2.0. It has 2.1B parameters. At 16-bit it needs about 5.1 GB of GPU memory, which fits on 1x MI300X from $1.85 an hour, at the lowest prices in the SAVRN Index.
Parameters2.1B
Context—
Weights—
Licenseapache-2.0
AccessOpen weights
Monthly Downloads—
Runs On
What it takes to serve MOSS-Audio-Tokenizer-v2 (2.1B parameters): the memory its weights need at each precision, and the cheapest way to rent enough data-center GPUs to hold them.
| Precision | Weights | Memory needed | Cheapest setup | Per hour | Also fits |
|---|---|---|---|---|---|
| 16-bit | 4.2 GB | 5.1 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
| 8-bit | 2.1 GB | 2.5 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
| 4-bit | 1.1 GB | 1.3 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
Memory is the weights at that precision plus 20% for the runtime and a short context; a long context needs more. Prices are the lowest on-demand hourly rates in the SAVRN Index, read Oct 20, 2026.
MOSS-Audio-Tokenizer-v2 on every accelerator the SAVRN Index prices, at every precision
Model Card
Identity and Version
- Repository
- Jinstudio/MOSS-Audio-Tokenizer-v2
- Publisher
- Le Jin
- Task
- Feature extraction
- Modality
- Text
- Library
- transformers
- Parameters
- 2.1B parameters
- Languages
- Not stated by the source
- Revision
- 9493fc55d183f739cb35c895ca5f54d1dc01861e
- First published
- 2026-10-20
- Last updated
- 2026-10-20
License and Download
- License
- apache-2.0
- Access
- Open weights, no gate
Download from Le Jin
Released by Le Jin through its official repository on Hugging Face. Read the license.
Built From
- Described by arXiv:2602.10934
Memory Requirements
| Precision | Weights in memory |
|---|---|
| 16-bit | 4.2 GB |
| 8-bit | 2.1 GB |
| 4-bit | 1.1 GB |
Weights only, from the published parameter count; the key-value cache and runtime add to this.
Questions About MOSS-Audio-Tokenizer-v2
How much GPU memory does MOSS-Audio-Tokenizer-v2 need?
About 5.1 GB at 16-bit and 1.3 GB at 4-bit: the weights (2.1B parameters) plus a working margin. A long context needs more.
What is the cheapest GPU to run MOSS-Audio-Tokenizer-v2 on?
At 16-bit, 1x MI300X from $1.85 an hour; at 4-bit, 1x MI300X from $1.85 an hour, at the lowest on-demand prices the SAVRN Index lists.
Can I use MOSS-Audio-Tokenizer-v2 commercially?
Yes. MOSS-Audio-Tokenizer-v2 is released under Apache License 2.0. The Apache License 2.0 is a permissive open-source license. It permits commercial use, modification and redistribution. It requires keeping the license and copyright notices and any NOTICE file, stating significant changes, and it includes an express patent grant from contributors.
Similar Models
This is the model card of a transformers model that has been pushed on the Hub. Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. Use the code below to get started with the model. Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).
Average score vs. parameter count across image (MIEB-Lite), video (MMEB-V), and audio (MAEB) benchmarks — jina-v5-omni-nano and jina-v5-omni-small define the open-weight frontier (Table 1 in the ArXiv report). jina-embeddings-v5-omni-small is a multimodal embedding model that accepts text, images, video, and audio and produces embeddings in a shared vector space aligned with the text-only jinaai/jina-embeddings-v5-text-small — so you can index with text and query with any modality, no reindexing. For a more compact alternative, see jinaai/jina-embeddings-v5-omni-nano. This is the base repository — it holds all task adapters (retrieval, classification, clustering, text-matching). For a…
Text, images, speech and environmental sounds in one embedding space. Vela Omni Mini supports multimodal search, routing and clustering with normalized vectors that can be compared directly. Scores are 0–100; higher is better. The comparison uses the same held-out evaluation examples, complete retrieval pools and 128-token text cap for both models. Bold marks improvement over the original large model. These known test pools are reused across releases. The common protocol uses labeled TRAIN prototypes for text classification and all matching positives for retrieval; it is separate from official MTEB classification. All 14 metrics, Macro-F1, exact counts and uncertainty. The primary metric…