SAVRN
Search Contact SAVRN

Open-weight model · Feature extraction

MicroGlot

by Athan Zhaohong Li athanzli/MicroGlot

MicroGlot is an open-weight model for feature extraction from Athan Zhaohong Li, released under Creative Commons Attribution 4.0. It has 3B parameters. At 16-bit it needs about 7.2 GB of GPU memory, which fits on 1x MI300X from $1.85 an hour, at the lowest prices in the SAVRN Index. It draws 327 downloads a month.

A taxonomy-informed sparse DNA foundation model for microbial genomics. MicroGlot is a 23-layer decoder-only mixture-of-experts transformer pretrained on 378.3 billion nucleotides from 3.70 million sequences across 99,700 microbial species, spanning bacteria…

Parameters3B
Context—
Weights17.9 GB
Licensecc-by-4.0
AccessOpen weights
Monthly Downloads327

Runs On

What it takes to serve MicroGlot (3B parameters): the memory its weights need at each precision, and the cheapest way to rent enough data-center GPUs to hold them.

PrecisionWeightsMemory neededCheapest setupPer hourAlso fits
16-bit 6.0 GB 7.2 GB 1x MI300X (192 GB)
Vultr
$1.85 1x H100 $1.99 · 1x MI325X $2.00
8-bit 3.0 GB 3.6 GB 1x MI300X (192 GB)
Vultr
$1.85 1x H100 $1.99 · 1x MI325X $2.00
4-bit 1.5 GB 1.8 GB 1x MI300X (192 GB)
Vultr
$1.85 1x H100 $1.99 · 1x MI325X $2.00

Memory is the weights at that precision plus 20% for the runtime and a short context; a long context needs more. Prices are the lowest on-demand hourly rates in the SAVRN Index, read Oct 7, 2026.

MicroGlot on every accelerator the SAVRN Index prices, at every precision

Model Card

By Athan Zhaohong Li, published under cc-by-4.0, revision 3a39eadab165.

A taxonomy-informed sparse DNA foundation model for microbial genomics.

MicroGlot is a 23-layer decoder-only mixture-of-experts transformer pretrained on 378.3 billion nucleotides from 3.70 million sequences across 99,700 microbial species, spanning bacteria, archaea, fungi, protists, viruses and plasmids. It encodes the taxonomic hierarchy as hyperbolic (Poincaré) embeddings and uses them both as an input token and to steer expert routing.

Paper: A Taxonomy-Informed Sparse DNA Foundation Model for Microbial Genomics\ Code: github.com/athanzli/MicroGlot

Models

Model Input Use it when Load with
MicroGlot DNA and its species most of your sequences have a known species from_pretrained("athanzli/MicroGlot", ...)
MicroGlot-plain DNA most of your sequences have no known species from_pretrained("athanzli/MicroGlot", subfolder="plain", ...)

A species is known if it is one of the 99,700 pretraining species (check with tokenizer.has_species(name)). For taxonomic classification tasks, use MicroGlot-plain, as conditioning on the species would leak the label.

Model details

Read the full model card (895 words)

Configuration

Architecture
MicroGlotForCausalLM
Layers
23
Hidden size
1,024
Feed-forward size
2,816
Vocabulary size
8,192
Experts active per token
1
RoPE base
500000
Stored precision
bfloat16
Model type
microglot

Identity and Version

Repository
athanzli/MicroGlot
Publisher
Athan Zhaohong Li
Task
Feature extraction
Modality
Text
Library
transformers
Parameters
3B parameters
Languages
dna
Revision
3a39eadab16596b67643d57e2e7fddbdac66f311
First published
2026-09-21
Last updated
2026-10-04

Files and Weights

21 files, 17.9 GB in total. The weights are 3 files totalling 17.9 GB in safetensors.

Weights3 files · 17.9 GB
Configuration8 files · 99.8 KB
Tokenizer6 files · 1.7 MB
Documentation2 files · 9.3 KB
Other1 file · 2.4 MB
Repository1 file · 1.5 KB
Every file
FileTypeSizeSHA-256
model.safetensorsWeights6.0 GB 6ebe0b54218a
plain/model.safetensorsWeights6.0 GB 256d8b6490a9
species_encoder/model.safetensorsWeights6.0 GB 2722cc080410
config.jsonConfiguration1.9 KB —
modeling_microglot.pyConfiguration81.3 KB —
plain/config.jsonConfiguration1.7 KB —
plain/special_tokens_map.jsonConfiguration417 B —
special_tokens_map.jsonConfiguration417 B —
species_encoder/config.jsonConfiguration1.7 KB —
species_encoder/special_tokens_map.jsonConfiguration417 B —
tokenization_microglot.pyConfiguration12.0 KB —
LICENSEDocumentation1.2 KB —
README.mdDocumentation8.1 KB —
species_vocab.txtOther2.4 MB —
.gitattributesRepository1.5 KB —
plain/tokenizer.jsonTokenizer578.8 KB —
plain/tokenizer_config.jsonTokenizer1.0 KB —
species_encoder/tokenizer.jsonTokenizer578.8 KB —
species_encoder/tokenizer_config.jsonTokenizer1.0 KB —
tokenizer.jsonTokenizer578.8 KB —
tokenizer_config.jsonTokenizer1.2 KB —

License and Download

License
cc-by-4.0
Access
Open weights, no gate
Download size
17.9 GB
Download from Athan Zhaohong Li

Released by Athan Zhaohong Li through its official repository on Hugging Face. Read the license.

Memory Requirements

PrecisionWeights in memory
As published17.9 GB
16-bit6.0 GB
8-bit3.0 GB
4-bit1.5 GB

Weights only, from the published parameter count; the key-value cache and runtime add to this.

Questions About MicroGlot

How much GPU memory does MicroGlot need?

About 7.2 GB at 16-bit and 1.8 GB at 4-bit: the weights (3B parameters) plus a working margin. A long context needs more.

What is the cheapest GPU to run MicroGlot on?

At 16-bit, 1x MI300X from $1.85 an hour; at 4-bit, 1x MI300X from $1.85 an hour, at the lowest on-demand prices the SAVRN Index lists.

Can I use MicroGlot commercially?

Yes. MicroGlot is released under Creative Commons Attribution 4.0. CC BY 4.0 permits sharing and adapting the work, including commercially, provided the creator is credited and changes are indicated.

Similar Models

Model · Feature extraction

Qwen3-Embedding-4B

Qwen

The Qwen3 Embedding model series is the latest proprietary model of the Qwen family, specifically designed for text embedding and ranking tasks. Building upon the dense foundational models of the Qwen3 series, it provides a comprehensive range of text embeddings and reranking models in various sizes (0.6B, 4B, and 8B). This series inherits the exceptional multilingual capabilities, long-text understanding, and reasoning skills of its foundational model. The Qwen3 Embedding series represents significant advancements in multiple text embedding and ranking tasks, including text retrieval, code retrieval, text classification, text clustering, and bitext mining. Exceptional Versatility: The…

Open weights apache-2.0 4B parameters 40,960 tokens sentence-transformers

Model · Feature extraction

Qwen3-Embedding-4B-W4A16-G128

Mou Geren

GPTQ Quantized Qwen/Qwen3-Embedding-4B with THUIR/T2Ranking and m-a-p/COIG-CQIA for calibration set. ~0.72% lost in C-MTEB. Evaluation performed with official code. pip install compressed-tensors optimum and auto-gptq / gptqmodel, then goto the official usage guide.

Open weights apache-2.0 4.1B parameters 40,960 tokens sentence-transformers

Model · Feature extraction

tiny-audio-granite-qwen

Alex Kroman

This is the model card of a transformers model that has been pushed on the Hub. Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. Use the code below to get started with the model. Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).

Open weights 1.9B parameters transformers

Model · Feature extraction

Vela-1.0-Omni-Mini

vLLM Semantic Router

Text, images, speech and environmental sounds in one embedding space. Vela Omni Mini supports multimodal search, routing and clustering with normalized vectors that can be compared directly. Scores are 0–100; higher is better. The comparison uses the same held-out evaluation examples, complete retrieval pools and 128-token text cap for both models. Bold marks improvement over the original large model. These known test pools are reused across releases. The common protocol uses labeled TRAIN prototypes for text classification and all matching positives for retrieval; it is separate from official MTEB classification. All 14 metrics, Macro-F1, exact counts and uncertainty. The primary metric…

Open weights apache-2.0 1.4B parameters pytorch

Model Type: Feature backbone; Params: 1136M (giant); Image size: 224 x 224 x 3; Patch size: 14 x 14 x 3; Repository: github.com:marrlab/DinoBloom; Original Weights: Zenodo.

Open weights apache-2.0 1.1B parameters timm

Model · Feature extraction

voxtral-mini-audio-extractor

Vxltxr Llc

A lightweight standalone audio feature extractor derived from Voxtral-Mini-3B-2507. This repository isolates the Whisper-based audio encoder and multi-modal projector from the original 3B language model. The extracted module is intended for offline audio preprocessing, dataset preparation, feature caching, and downstream multimodal training pipelines. The language-model decoder is not included. It does not contain the 3B LLaMA language-model decoder. For offline preprocessing, loading the complete multimodal language model is unnecessary when the only required output is the projected audio representation. This standalone checkpoint can therefore be used as a dedicated audio feature…

Open weights apache-2.0 662M parameters