Open-weight model
gelatwo-xlam-json-monarch-16384
by Meihua Dang mhdang/gelatwo-xlam-json-monarch-16384
A hidden Markov model distilled from meta-llama/Llama-3.1-8B-Instruct, used as the tractable proposal for constrained tool calling on xLAM (JSON template) in Mitigating Bias in Locally Constrained Decoding via Tractable Proposals (arXiv:2606.01926).
Runs On
What it takes to serve gelatwo-xlam-json-monarch-16384 (2.1B parameters): the memory its weights need at each precision, and the cheapest way to rent enough data-center GPUs to hold them.
| Precision | Weights | Memory needed | Cheapest setup | Per hour | Also fits |
|---|---|---|---|---|---|
| 16-bit | 4.2 GB | 5.1 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
| 8-bit | 2.1 GB | 2.5 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
| 4-bit | 1.1 GB | 1.3 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
Memory is the weights at that precision plus 20% for the runtime and a short context; a long context needs more. Prices are the lowest on-demand hourly rates in the SAVRN Index, read Sep 18, 2026.
Model Card
By Meihua Dang, published under mit, revision 6a30d5121d22.
A hidden Markov model distilled from meta-llama/Llama-3.1-8B-Instruct, used as the tractable proposal for constrained tool calling on xLAM (JSON template) in Mitigating Bias in Locally Constrained Decoding via Tractable Proposals (arXiv:2606.01926). Locally constrained decoding (LCD) enforces a constraint by masking next tokens one step at a time, which biases the samples it draws. P-GCD instead combines this HMM with a tensorized finite automaton to form a proposal that carries both the logical and the probabilistic information about the target distribution, and samples from it with sequential Monte Carlo. The HMM supplies the lookahead the This is not a fine-tune of the base LM and shares…
Read Meihua Dang's full model card
A hidden Markov model distilled from meta-llama/Llama-3.1-8B-Instruct, used as the tractable
proposal for constrained tool calling on xLAM (JSON template) in Mitigating Bias in Locally Constrained Decoding via
Tractable Proposals (arXiv:2606.01926).
Locally constrained decoding (LCD) enforces a constraint by masking next tokens one step at a time, which biases the samples it draws. P-GCD instead combines this HMM with a tensorized finite automaton to form a proposal that carries both the logical and the probabilistic information about the target distribution, and samples from it with sequential Monte Carlo. The HMM supplies the lookahead the automaton alone cannot: how likely the constraint is to be satisfiable from here.
Architecture
| Transition | Monarch-factorized, 128x128 blocks |
| Hidden states | 16384 |
| Vocabulary | 128256 (matches meta-llama/Llama-3.1-8B-Instruct) |
| Class | gelatwo.hmm.MonarchHMM |
This is not a fine-tune of the base LM and shares none of its weights: it is a separate model, trained by EM on sequences sampled from that LM, and it reuses the LM's tokenizer so their vocabularies line up at decoding time.
Usage
Install the reference implementation, github.com/MhDang/gelatwo, then:
from gelatwo.hmm import MonarchHMM
hmm = MonarchHMM.from_pretrained("mhdang/gelatwo-xlam-json-monarch-16384")
configs/xlam.yaml in the reference implementation already points at
this checkpoint, so the xLAM (JSON template) runs need no override.
Citation
@inproceedings{dang2026mitigating,
title = {Mitigating Bias in Locally Constrained Decoding via Tractable Proposals},
author = {Dang, Meihua and Song, Linxin and Zhang, Honghua and Zhao, Jieyu and Van den Broeck, Guy and Ermon, Stefano},
booktitle = {Proceedings of the 43rd International Conference on Machine Learning (ICML)},
year = {2026},
}
Configuration
- Vocabulary size
- 128,256
Identity and Version
- Repository
- mhdang/gelatwo-xlam-json-monarch-16384
- Publisher
- Meihua Dang
- Task
- Not stated by the source
- Modality
- Other
- Library
- Not stated by the source
- Parameters
- 2.1B parameters
- Languages
- Not stated by the source
- Revision
- 6a30d5121d2227da4cc705a803ef009161384d08
- First published
- 2026-09-18
- Last updated
- 2026-09-18
Files and Weights
4 files, 8.4 GB in total. The weights are 1 file totalling 8.4 GB in safetensors.
Every file
| File | Type | Size | SHA-256 |
|---|---|---|---|
| model.safetensors | Weights | 8.4 GB | e6f3787c5fcb |
| config.json | Configuration | 142 B | — |
| README.md | Documentation | 2.1 KB | — |
| .gitattributes | Repository | 1.5 KB | — |
License and Download
- License
- mit
- Access
- Open weights, no gate
- Download size
- 8.4 GB
Released by Meihua Dang through its official repository on Hugging Face. Read the license.
Built From
- Described by arXiv:2606.01926
Memory Requirements
| Precision | Weights in memory |
|---|---|
| As published | 8.4 GB |
| 16-bit | 4.2 GB |
| 8-bit | 2.1 GB |
| 4-bit | 1.1 GB |
Weights only, from the published parameter count; the key-value cache and runtime add to this.
Questions About gelatwo-xlam-json-monarch-16384
How much GPU memory does gelatwo-xlam-json-monarch-16384 need?
About 5.1 GB at 16-bit and 1.3 GB at 4-bit: the weights (2.1B parameters) plus a working margin. A long context needs more.
What is the cheapest GPU to run gelatwo-xlam-json-monarch-16384 on?
At 16-bit, 1x MI300X from $1.85 an hour; at 4-bit, 1x MI300X from $1.85 an hour, at the lowest on-demand prices the SAVRN Index lists.
Can I use gelatwo-xlam-json-monarch-16384 commercially?
Yes. gelatwo-xlam-json-monarch-16384 is released under MIT License. The MIT License is a short permissive license. It permits commercial use, modification and redistribution, provided the copyright notice and permission notice are included.