A hidden Markov model distilled from meta-llama/Llama-3.1-8B-Instruct, used as the tractable proposal for SQL-constrained text-to-SQL on Spider in Mitigating Bias in Locally Constrained Decoding via Tractable Proposals (arXiv:2606.01926).
Runs On
What it takes to serve gelatwo-spider-monarch-1024 (131M parameters): the memory its weights need at each precision, and the cheapest way to rent enough data-center GPUs to hold them.
| Precision | Weights | Memory needed | Cheapest setup | Per hour | Also fits |
|---|---|---|---|---|---|
| 16-bit | 0.3 GB | 0.3 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
| 8-bit | 0.1 GB | 0.2 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
| 4-bit | 0.1 GB | 0.1 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
Memory is the weights at that precision plus 20% for the runtime and a short context; a long context needs more. Prices are the lowest on-demand hourly rates in the SAVRN Index, read Sep 18, 2026.
Model Card
By Meihua Dang, published under mit, revision ed24571a5aac.
A hidden Markov model distilled from meta-llama/Llama-3.1-8B-Instruct, used as the tractable proposal for SQL-constrained text-to-SQL on Spider in Mitigating Bias in Locally Constrained Decoding via Tractable Proposals (arXiv:2606.01926). Locally constrained decoding (LCD) enforces a constraint by masking next tokens one step at a time, which biases the samples it draws. P-GCD instead combines this HMM with a tensorized finite automaton to form a proposal that carries both the logical and the probabilistic information about the target distribution, and samples from it with sequential Monte Carlo. The HMM supplies the lookahead the This is not a fine-tune of the base LM and shares none of…
Read Meihua Dang's full model card
A hidden Markov model distilled from meta-llama/Llama-3.1-8B-Instruct, used as the tractable
proposal for SQL-constrained text-to-SQL on Spider in Mitigating Bias in Locally Constrained Decoding via
Tractable Proposals (arXiv:2606.01926).
Locally constrained decoding (LCD) enforces a constraint by masking next tokens one step at a time, which biases the samples it draws. P-GCD instead combines this HMM with a tensorized finite automaton to form a proposal that carries both the logical and the probabilistic information about the target distribution, and samples from it with sequential Monte Carlo. The HMM supplies the lookahead the automaton alone cannot: how likely the constraint is to be satisfiable from here.
Architecture
| Transition | Monarch-factorized, 32x32 blocks |
| Hidden states | 1024 |
| Vocabulary | 128256 (matches meta-llama/Llama-3.1-8B-Instruct) |
| Class | gelatwo.hmm.MonarchHMM |
This is not a fine-tune of the base LM and shares none of its weights: it is a separate model, trained by EM on sequences sampled from that LM, and it reuses the LM's tokenizer so their vocabularies line up at decoding time.
Usage
Install the reference implementation, github.com/MhDang/gelatwo, then:
from gelatwo.hmm import MonarchHMM
hmm = MonarchHMM.from_pretrained("mhdang/gelatwo-spider-monarch-1024")
configs/spider.yaml in the reference implementation already points at
this checkpoint, so the Spider runs need no override.
Citation
@inproceedings{dang2026mitigating,
title = {Mitigating Bias in Locally Constrained Decoding via Tractable Proposals},
author = {Dang, Meihua and Song, Linxin and Zhang, Honghua and Zhao, Jieyu and Van den Broeck, Guy and Ermon, Stefano},
booktitle = {Proceedings of the 43rd International Conference on Machine Learning (ICML)},
year = {2026},
}
Configuration
- Vocabulary size
- 128,256
Identity and Version
- Repository
- mhdang/gelatwo-spider-monarch-1024
- Publisher
- Meihua Dang
- Task
- Not stated by the source
- Modality
- Other
- Library
- Not stated by the source
- Parameters
- 131M parameters
- Languages
- Not stated by the source
- Revision
- ed24571a5aac4ab79c65f7499ec8d6860706b0b2
- First published
- 2026-09-18
- Last updated
- 2026-09-18
Files and Weights
4 files, 525.6 MB in total. The weights are 1 file totalling 525.6 MB in safetensors.
Every file
| File | Type | Size | SHA-256 |
|---|---|---|---|
| model.safetensors | Weights | 525.6 MB | 7d9536e63bfe |
| config.json | Configuration | 139 B | — |
| README.md | Documentation | 2.1 KB | — |
| .gitattributes | Repository | 1.5 KB | — |
License and Download
- License
- mit
- Access
- Open weights, no gate
- Download size
- 525.6 MB
Released by Meihua Dang through its official repository on Hugging Face. Read the license.
Built From
- Described by arXiv:2606.01926
Memory Requirements
| Precision | Weights in memory |
|---|---|
| As published | 525.6 MB |
| 16-bit | 0.3 GB |
| 8-bit | 0.1 GB |
| 4-bit | 0.1 GB |
Weights only, from the published parameter count; the key-value cache and runtime add to this.
Questions About gelatwo-spider-monarch-1024
How much GPU memory does gelatwo-spider-monarch-1024 need?
About 0.3 GB at 16-bit and 0.1 GB at 4-bit: the weights (131M parameters) plus a working margin. A long context needs more.
What is the cheapest GPU to run gelatwo-spider-monarch-1024 on?
At 16-bit, 1x MI300X from $1.85 an hour; at 4-bit, 1x MI300X from $1.85 an hour, at the lowest on-demand prices the SAVRN Index lists.
Can I use gelatwo-spider-monarch-1024 commercially?
Yes. gelatwo-spider-monarch-1024 is released under MIT License. The MIT License is a short permissive license. It permits commercial use, modification and redistribution, provided the copyright notice and permission notice are included.