A hidden Markov model distilled from mhdang/gelatwo-common-gen-gpt2-large, used as the tractable proposal for keyword-constrained generation on CommonGen in Mitigating Bias in Locally Constrained Decoding via Tractable Proposals (arXiv:2606.01926).
Runs On
What it takes to serve gelatwo-common-gen-dense-4096 (223M parameters): the memory its weights need at each precision, and the cheapest way to rent enough data-center GPUs to hold them.
| Precision | Weights | Memory needed | Cheapest setup | Per hour | Also fits |
|---|---|---|---|---|---|
| 16-bit | 0.4 GB | 0.5 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
| 8-bit | 0.2 GB | 0.3 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
| 4-bit | 0.1 GB | 0.1 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
Memory is the weights at that precision plus 20% for the runtime and a short context; a long context needs more. Prices are the lowest on-demand hourly rates in the SAVRN Index, read Sep 18, 2026.
Model Card
By Meihua Dang, published under mit, revision e662b9e5063a.
A hidden Markov model distilled from mhdang/gelatwo-common-gen-gpt2-large, used as the tractable proposal for keyword-constrained generation on CommonGen in Mitigating Bias in Locally Constrained Decoding via Tractable Proposals (arXiv:2606.01926). Locally constrained decoding (LCD) enforces a constraint by masking next tokens one step at a time, which biases the samples it draws. P-GCD instead combines this HMM with a tensorized finite automaton to form a proposal that carries both the logical and the probabilistic information about the target distribution, and samples from it with sequential Monte Carlo. The HMM supplies the lookahead the This is not a fine-tune of the base LM and shares…
Read Meihua Dang's full model card
A hidden Markov model distilled from mhdang/gelatwo-common-gen-gpt2-large, used as the tractable
proposal for keyword-constrained generation on CommonGen in Mitigating Bias in Locally Constrained Decoding via
Tractable Proposals (arXiv:2606.01926).
Locally constrained decoding (LCD) enforces a constraint by masking next tokens one step at a time, which biases the samples it draws. P-GCD instead combines this HMM with a tensorized finite automaton to form a proposal that carries both the logical and the probabilistic information about the target distribution, and samples from it with sequential Monte Carlo. The HMM supplies the lookahead the automaton alone cannot: how likely the constraint is to be satisfiable from here.
Architecture
| Transition | Dense |
| Hidden states | 4096 |
| Vocabulary | 50257 (matches mhdang/gelatwo-common-gen-gpt2-large) |
| Class | gelatwo.hmm.HMM |
This is not a fine-tune of the base LM and shares none of its weights: it is a separate model, trained by EM on sequences sampled from that LM, and it reuses the LM's tokenizer so their vocabularies line up at decoding time.
Usage
Install the reference implementation, github.com/MhDang/gelatwo, then:
from gelatwo.hmm import HMM
hmm = HMM.from_pretrained("mhdang/gelatwo-common-gen-dense-4096")
configs/common-gen.yaml in the reference implementation already points at
this checkpoint, so the CommonGen runs need no override.
Citation
@inproceedings{dang2026mitigating,
title = {Mitigating Bias in Locally Constrained Decoding via Tractable Proposals},
author = {Dang, Meihua and Song, Linxin and Zhang, Honghua and Zhao, Jieyu and Van den Broeck, Guy and Ermon, Stefano},
booktitle = {Proceedings of the 43rd International Conference on Machine Learning (ICML)},
year = {2026},
}
Configuration
- Vocabulary size
- 50,257
Identity and Version
- Repository
- mhdang/gelatwo-common-gen-dense-4096
- Publisher
- Meihua Dang
- Task
- Not stated by the source
- Modality
- Other
- Library
- Not stated by the source
- Parameters
- 223M parameters
- Languages
- Not stated by the source
- Revision
- e662b9e5063ac80bca8f918796ba03659e184e95
- First published
- 2026-09-18
- Last updated
- 2026-09-18
Files and Weights
4 files, 890.6 MB in total. The weights are 1 file totalling 890.6 MB in safetensors.
Every file
| File | Type | Size | SHA-256 |
|---|---|---|---|
| model.safetensors | Weights | 890.6 MB | 2be4b211b652 |
| config.json | Configuration | 75 B | — |
| README.md | Documentation | 2.1 KB | — |
| .gitattributes | Repository | 1.5 KB | — |
License and Download
- License
- mit
- Access
- Open weights, no gate
- Download size
- 890.6 MB
Released by Meihua Dang through its official repository on Hugging Face. Read the license.
Built From
- Described by arXiv:2606.01926
Memory Requirements
| Precision | Weights in memory |
|---|---|
| As published | 890.6 MB |
| 16-bit | 0.4 GB |
| 8-bit | 0.2 GB |
| 4-bit | 0.1 GB |
Weights only, from the published parameter count; the key-value cache and runtime add to this.
Questions About gelatwo-common-gen-dense-4096
How much GPU memory does gelatwo-common-gen-dense-4096 need?
About 0.5 GB at 16-bit and 0.1 GB at 4-bit: the weights (223M parameters) plus a working margin. A long context needs more.
What is the cheapest GPU to run gelatwo-common-gen-dense-4096 on?
At 16-bit, 1x MI300X from $1.85 an hour; at 4-bit, 1x MI300X from $1.85 an hour, at the lowest on-demand prices the SAVRN Index lists.
Can I use gelatwo-common-gen-dense-4096 commercially?
Yes. gelatwo-common-gen-dense-4096 is released under MIT License. The MIT License is a short permissive license. It permits commercial use, modification and redistribution, provided the copyright notice and permission notice are included.