h2o-lightning-31b is a model for image and text to text from H2O.ai, released under other (access requested at publisher). It has 31.3B parameters. At 16-bit it needs about 75.1 GB of GPU memory, which fits on 1x MI300X from $1.85 an hour, at the lowest prices in the SAVRN Index. It draws 404 downloads a month.
H2O-Lightning-31B is a 31B-parameter decision model from H2O.ai, built on google/gemma-4-31B-it. It answers typed decision questions about a record (a document, a ticket, a policy, a conversation), and about images that come with the record.
Parameters31.3B
Context—
Weights62.5 GB
Licenseother
AccessAccess requested at publisher
Monthly Downloads404
Runs On
What it takes to serve h2o-lightning-31b (31.3B parameters): the memory its weights need at each precision, and the cheapest way to rent enough data-center GPUs to hold them.
| Precision | Weights | Memory needed | Cheapest setup | Per hour | Also fits |
|---|---|---|---|---|---|
| 16-bit | 62.5 GB | 75.1 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
| 8-bit | 31.3 GB | 37.5 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
| 4-bit | 15.6 GB | 18.8 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
Memory is the weights at that precision plus 20% for the runtime and a short context; a long context needs more. Prices are the lowest on-demand hourly rates in the SAVRN Index, read Oct 23, 2026.
h2o-lightning-31b on every accelerator the SAVRN Index prices, at every precision
Model Card
H2O-Lightning-31B is a 31B-parameter decision model from H2O.ai, built on google/gemma-4-31B-it. It answers typed decision questions about a record (a document, a ticket, a policy, a conversation), and about images that come with the record. It returns a - yes/no (noul): gives the probability that a statement is true. - #1 of 167 — Intelligence (open and API models within the board's cost and speed caps), JevBench board release v1.6.39, 2026-10-22; ahead of Chon-1.0-12B (79.86) (details) - #2 of 196 — Composite (open-weight models; behind only our H2O-Lightning-4B), JevBench board release v1.6.39, 2026-10-22; ahead of Torchcast Decision 4B (73.03) (details) - #1 of 65 — Capability (all…
Excerpt from the card by H2O.ai, licensed other.
Identity and Version
- Repository
- h2oai/h2o-lightning-31b
- Publisher
- H2O.ai
- Task
- Image and text to text
- Modality
- Image and text
- Library
- transformers
- Parameters
- 31.3B parameters
- Languages
- Not stated by the source
- Revision
- 69c3d789e061d663964d08a0c4a04aee603e6219
- First published
- 2026-10-20
- Last updated
- 2026-10-23
Files and Weights
46 files, 62.6 GB in total. The weights are 2 files totalling 62.5 GB in safetensors.
Weights2 files · 62.5 GB
Configuration16 files · 659.8 KB
Tokenizer2 files · 32.2 MB
Documentation7 files · 105.7 KB
Other18 files · 3.3 MB
Repository1 file · 2.4 KB
Every file
| File | Type | Size | SHA-256 |
|---|---|---|---|
| model-00001-of-00002.safetensors | Weights | 49.9 GB | — |
| model-00002-of-00002.safetensors | Weights | 12.6 GB | — |
| assets.py | Configuration | 9.7 KB | — |
| box.py | Configuration | 19.0 KB | — |
| config.json | Configuration | 5.5 KB | — |
| di_tools/make_shards.py | Configuration | 1.4 KB | — |
| generation_config.json | Configuration | 208 B | — |
| lightning_31b_shim.py | Configuration | 213.1 KB | — |
| model.safetensors.index.json | Configuration | 120.2 KB | — |
| pooled_readout.py | Configuration | 29.6 KB | — |
| processor_config.json | Configuration | 1.7 KB | — |
| selector_v2.py | Configuration | 13.5 KB | — |
| serve_config.json | Configuration | 11.8 KB | — |
| serve_config_two_way.json | Configuration | 16.6 KB | — |
| state_rag.py | Configuration | 4.2 KB | — |
| test_security.py | Configuration | 10.6 KB | — |
| test_shim.py | Configuration | 195.1 KB | — |
| warmup.py | Configuration | 7.6 KB | — |
| LICENSE | Documentation | 12.4 KB | — |
| LICENSE-APACHE-2.0 | Documentation | 11.3 KB | — |
| LICENSE-cygnet | Documentation | 1.1 KB | — |
| LICENSE-flexdm | Documentation | 1.1 KB | — |
| LICENSE-wcole3 | Documentation | 1.1 KB | — |
| NOTICE | Documentation | 3.4 KB | — |
| README.md | Documentation | 75.3 KB | — |
| assets/banner.png | Other | 1.6 MB | — |
| assets/di_board_31b.png | Other | 479.6 KB | — |
| assets/di_board_31b.svg | Other | 51.3 KB | — |
| assets/jbboard_31b_capability_all_2026-10-22.png | Other | 175.7 KB | — |
| assets/jbboard_31b_capability_open_2026-10-22.png | Other | 154.2 KB | — |
| assets/jbboard_31b_composite_all_2026-10-22.png | Other | 205.3 KB | — |
| assets/jbboard_31b_composite_open_2026-10-22.png | Other | 147.4 KB | — |
| assets/jbboard_31b_image_capability_all_2026-10-22.png | Other | 115.4 KB | — |
| assets/jbboard_31b_image_composite_all_2026-10-22.png | Other | 136.3 KB | — |
| assets/jbboard_31b_image_composite_open_2026-10-22.png | Other | 115.5 KB | — |
| chat_template.jinja | Other | 18.7 KB | — |
| di_tools/di_text_sharded.sh | Other | 1.3 KB | — |
| engine/package.SHA256SUMS | Other | 1.0 KB | — |
| engine/vllm_per_request_attention.SHA256SUMS | Other | 1.8 KB | — |
| example_receipt.png | Other | 31.1 KB | — |
| run_decision_index.sh | Other | 6.9 KB | — |
| run_default.sh | Other | 404 B | — |
| serve.sh | Other | 18.0 KB | — |
| .gitattributes | Repository | 2.4 KB | — |
| tokenizer.json | Tokenizer | 32.2 MB | — |
| tokenizer_config.json | Tokenizer | 3.1 KB | — |
License and Download
- License
- other
- Access
- Access requested at publisher
- Download size
- 62.5 GB
Request access from H2O.ai
H2O.ai grants access through its official repository on Hugging Face. Read the license.
Built From
- Derived from google/gemma-4-31B-it
Memory Requirements
| Precision | Weights in memory |
|---|---|
| As published | 62.5 GB |
| 16-bit | 62.5 GB |
| 8-bit | 31.3 GB |
| 4-bit | 15.6 GB |
Weights only, from the published parameter count; the key-value cache and runtime add to this.
Questions About h2o-lightning-31b
How much GPU memory does h2o-lightning-31b need?
About 75.1 GB at 16-bit and 18.8 GB at 4-bit: the weights (31.3B parameters) plus a working margin. A long context needs more.
What is the cheapest GPU to run h2o-lightning-31b on?
At 16-bit, 1x MI300X from $1.85 an hour; at 4-bit, 1x MI300X from $1.85 an hour, at the lowest on-demand prices the SAVRN Index lists.
What license is h2o-lightning-31b released under?
other, as its publisher declares it. Read the license text before commercial use.
Similar Models
Gemma is a family of open models built by Google DeepMind. Gemma 4 models are multimodal, handling text and image input (with audio supported on E2B, E4B, and 12B) and generating text output. This release includes open-weights models in both pre-trained and instruction-tuned variants. Gemma 4 features a context window of up to 256K tokens and maintains multilingual support in over 140 languages. Featuring both Dense and Mixture-of-Experts (MoE) architectures, Gemma 4 is well-suited for tasks like text generation, coding, and reasoning. The models are available in five distinct sizes: E2B, E4B, 12B, 26B A4B, and 31B. Their diverse sizes make them deployable in environments ranging from…
Gemma is a family of open models built by Google DeepMind. Gemma 4 models are multimodal, handling text and image input (with audio supported on E2B, E4B, and 12B) and generating text output. This release includes open-weights models in both pre-trained and instruction-tuned variants. Gemma 4 features a context window of up to 256K tokens and maintains multilingual support in over 140 languages. Featuring both Dense and Mixture-of-Experts (MoE) architectures, Gemma 4 is well-suited for tasks like text generation, coding, and reasoning. The models are available in five distinct sizes: E2B, E4B, 12B, 26B A4B, and 31B. Their diverse sizes make them deployable in environments ranging from…
Quantized version of Ateron/Gemma-4-Writers-31B. Offline-quantized W8A16 FP8 in the float-quantized format: weights in float8e4m3fn with per-output-channel symmetric scales, activations kept in bf16/fp16. For each linear layer, every output row gets its own scale starting from amax / 448, refined by an MSE clip search over ~9 clip fractions (0.8–1.0× amax) picking the lowest-error scale per row. Weights are then quantized q = e4m3(w / scale) with round-to-nearest and saturation. This per-channel + clipping scheme gives better SNR than vLLM's online per-tensor --quantization fp8 path. Only 2D linear projection weights are quantized (attention q/k/v/o, MLP gate/up/down). Embeddings, norms…
Quantized version of Ateron/Gemma-4-Writers-31B-V2. Offline-quantized W8A16 FP8 in the float-quantized format: weights in float8e4m3fn with per-output-channel symmetric scales, activations kept in bf16/fp16. For each linear layer, every output row gets its own scale starting from amax / 448, refined by an MSE clip search over ~9 clip fractions (0.8–1.0× amax) picking the lowest-error scale per row. Weights are then quantized q = e4m3(w / scale) with round-to-nearest and saturation. This per-channel + clipping scheme gives better SNR than vLLM's online per-tensor --quantization fp8 path. Only 2D linear projection weights are quantized (attention q/k/v/o, MLP gate/up/down). Embeddings, norms…
