Open-weight model
vision-language-pretraining-v1
by Joaquín A. González williamgwg/vision-language-pretraining-v1
vision-language-pretraining-v1 is an open-weight model from Joaquín A. González, released under Creative Commons Attribution 4.0. It has 24,832 parameters and a 128-token context. At 16-bit it needs about 0 GB of GPU memory, which fits on 1x MI300X from $1.85 an hour, at the lowest prices in the SAVRN Index.
This is an exploratory note for Vision Language Pretraining. It records the intended comparison, likely confounders, and reproducibility requirements before any benchmark result is reported.
Runs On
What it takes to serve vision-language-pretraining-v1 (24,832 parameters): the memory its weights need at each precision, and the cheapest way to rent enough data-center GPUs to hold them.
| Precision | Weights | Memory needed | Cheapest setup | Per hour | Also fits |
|---|---|---|---|---|---|
| 16-bit | 0.0 GB | 0.0 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
| 8-bit | 0.0 GB | 0.0 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
| 4-bit | 0.0 GB | 0.0 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
Memory is the weights at that precision plus 20% for the runtime and a short context; a long context needs more. Prices are the lowest on-demand hourly rates in the SAVRN Index, read Oct 18, 2026.
vision-language-pretraining-v1 on every accelerator the SAVRN Index prices, at every precision
Model Card
By Joaquín A. González, published under cc-by-4.0, revision 68af4c186b55.
This is an exploratory note for Vision Language Pretraining. It records the intended comparison, likely confounders, and reproducibility requirements before any benchmark result is reported. - the scope of the research question and likely confounders - a proposed comparison with matched baselines - concrete evaluation context such as task-appropriate public benchmarks named in the main note - reproducibility checks, failure modes, and open questions - topic-relevant references Start with papernotes.md for the full note. Sections labeled as plans or hypotheses should not be interpreted as experimental results. If results are added later, they should include dataset versions, commands, seeds…
Read Joaquín A. González's full model card
Notes on Vision Language Pretraining
Repository summary
This is an exploratory note for Vision Language Pretraining. It records the intended comparison, likely confounders, and reproducibility requirements before any benchmark result is reported.
What is covered
- the scope of the research question and likely confounders
- a proposed comparison with matched baselines
- concrete evaluation context such as task-appropriate public benchmarks named in the main note
- reproducibility checks, failure modes, and open questions
- topic-relevant references
How to read this repository
Start with paper_notes.md for the full note. Sections labeled as plans or hypotheses should not be interpreted as experimental results. If results are added later, they should include dataset versions, commands, seeds, hardware, and raw logs.
Scope and limitations
The note is intentionally exploratory. It does not claim benchmark improvements, completed ablations, released code, or a trained checkpoint. References and proposed datasets provide a starting point for verification rather than evidence that the study has already been run.
Files
paper_notes.md— primary artifactREADME.md— this documentation
License
Released under cc-by-4.0. Review the source-data terms separately when this repository is used with external datasets.
Configuration
- Architecture
- CustomResearchModel
- Context length (tokens)
- 128
- Layers
- 8
- Hidden size
- 192
- Feed-forward size
- 384
- Attention heads
- 4
- Model type
- transformer
Identity and Version
- Repository
- williamgwg/vision-language-pretraining-v1
- Publisher
- Joaquín A. González
- Task
- Not stated by the source
- Modality
- Other
- Library
- Not stated by the source
- Parameters
- 24,832 parameters
- Languages
- Not stated by the source
- Revision
- 68af4c186b55755af4f5f01245ce13c04240083d
- First published
- 2026-09-24
- Last updated
- 2026-09-24
Files and Weights
6 files, 106.5 KB in total. The weights are 1 file totalling 99.8 KB in safetensors.
Every file
| File | Type | Size | SHA-256 |
|---|---|---|---|
| model.safetensors | Weights | 99.8 KB | 1758a04d5fd5 |
| config.json | Configuration | 440 B | — |
| training_args.json | Configuration | 187 B | — |
| README.md | Documentation | 1.5 KB | — |
| paper_notes.md | Documentation | 3.1 KB | — |
| .gitattributes | Repository | 1.5 KB | — |
License and Download
- License
- cc-by-4.0
- Access
- Open weights, no gate
- Download size
- 99.8 KB
Released by Joaquín A. González through its official repository on Hugging Face. Read the license.
Memory Requirements
| Precision | Weights in memory |
|---|---|
| As published | 99.8 KB |
| 16-bit | 0.0 GB |
| 8-bit | 0.0 GB |
| 4-bit | 0.0 GB |
Weights only, from the published parameter count; the key-value cache and runtime add to this.
Questions About vision-language-pretraining-v1
How much GPU memory does vision-language-pretraining-v1 need?
About 0 GB at 16-bit and 0 GB at 4-bit: the weights (24,832 parameters) plus a working margin. A long context needs more.
What is the cheapest GPU to run vision-language-pretraining-v1 on?
At 16-bit, 1x MI300X from $1.85 an hour; at 4-bit, 1x MI300X from $1.85 an hour, at the lowest on-demand prices the SAVRN Index lists.
Can I use vision-language-pretraining-v1 commercially?
Yes. vision-language-pretraining-v1 is released under Creative Commons Attribution 4.0. CC BY 4.0 permits sharing and adapting the work, including commercially, provided the creator is credited and changes are indicated.
What is vision-language-pretraining-v1's context length?
128 tokens, from the maximum position embeddings in its published configuration.