Open-weight model
efficientformer-retrieval-lab4
by Alvinng05 alvinng05/efficientformer-retrieval-lab4
efficientformer-retrieval-lab4 is an open-weight model from Alvinng05, released under Apache License 2.0. It has 24,832 parameters and a 256-token context. At 16-bit it needs about 0 GB of GPU memory, which fits on 1x MI300X from $1.85 an hour, at the lowest prices in the SAVRN Index.
This is an experimental Efficientformer codebase for Retrieval. It keeps the base setup intentionally manageable so architecture changes can be inspected before a full training run.
Runs On
What it takes to serve efficientformer-retrieval-lab4 (24,832 parameters): the memory its weights need at each precision, and the cheapest way to rent enough data-center GPUs to hold them.
| Precision | Weights | Memory needed | Cheapest setup | Per hour | Also fits |
|---|---|---|---|---|---|
| 16-bit | 0.0 GB | 0.0 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
| 8-bit | 0.0 GB | 0.0 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
| 4-bit | 0.0 GB | 0.0 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
Memory is the weights at that precision plus 20% for the runtime and a short context; a long context needs more. Prices are the lowest on-demand hourly rates in the SAVRN Index, read Oct 18, 2026.
efficientformer-retrieval-lab4 on every accelerator the SAVRN Index prices, at every precision
Model Card
By Alvinng05, published under apache-2.0, revision 59036c9321b2.
This is an experimental Efficientformer codebase for Retrieval. It keeps the base setup intentionally manageable so architecture changes can be inspected before a full training run. - The Python file contains the model and runnable example or training entry point. - config.json records the generated architecture settings. - trainingargs.json records the default experiment recipe. - model.safetensors is a valid initialization checkpoint for smoke tests; it is not presented as a trained benchmark checkpoint. - No benchmark score is claimed in this repository. The included configuration uses rmsprop with a onecycle schedule. These are starting values in the script, not evidence of a completed…
Read Alvinng05's full model card
Efficientformer for Retrieval
Overview
This is an experimental Efficientformer codebase for Retrieval. It keeps the base setup intentionally manageable so architecture changes can be inspected before a full training run.
Repository status
- The Python file contains the model and runnable example or training entry point.
config.jsonrecords the generated architecture settings.training_args.jsonrecords the default experiment recipe.model.safetensorsis a valid initialization checkpoint for smoke tests; it is not presented as a trained benchmark checkpoint.- No benchmark score is claimed in this repository.
Architecture
| Item | Value |
|---|---|
| Architecture | Efficientformer |
| Scale | base |
| Attention | multi query |
| Fusion | co attention |
| Activation | approx gelu |
| Normalization | layernorm |
Default experiment recipe
The included configuration uses rmsprop with a onecycle schedule. These are starting values in the script, not evidence of a completed run. For a meaningful evaluation, train all baselines with the same data exposure, tuning budget, and random seeds.
Quick check
python main.py --help
Inspect the script's __main__ block for its generated smoke-test example. Because this is a custom implementation, generic automatic loading APIs require an explicit adapter before use.
Evaluation guidance
A useful first evaluation would use Flickr30k, report the task metric across at least three seeds, and include a matched-capacity baseline. Keep training logs and environment versions with any published result.
Limitations
The initialization checkpoint has not been trained or audited for robustness, fairness, or domain transfer. The implementation should be treated as an experimental starting point. Results from a future trained checkpoint must be documented separately from the defaults shipped here.
Files
main.py— primary artifactREADME.md— this documentationconfig.json— architecture configurationtraining_args.json— default experiment settingsmodel.safetensors— initialization checkpoint
License
Released under apache-2.0. Review the source-data terms separately when this repository is used with external datasets.
Configuration
- Architecture
- CustomResearchModel
- Context length (tokens)
- 256
- Layers
- 3
- Hidden size
- 192
- Feed-forward size
- 768
- Attention heads
- 4
- Model type
- efficientformer
Identity and Version
- Repository
- alvinng05/efficientformer-retrieval-lab4
- Publisher
- Alvinng05
- Task
- Not stated by the source
- Modality
- Other
- Library
- Not stated by the source
- Parameters
- 24,832 parameters
- Languages
- Not stated by the source
- Revision
- 59036c9321b2bf438e28c7fc6bf2351953a818f1
- First published
- 2026-10-02
- Last updated
- 2026-10-02
Files and Weights
6 files, 109.1 KB in total. The weights are 1 file totalling 99.8 KB in safetensors.
Every file
| File | Type | Size | SHA-256 |
|---|---|---|---|
| model.safetensors | Weights | 99.8 KB | 1758a04d5fd5 |
| config.json | Configuration | 455 B | — |
| main.py | Configuration | 4.7 KB | — |
| training_args.json | Configuration | 192 B | — |
| README.md | Documentation | 2.4 KB | — |
| .gitattributes | Repository | 1.5 KB | — |
License and Download
- License
- apache-2.0
- Access
- Open weights, no gate
- Download size
- 99.8 KB
Released by Alvinng05 through its official repository on Hugging Face. Read the license.
Memory Requirements
| Precision | Weights in memory |
|---|---|
| As published | 99.8 KB |
| 16-bit | 0.0 GB |
| 8-bit | 0.0 GB |
| 4-bit | 0.0 GB |
Weights only, from the published parameter count; the key-value cache and runtime add to this.
Questions About efficientformer-retrieval-lab4
How much GPU memory does efficientformer-retrieval-lab4 need?
About 0 GB at 16-bit and 0 GB at 4-bit: the weights (24,832 parameters) plus a working margin. A long context needs more.
What is the cheapest GPU to run efficientformer-retrieval-lab4 on?
At 16-bit, 1x MI300X from $1.85 an hour; at 4-bit, 1x MI300X from $1.85 an hour, at the lowest on-demand prices the SAVRN Index lists.
Can I use efficientformer-retrieval-lab4 commercially?
Yes. efficientformer-retrieval-lab4 is released under Apache License 2.0. The Apache License 2.0 is a permissive open-source license. It permits commercial use, modification and redistribution. It requires keeping the license and copyright notices and any NOTICE file, stating significant changes, and it includes an express patent grant from contributors.
What is efficientformer-retrieval-lab4's context length?
256 tokens, from the maximum position embeddings in its published configuration.