PIGEONS is an open-weight model from PIGEONS, released under Apache License 2.0. It has 35.3B parameters. At 16-bit it needs about 84.6 GB of GPU memory, which fits on 1x MI300X from $1.85 an hour, at the lowest prices in the SAVRN Index.
Detailed video captioning has become an essential semantic interface for multi-modal understanding and generation.
Runs On
What it takes to serve PIGEONS (35.3B parameters): the memory its weights need at each precision, and the cheapest way to rent enough data-center GPUs to hold them.
| Precision | Weights | Memory needed | Cheapest setup | Per hour | Also fits |
|---|---|---|---|---|---|
| 16-bit | 70.5 GB | 84.6 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x MI325X $2.00 · 1x MI355X $2.59 |
| 8-bit | 35.3 GB | 42.3 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
| 4-bit | 17.6 GB | 21.2 GB | 1x MI300X (192 GB) Vultr |
$1.85 | 1x H100 $1.99 · 1x MI325X $2.00 |
Memory is the weights at that precision plus 20% for the runtime and a short context; a long context needs more. Prices are the lowest on-demand hourly rates in the SAVRN Index, read Oct 18, 2026.
PIGEONS on every accelerator the SAVRN Index prices, at every precision
Model Card
By PIGEONS, published under apache-2.0, revision 54259242fdf6.
Detailed video captioning has become an essential semantic interface for multi-modal understanding and generation. While existing methods mainly target short videos, extending them to long videos remains challenging, primarily due to limited context windows of current video understanding models. A naive solution is to adopt segment-wise captioning followed by concatenation, yet this paradigm often suffers from inconsistent subject references and fragmented long-range narratives. To mitigate these issues, we introduce PIGEONS, an incremental long-video captioning model built upon a Persistent Identity Gallery and an Evolving Overarching Narrative Summary. As illustrated in the figure below…
Read PIGEONS's full model card
PIGEONS: Incremental Long-Video Captioning with Persistent Identity Gallery and Evolving Overarching Narrative Summary
Overview
Detailed video captioning has become an essential semantic interface for multi-modal understanding and generation. While existing methods mainly target short videos, extending them to long videos remains challenging, primarily due to limited context windows of current video understanding models. A naive solution is to adopt segment-wise captioning followed by concatenation, yet this paradigm often suffers from inconsistent subject references and fragmented long-range narratives. To mitigate these issues, we introduce PIGEONS, an incremental long-video captioning model built upon a Persistent Identity Gallery and an Evolving Overarching Narrative Summary.
As illustrated in the figure below, before caption generation, PIGEONS scans the entire video to construct an identity gallery that captures key subjects along with their visual references, thereby providing a stable referential anchor. The model then performs clip-by-clip captioning conditioned on the persistent gallery and an evolving overarching narrative summary, which compactly maintains and updates essential long-range contextual information after each captioning step.
To enable these capabilities, we develop a three-stage training strategy that integrates supervised fine-tuning and reinforcement learning. Extensive experiments demonstrate that PIGEONS markedly improves long-video captioning performance, achieving superior accuracy, completeness, and subject referential consistency over existing methods, while maintaining competitive performance on short-video captioning.
Getting Started
Please refer to our Github repository for more details.
Citation
If you find our work helpful for your research, please consider giving a star and citing our paper. We appreciate your support!
TODO
Configuration
- Architecture
- Qwen3OmniMoeForConditionalGeneration
- Hidden size
- 1,024
- Model type
- qwen3_omni_moe
Identity and Version
- Repository
- PIGEONS-Captioner/PIGEONS
- Publisher
- PIGEONS
- Task
- Not stated by the source
- Modality
- Other
- Library
- Not stated by the source
- Parameters
- 35.3B parameters
- Languages
- Not stated by the source
- Revision
- 54259242fdf6c5311ceb3785305b5f297f4fb315
- First published
- 2026-10-12
- Last updated
- 2026-10-12
Files and Weights
25 files, 70.5 GB in total. The weights are 15 files totalling 70.5 GB in safetensors.
Every file
| File | Type | Size | SHA-256 |
|---|---|---|---|
| model-00001-of-00015.safetensors | Weights | 5.0 GB | 3519593b91c4 |
| model-00002-of-00015.safetensors | Weights | 5.0 GB | 11e82a4d5ffa |
| model-00003-of-00015.safetensors | Weights | 5.0 GB | 9ccc35f47d6f |
| model-00004-of-00015.safetensors | Weights | 5.0 GB | e15551a571dc |
| model-00005-of-00015.safetensors | Weights | 5.0 GB | a35216876f90 |
| model-00006-of-00015.safetensors | Weights | 5.0 GB | 4611851ee89b |
| model-00007-of-00015.safetensors | Weights | 5.0 GB | 0e71bb0b5461 |
| model-00008-of-00015.safetensors | Weights | 5.0 GB | d4b70bb1cc3f |
| model-00009-of-00015.safetensors | Weights | 5.0 GB | ef0380834069 |
| model-00010-of-00015.safetensors | Weights | 5.0 GB | 09f0169e33b9 |
| model-00011-of-00015.safetensors | Weights | 5.0 GB | 408ca37fdce2 |
| model-00012-of-00015.safetensors | Weights | 5.0 GB | 473db17f330f |
| model-00013-of-00015.safetensors | Weights | 5.0 GB | 8023db045338 |
| model-00014-of-00015.safetensors | Weights | 5.0 GB | 184d98215edb |
| model-00015-of-00015.safetensors | Weights | 622.9 MB | c479f4a67a37 |
| config.json | Configuration | 11.6 KB | — |
| model.safetensors.index.json | Configuration | 2.7 MB | — |
| preprocessor_config.json | Configuration | 603 B | — |
| processor_config.json | Configuration | 2.4 KB | — |
| README.md | Documentation | 2.6 KB | — |
| assets/main.png | Other | 2.1 MB | 79c84f94e535 |
| chat_template.jinja | Other | 6.5 KB | — |
| .gitattributes | Repository | 1.6 KB | — |
| tokenizer.json | Tokenizer | 11.4 MB | a7d41145c408 |
| tokenizer_config.json | Tokenizer | 1.0 KB | — |
License and Download
- License
- apache-2.0
- Access
- Open weights, no gate
- Download size
- 70.5 GB
Released by PIGEONS through its official repository on Hugging Face. Read the license.
Built From
- Derived from Qwen/Qwen3-Omni-30B-A3B-Instruct
Memory Requirements
| Precision | Weights in memory |
|---|---|
| As published | 70.5 GB |
| 16-bit | 70.5 GB |
| 8-bit | 35.3 GB |
| 4-bit | 17.6 GB |
Weights only, from the published parameter count; the key-value cache and runtime add to this.
Questions About PIGEONS
How much GPU memory does PIGEONS need?
About 84.6 GB at 16-bit and 21.2 GB at 4-bit: the weights (35.3B parameters) plus a working margin. A long context needs more.
What is the cheapest GPU to run PIGEONS on?
At 16-bit, 1x MI300X from $1.85 an hour; at 4-bit, 1x MI300X from $1.85 an hour, at the lowest on-demand prices the SAVRN Index lists.
Can I use PIGEONS commercially?
Yes. PIGEONS is released under Apache License 2.0. The Apache License 2.0 is a permissive open-source license. It permits commercial use, modification and redistribution. It requires keeping the license and copyright notices and any NOTICE file, stating significant changes, and it includes an express patent grant from contributors.