每个目录是一次训练：a.pt 是可训练状态（仅 FFN 三矩阵，264.2M），a.step1000.pt 是中间存点， 门控的 run 另有 folded.pt（折叠成逐层宽度，评测用这个），train.log 含每步损失与留出集曲线。 它是旧批次的门控 75%（原名 c32kgate2304）：旧数据池（150M token，同一份数据过 3.49 遍， 预期分数与逐任务明细在仓库的 EVALUATING.md，摘要： 评测它必须用 --window 128，因为它是按 128–256 训的。用 512…
Parameters—
Context—
Weights157.7 GB
Licenseapache-2.0
AccessOpen weights
Monthly Downloads—
Model Card
By Vincent, published under apache-2.0, revision 551edcb8cbf8.
每个目录是一次训练：a.pt 是可训练状态（仅 FFN 三矩阵，264.2M），a.step1000.pt 是中间存点， 门控的 run 另有 folded.pt（折叠成逐层宽度，评测用这个），train.log 含每步损失与留出集曲线。 它是旧批次的门控 75%（原名 c32kgate2304）：旧数据池（150M token，同一份数据过 3.49 遍， 预期分数与逐任务明细在仓库的 EVALUATING.md，摘要： 评测它必须用 --window 128，因为它是按 128–256 训的。用 512 评它会低估 —— 那是它没训练过的 regime。 不要把它的分数当作方法的结果。 新批次的门控 75%（b2gate2304）用新数据池和连续窗口重训，
Read Vincent's full model card
prefill-assistant checkpoints
伴随 zywang624/prefill-assistant。
每个目录是一次训练：a.pt 是可训练状态（仅 FFN 三矩阵，264.2M），a.step1000.pt 是中间存点，
门控的 run 另有 folded.pt（折叠成逐层宽度，评测用这个），train.log 含每步损失与留出集曲线。
目录
| 目录 | 是什么 | 评测窗口 |
|---|---|---|
b2_w3072 |
新批次的全宽对照。新数据池（1.05B token，epoch 0.4991）、窗口 [512, 1024] 连续采样 | 512 |
CALIBRATION-ONLY_gate75_old-protocol |
仅用于校验评测流程，不是要展示的 75% 结果 | 128 |
关于 CALIBRATION-ONLY_gate75_old-protocol
它是旧批次的门控 75%（原名 c32k_gate2304）：旧数据池（150M token，同一份数据过 3.49 遍，
只覆盖 PG19 的 2/23 个分片）、窗口 128–256 离散两点。它之所以在这里，只有一个理由 ——
它在当前评测口径下的每个子任务分数都测过，所以另一台机器可以用它验证自己的流程是否正确。
预期分数与逐任务明细在仓库的 EVALUATING.md，摘要：
| RULER @32768，13 项均值 | 0.5901 |
| LongBench 宏平均 | 32.5819 |
评测它必须用 --window 128，因为它是按 128–256 训的。用 512 评它会低估 —— 那是它没训练过的 regime。
不要把它的分数当作方法的结果。 新批次的门控 75%（b2_gate2304）用新数据池和连续窗口重训，
训完会补进这个仓库，那个才是要展示的数。
数据
训练与评测数据在 mississippiu/prefill-assistant-data。
Identity and Version
- Repository
- mississippiu/prefill-assistant-models
- Publisher
- Vincent
- Task
- Not stated by the source
- Modality
- Other
- Library
- Not stated by the source
- Parameters
- Not stated by the source
- Languages
- Not stated by the source
- Revision
- 551edcb8cbf8bf4cf21ad3271df7533ca2b16d57
- First published
- 2026-09-10
- Last updated
- 2026-09-18
Files and Weights
145 files, 157.7 GB in total. The weights are 94 files totalling 157.7 GB in pt.
Weights94 files · 157.7 GB
Documentation3 files · 4.4 KB
Other47 files · 136.3 KB
Repository1 file · 1.5 KB
Every file
| File | Type | Size | SHA-256 |
|---|---|---|---|
| CALIBRATION-ONLY_gate75_old-protocol/a.pt | Weights | 528.9 MB | 1ac9ee059c84 |
| CALIBRATION-ONLY_gate75_old-protocol/a.step1000.pt | Weights | 528.9 MB | 91d966d2c6a1 |
| CALIBRATION-ONLY_gate75_old-protocol/folded.pt | Weights | 1.1 GB | 1846eb8864e9 |
| CALIBRATION-ONLY_gate75_old-protocol/folded.step1000.pt | Weights | 1.1 GB | 879e1753f26d |
| b2_1.7b_gate3840_4gh200_chunked/a.pt | Weights | 6.3 GB | fbd4d112a43c |
| b2_1.7b_gate3840_4gh200_chunked/folded.pt | Weights | 4.2 GB | d413db0c0b13 |
| b2_1.7b_inv3840_4gh200_chunked/a.pt | Weights | 1.6 GB | 6c8597c22c1f |
| b2_1.7b_w3840_4gh200_chunked/a.pt | Weights | 1.6 GB | 0030261b2b8a |
| b2_decodeonly2304_v2/a.pt | Weights | 528.5 MB | 9ef3569eff37 |
| b2_decodeonly2304_v2/a.step1000.pt | Weights | 528.5 MB | 54d2a03ec7a3 |
| b2_forcehalf1536/a.pt | Weights | 528.5 MB | 3f6b1892ba1f |
| b2_forcehalf1536/a.step1000.pt | Weights | 528.5 MB | 35bbd5b5d991 |
| b2_gate1536/a.pt | Weights | 528.9 MB | 088ab16ad9b3 |
| b2_gate1536/a.step1000.pt | Weights | 528.9 MB | f33296a01142 |
| b2_gate1536/folded.pt | Weights | 1.1 GB | 2e69c89e5a9e |
| b2_gate1536/folded.step1000.pt | Weights | 1.1 GB | 1bdb162c5560 |
| b2_gate1920/a.pt | Weights | 528.9 MB | c21566d46f47 |
| b2_gate1920/a.step1000.pt | Weights | 528.9 MB | 4b46c136fe5b |
| b2_gate1920/folded.pt | Weights | 1.1 GB | 7436211b28ad |
| b2_gate1920/folded.step1000.pt | Weights | 1.1 GB | a288bc93b2da |
| b2_gate2304/a.pt | Weights | 528.9 MB | f07eed8a7b03 |
| b2_gate2304/a.step1000.pt | Weights | 528.9 MB | 96a257d61673 |
| b2_gate2304/folded.pt | Weights | 1.1 GB | 601dc853e464 |
| b2_gate2304/folded.step1000.pt | Weights | 1.1 GB | db95b7e0f333 |
| b2_gate2304_4gh200/a.pt | Weights | 528.9 MB | 14c2090f9fe0 |
| b2_gate2304_4gh200/a.step1000.pt | Weights | 528.9 MB | 98cb8c042346 |
| b2_gate2304_4gh200/a.step2000.pt | Weights | 528.9 MB | 7a3dc86b15a6 |
| b2_gate2304_4gh200/a.step3000.pt | Weights | 528.9 MB | d96a280987cc |
| b2_gate2304_4gh200/folded.pt | Weights | 1.1 GB | 59ed9b6589f7 |
| b2_gate2304_4gh200/folded.step1000.pt | Weights | 1.1 GB | 3ec109ac04c7 |
| b2_gate2304_4gh200/folded.step2000.pt | Weights | 1.1 GB | ee8821b74e5d |
| b2_gate2304_4gh200/folded.step3000.pt | Weights | 1.1 GB | 25012f301b89 |
| b2_gate2688/a.pt | Weights | 528.9 MB | 38b7ca7847fb |
| b2_gate2688/a.step1000.pt | Weights | 528.9 MB | be6059efedf4 |
| b2_gate2688/folded.pt | Weights | 1.1 GB | 09d00d7f06d4 |
| b2_gate2688/folded.step1000.pt | Weights | 1.1 GB | b5800c0514d7 |
| b2_inv1536/a.pt | Weights | 528.5 MB | 775059f920cb |
| b2_inv1536/a.step1000.pt | Weights | 528.5 MB | 3cf03e410cfb |
| b2_inv1920/a.pt | Weights | 528.5 MB | a08cd638446a |
| b2_inv1920/a.step1000.pt | Weights | 528.5 MB | e5a2e147deff |
| b2_inv2304/a.pt | Weights | 528.5 MB | 80c9d4be6636 |
| b2_inv2304/a.step1000.pt | Weights | 528.5 MB | 3735f9da5963 |
| b2_inv2688/a.pt | Weights | 528.5 MB | dc85348b00c5 |
| b2_inv2688/a.step1000.pt | Weights | 528.5 MB | 0c868c2bdf6c |
| b2_w1536/a.pt | Weights | 528.5 MB | 89b08142c493 |
| b2_w1536/a.step1000.pt | Weights | 528.5 MB | 966ec436882c |
| b2_w1920/a.pt | Weights | 528.5 MB | d6b4788382bd |
| b2_w1920/a.step1000.pt | Weights | 528.5 MB | 4563a4fa27ec |
| b2_w2304/a.pt | Weights | 528.5 MB | 25071668448f |
| b2_w2304/a.step1000.pt | Weights | 528.5 MB | ec82d0273310 |
| b2_w2688/a.pt | Weights | 528.5 MB | 51e0c4437680 |
| b2_w2688/a.step1000.pt | Weights | 528.5 MB | d647b16154bc |
| b2_w3072/a.pt | Weights | 528.5 MB | 665bf95e3058 |
| b2_w3072/a.step1000.pt | Weights | 528.5 MB | 07aee2c73bd0 |
| llama1b_gate5120/a.pt | Weights | 1.6 GB | 5fbd70c5a40a |
| llama1b_gate5120/a.step1000.pt | Weights | 1.6 GB | 17b2daae3812 |
| llama1b_gate5120/folded.pt | Weights | 3.2 GB | 6427c3375580 |
| llama1b_gate5120/folded.step1000.pt | Weights | 3.2 GB | 040fc6242d6e |
| llama1b_gate6144/a.pt | Weights | 1.6 GB | c450154e3884 |
| llama1b_gate6144/a.step1000.pt | Weights | 1.6 GB | d4e014db3e9c |
| llama1b_gate6144/folded.pt | Weights | 3.2 GB | 036c37815086 |
| llama1b_gate6144/folded.step1000.pt | Weights | 3.2 GB | c1ec33308833 |
| llama1b_gate6144_base/a.pt | Weights | 1.6 GB | ce165844e2c0 |
| llama1b_gate6144_base/a.step1000.pt | Weights | 1.6 GB | 56de83e35906 |
| llama1b_gate6144_base/folded.pt | Weights | 3.2 GB | ca3c44681235 |
| llama1b_gate6144_base/folded.step1000.pt | Weights | 3.2 GB | 173ca847af19 |
| llama1b_gate7168/a.pt | Weights | 1.6 GB | 4d7080d04d9f |
| llama1b_gate7168/a.step1000.pt | Weights | 1.6 GB | e915cc434b65 |
| llama1b_gate7168/folded.pt | Weights | 3.2 GB | e331631bddc4 |
| llama1b_gate7168/folded.step1000.pt | Weights | 3.2 GB | d260fbdde848 |
| llama1b_inv6144/a.pt | Weights | 1.6 GB | 7cc7c85effe8 |
| llama1b_inv6144/a.step1000.pt | Weights | 1.6 GB | fa65b4793170 |
| llama1b_inv6144_base/a.pt | Weights | 1.6 GB | de2f9168b1bf |
| llama1b_inv6144_base/a.step1000.pt | Weights | 1.6 GB | a2629dc84e7a |
| llama1b_w5120/a.pt | Weights | 1.6 GB | 25afa225badf |
| llama1b_w5120/a.step1000.pt | Weights | 1.6 GB | 2e92988fa48a |
| llama1b_w6144/a.pt | Weights | 1.6 GB | f8162eae63f1 |
| llama1b_w6144/a.step1000.pt | Weights | 1.6 GB | b1ae8b836d4c |
| llama1b_w6144_base/a.pt | Weights | 1.6 GB | 513de2f78b9d |
| llama1b_w6144_base/a.step1000.pt | Weights | 1.6 GB | af39598463b9 |
| llama1b_w7168/a.pt | Weights | 1.6 GB | 5d633d5e9992 |
| llama1b_w7168/a.step1000.pt | Weights | 1.6 GB | 294d80ca5dca |
| llama1b_w8192/a.pt | Weights | 1.6 GB | 403f229074d0 |
| llama1b_w8192/a.step1000.pt | Weights | 1.6 GB | 244475b466f4 |
| llama1b_w8192_base/a.pt | Weights | 1.6 GB | ae0b1bcd5e9e |
| llama1b_w8192_base/a.step1000.pt | Weights | 1.6 GB | 807290ff29e1 |
| qwen4b_gate6080/a.pt | Weights | 5.4 GB | 1ff9bc25b721 |
| qwen4b_gate6080/a.step1000.pt | Weights | 5.4 GB | c5cbc42525fc |
| qwen4b_gate6080/folded.pt | Weights | 10.8 GB | fd4dd21bf586 |
| qwen4b_gate6080/folded.step1000.pt | Weights | 10.8 GB | a7d8146e383d |
| qwen4b_w9728/a.pt | Weights | 5.4 GB | 91e46d5c4e90 |
| qwen4b_w9728/a.step1000.pt | Weights | 5.4 GB | af186f27c2fc |
| resumetest_0.6b_gate2304_4gh200_chunked/a.pt | Weights | 1.6 GB | 4418a89661f9 |
| resumetest_0.6b_gate2304_4gh200_chunked/folded.pt | Weights | 1.1 GB | 706b1effa832 |
| README.md | Documentation | 1.8 KB | — |
| b2_1.7b_inv3840_4gh200_chunked/INVALID.md | Documentation | 1.3 KB | — |
| b2_1.7b_w3840_4gh200_chunked/INVALID.md | Documentation | 1.3 KB | — |
| CALIBRATION-ONLY_gate75_old-protocol/fold.log | Other | 362 B | — |
| CALIBRATION-ONLY_gate75_old-protocol/train.log | Other | 2.9 KB | — |
| b2_1.7b_gate3840_4gh200_chunked/fold.log | Other | 1.0 KB | — |
| b2_decodeonly2304_v2/train.log | Other | 4.0 KB | — |
| b2_forcehalf1536/train.log | Other | 3.9 KB | — |
| b2_gate1536/fold.log | Other | 360 B | — |
| b2_gate1536/train.log | Other | 4.3 KB | — |
| b2_gate1920/fold.log | Other | 360 B | — |
| b2_gate1920/train.log | Other | 4.3 KB | — |
| b2_gate2304/fold.log | Other | 360 B | — |
| b2_gate2304/train.log | Other | 4.3 KB | — |
| b2_gate2304_4gh200/fold.log | Other | 319 B | — |
| b2_gate2304_4gh200/fold.step1000.log | Other | 328 B | — |
| b2_gate2304_4gh200/fold.step2000.log | Other | 328 B | — |
| b2_gate2304_4gh200/fold.step3000.log | Other | 328 B | — |
| b2_gate2304_4gh200/train.log | Other | 6.7 KB | — |
| b2_gate2688/fold.log | Other | 363 B | — |
| b2_gate2688/train.log | Other | 4.3 KB | — |
| b2_inv1536/train.log | Other | 4.0 KB | — |
| b2_inv1920/train.log | Other | 4.0 KB | — |
| b2_inv2304/train.log | Other | 4.0 KB | — |
| b2_inv2688/train.log | Other | 4.0 KB | — |
| b2_w1536/train.log | Other | 3.9 KB | — |
| b2_w1920/train.log | Other | 3.9 KB | — |
| b2_w2304/train.log | Other | 3.9 KB | — |
| b2_w2688/train.log | Other | 3.9 KB | — |
| b2_w3072/train.log | Other | 3.8 KB | — |
| llama1b_gate5120/fold.log | Other | 365 B | — |
| llama1b_gate5120/train.log | Other | 4.3 KB | — |
| llama1b_gate6144/fold.log | Other | 365 B | — |
| llama1b_gate6144/train.log | Other | 4.3 KB | — |
| llama1b_gate6144_base/fold.log | Other | 365 B | — |
| llama1b_gate6144_base/train.log | Other | 4.3 KB | — |
| llama1b_gate7168/fold.log | Other | 368 B | — |
| llama1b_gate7168/train.log | Other | 4.3 KB | — |
| llama1b_inv6144/train.log | Other | 4.0 KB | — |
| llama1b_inv6144_base/train.log | Other | 4.0 KB | — |
| llama1b_w5120/train.log | Other | 3.9 KB | — |
| llama1b_w6144/train.log | Other | 3.9 KB | — |
| llama1b_w6144_base/train.log | Other | 3.9 KB | — |
| llama1b_w7168/train.log | Other | 3.9 KB | — |
| llama1b_w8192/train.log | Other | 3.9 KB | — |
| llama1b_w8192_base/train.log | Other | 3.9 KB | — |
| qwen4b_gate6080/fold.log | Other | 1.0 KB | — |
| qwen4b_gate6080/train.log | Other | 5.6 KB | — |
| qwen4b_w9728/train.log | Other | 5.2 KB | — |
| resumetest_0.6b_gate2304_4gh200_chunked/fold.log | Other | 339 B | — |
| .gitattributes | Repository | 1.5 KB | — |
License and Download
- License
- apache-2.0
- Access
- Open weights, no gate
- Download size
- 157.7 GB
Download from Vincent
Released by Vincent through its official repository on Hugging Face. Read the license.
Memory Requirements
| Precision | Weights in memory |
|---|---|
| As published | 157.7 GB |
Weights only, from the published parameter count; the key-value cache and runtime add to this.
Questions About prefill-assistant-models
Can I use prefill-assistant-models commercially?
Yes. prefill-assistant-models is released under Apache License 2.0. The Apache License 2.0 is a permissive open-source license. It permits commercial use, modification and redistribution. It requires keeping the license and copyright notices and any NOTICE file, stating significant changes, and it includes an express patent grant from contributors.