Qwen3-VL-8B Hebrew — v20a (KTIV v2 + first grounding tasks)
Cairo Genizah AI Transcription — formal analysis and write-up of the project
The full account of the methods, training data, benchmarks and error analysis behind the Cairo Genizah AI transcription models. Search the transcribed corpus at Cairo Genizah AI.
Part of the Qwen3-VL-8B Hebrew / Cairo Genizah fine-tune series (https://github.com/AIStream-Peelout/historical-document-analysis). Flagship: v21b step 1200.
This version
Status: superseded by v21b (which warm-starts from this checkpoint). Merged bf16 weights of step 1800
are published as isaacmg/qwen3-vl-8b-hebrew-v20a-merged.
- Trained 2026-09-03/05 in two sessions (steps 100–600, then 700–1800 until the session cap); warm
start v19a step 1300.
- What was new:
genizah_ktiv_v2 — 30,991 rows from 1,587 manuscripts / 3,698 pages, 2.19M
transcribed letters (2.7× v19a's KTIV slice) with repaired layout ground truth — and the first grounding
and layout families with boxes in 0–1000 coordinates: locate (6,613 rows), read_box (3,242),
layout_qa (3,301), grounded_page (268), at an 8 % share of the mixture.
- Recipe: shared recipe, merger frozen (
colab/genizah_v20a.ipynb). Eval loss 0.746 → 0.673.
| benchmark |
step 1800 |
step 1500 |
| religious CER* / F1 / readOK |
0.184 / 0.849 / 0.647 (two-column 0.218) |
0.196 / 0.851 |
| PGP CER* / F1 |
0.183 / 0.875 (Judaeo-Arabic F1 0.872) |
0.178 / 0.876 |
| grounding |
locate 53/72 hits, median IoU 0.47; read-box CER 0.13; grounded pages 21/24 parsed, line IoU 0.37 |
Finding: the first real transcription gain of the series and the first model that can point at
text. Its line boxes follow a layout prior rather than the ink (boxes on the right line 58 % of the time),
which set the agenda for the direct-grounding tasks of v21.
Canonical revisions: step 1800 af9df6a0ad4743bc8493a7cf0c14bc3b4b796dd8; step 1500
91e940671e6a9fb164319a9aabf3ae77d9fe8eaa.
The series at a glance
All versions are QLoRA adapters on Qwen/Qwen3-VL-8B-Instruct (r16/α16 on the language model and every
vision-tower block, merger frozen unless stated; batch 1 × 8, AdamW-8bit, LR 5e-5 cosine, image budget
6.5–7 MP, eval/save every 100 steps), trained on Colab A100s and evaluated with one decode at T=0.1
through an 8-bit MLX conversion. Two benchmarks recur below: religious = 140 held-out KTIV Talmud
pages (33 two-column), PGP = 131 held-out documentary fragments from the Princeton Geniza Project
(82 Judaeo-Arabic). CER* is the median character error rate over substantive outputs; page-level flips
between versions are within decode noise, medians are the claim. The flagship is v21b step 1200.
| version |
trained |
data change |
recipe change |
religious CER* |
PGP CER* |
status |
| v19a |
2026-08-20/21 |
+ KTIV v1 (8.3k rows), synthetic Hebrew v3 |
base recipe, merger frozen |
0.216 |
0.196 |
former flagship, superseded |
| v19b |
2026-08-28/29 |
= v19a |
merger r16 LoRA |
0.234 |
0.186 |
control, not adopted |
| v19c |
2026-08-30 → 09-01 |
= v19a |
merger trained full-weight |
0.225 |
0.189 |
not adopted |
| v20a |
2026-09-03/05 |
KTIV v2 (2.7× letters) + 8 % grounding |
warm start v19a-1300 |
0.184 |
0.183 |
superseded; merged weights published |
| v20b |
2026-09-06/09 |
= v20a |
merger r16 LoRA |
n/a |
n/a |
dead-end arm (boxes worse) |
| v21 |
2026-09-11 |
KTIV v3, 20 % grounding, streamed |
warm start v20a-1800 |
— |
— |
scrap: trained without images |
| v21b |
2026-09-11/15 |
= v21 |
dispatch fix + dataloader gate |
0.165 |
0.177 |
flagship / model of record |
| v22a |
2026-09-25/26 |
+ PGP editions, documentary grounding, extractive QA; no synthetic / Talmud |
warm start v21b-1200, map-style data |
0.186 |
0.164 |
pilot; documentary best, religious regression |
Baselines on the same tables: the best Kraken setup reads religious pages at CER 0.30 and PGP at 0.25;
stock Qwen3-VL-8B produces a substantive transcription on 2 % of PGP pages (F1 0.01).
Training data comes from the KTIV project of the National Library of Israel (literary manuscripts and
transcriptions), from published editions of Princeton Geniza Project documents, and from synthetic renders;
see each card's data section. Code, builders and the evaluation harness: https://github.com/AIStream-Peelout/historical-document-analysis.
Loading
The repo root holds the adapter of the latest saved step; every step is a commit ("step N" = root
adapter, "step N, checkpoint" = the same plus last-checkpoint/ with optimizer state). Pin the revision
you want:
from transformers import AutoModelForImageTextToText, AutoProcessor
from peft import PeftModel
base = AutoModelForImageTextToText.from_pretrained("Qwen/Qwen3-VL-8B-Instruct", torch_dtype="bfloat16", device_map="auto")
model = PeftModel.from_pretrained(base, "isaacmg/qwen3-vl-8b-hebrew-v20a-ckpt", revision="af9df6a0ad4743bc8493a7cf0c14bc3b4b796dd8") # step 1800
processor = AutoProcessor.from_pretrained("Qwen/Qwen3-VL-8B-Instruct", min_pixels=6_500_000, max_pixels=7_000_000)
Prompt the model the way it was trained: one page image plus a plain instruction such as
"Transcribe this manuscript page line by line." For the grounding tasks (from v20a on) use the exact
prompts in the builders (src/finetuning/qwen_hebrew/build_ktiv_dataset.py); boxes are [x1, y1, x2, y2]
normalised to 0–1000 of the original image.
Data rights
Manuscript images remain the property of their holding institutions; the KTIV transcriptions are the
National Library of Israel's; the Princeton Geniza Project's editions are used under their research terms;
the Talmud replay set (v19a–v21b) consists of scans marked for non-commercial use. This adapter is released
for research on handwritten-text recognition. Credit NLI KTIV and the Princeton Geniza Project when
you build on it.
