qwen3-vl-8b-hebrew-v20b-ckpt

Qwen3-VL-8B Hebrew — v20b (merger-LoRA arm of v20; dead end)

Cairo Genizah AI Transcription — formal analysis and write-up of the project The full account of the methods, training data, benchmarks and error analysis behind the Cairo Genizah AI transcription models. Search the transcribed corpus at Cairo Genizah AI.

Part of the Qwen3-VL-8B Hebrew / Cairo Genizah fine-tune series (https://github.com/AIStream-Peelout/historical-document-analysis). Flagship: v21b step 1200.

This version

Status: dead-end arm, kept for the record. Use v21b.

• Trained 2026-09-06/09 (steps 100–200, then 300–2000); the only run in the series that completed its 2,000 steps. Warm start v19a step 1300; data identical to v20a, so it is a clean A/B.
• The one change: r16 LoRA on the merger linears (colab/genizah_v20b.ipynb), to test whether the merger is the lever for better boxes. Eval loss 0.750 → 0.668.

Finding: merger LoRA made the box geometry slightly worse and the transcription no better; the lever for accurate boxes is direct grounding data, not merger capacity. Final step 2000 4cae9d52d9cd167cb0d1ddd0248ed728c7308f29.

The series at a glance

All versions are QLoRA adapters on Qwen/Qwen3-VL-8B-Instruct (r16/α16 on the language model and every vision-tower block, merger frozen unless stated; batch 1 × 8, AdamW-8bit, LR 5e-5 cosine, image budget 6.5–7 MP, eval/save every 100 steps), trained on Colab A100s and evaluated with one decode at T=0.1 through an 8-bit MLX conversion. Two benchmarks recur below: religious = 140 held-out KTIV Talmud pages (33 two-column), PGP = 131 held-out documentary fragments from the Princeton Geniza Project (82 Judaeo-Arabic). CER* is the median character error rate over substantive outputs; page-level flips between versions are within decode noise, medians are the claim. The flagship is v21b step 1200.

Baselines on the same tables: the best Kraken setup reads religious pages at CER 0.30 and PGP at 0.25; stock Qwen3-VL-8B produces a substantive transcription on 2 % of PGP pages (F1 0.01). Training data comes from the KTIV project of the National Library of Israel (literary manuscripts and transcriptions), from published editions of Princeton Geniza Project documents, and from synthetic renders; see each card's data section. Code, builders and the evaluation harness: https://github.com/AIStream-Peelout/historical-document-analysis.

Loading

The repo root holds the adapter of the latest saved step; every step is a commit ("step N" = root adapter, "step N, checkpoint" = the same plus last-checkpoint/ with optimizer state). Pin the revision you want:

from transformers import AutoModelForImageTextToText, AutoProcessor
from peft import PeftModel

base = AutoModelForImageTextToText.from_pretrained("Qwen/Qwen3-VL-8B-Instruct", torch_dtype="bfloat16", device_map="auto")
model = PeftModel.from_pretrained(base, "isaacmg/qwen3-vl-8b-hebrew-v20b-ckpt", revision="4cae9d52d9cd167cb0d1ddd0248ed728c7308f29")  # step 2000
processor = AutoProcessor.from_pretrained("Qwen/Qwen3-VL-8B-Instruct", min_pixels=6_500_000, max_pixels=7_000_000)

Prompt the model the way it was trained: one page image plus a plain instruction such as "Transcribe this manuscript page line by line." For the grounding tasks (from v20a on) use the exact prompts in the builders (src/finetuning/qwen_hebrew/build_ktiv_dataset.py); boxes are [x1, y1, x2, y2] normalised to 0–1000 of the original image.

Data rights

Manuscript images remain the property of their holding institutions; the KTIV transcriptions are the National Library of Israel's; the Princeton Geniza Project's editions are used under their research terms; the Talmud replay set (v19a–v21b) consists of scans marked for non-commercial use. This adapter is released for research on handwritten-text recognition. Credit NLI KTIV and the Princeton Geniza Project when you build on it.