Tesseract model for Karakalpak (Cyrillic) — v0.1
Tesseract 5 LSTM model for printed Karakalpak in the Cyrillic alphabet, including the letters Ә Ғ Қ Ң Ө Ү Ў Ҳ. Fine-tuned from Tesseract's Kazakh kaz (tessdata_best). For Latin-script Karakalpak use kaa-ml/tesseract-kaa-latn (CER 4.7%).
Before this model, the usual choice for Karakalpak Cyrillic was the Kazakh model. It has no ў and ҳ, so it silently writes у and х instead — the text looks right but is misspelled. The Russian model loses all eight Karakalpak letters.
Benchmark
10 scanned pages of an old printed Karakalpak book (never used in training), --psm 4. CER = character error rate (lower is better). Letter recall = share of ә ғ қ ң ө ү ў ҳ in the reference that the model output contains.
| Model |
CER |
Karakalpak letter recall |
Speed (page) |
this model, best/ |
12.87% |
92.3% |
0.73 s |
this model, fast/ |
13.64% |
90.7% |
0.62 s |
Tesseract kaz (tessdata_best) |
12.64% |
88.8% |
— |
Tesseract kaz (tessdata_fast) |
13.37% |
87.6% |
— |
Tesseract rus |
17.1%* |
0% |
— |
* on all 15 annotated pages.
Honest summary: this version fixes the Karakalpak letters (92% vs 89%), but its overall CER on old book scans is still on par with the Kazakh model (+0.2 points). Most remaining errors are layout and punctuation on degraded pages. v0.2 will add more real scanned lines, which gave the largest gains so far; heavier synthetic scan simulation did not help.
Usage
Download best/kaa_cyrl.traineddata into a folder and point Tesseract at it:
tesseract page.png out -l kaa_cyrl --tessdata-dir ./tessdata --psm 4
Mixed Karakalpak/Russian documents: -l kaa_cyrl+rus.
import pytesseract
from huggingface_hub import hf_hub_download
from PIL import Image
path = hf_hub_download("kaa-ml/tesseract-kaa-cyrl", "best/kaa_cyrl.traineddata")
text = pytesseract.image_to_string(
Image.open("page.png"),
lang="kaa_cyrl",
config=f"--tessdata-dir {path.rsplit('/', 1)[0]} --psm 4",
)
Scan at 300 DPI for best results.
Training
- Start model:
kaz from tessdata_best (Apache-2.0).
- Data: 8,000 synthetic lines with natural letter distribution (
kaa-ml/kaa-cyrl-ocr-synthetic, synthetic_natural; Karakalpak news text from kknews.uz, CC-BY-4.0) + 182 real scanned lines from an old printed book (×30 augmented copies).
- tesstrain, 20,000 iterations, learning rate 1e-4, on a MacBook CPU.
- Things that did not help and are not in this model: 50% oversampling of lines with rare letters (the model started hallucinating
ў), a Karakalpak word dictionary built from news text (CER 13.10%), Augraphy scan simulation with old-style serif fonts (13.08%).
Versions
License
Apache-2.0 (same as Tesseract's traineddata). Synthetic training text comes from kknews.uz under CC-BY-4.0.
