PP-FormulaNet_plus-L for Core ML (OhMyShot model pack)
PaddlePaddle's PP-FormulaNet_plus-L, which
reads a picture of a formula as LaTeX, converted to Core ML ML programs with 8-bit weights for
OhMyShot, a screenshot app for macOS, which downloads them as a model pack to
copy the formula in a selection as LaTeX. The weights are PaddlePaddle's, quantized; see LICENSE.txt for the license and
what was changed.
| File |
What it is |
encoder.mlpackage |
Input pixel_values, 1 × 3 × 768 × 768; outputs cross_k and cross_v, 8 × 1 × 16 × 144 × 32: each decoder layer's cross-attention keys and values. |
decoder.mlpackage |
One step: inputs ids (1 × 1, the last token), pos (1, its position), mask (1 × 1 × 1 × 1024, 0 for positions up to pos, -∞ after), cross_k, cross_v; output next, the likeliest next token, and logits. States k_cache and v_cache hold the self-attention keys and values. |
tokens.txt |
The tokenizer's tokens, one a line in id order (GPT-2 byte-level characters); ids below 23 are special. |
config.json |
Pre- and post-processing: inversion of dark pictures, cropping, scaling, normalization, the start and end tokens. |
LICENSE.txt |
Apache License 2.0, with what was changed. |
Decoding starts from token 2 at position 0 and ends when the decoder gives 2. Run on the CPU (.cpuOnly).
Made with Tools/model-packs/formula_pack.py from this upstream revision:
- https://huggingface.co/PaddlePaddle/PP-FormulaNet_plus-L_safetensors/tree/6cc004a8e91ea3f3819b05317f40b290cbe99c2b
Checked against the OhMyShot formula corpus (Tools/l3-spike: 40 formulas drawn with LaTeX): read through these models
on the CPU, with the app's own decoding, the share whose LaTeX draws exactly the formula drawn, by picture: 95.0% at
2x (192 dpi), 82.5% at 1x (96 dpi), 95.0% in dark mode (inverted first, as config.json has it), 75.0% in STIX Two Math, and
90.0% in TeX Gyre Pagella Math.
