SigLIP 2 base (256 px) for Core ML
Core ML conversion of Google's SigLIP 2
google/siglip2-base-patch16-256, with both towers:
an image encoder and a text encoder, each returning an L2-normalized 768-d embedding. Labels are plain text chosen
at run time, so it classifies images zero-shot. Google authored the Apache-2.0 source model; Fluid Inference
converted it. The image encoder runs entirely on the Apple Neural Engine.
Packages
| Package |
Input |
Output |
Size |
siglip2-base-patch16-256-image-fp16.mlpackage |
pixel_values float32 [1, 3, 256, 256] |
image_embeds float32 [1, 768] |
176 MB |
siglip2-base-patch16-256-text-fp16.mlpackage |
input_ids int32 [1, 64] |
text_embeds float32 [1, 768] |
539 MB |
fp16 weights, macOS 14 / iOS 17 or newer. config.json holds the preprocessing and scoring constants:
- Image: resize to 256 × 256 (bilinear, antialiased like PIL), scale to [0, 1], then
(x − 0.5) / 0.5.
- Text: lowercase, Gemma tokenizer (
tokenizer.json), append <eos>, pad with <pad> to 64 tokens. SigLIP was
trained without an attention mask, so the padding must match.
- Scores:
cos = image_embeds · text_embeds. sigmoid(logit_scale · cos + logit_bias) (112.90, −16.77) gives
an independent probability per label; softmax or argmax over cos picks one label. Embed the labels once and
reuse them for every image.
Accuracy
ImageNet-1k zero-shot, all 50,000 test images (clip-benchmark/wds_imagenet1k, prompt
this is a photo of {class}.), fp16 on CPU + Neural Engine against the fp32 PyTorch model:
|
PyTorch fp32 |
Core ML fp16 |
| Top-1 |
76.79% |
76.76% |
| Same prediction as PyTorch |
— |
99.32% |
| Image embedding cosine, mean / min |
— |
0.99990 / 0.975 |
Google reports 79.1% with its own class names and prompts; the single-prompt protocol here is lower for both
backends. Oxford-IIIT Pets test (3,669 photos, 37 breeds): Core ML 94.77%, PyTorch 94.74%, 99.89% identical.
Speed and memory
Apple M5 Pro (24 GB), macOS 27. Image encoder alone, one call at a time: 5.2 ms on CPU + Neural Engine
(100% of ops on the ANE), 3.5 ms CPU + GPU, versus 19.1 ms for PyTorch fp32 on MPS.
End to end on 7,349 Oxford-IIIT Pets photos (JPEG decode, resize, encoder, scoring against 37 cached breed
prompts), same photos and prompts, measured after model load:
|
Core ML (Swift, 4 photos in flight) |
PyTorch fp32, MPS, batch 32 |
| Time |
36.3 s |
102.2 s |
| Photos per second |
202 |
72 |
| Peak memory (process) |
262 MB |
4.24 GB |
| Model on disk |
715 MB |
1.50 GB |
| Accuracy |
94.26% |
94.11% |
PyTorch batch 1: 153.5 s, 48 photos/s, 3.64 GB. Reports: reports/.
Use from Swift
FluidUse wraps both packages (SigLIP2Manager, Swift Gemma tokenizer
with token ids identical to the Python tokenizer) and includes the photo-sorting demo ImageSortDemo.
Conversion scripts: FluidInference/mobius models/emb/siglip2/coreml.
