# Zero Shot Image Classification Models: Open-Weight AI Source: https://savrn.com/models/tasks/zero-shot-image-classification Markdown alternate of the page above; the site index is https://savrn.com/llms.txt --- SAVRN Model Hub · Models by Task # Zero Shot Image Classification Models 21 open-weight zero shot image classification models in the SAVRN Model Hub, with Google, OpenAI and LAION eV publishing the most. 21Models 9Publishers 151M to 9BParameter range 5Licenses ## SAVRN's Take Hand one of these models a photo and a few text labels and it picks the label that fits, with no training run for that label set. They also serve as the vision encoder inside a larger vision-language model. Seventeen models sit in this category, twelve priced in the Index. Google publishes seven, OpenAI four, LAION eV three, yet the download chart belongs to OpenAI: clip-vit-base-patch32 pulls 21.8 million a month, more than double the 8.3 million for the 428-million-parameter clip-vit-large-patch14. Running them is light duty. The smallest, LAION's CLIP-ViT-B-32 at 151 million parameters, needs 0.4 GB at 16-bit. The largest, Google's siglip2-giant-opt-patch16-384 at 1.9 billion, needs 4.5 GB, or 1.1 GB at 4-bit. Every priced entry in the top downloads lands on the same cheapest host, one MI300X at $1.85 an hour, so the hardware question is throughput, not fit: how many images an hour does it take to justify that host. The 77-token context length on the CLIP checkpoints caps each label at a short phrase. Seven models carry Apache 2.0, including the SigLIP 2 entries, and four carry MIT, including the LAION checkpoints and fashion-clip, a 151-million-parameter fine-tune of LAION's ViT-B-32. Four list no license, the top-of-chart OpenAI CLIP checkpoints among them, so settle that with the publisher before a paid service goes live. One sits under CC BY-NC-ND 4.0, the non-commercial, no-derivatives license. Our reading: SigLIP 2 pairs a clean license with a 375 million to 1.9 billion parameter spread, while the OpenAI checkpoints bring the install base and an open license question. SAVRN Research, 2026-09-18 ## Most Downloaded | Model | Publisher | Parameters | License | Monthly downloads | Cheapest GPUs at 16-bit | | --- | --- | --- | --- | --- | --- | | [clip-vit-base-patch32](https://savrn.com/models/clip-vit-base-patch32) | OpenAI | — | Not stated | 16.9M | — | | [clip-vit-large-patch14](https://savrn.com/models/clip-vit-large-patch14) | OpenAI | 428M | Not stated | 6.9M | 1x MI300X, $1.85/hr | | [CLIP-ViT-B-32-laion2B-s34B-b79K](https://savrn.com/models/clip-vit-b-32-laion2b-s34b-b79k) | LAION eV | 151M | mit | 4M | 1x MI300X, $1.85/hr | | [clip-vit-large-patch14-336](https://savrn.com/models/clip-vit-large-patch14-336) | OpenAI | — | Not stated | 2.8M | — | | [siglip2-giant-opt-patch16-384](https://savrn.com/models/siglip2-giant-opt-patch16-384) | Google | 1.9B | apache-2.0 | 1.9M | 1x MI300X, $1.85/hr | | [CLIP-convnext_base_w-laion2B-s13B-b82K-augreg](https://savrn.com/models/clip-convnext-base-w-laion2b-s13b-b82k-augreg) | LAION eV | — | mit | 1.8M | — | | [siglip2-base-patch16-224](https://savrn.com/models/siglip2-base-patch16-224) | Google | 375M | apache-2.0 | 1.7M | 1x MI300X, $1.85/hr | | [clip-vit-base-patch16](https://savrn.com/models/clip-vit-base-patch16) | OpenAI | — | Not stated | 1.5M | — | | [siglip2-base-patch16-256](https://savrn.com/models/siglip2-base-patch16-256) | Google | 375M | apache-2.0 | 1.4M | 1x MI300X, $1.85/hr | | [CLIP-ViT-L-14-laion2B-s32B-b82K](https://savrn.com/models/clip-vit-l-14-laion2b-s32b-b82k) | LAION eV | 428M | mit | 1.3M | 1x MI300X, $1.85/hr | ## Licenses | License | Models | Commercial use | | --- | --- | --- | | [apache-2.0](https://savrn.com/models/licenses/apache-2-0) | 9 | Yes | | not stated | 5 | Not stated | | [mit](https://savrn.com/models/licenses/mit) | 5 | Yes | | [cc-by-nc-4.0](https://savrn.com/models/licenses/cc-by-nc-4-0) | 1 | Not without separate permission | | [cc-by-4.0](https://savrn.com/models/licenses/cc-by-4-0) | 1 | Yes | ## Who Publishes Them | Publisher | Models | | --- | --- | | [Google](https://savrn.com/model-publishers/google) | 7 | | [OpenAI](https://savrn.com/model-publishers/openai) | 4 | | [LAION eV](https://savrn.com/model-publishers/laion) | 3 | | [Siyuan Yan](https://savrn.com/model-publishers/redlessone) | 2 | | [SCX.ai Holdings Limited](https://savrn.com/model-publishers/scx-admin) | 1 | | [Patrick John Chia](https://savrn.com/model-publishers/patrickjohncyh) | 1 | ## All 21 Models Model · Zero shot image classification ### [clip-vit-base-patch32](https://savrn.com/models/clip-vit-base-patch32) [OpenAI](https://savrn.com/model-publishers/openai) Disclaimer: The model card is taken and modified from the official CLIP repository, it can be found here. The CLIP model was developed by researchers at OpenAI to learn about what contributes to robustness in computer vision tasks. The model was also developed to test the ability of models to generalize to arbitrary image classification tasks in a zero-shot manner. It was not developed for general model deployment - to deploy models like CLIP, researchers will first need to carefully study their capabilities in relation to the specific context they’re being deployed within. January 2021 The model uses a ViT-B/32 Transformer architecture as an image encoder and uses a masked self-attention… Open weights 77 tokens transformers [View model](https://savrn.com/models/clip-vit-base-patch32) Model · Zero shot image classification ### [clip-vit-large-patch14](https://savrn.com/models/clip-vit-large-patch14) [OpenAI](https://savrn.com/model-publishers/openai) Disclaimer: The model card is taken and modified from the official CLIP repository, it can be found here. The CLIP model was developed by researchers at OpenAI to learn about what contributes to robustness in computer vision tasks. The model was also developed to test the ability of models to generalize to arbitrary image classification tasks in a zero-shot manner. It was not developed for general model deployment - to deploy models like CLIP, researchers will first need to carefully study their capabilities in relation to the specific context they’re being deployed within. January 2021 The base model uses a ViT-L/14 Transformer architecture as an image encoder and uses a masked… Open weights 428M parameters 77 tokens transformers [View model](https://savrn.com/models/clip-vit-large-patch14) Model · Zero shot image classification ### [CLIP-ViT-B-32-laion2B-s34B-b79K](https://savrn.com/models/clip-vit-b-32-laion2b-s34b-b79k) [LAION eV](https://savrn.com/model-publishers/laion) A CLIP ViT-B/32 model trained with the LAION-2B English subset of LAION-5B (https://laion.ai/blog/laion-5b/) using OpenCLIP (https://github.com/mlfoundations/openclip). Model training done by Romain Beaumont on the stability.ai cluster. As per the original OpenAI CLIP model card, this model is intended as a research output for research communities. We hope that this model will enable researchers to better understand and explore zero-shot, arbitrary image classification. We also hope it can be used for interdisciplinary studies of the potential impact of such model. The OpenAI CLIP paper includes a discussion of potential downstream impacts to provide an example for this sort of analysis.… Open weights mit 151M parameters 77 tokens open_clip [View model](https://savrn.com/models/clip-vit-b-32-laion2b-s34b-b79k) Model · Zero shot image classification ### [siglip2-giant-opt-patch16-384](https://savrn.com/models/siglip2-giant-opt-patch16-384) [Google](https://savrn.com/model-publishers/google) SigLIP 2 extends the pretraining objective of SigLIP with prior, independently developed techniques into a unified recipe, for improved semantic understanding, localization, and dense features. You can use the raw model for tasks like zero-shot image classification and image-text retrieval, or as a vision encoder for VLMs (and other vision tasks). Here is how to use this model to perform zero-shot image classification: You can encode an image using the Vision Tower like so: For more code examples, we refer to the siglip documentation. SigLIP 2 adds some clever training objectives on top of SigLIP: SigLIP 2 is pre-trained on the WebLI dataset (Chen et al., 2023). The model was trained on up… Open weights apache-2.0 1.9B parameters transformers [View model](https://savrn.com/models/siglip2-giant-opt-patch16-384) Model · Zero shot image classification ### [CLIP-convnext_base_w-laion2B-s13B-b82K-augreg](https://savrn.com/models/clip-convnext-base-w-laion2b-s13b-b82k-augreg) [LAION eV](https://savrn.com/model-publishers/laion) A series of CLIP ConvNeXt-Base (w/ wide embed dim) models trained on subsets LAION-5B (https://laion.ai/blog/laion-5b/) using OpenCLIP (https://github.com/mlfoundations/openclip). Explore an alternative to ViT and ResNet (w/ AttentionPooling) CLIP models that scales well with model size and image resolution First known ConvNeXt CLIP models trained at scale in the range of CLIP ViT-B/16 and RN50x4 models First released model weights exploring increase of augmentation + regularization for image tower via adding (greater scale range of RRC, random erasing, stochastic depth) The models utilize the timm ConvNeXt-Base model (convnextbase) as the image tower, and the same text tower as the RN50x4… Open weights mit open_clip [View model](https://savrn.com/models/clip-convnext-base-w-laion2b-s13b-b82k-augreg) Model · Zero shot image classification ### [siglip2-base-patch16-224](https://savrn.com/models/siglip2-base-patch16-224) [Google](https://savrn.com/model-publishers/google) SigLIP 2 extends the pretraining objective of SigLIP with prior, independently developed techniques into a unified recipe, for improved semantic understanding, localization, and dense features. You can use the raw model for tasks like zero-shot image classification and image-text retrieval, or as a vision encoder for VLMs (and other vision tasks). Here is how to use this model to perform zero-shot image classification: You can encode an image using the Vision Tower like so: For more code examples, we refer to the siglip documentation. SigLIP 2 adds some clever training objectives on top of SigLIP: SigLIP 2 is pre-trained on the WebLI dataset (Chen et al., 2023). The model was trained on up… Open weights apache-2.0 375M parameters transformers [View model](https://savrn.com/models/siglip2-base-patch16-224) Model · Zero shot image classification ### [clip-vit-base-patch16](https://savrn.com/models/clip-vit-base-patch16) [OpenAI](https://savrn.com/model-publishers/openai) Disclaimer: The model card is taken and modified from the official CLIP repository, it can be found here. The CLIP model was developed by researchers at OpenAI to learn about what contributes to robustness in computer vision tasks. The model was also developed to test the ability of models to generalize to arbitrary image classification tasks in a zero-shot manner. It was not developed for general model deployment - to deploy models like CLIP, researchers will first need to carefully study their capabilities in relation to the specific context they’re being deployed within. January 2021 The base model uses a ViT-B/16 Transformer architecture as an image encoder and uses a masked… Open weights 77 tokens transformers [View model](https://savrn.com/models/clip-vit-base-patch16) Model · Zero shot image classification ### [siglip2-base-patch16-256](https://savrn.com/models/siglip2-base-patch16-256) [Google](https://savrn.com/model-publishers/google) SigLIP 2 extends the pretraining objective of SigLIP with prior, independently developed techniques into a unified recipe, for improved semantic understanding, localization, and dense features. You can use the raw model for tasks like zero-shot image classification and image-text retrieval, or as a vision encoder for VLMs (and other vision tasks). Here is how to use this model to perform zero-shot image classification: You can encode an image using the Vision Tower like so: For more code examples, we refer to the siglip documentation. SigLIP 2 adds some clever training objectives on top of SigLIP: SigLIP 2 is pre-trained on the WebLI dataset (Chen et al., 2023). The model was trained on up… Open weights apache-2.0 375M parameters transformers [View model](https://savrn.com/models/siglip2-base-patch16-256) Model · Zero shot image classification ### [CLIP-ViT-L-14-laion2B-s32B-b82K](https://savrn.com/models/clip-vit-l-14-laion2b-s32b-b82k) [LAION eV](https://savrn.com/model-publishers/laion) A CLIP ViT L/14 model trained with the LAION-2B English subset of LAION-5B (https://laion.ai/blog/laion-5b/) using OpenCLIP (https://github.com/mlfoundations/openclip). Model training ('babysitting') done by Ross Wightman on the JUWELS Booster supercomputer. See acknowledgements below. As per the original OpenAI CLIP model card, this model is intended as a research output for research communities. We hope that this model will enable researchers to better understand and explore zero-shot, arbitrary image classification. We also hope it can be used for interdisciplinary studies of the potential impact of such model. The OpenAI CLIP paper includes a discussion of potential downstream impacts… Open weights mit 428M parameters 77 tokens open_clip [View model](https://savrn.com/models/clip-vit-l-14-laion2b-s32b-b82k) Model · Zero shot image classification ### [fashion-clip](https://savrn.com/models/fashion-clip) [Patrick John Chia](https://savrn.com/model-publishers/patrickjohncyh) UPDATE (10/03/23): We have updated the model! We found that laion/CLIP-ViT-B-32-laion2B-s34B-b79K checkpoint (thanks Bin!) worked better than original OpenAI CLIP on Fashion. We thus fine-tune a newer (and better!) version of FashionCLIP (henceforth FashionCLIP 2.0), while keeping the architecture the same. We postulate that the perofrmance gains afforded by laion/CLIP-ViT-B-32-laion2B-s34B-b79K are due to the increased training data (5x OpenAI CLIP data). Our thesis, however, remains the same -- fine-tuning laion/CLIP on our fashion dataset improved zero-shot perofrmance across our benchmarks. See the below table comparing weighted macro F1 score across models. FashionCLIP is a CLIP-based… Open weights mit 151M parameters 77 tokens transformers [View model](https://savrn.com/models/fashion-clip) Model · Zero shot image classification ### [siglip-base-patch16-224](https://savrn.com/models/siglip-base-patch16-224) [Google](https://savrn.com/model-publishers/google) SigLIP model pre-trained on WebLi at resolution 224x224. It was introduced in the paper Sigmoid Loss for Language Image Pre-Training by Zhai et al. and first released in this repository. Disclaimer: The team releasing SigLIP did not write a model card for this model so this model card has been written by the Hugging Face team. SigLIP is CLIP, a multimodal model, with a better loss function. The sigmoid loss operates solely on image-text pairs and does not require a global view of the pairwise similarities for normalization. This allows further scaling up the batch size, while also performing better at smaller batch sizes. A TLDR of SigLIP by one of the authors can be found here. You can use… Open weights apache-2.0 203M parameters transformers [View model](https://savrn.com/models/siglip-base-patch16-224) Model · Zero shot image classification ### [siglip2-base-patch16-naflex](https://savrn.com/models/siglip2-base-patch16-naflex) [Google](https://savrn.com/model-publishers/google) SigLIP 2 extends the pretraining objective of SigLIP with prior, independently developed techniques into a unified recipe, for improved semantic understanding, localization, and dense features. You can use the raw model for tasks like zero-shot image classification and image-text retrieval, or as a vision encoder for VLMs (and other vision tasks). Here is how to use this model to perform zero-shot image classification: You can encode an image using the Vision Tower like so: For more code examples, we refer to the siglip2 documentation. SigLIP 2 adds some clever training objectives on top of SigLIP: SigLIP 2 is pre-trained on the WebLI dataset (Chen et al., 2023). The model was trained on up… Open weights apache-2.0 375M parameters transformers [View model](https://savrn.com/models/siglip2-base-patch16-naflex) Model · Zero shot image classification ### [siglip2-so400m-patch14-384](https://savrn.com/models/siglip2-so400m-patch14-384) [Google](https://savrn.com/model-publishers/google) SigLIP 2 extends the pretraining objective of SigLIP with prior, independently developed techniques into a unified recipe, for improved semantic understanding, localization, and dense features. You can use the raw model for tasks like zero-shot image classification and image-text retrieval, or as a vision encoder for VLMs (and other vision tasks). Here is how to use this model to perform zero-shot image classification: You can encode an image using the Vision Tower like so: For more code examples, we refer to the siglip documentation. SigLIP 2 adds some clever training objectives on top of SigLIP: SigLIP 2 is pre-trained on the WebLI dataset (Chen et al., 2023). The model was trained on up… Open weights apache-2.0 1.1B parameters transformers [View model](https://savrn.com/models/siglip2-so400m-patch14-384) Model · Zero shot image classification ### [siglip-so400m-patch14-384](https://savrn.com/models/siglip-so400m-patch14-384) [Google](https://savrn.com/model-publishers/google) SigLIP model pre-trained on WebLi at resolution 384x384. It was introduced in the paper Sigmoid Loss for Language Image Pre-Training by Zhai et al. and first released in this repository. This model has the SoViT-400m architecture, which is the shape-optimized version as presented in Getting ViT in Shape: Scaling Laws for Compute-Optimal Model Design by Alabdulmohsin et al. Disclaimer: The team releasing SigLIP did not write a model card for this model so this model card has been written by the Hugging Face team. SigLIP is CLIP, a multimodal model, with a better loss function. The sigmoid loss operates solely on image-text pairs and does not require a global view of the pairwise similarities… Open weights apache-2.0 878M parameters transformers [View model](https://savrn.com/models/siglip-so400m-patch14-384) Model · Zero shot image classification ### [clip-vit-large-patch14-336](https://savrn.com/models/clip-vit-large-patch14-336) [OpenAI](https://savrn.com/model-publishers/openai) This model was trained from scratch on an unknown dataset. It achieves the following results on the evaluation set: The following hyperparameters were used during training: - trainingprecision: float32 - Transformers 4.21.3 - TensorFlow 2.8.2 - Tokenizers 0.12.1 Open weights 77 tokens transformers [View model](https://savrn.com/models/clip-vit-large-patch14-336) Model · Zero shot image classification ### [DermLIP_ViT-B-16](https://savrn.com/models/dermlip-vit-b-16) [Siyuan Yan](https://savrn.com/model-publishers/redlessone) DermLIP is a vision-language model for dermatology, trained on the Derm1M dataset—the largest dermatological image-text corpus to date. python import openclip from PIL import Image import torch model,, preprocess = openclip.createmodelandtransforms( 'hf-hub:redlessone/DermLIPViT-B-16' model.eval() tokenizer = openclip.gettokenizer('hf-hub:redlessone/DermLIPViT-B-16') image = preprocess(Image.open("yourskinimage.png")).unsqueeze(0) PADCLASSNAMES = [ "nevus", "basal cell carcinoma", "actinic keratosis", "seborrheic keratosis", "squamous cell carcinoma", "melanoma" template = lambda c: f'This is a skin image of {c}' text = tokenizer([template(c) for c in PADCLASSNAMES]) with torch.nograd()… Open weights cc-by-4.0 transformers [View model](https://savrn.com/models/dermlip-vit-b-16) Model · Zero shot image classification ### [DermFM-Zero](https://savrn.com/models/dermfm-zero) [Siyuan Yan](https://savrn.com/model-publishers/redlessone) DermFM-Zero: A Vision-Language Foundation Model for Zero-shot Clinical Collaboration and Automated Concept Discovery in Dermatology DermFM-Zero pairs a PanDerm ViT-L/16 vision encoder with a knowledge-enhanced PubMedBERT text encoder, aligned on dermatology image–text pairs with multi-aspect knowledge contrastive learning. It classifies dermoscopic and clinical images from text prompts without task-specific training, retrieves images from text and text from images, serves as a backbone for multimodal fine-tuning and VQA, and exposes interpretable concepts through sparse autoencoders. In the paper, it was evaluated on diverse benchmarks and in three multinational reader studies with 761… Access requested at publisher cc-by-nc-4.0 open_clip [View model](https://savrn.com/models/dermfm-zero) Model · Zero shot image classification ### [siglip_openclip_m_16_general](https://savrn.com/models/siglip-openclip-m-16-general) [Xhafer Epiroti](https://savrn.com/model-publishers/bn22) Open weights mit open_clip [View model](https://savrn.com/models/siglip-openclip-m-16-general) Model · Zero shot image classification ### [Apex-large](https://savrn.com/models/apex-large) [SCX.ai Holdings Limited](https://savrn.com/model-publishers/scx-admin) Open weights apache-2.0 9B parameters gliclass [View model](https://savrn.com/models/apex-large) Model · Zero shot image classification ### [clip-vit-base-patch16](https://savrn.com/models/clip-vit-base-patch16-2) [Dang Quoc Dung](https://savrn.com/model-publishers/dungdq1) https://huggingface.co/openai/clip-vit-base-patch16 with ONNX weights to be compatible with Transformers.js. If you haven't already, you can install the Transformers.js JavaScript library from NPM using: Example: Perform zero-shot image classification with the pipeline API. Example: Compute vision embeddings with CLIPVisionModelWithProjection. Note: Having a separate repo for ONNX weights is intended to be a temporary solution until WebML gains more traction. If you would like to make your models web-ready, we recommend converting to ONNX using Optimum and structuring your repo like this one (with ONNX weights located in a subfolder named onnx). Open weights 77 tokens transformers.js [View model](https://savrn.com/models/clip-vit-base-patch16-2) Model · Zero shot image classification ### [siglip2-base-patch16-256-coreml](https://savrn.com/models/siglip2-base-patch16-256-coreml) [Fluid Inference Community](https://savrn.com/model-publishers/fluidinference) Core ML conversion of Google's SigLIP 2 google/siglip2-base-patch16-256, with both towers: an image encoder and a text encoder, each returning an L2-normalized 768-d embedding. Labels are plain text chosen at run time, so it classifies images zero-shot. Google authored the Apache-2.0 source model; Fluid Inference converted it. The image encoder runs entirely on the Apple Neural Engine. fp16 weights, macOS 14 / iOS 17 or newer. config.json holds the preprocessing and scoring constants: trained without an attention mask, so the padding must match. an independent probability per label; softmax or argmax over cos picks one label. Embed the labels once and reuse them for every image. ImageNet-1k… Open weights apache-2.0 coremltools [View model](https://savrn.com/models/siglip2-base-patch16-256-coreml) ## Questions ### Which Zero shot image classification models are most downloaded? By monthly downloads reported by the Hugging Face Hub: clip-vit-base-patch32 (16.9M); clip-vit-large-patch14 (6.9M); CLIP-ViT-B-32-laion2B-s34B-b79K (4M). ## Other Tasks - [Text Generation](https://savrn.com/models/tasks/text-generation) 1548 - [Image and Text to Text](https://savrn.com/models/tasks/image-and-text-to-text) 577 - [Feature Extraction](https://savrn.com/models/tasks/feature-extraction) 398 - [Robotics](https://savrn.com/models/tasks/robotics) 398 - [Text Classification](https://savrn.com/models/tasks/text-classification) 331 - [Text to Image](https://savrn.com/models/tasks/text-to-image) 232 - [Speech Recognition](https://savrn.com/models/tasks/speech-recognition) 187 - [Text to Speech](https://savrn.com/models/tasks/text-to-speech) 145 - [Image Classification](https://savrn.com/models/tasks/image-classification) 139 - [Audio Classification](https://savrn.com/models/tasks/audio-classification) 130 - [Sentence Similarity](https://savrn.com/models/tasks/sentence-similarity) 103 - [Reinforcement Learning](https://savrn.com/models/tasks/reinforcement-learning) 100 [See all](https://savrn.com/models/tasks)