CLIP-ViT-B-32-laion2B-s34B-b79K

Model · Zero shot image classification

fashion-clip

Patrick John Chia

UPDATE (10/03/23): We have updated the model! We found that laion/CLIP-ViT-B-32-laion2B-s34B-b79K checkpoint (thanks Bin!) worked better than original OpenAI CLIP on Fashion. We thus fine-tune a newer (and better!) version of FashionCLIP (henceforth FashionCLIP 2.0), while keeping the architecture the same. We postulate that the perofrmance gains afforded by laion/CLIP-ViT-B-32-laion2B-s34B-b79K are due to the increased training data (5x OpenAI CLIP data). Our thesis, however, remains the same -- fine-tuning laion/CLIP on our fashion dataset improved zero-shot perofrmance across our benchmarks. See the below table comparing weighted macro F1 score across models. FashionCLIP is a CLIP-based…

Model · Zero shot image classification

siglip-base-patch16-224

Google

SigLIP model pre-trained on WebLi at resolution 224x224. It was introduced in the paper Sigmoid Loss for Language Image Pre-Training by Zhai et al. and first released in this repository. Disclaimer: The team releasing SigLIP did not write a model card for this model so this model card has been written by the Hugging Face team. SigLIP is CLIP, a multimodal model, with a better loss function. The sigmoid loss operates solely on image-text pairs and does not require a global view of the pairwise similarities for normalization. This allows further scaling up the batch size, while also performing better at smaller batch sizes. A TLDR of SigLIP by one of the authors can be found here. You can use…

Model · Zero shot image classification

clip-vit-base-patch32

OpenAI

Disclaimer: The model card is taken and modified from the official CLIP repository, it can be found here. The CLIP model was developed by researchers at OpenAI to learn about what contributes to robustness in computer vision tasks. The model was also developed to test the ability of models to generalize to arbitrary image classification tasks in a zero-shot manner. It was not developed for general model deployment - to deploy models like CLIP, researchers will first need to carefully study their capabilities in relation to the specific context they’re being deployed within. January 2021 The model uses a ViT-B/32 Transformer architecture as an image encoder and uses a masked self-attention…

Model · Zero shot image classification

clip-vit-large-patch14-336

OpenAI

This model was trained from scratch on an unknown dataset. It achieves the following results on the evaluation set: The following hyperparameters were used during training: - trainingprecision: float32 - Transformers 4.21.3 - TensorFlow 2.8.2 - Tokenizers 0.12.1

Model · Zero shot image classification

clip-vit-base-patch16

OpenAI

Disclaimer: The model card is taken and modified from the official CLIP repository, it can be found here. The CLIP model was developed by researchers at OpenAI to learn about what contributes to robustness in computer vision tasks. The model was also developed to test the ability of models to generalize to arbitrary image classification tasks in a zero-shot manner. It was not developed for general model deployment - to deploy models like CLIP, researchers will first need to carefully study their capabilities in relation to the specific context they’re being deployed within. January 2021 The base model uses a ViT-B/16 Transformer architecture as an image encoder and uses a masked…

Model · Zero shot image classification

CLIP-convnext_base_w-laion2B-s13B-b82K-augreg

LAION eV

A series of CLIP ConvNeXt-Base (w/ wide embed dim) models trained on subsets LAION-5B (https://laion.ai/blog/laion-5b/) using OpenCLIP (https://github.com/mlfoundations/openclip). Explore an alternative to ViT and ResNet (w/ AttentionPooling) CLIP models that scales well with model size and image resolution First known ConvNeXt CLIP models trained at scale in the range of CLIP ViT-B/16 and RN50x4 models First released model weights exploring increase of augmentation + regularization for image tower via adding (greater scale range of RRC, random erasing, stochastic depth) The models utilize the timm ConvNeXt-Base model (convnextbase) as the image tower, and the same text tower as the RN50x4…