SAVRN
Search Contact SAVRN

Open-weight model · Zero shot image classification

fashion-clip

by Patrick John Chia patrickjohncyh/fashion-clip

UPDATE (10/03/23): We have updated the model! We found that laion/CLIP-ViT-B-32-laion2B-s34B-b79K checkpoint (thanks Bin!) worked better than original OpenAI CLIP on Fashion.

Parameters151M
Context77
Weights1.8 GB
Licensemit
AccessOpen weights
Monthly Downloads2.2M

Runs On

What it takes to serve fashion-clip (151M parameters): the memory its weights need at each precision, and the cheapest way to rent enough data-center GPUs to hold them.

PrecisionWeightsMemory neededCheapest setupPer hourAlso fits
16-bit 0.3 GB 0.4 GB 1x MI300X (192 GB)
Vultr		 $1.85 1x H100 $1.99 · 1x MI325X $2.00
8-bit 0.2 GB 0.2 GB 1x MI300X (192 GB)
Vultr		 $1.85 1x H100 $1.99 · 1x MI325X $2.00
4-bit 0.1 GB 0.1 GB 1x MI300X (192 GB)
Vultr		 $1.85 1x H100 $1.99 · 1x MI325X $2.00

Memory is the weights at that precision plus 20% for the runtime and a short context; a long context needs more. Prices are the lowest on-demand hourly rates in the SAVRN Index, read Sep 18, 2026.

Model Card

By Patrick John Chia, published under mit, revision 7e3ba62ce16b.

Model Card: Fashion CLIP

Disclaimer: The model card adapts the model card from here.

Model Details

UPDATE (10/03/23): We have updated the model! We found that laion/CLIP-ViT-B-32-laion2B-s34B-b79K checkpoint (thanks Bin!) worked better than original OpenAI CLIP on Fashion. We thus fine-tune a newer (and better!) version of FashionCLIP (henceforth FashionCLIP 2.0), while keeping the architecture the same. We postulate that the perofrmance gains afforded by laion/CLIP-ViT-B-32-laion2B-s34B-b79K are due to the increased training data (5x OpenAI CLIP data). Our thesis, however, remains the same -- fine-tuning laion/CLIP on our fashion dataset improved zero-shot perofrmance across our benchmarks. See the below table comparing weighted macro F1 score across models.

Model FMNIST KAGL DEEP
OpenAI CLIP 0.66 0.63 0.45
FashionCLIP 0.74 0.67 0.48
Laion CLIP 0.78 0.71 0.58
FashionCLIP 2.0 0.83 0.73 0.62

Read the full model card (732 words)

Configuration

Architecture
CLIPModel
Context length (tokens)
77
Layers
12
Hidden size
512
Feed-forward size
2,048
Attention heads
8
Vocabulary size
49,408
Stored precision
float32
Model type
clip

Identity and Version

Repository
patrickjohncyh/fashion-clip
Publisher
Patrick John Chia
Task
Zero shot image classification
Modality
Other
Library
transformers
Parameters
151M parameters
Languages
en
Revision
7e3ba62ce16b379a1ab479346b66f192e76f51b7
First published
2023-02-21
Last updated
2024-09-17

Files and Weights

19 files, 1.8 GB in total. The weights are 3 files totalling 1.8 GB in bin, onnx, safetensors.

Weights3 files · 1.8 GB
Configuration6 files · 6.7 KB
Tokenizer8 files · 7.2 MB
Documentation1 file · 7.0 KB
Repository1 file · 1.5 KB
Every file
FileTypeSizeSHA-256
model.safetensorsWeights605.2 MB 4977e3a54929
onnx/model.onnxWeights605.8 MB dc4c724479e4
pytorch_model.binWeights605.2 MB 5adfac18a5ed
config.jsonConfiguration4.5 KB —
onnx/config.jsonConfiguration455 B —
onnx/preprocessor_config.jsonConfiguration468 B —
onnx/special_tokens_map.jsonConfiguration588 B —
preprocessor_config.jsonConfiguration316 B —
special_tokens_map.jsonConfiguration389 B —
README.mdDocumentation7.0 KB —
.gitattributesRepository1.5 KB —
merges.txtTokenizer524.7 KB —
onnx/merges.txtTokenizer524.6 KB —
onnx/tokenizer.jsonTokenizer2.2 MB —
onnx/tokenizer_config.jsonTokenizer772 B —
onnx/vocab.jsonTokenizer862.3 KB —
tokenizer.jsonTokenizer2.2 MB —
tokenizer_config.jsonTokenizer568 B —
vocab.jsonTokenizer862.3 KB —

License and Download

License
mit
Access
Open weights, no gate
Download size
1.8 GB
Download from Patrick John Chia

Released by Patrick John Chia through its official repository on Hugging Face. Read the license.

Memory Requirements

PrecisionWeights in memory
As published1.8 GB
16-bit0.3 GB
8-bit0.2 GB
4-bit0.1 GB

Weights only, from the published parameter count; the key-value cache and runtime add to this.

Compare fashion-clip

Questions About fashion-clip

How much GPU memory does fashion-clip need?

About 0.4 GB at 16-bit and 0.1 GB at 4-bit: the weights (151M parameters) plus a working margin. A long context needs more.

What is the cheapest GPU to run fashion-clip on?

At 16-bit, 1x MI300X from $1.85 an hour; at 4-bit, 1x MI300X from $1.85 an hour, at the lowest on-demand prices the SAVRN Index lists.

Can I use fashion-clip commercially?

Yes. fashion-clip is released under MIT License. The MIT License is a short permissive license. It permits commercial use, modification and redistribution, provided the copyright notice and permission notice are included.

What is fashion-clip's context length?

77 tokens, from the maximum position embeddings in its published configuration.

Similar Models

Model · Zero shot image classification

CLIP-ViT-B-32-laion2B-s34B-b79K

LAION eV

A CLIP ViT-B/32 model trained with the LAION-2B English subset of LAION-5B (https://laion.ai/blog/laion-5b/) using OpenCLIP (https://github.com/mlfoundations/openclip). Model training done by Romain Beaumont on the stability.ai cluster. As per the original OpenAI CLIP model card, this model is intended as a research output for research communities. We hope that this model will enable researchers to better understand and explore zero-shot, arbitrary image classification. We also hope it can be used for interdisciplinary studies of the potential impact of such model. The OpenAI CLIP paper includes a discussion of potential downstream impacts to provide an example for this sort of analysis.…

Open weights mit 151M parameters 77 tokens open_clip
View model

Model · Zero shot image classification

siglip-base-patch16-224

Google

SigLIP model pre-trained on WebLi at resolution 224x224. It was introduced in the paper Sigmoid Loss for Language Image Pre-Training by Zhai et al. and first released in this repository. Disclaimer: The team releasing SigLIP did not write a model card for this model so this model card has been written by the Hugging Face team. SigLIP is CLIP, a multimodal model, with a better loss function. The sigmoid loss operates solely on image-text pairs and does not require a global view of the pairwise similarities for normalization. This allows further scaling up the batch size, while also performing better at smaller batch sizes. A TLDR of SigLIP by one of the authors can be found here. You can use…

Open weights apache-2.0 203M parameters transformers
View model

Model · Zero shot image classification

clip-vit-base-patch32

OpenAI

Disclaimer: The model card is taken and modified from the official CLIP repository, it can be found here. The CLIP model was developed by researchers at OpenAI to learn about what contributes to robustness in computer vision tasks. The model was also developed to test the ability of models to generalize to arbitrary image classification tasks in a zero-shot manner. It was not developed for general model deployment - to deploy models like CLIP, researchers will first need to carefully study their capabilities in relation to the specific context they’re being deployed within. January 2021 The model uses a ViT-B/32 Transformer architecture as an image encoder and uses a masked self-attention…

Open weights 77 tokens transformers
View model

Model · Zero shot image classification

clip-vit-large-patch14-336

OpenAI

This model was trained from scratch on an unknown dataset. It achieves the following results on the evaluation set: The following hyperparameters were used during training: - trainingprecision: float32 - Transformers 4.21.3 - TensorFlow 2.8.2 - Tokenizers 0.12.1

Open weights 77 tokens transformers
View model

Model · Zero shot image classification

clip-vit-base-patch16

OpenAI

Disclaimer: The model card is taken and modified from the official CLIP repository, it can be found here. The CLIP model was developed by researchers at OpenAI to learn about what contributes to robustness in computer vision tasks. The model was also developed to test the ability of models to generalize to arbitrary image classification tasks in a zero-shot manner. It was not developed for general model deployment - to deploy models like CLIP, researchers will first need to carefully study their capabilities in relation to the specific context they’re being deployed within. January 2021 The base model uses a ViT-B/16 Transformer architecture as an image encoder and uses a masked…

Open weights 77 tokens transformers
View model

Model · Zero shot image classification

CLIP-convnext_base_w-laion2B-s13B-b82K-augreg

LAION eV

A series of CLIP ConvNeXt-Base (w/ wide embed dim) models trained on subsets LAION-5B (https://laion.ai/blog/laion-5b/) using OpenCLIP (https://github.com/mlfoundations/openclip). Explore an alternative to ViT and ResNet (w/ AttentionPooling) CLIP models that scales well with model size and image resolution First known ConvNeXt CLIP models trained at scale in the range of CLIP ViT-B/16 and RN50x4 models First released model weights exploring increase of augmentation + regularization for image tower via adding (greater scale range of RRC, random erasing, stochastic depth) The models utilize the timm ConvNeXt-Base model (convnextbase) as the image tower, and the same text tower as the RN50x4…

Open weights mit open_clip
View model