Open-Weight Models Under MIT License

Model · Image to text

kosmos-2-patch14-224

Microsoft

This Hub repository contains a HuggingFace's transformers implementation of the original Kosmos-2 model from Microsoft. Use the code below to get started with the model. This model is capable of performing different tasks through changing the prompts. First, let's define a function to run a prompt. Here are the tasks Kosmos-2 could perform: Once you have the entities, you can use the following helper function to draw their bounding bboxes on the image

Model · Image segmentation

oneformer_ade20k_swin_large

SHI Labs

OneFormer model trained on the ADE20k dataset (large-sized version, Swin backbone). It was introduced in the paper OneFormer: One Transformer to Rule Universal Image Segmentation by Jain et al. and first released in this repository. OneFormer is the first multi-task universal image segmentation framework. It needs to be trained only once with a single universal architecture, a single model, and on a single dataset, to outperform existing specialized models across semantic, instance, and panoptic segmentation tasks. OneFormer uses a task token to condition the model on the task in focus, making the architecture task-guided for training, and task-dynamic for inference, all with a single…

Model · Summarization

bart-large-cnn-samsum

Philipp Schmid

This model was trained using Amazon SageMaker and the new Hugging Face Deep Learning container.

Model · Image classification

swin_base_patch4_window12_384.ms_in22k_ft_in1k

PyTorch Image Models

A Swin Transformer image classification model. Pretrained on ImageNet-22k and fine-tuned on ImageNet-1k by paper authors. Explore the dataset and runtime metrics of this model in timm model results.

Model · Image text to image

HiDream-O1-Image-FP8

DRBAPH

This is the FP8 mixed-precision quantization of HiDream-O1-Image for use with ComfyUI. By quantizing to 8-bit floats, the model fits comfortably within ~10 GB of VRAM — making it accessible on 12 GB GPUs (RTX 3080/4070/4080, etc.) with minimal quality trade-off. This is the recommended variant for GPUs with less than 16 GB VRAM. Tested on 12 GB cards at 2048 × 2048 resolution. Or install via ComfyUI Manager by searching for HiDream O1. Open ComfyUI and use the workflow provided in the custom node repository. Point the model loader to HiDream-O1-Image-fp8. HiDream-O1-Image is a natively unified image generative foundation model built on a Pixel-level Unified Transformer (UiT) — no external…

Model · Image segmentation

oneformer_cityscapes_swin_large

SHI Labs

OneFormer model trained on the Cityscapes dataset (large-sized version, Swin backbone). It was introduced in the paper OneFormer: One Transformer to Rule Universal Image Segmentation by Jain et al. and first released in this repository. OneFormer is the first multi-task universal image segmentation framework. It needs to be trained only once with a single universal architecture, a single model, and on a single dataset, to outperform existing specialized models across semantic, instance, and panoptic segmentation tasks. OneFormer uses a task token to condition the model on the task in focus, making the architecture task-guided for training, and task-dynamic for inference, all with a single…

Model · Translation

mbart-large-en-ro

AI at Meta

This is mbart-large-cc25, finetuned on wmtenro. It scores BLEU 28.1 without post processing and BLEU 38 with postprocessing. Instructions in romanianpostprocessing.md Original Code: https://github.com/pytorch/fairseq/tree/master/examples/mbart Docs: https://huggingface.co/transformers/master/modeldoc/mbart.html

Model · Video classification

vivit-b-16x2-kinetics400

Google

ViViT model as introduced in the paper ViViT: A Video Vision Transformer by Arnab et al. and first released in this repository. Disclaimer: The team releasing ViViT did not write a model card for this model so this model card has been written by the Hugging Face team. ViViT is an extension of the Vision Transformer (ViT) to video. We refer to the paper for details. The model is mostly meant to intended to be fine-tuned on a downstream task, like video classification. See the model hub to look for fine-tuned versions on a task that interests you. For code examples, we refer to the documentation.

Model · Image segmentation

oneformer_coco_swin_large

SHI Labs

OneFormer model trained on the COCO dataset (large-sized version, Swin backbone). It was introduced in the paper OneFormer: One Transformer to Rule Universal Image Segmentation by Jain et al. and first released in this repository. OneFormer is the first multi-task universal image segmentation framework. It needs to be trained only once with a single universal architecture, a single model, and on a single dataset, to outperform existing specialized models across semantic, instance, and panoptic segmentation tasks. OneFormer uses a task token to condition the model on the task in focus, making the architecture task-guided for training, and task-dynamic for inference, all with a single model.…

Model · Image segmentation

oneformer_ade20k_swin_tiny

SHI Labs

OneFormer model trained on the ADE20k dataset (tiny-sized version, Swin backbone). It was introduced in the paper OneFormer: One Transformer to Rule Universal Image Segmentation by Jain et al. and first released in this repository. OneFormer is the first multi-task universal image segmentation framework. It needs to be trained only once with a single universal architecture, a single model, and on a single dataset, to outperform existing specialized models across semantic, instance, and panoptic segmentation tasks. OneFormer uses a task token to condition the model on the task in focus, making the architecture task-guided for training, and task-dynamic for inference, all with a single model.…

Donut model pre-trained-only. It was introduced in the paper OCR-free Document Understanding Transformer by Geewok et al. and first released in this repository. Disclaimer: The team releasing Donut did not write a model card for this model so this model card has been written by the Hugging Face team. Donut consists of a vision encoder (Swin Transformer) and a text decoder (BART). Given an image, the encoder first encodes the image into a tensor of embeddings (of shape batchsize, seqlen, hiddensize), after which the decoder autoregressively generates text, conditioned on the encoding of the encoder. This model is meant to be fine-tuned on a downstream task, like document image classification…

This repo contains the inference code to use pretrained human voice gender classifier. - You could also try Huggingface online demo. First, clone the original github repository and install the packages via pip. For those who need pretrained weights, please download it in here State-of-the-art speaker verification model already produces good representation of the speaker's gender. I used the pretrained ECAPA-TDNN from TaoRuijie's repository, added one linear layer to make two-class classifier, and finetuned the model with the VoxCeleb2 dev set. The model achieved 98.7% accuracy on the VoxCeleb1 identification test split. I would like to note the training dataset I've used for this model…

VibeVoice is a novel framework designed for generating expressive, long-form, multi-speaker conversational audio, such as podcasts, from text. It addresses significant challenges in traditional Text-to-Speech (TTS) systems, particularly in scalability, speaker consistency, and natural turn-taking. A core innovation of VibeVoice is its use of continuous speech tokenizers (Acoustic and Semantic) operating at an ultra-low frame rate of 7.5 Hz. These tokenizers efficiently preserve audio fidelity while significantly boosting computational efficiency for processing long sequences. VibeVoice employs a next-token diffusion framework, leveraging a Large Language Model (LLM) to understand textual…

Model · Zero-shot classification

deberta-v3-xsmall-zeroshot-v1.1-all-33

Moritz Borrett-Laurer (formerly Laurer)

This model was fine-tuned using the same pipeline as described in the model card for MoritzLaurer/deberta-v3-large-zeroshot-v1.1-all-33 and in this paper. The foundation model is microsoft/deberta-v3-xsmall. The model only has 22 million backbone parameters and 128 million vocabulary parameters. The backbone parameters are the main parameters active during inference, providing a significant speedup over larger models. The model is 142 MB small. This model was trained to provide a small and highly efficient zeroshot option, especially for edge devices or in-browser use-cases with transformers.js. For usage instructions and other details refer to this model card…

Model · Text to speech

MeloTTS-Korean

MyShell.ai

MeloTTS is a high-quality multi-lingual text-to-speech library by MyShell.ai. Supported languages include: - The Chinese speaker supports mixed Chinese and English. - Fast enough for CPU real-time inference. An unofficial live demo is hosted on Hugging Face Spaces. There are hundreds of TTS models on MyShell, much more than MeloTTS. See examples here. More can be found at the widget center of MyShell.ai. Follow the installation steps here before using the following snippet: Open Source AI Grant We are actively sponsoring open-source AI projects. The sponsorship includes GPU resources, fundings and intellectual support (collaboration with top research labs). We welcome both reseach and…

Model · Image segmentation

coco_panoptic_eomt_large_640

Mobile Perception Systems Lab

EoMT (Encoder-only Mask Transformer) is a Vision Transformer (ViT) architecture designed for high-quality and efficient image segmentation. It was introduced in the CVPR 2025 highlight paper: by Tommie Kerssies, Niccolò Cavagnero, Alexander Hermans, Narges Norouzi, Giuseppe Averta, Bastian Leibe, Gijs Dubbelman, and Daan de Geus. The original implementation can be found in this repository. The HuggingFace model page is available at this link. Here is how to use this model for Panotpic Segmentation: If you find our work useful, please consider citing us as

Model · Image segmentation

upernet-convnext-tiny

OpenMMLab

UperNet framework for semantic segmentation, leveraging a ConvNeXt backbone. UperNet was introduced in the paper Unified Perceptual Parsing for Scene Understanding by Xiao et al. Combining UperNet with a ConvNeXt backbone was introduced in the paper A ConvNet for the 2020s. Disclaimer: The team releasing UperNet + ConvNeXt did not write a model card for this model so this model card has been written by the Hugging Face team. UperNet is a framework for semantic segmentation. It consists of several components, including a backbone, a Feature Pyramid Network (FPN) and a Pyramid Pooling Module (PPM). Any visual backbone can be plugged into the UperNet framework. The framework predicts a…

Model · Translation

ProstT5

Rostlab

ProstT5 is a protein language model (pLM) which can translate between protein sequence and structure. ProstT5 (Protein structure-sequence T5) is based on ProtT5-XL-U50, a T5 model trained on encoding protein sequences using span corruption applied on billions of protein sequences. ProstT5 finetunes ProtT5-XL-U50 on translating between protein sequence and structure using 17M proteins with high-quality 3D structure predictions from the AlphaFoldDB. Protein structure is converted from 3D to 1D using the 3Di-tokens introduced by Foldseek. In a first step, ProstT5 learnt to represent the newly introduced 3Di-tokens by continuing the original span-denoising objective applied on 3Di- and amino…

Model · Image to text

GLM-OCR-4bit

MLX Community

This model was converted to MLX format from zai-org/GLM-OCR using mlx-vlm version 0.3.10. Refer to the original model card for more details on the model.

https://github.com/vibevoice-community/VibeVoice VibeVoice is a novel framework designed for generating expressive, long-form, multi-speaker conversational audio, such as podcasts, from text. It addresses significant challenges in traditional Text-to-Speech (TTS) systems, particularly in scalability, speaker consistency, and natural turn-taking. A core innovation of VibeVoice is its use of continuous speech tokenizers (Acoustic and Semantic) operating at an ultra-low frame rate of 7.5 Hz. These tokenizers efficiently preserve audio fidelity while significantly boosting computational efficiency for processing long sequences. VibeVoice employs a next-token diffusion framework, leveraging a…

Model · Image segmentation

BiRefNet_HR

Peng Zheng

This repo is the official implementation of "Bilateral Reference for High-Resolution Dichotomous Image Segmentation" (CAAI AIR 2024). Visit our GitHub repo: https://github.com/ZhengPeng7/BiRefNet for more details -- codes, docs, and model zoo! This repo contains the weights of BiRefNet proposed in our paper, which has achieved the SOTA performance on three tasks (DIS, HRSOD, and COD). Go to my GitHub page for BiRefNet codes and the latest updates: https://github.com/ZhengPeng7/BiRefNet:) + Online Image Inference on Colab: + Online Inference with GUI on Hugging Face with adjustable resolutions: + Inference and evaluation of your given weights: + Many thanks to @freepik for their generous…

Model · Text to speech

speecht5_tts

Microsoft

SpeechT5 model fine-tuned for speech synthesis (text-to-speech) on LibriTTS. This model was introduced in SpeechT5: Unified-Modal Encoder-Decoder Pre-Training for Spoken Language Processing by Junyi Ao, Rui Wang, Long Zhou, Chengyi Wang, Shuo Ren, Yu Wu, Shujie Liu, Tom Ko, Qing Li, Yu Zhang, Zhihua Wei, Yao Qian, Jinyu Li, Furu Wei. SpeechT5 was first released in this repository, original weights. The license used is MIT. Motivated by the success of T5 (Text-To-Text Transfer Transformer) in pre-trained natural language processing models, we propose a unified-modal SpeechT5 framework that explores the encoder-decoder pre-training for self-supervised speech/text representation learning. The…

Model · Text to speech

MeloTTS-Chinese

MyShell.ai

MeloTTS is a high-quality multi-lingual text-to-speech library by MyShell.ai. Supported languages include: - The Chinese speaker supports mixed Chinese and English. - Fast enough for CPU real-time inference. An unofficial live demo is hosted on Hugging Face Spaces. There are hundreds of TTS models on MyShell, much more than MeloTTS. See examples here. More can be found at the widget center of MyShell.ai. Follow the installation steps here before using the following snippet: Open Source AI Grant We are actively sponsoring open-source AI projects. The sponsorship includes GPU resources, fundings and intellectual support (collaboration with top research labs). We welcome both reseach and…

Model · Text classification

DeBERTa-v3-base-mnli-fever-docnli-ling-2c

Moritz Borrett-Laurer (formerly Laurer)

This model was trained on 1.279.665 hypothesis-premise pairs from 8 NLI datasets: MultiNLI, Fever-NLI, LingNLI and DocNLI (which includes ANLI, QNLI, DUC, CNN/DailyMail, Curation). It is the only model in the model hub trained on 8 NLI datasets, including DocNLI with very long texts to learn long range reasoning. Note that the model was trained on binary NLI to predict either "entailment" or "not-entailment". The DocNLI merges the classes "neural" and "contradiction" into "not-entailment" to enable the inclusion of the DocNLI dataset. The base model is DeBERTa-v3-base from Microsoft. The v3 variant of DeBERTa substantially outperforms previous versions of the model by including a different…

Model · Zero-shot classification

multilingual-MiniLMv2-L12-mnli-xnli

Moritz Borrett-Laurer (formerly Laurer)

multilingual - zero-shot-classification - text-classification - nli - pytorch - accuracy - multinli - xnli pipelinetag: zero-shot-classification candidatelabels: "politics, economy, entertainment, environment" This multilingual model can perform natural language inference (NLI) on 100+ languages and is therefore also suitable for multilingual zero-shot classification. The underlying multilingual-MiniLM-L12 model was created by Microsoft and was distilled from XLM-RoBERTa-large (see details in the original paper and newer information in this repo). The model was then fine-tuned on the XNLI dataset, which contains hypothesis-premise pairs from 15 languages, as well as the English MNLI…

Model · Zero-shot classification

deberta-v3-large-zeroshot-v1

Moritz Borrett-Laurer (formerly Laurer)

The model is designed for zero-shot classification with the Hugging Face pipeline. The model should be substantially better at zero-shot classification than my other zero-shot models on the The model can do one universal task: determine whether a hypothesis is true or nottrue given a text (also called entailment vs. notentailment). This task format is based on the Natural Language Inference task (NLI). The task is so universal that any classification task can be reformulated into the task. The model was trained on a mixture of 27 tasks and 310 classes that have been reformatted into this universal format. 1. 26 classification tasks with ~400k texts: 'amazonpolarity', 'imdb', 'appreviews'…

Model · Time series forecasting

MOMENT-1-base

Auton Lab

MOMENT is a family of foundation models for general-purpose time-series analysis. The models in this family (1) serve as a building block for diverse time-series analysis tasks (e.g., forecasting, classification, anomaly detection, and imputation, etc.), (2) are effective out-of-the-box, i.e., with no (or few) task-specific exemplars (enabling e.g., zero-shot forecasting, few-shot classification, etc.), and (3) are tunable using in-distribution and task-specific data to improve performance. For details on MOMENT models, training data, and experimental results, please refer to the paper MOMENT: A Family of Open Time-series Foundation Models. Recommended Python Version: Python 3.11 (support…

Model · Time series forecasting

MOMENT-1-small

Auton Lab

MOMENT is a family of foundation models for general-purpose time-series analysis. The models in this family (1) serve as a building block for diverse time-series analysis tasks (e.g., forecasting, classification, anomaly detection, and imputation, etc.), (2) are effective out-of-the-box, i.e., with no (or few) task-specific exemplars (enabling e.g., zero-shot forecasting, few-shot classification, etc.), and (3) are tunable using in-distribution and task-specific data to improve performance. For details on MOMENT models, training data, and experimental results, please refer to the paper MOMENT: A Family of Open Time-series Foundation Models. Recommended Python Version: Python 3.11 (support…

Model · Zero-shot classification

deberta-v3-base-zeroshot-v2.0

Moritz Borrett-Laurer (formerly Laurer)

Models in this series are designed for efficient zeroshot classification with the Hugging Face pipeline. These models can do classification without training data and run on both GPUs and CPUs. An overview of the latest zeroshot classifiers is available in my Zeroshot Classifier Collection. The main update of this zeroshot-v2.0 series of models is that several models are trained on fully commercially-friendly data for users with strict license requirements. These models can do one universal classification task: determine whether a hypothesis is "true" or "not true" given a text (entailment vs. notentailment). This task format is based on the Natural Language Inference task (NLI). The task is…

Model · Image segmentation

BiRefNet_HR-matting

Peng Zheng

This repo is the official implementation of "Bilateral Reference for High-Resolution Dichotomous Image Segmentation" (CAAI AIR 2024). Visit our GitHub repo: https://github.com/ZhengPeng7/BiRefNet for more details -- codes, docs, and model zoo! This repo contains the weights of BiRefNet proposed in our paper, which has achieved the SOTA performance on three tasks (DIS, HRSOD, and COD). Go to my GitHub page for BiRefNet codes and the latest updates: https://github.com/ZhengPeng7/BiRefNet:) + Online Image Inference on Colab: + Online Inference with GUI on Hugging Face with adjustable resolutions: + Inference and evaluation of your given weights: + Many thanks to @freepik for their generous…

Model · Video classification

vjepa2-vitl-fpc16-256-ssv2

AI at Meta

A frontier video understanding model developed by FAIR, Meta, which extends the pretraining objectives of VJEPA, resulting in state-of-the-art video understanding capabilities, leveraging data and model sizes at scale. The code is released in this repository. This is V-JEPA 2 ViT-L 256 model with video classification head pretrained on Something-Something-V2 dataset. To run V-JEPA 2 model, ensure you have installed the latest transformers

Model · Time series forecasting

MOMENT-1-large

Auton Lab

MOMENT is a family of foundation models for general-purpose time-series analysis. The models in this family (1) serve as a building block for diverse time-series analysis tasks (e.g., forecasting, classification, anomaly detection, and imputation, etc.), (2) are effective out-of-the-box, i.e., with no (or few) task-specific exemplars (enabling e.g., zero-shot forecasting, few-shot classification, etc.), and (3) are tunable using in-distribution and task-specific data to improve performance. For details on MOMENT models, training data, and experimental results, please refer to the paper MOMENT: A Family of Open Time-series Foundation Models. Recommended Python Version: Python 3.11 (support…

Model · Question answering

bert-medium-squad2-distilled

Deepset

Haystack version 1.x distillation feature was used for training. deepset/bert-large-uncased-whole-word-masking-squad2 was used as the teacher model. Haystack is an AI orchestration framework to build customizable, production-ready LLM applications. You can use this model in Haystack to do extractive question answering on documents. To load and run the model with Haystack: For a complete example with an extractive question answering pipeline that scales over many documents, check out the corresponding Haystack tutorial. - Timo Möller: timo.moeller [at] deepset.ai deepset is the company behind the production-ready open-source AI framework Haystack. We also have a Discord community open to…

Model · Video classification

vjepa2.1-vit-large-384

Antonio Apicella

A HuggingFace-format conversion of Meta AI's V-JEPA 2.1 ViT-L/16 video encoder and predictor, operating at 384x384 resolution. The weights are Meta's, copied without modification. This repository provides the transformers-compatible packaging plus a documented numerical validation against the original implementation. An equivalent community port already exists (Dev-Jahn/vjepa2.1-vitl-fpc64-384). This repository adds an independently reproduced conversion together with the validation results below. Forward outputs match the existing port to all reported digits. The only structural change is that the fused QKV projection of each attention block is split into separate query / key / value…

Model · Image to text

git-base

Microsoft

GIT (short for GenerativeImage2Text) model, base-sized version. It was introduced in the paper GIT: A Generative Image-to-text Transformer for Vision and Language by Wang et al. and first released in this repository. Disclaimer: The team releasing GIT did not write a model card for this model so this model card has been written by the Hugging Face team. GIT is a Transformer decoder conditioned on both CLIP image tokens and text tokens. The model is trained using "teacher forcing" on a lot of (image, text) pairs. The goal for the model is simply to predict the next text token, giving the image tokens and previous text tokens. The model has full access to (i.e. a bidirectional attention mask…

Model · Image segmentation

upernet-swin-large

OpenMMLab

UperNet framework for semantic segmentation, leveraging a Swin Transformer backbone. UperNet was introduced in the paper Unified Perceptual Parsing for Scene Understanding by Xiao et al. Combining UperNet with a Swin Transformer backbone was introduced in the paper Swin Transformer: Hierarchical Vision Transformer using Shifted Windows. Disclaimer: The team releasing UperNet + Swin Transformer did not write a model card for this model so this model card has been written by the Hugging Face team. UperNet is a framework for semantic segmentation. It consists of several components, including a backbone, a Feature Pyramid Network (FPN) and a Pyramid Pooling Module (PPM). Any visual backbone can…

Model · Audio classification

WeSpeaker-ResNet34-LM-MLX

Ivan

MLX-compatible weights for WeSpeaker ResNet34-LM, converted from the pyannote speaker embedding model with BatchNorm fused into Conv2d. WeSpeaker ResNet34-LM is a speaker embedding model (~6.6M params) that produces 256-dimensional L2-normalized speaker embeddings from audio. Trained on VoxCeleb for speaker verification and diarization. BatchNorm is fused into Conv2d at conversion time — no BN layers in the MLX model. Part of speech-swift. Converts the original pyannote/wespeaker-voxceleb-resnet34-LM checkpoint using a custom unpickler (no pyannote.audio dependency required). Key transformations: - Fuse BatchNorm into Conv2d: wfused = w × γ/√(σ²+ε), bfused = β − μ×γ/√(σ²+ε) - Transpose…

Model · Image and text to text

Florence-2-base-ft

ONNX Community

https://huggingface.co/microsoft/Florence-2-base-ft with ONNX weights to be compatible with Transformers.js. If you haven't already, you can install the Transformers.js JavaScript library from NPM using: Example: Perform image captioning with onnx-community/Florence-2-base-ft. We also released an online demo, which you can try yourself: https://huggingface.co/spaces/Xenova/florence2-webgpu Note: Having a separate repo for ONNX weights is intended to be a temporary solution until WebML gains more traction. If you would like to make your models web-ready, we recommend converting to ONNX using Optimum and structuring your repo like this one (with ONNX weights located in a subfolder named onnx).

Model · Any to any

Janus-Pro-1B

DeepSeek-community

Janus-Pro is a novel autoregressive framework that unifies multimodal understanding and generation. It addresses the limitations of previous approaches by decoupling visual encoding into separate pathways, while still utilizing a single, unified transformer architecture for processing. The decoupling not only alleviates the conflict between the visual encoder’s roles in understanding and generation, but also enhances the framework’s flexibility. Janus-Pro surpasses previous unified model and matches or exceeds the performance of task-specific models. The simplicity, high flexibility, and effectiveness of Janus-Pro make it a strong candidate for next-generation unified multimodal models.…

Model · Question answering

mobilebert-uncased-squad-v2

Qingqing Cao

MobileBERT is a thin version of BERTLARGE, while equipped with bottleneck structures and a carefully designed balance between self-attentions and feed-forward networks. This model was fine-tuned from the HuggingFace checkpoint google/mobilebert-uncased on SQuAD2.0. CPU: Intel(R) Core(TM) i7-6800K CPU @ 3.40GHz Memory: 32 GiB GPUs: 2 GeForce GTX 1070, each with 8GiB memory GPU driver: 418.87.01, CUDA: 10.1 It took about 3.5 hours to finish. Note that the above results didn't involve any hyperparameter search.

Donut model fine-tuned on DocVQA. It was introduced in the paper OCR-free Document Understanding Transformer by Geewok et al. and first released in this repository. Disclaimer: The team releasing Donut did not write a model card for this model so this model card has been written by the Hugging Face team. Donut consists of a vision encoder (Swin Transformer) and a text decoder (BART). Given an image, the encoder first encodes the image into a tensor of embeddings (of shape batchsize, seqlen, hiddensize), after which the decoder autoregressively generates text, conditioned on the encoding of the encoder. This model is fine-tuned on DocVQA, a document visual question answering dataset. We…

Model · Image to text

mscoco_finetuned_CoCa-ViT-L-14-laion2B-s13B-b90k

LAION eV

pipelinetag: image-to-text

Model · Video classification

xclip-base-patch32-16-frames

Microsoft

X-CLIP model (base-sized, patch resolution of 32) trained fully-supervised on Kinetics-400. It was introduced in the paper Expanding Language-Image Pretrained Models for General Video Recognition by Ni et al. and first released in this repository. This model was trained using 16 frames per video, at a resolution of 224x224. Disclaimer: The team releasing X-CLIP did not write a model card for this model so this model card has been written by the Hugging Face team. X-CLIP is a minimal extension of CLIP for general video-language understanding. The model is trained in a contrastive way on (video, text) pairs. This allows the model to be used for tasks like zero-shot, few-shot or fully…

Model · Image segmentation

BiRefNet-portrait

Peng Zheng

Check the main BiRefNet model repo for more info and how to use it: https://huggingface.co/ZhengPeng7/BiRefNet/blob/main/README.md Also check the GitHub repo of BiRefNet for all things you may want: https://github.com/ZhengPeng7/BiRefNet + Many thanks to @fal for their generous support on GPU resources for training this BiRefNet for portrait matting.

Model · Image segmentation

BiRefNet_dynamic

Peng Zheng

For performance of different epochs, check the evalresults-xxx folder for it on my google drive. This repo is the official implementation of "Bilateral Reference for High-Resolution Dichotomous Image Segmentation" (CAAI AIR 2024). Visit our GitHub repo: https://github.com/ZhengPeng7/BiRefNet for more details -- codes, docs, and model zoo! This repo contains the weights of BiRefNet proposed in our paper, which has achieved the SOTA performance on three tasks (DIS, HRSOD, and COD). Go to my GitHub page for BiRefNet codes and the latest updates: https://github.com/ZhengPeng7/BiRefNet:) + Online Image Inference on Colab: + Online Inference with GUI on Hugging Face with adjustable resolutions: +…

Model · Audio classification

wavlm-emotion-russian-resd

Aniemore

Speech emotion recognition for Russian over seven classes: anger, disgust, enthusiasm, fear, happiness, neutral, sadness. Fine-tuned from jonatasgrosman/expw2v2truwavlms363 on Aniemore/resd. Audio resampled to 16 kHz mono, clips capped at 12 s, normalized per utterance, padding masked. UA is macro-averaged recall, WA is accuracy, F1 is macro-averaged. All three test sets went through the same harness, so the rows are comparable to each other. The RESD split matches fold 1 of EmoBox bit for bit. The top entry there is WavLM-large at WA 56.47 / UA 55.87 / F1 55.82. These numbers are higher, but the training protocol differs — EmoBox freezes the encoder and trains a probe, this is a full…

Model · Video classification

vivit-b-16x2

Google

ViViT model as introduced in the paper ViViT: A Video Vision Transformer by Arnab et al. and first released in this repository. Disclaimer: The team releasing ViViT did not write a model card for this model so this model card has been written by the Hugging Face team. ViViT is an extension of the Vision Transformer (ViT) to video. We refer to the paper for details. The model is mostly meant to intended to be fine-tuned on a downstream task, like video classification. See the model hub to look for fine-tuned versions on a task that interests you. For code examples, we refer to the documentation.

Model · Image segmentation

BiRefNet-matting

Peng Zheng

Check the main BiRefNet model repo for more info and how to use it: https://huggingface.co/ZhengPeng7/BiRefNet/blob/main/README.md Also check the GitHub repo of BiRefNet for all things you may want: https://github.com/ZhengPeng7/BiRefNet + Many thanks to @freepik for their generous support on GPU resources for training this model!

Model · Zero-shot classification

deberta-v3-base-zeroshot-v1.1-all-33

Moritz Borrett-Laurer (formerly Laurer)

The model is designed for zero-shot classification with the Hugging Face pipeline. The model can do one universal classification task: determine whether a hypothesis is "true" or "not true" given a text (entailment vs. notentailment). This task format is based on the Natural Language Inference task (NLI). The task is so universal that any classification task can be reformulated into this task. A detailed description of how the model was trained and how it can be used is available in this paper. The model was trained on a mixture of 33 datasets and 387 classes that have been reformatted into this universal format. 1. Five NLI datasets with ~885k texts: "mnli", "anli", "fever", "wanli"…

Model · Image to text

pix2text-mfr

Breezedeus

Mathematical Formula Recognition (MFR) model from Pix2Text (P2T). This MFR model utilizes the TrOCR architecture developed by Microsoft, starting with its initial values and retrained using a dataset of mathematical formula images. The resulting MFR model can be used to convert images of mathematical formulas into LaTeX text representation. More detailed can be found: Pix2Text V1.0 New Release: The Best Open-Source Formula Recognition Model | Breezedeus.com. 此 MFR 模型使用了微软的 TrOCR 架构，以其为初始值并利用数学公式图片数据集进行了重新训练。 获得的 MFR 模型可用于把数学公式图片转换为 LaTeX 文本表示。更多细节请见：Pix2Text V1.0 新版发布：最好的开源公式识别模型 | Breezedeus.com。 - 用途：此模型为数学公式识别模型，它可以把输入的数学公式图片转换为 LaTeX 文本表示。 This method doesn't need to install pix2text…

Model · Text generation

Crowd-v1

Vxtzq

CrowdGPT's first community-distributed language model architecture. Crowd-v1 is the first official model architecture released for CrowdGPT, a community-driven distributed AI project. Unlike a conventional pretrained model release, Crowd-v1 is distributed with randomly initialized weights. The purpose of this release is to provide a common model definition and weight format that CrowdGPT clients can download and collectively train. The model is designed to be consumed by the CrowdGPT distributed training infrastructure, where individual participants contribute compute toward training a shared model. Crowd-v1 contains approximately 1 billion parameters. Grouped-Query Attention (GQA) Crowd-v1…

Model · Audio classification

sonics-spectttra-alpha-120s

Awsaf

The recent surge in AI-generated songs presents exciting possibilities and challenges. These innovations necessitate the ability to distinguish between human-composed and synthetic songs to safeguard artistic integrity and protect human musical artistry. Existing research and datasets in fake song detection only focus on singing voice deepfake detection (SVDD), where the vocals are AI-generated but the instrumental music is sourced from real songs. However, these approaches are inadequate for detecting contemporary end-to-end artificial songs where all components (vocals, music, lyrics, and style) could be AI-generated. Additionally, existing datasets lack music-lyrics diversity…

Model · Zero-shot classification

DeBERTa-v3-xsmall-mnli-fever-anli-ling-binary

Moritz Borrett-Laurer (formerly Laurer)

This model was trained on 782 357 hypothesis-premise pairs from 4 NLI datasets: MultiNLI, Fever-NLI, LingNLI and ANLI. Note that the model was trained on binary NLI to predict either "entailment" or "not-entailment". This is specifically designed for zero-shot classification, where the difference between "neutral" and "contradiction" is irrelevant. The base model is DeBERTa-v3-xsmall from Microsoft. The v3 variant of DeBERTa substantially outperforms previous versions of the model by including a different pre-training objective, see the DeBERTa-V3 paper. For highest performance (but less speed), I recommend using…

Model · Image to text

GLM-OCR

Unsloth AI

Join our WeChat and Discord community Use GLM-OCR's API GLM-OCR is a multimodal OCR model for complex document understanding, built on the GLM-V encoder–decoder architecture. It introduces Multi-Token Prediction (MTP) loss and stable full-task reinforcement learning to improve training efficiency, recognition accuracy, and generalization. The model integrates the CogViT visual encoder pre-trained on large-scale image–text data, a lightweight cross-modal connector with efficient token downsampling, and a GLM-0.5B language decoder. Combined with a two-stage pipeline of layout analysis and parallel recognition based on PP-DocLayout-V3, GLM-OCR delivers robust and high-quality OCR performance…

Model · Audio classification

Silero-VAD-v6.2.1-MLX

Ivan

MLX port of snakers4/silero-vad tag v6.2.1 for voice activity detection on Apple Silicon. Measured with speech-swift release tests on Apple Silicon using a 20 s 16 kHz speech fixture, 625 streaming chunks. Parity against the matching CoreML v6.2.1 export: The exported safetensors were also checked tensor-by-tensor against the upstream v6.2.1 JIT state dict after conversion; the maximum absolute tensor difference was 0. Converted from snakers4/silero-vad tag v6.2.1. The upstream project is MIT licensed. - speech-swift - Apple SDK - Docs - install and CLI docs - soniqo.audio - website - blog - blog

Model · Zero-shot classification

multilingual-MiniLMv2-L6-mnli-xnli

Moritz Borrett-Laurer (formerly Laurer)

multilingual - zero-shot-classification - text-classification - nli - pytorch - accuracy - multinli - xnli pipelinetag: zero-shot-classification candidatelabels: "politics, economy, entertainment, environment" This multilingual model can perform natural language inference (NLI) on 100+ languages and is therefore also suitable for multilingual zero-shot classification. The underlying multilingual-MiniLM-L6 model was created by Microsoft and was distilled from XLM-RoBERTa-large (see details in the original paper and newer information in this repo). The model was then fine-tuned on the XNLI dataset, which contains hypothesis-premise pairs from 15 languages, as well as the English MNLI dataset.…

Model · Robotics

openvla-7b-oft-finetuned-libero-spatial

Moo Jin Kim

This repository contains the OpenVLA-OFT checkpoint for LIBERO-Spatial, as described in Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Success. OpenVLA-OFT significantly improves upon the base OpenVLA model by incorporating optimized fine-tuning techniques. See here for other OpenVLA-OFT checkpoints: https://huggingface.co/moojink?searchmodels=oft This example demonstrates generating an action chunk using a pretrained OpenVLA-OFT checkpoint. Ensure you have set up the conda environment as described in the GitHub README.

Model · Image segmentation

oneformer_ade20k_dinat_large

SHI Labs

OneFormer model trained on the ADE20k dataset (large-sized version, Dinat backbone). It was introduced in the paper OneFormer: One Transformer to Rule Universal Image Segmentation by Jain et al. and first released in this repository. OneFormer is the first multi-task universal image segmentation framework. It needs to be trained only once with a single universal architecture, a single model, and on a single dataset, to outperform existing specialized models across semantic, instance, and panoptic segmentation tasks. OneFormer uses a task token to condition the model on the task in focus, making the architecture task-guided for training, and task-dynamic for inference, all with a single…

Model · Image segmentation

birefnet-lite-512

Studio Ludens

A 512×512 ONNX re-export of ZhengPeng7/BiRefNetlite that actually runs in a browser — solving the OOM wall that blocks every 1024×1024 variant from loading in onnxruntime-web. Drop it in with @huggingface/transformers to get high-quality alpha mattes entirely client-side, with no server round-trip. Used in production by Repper for per-motif matte refinement during foreground extraction. The 1024×1024 ONNX variants — including onnx-community/BiRefNetlite-ONNX — fail in every browser backend we tested: Root cause: BiRefNetlite's decoder produces very large intermediate tensors at 1024×1024 (multi-scale feature maps with 1024-way concatenations). The onnxruntime-web WASM heap is hardcoded at…