Search public pages, research tools, and SAVRN solutions.
SAVRN Model Hub · Models by Task
60 models in the SAVRN Model Hub for audio classification, from publishers including AI at Meta, Superb, SpeechBrain, Massachusetts Institute of Technology.
60 models.
LAION-CLAP was trained on LAION-audio-630k The abstract of the paper states that: You can use this model for zero shot audio classification or extracting audio and/or textual features. You can also get the audio and text embeddings using ClapModel If you are using this model for your work, please consider citing the original paper
The model expects a raw audio signal as input and outputs predictions for age in a range of approximately 0...1 (0...100 years) and gender expressing the probababilty for being child, female, or male. In addition, it also provides the pooled states of the last transformer layer. The model was created by fine-tuning Wav2Vec2-Large-Robust Timit and For this version of the model we trained all 24 transformer layers. An ONNX export of the model is available from Further details are given in the associated paper and tutorial.
Please note that this model is for research purpose only. A commercial license for a model that has been trained on much more data can be acquired with audEERING. The model expects a raw audio signal as input, and outputs predictions for arousal, dominance and valence in a range of approximately 0...1. In addition, it provides the pooled states of the last transformer layer. The model was created by fine-tuning The model was pruned from 24 to 12 transformer layers before fine-tuning. An ONNX export of the model is available from doi:10.5281/zenodo.6221127. Further details are given in the associated paper and tutorial.
Model · Audio classification
This model is a fine-tuned version of facebook/wav2vec2-xls-r-300m on Librispeech-clean-100 for gender recognition. It achieves the following results on the evaluation set: The Librispeech-clean-100 dataset was used to train the model, with 70% of the data used for training, 10% for validation, and 20% for testing. The following hyperparameters were used during training: - learningrate: 3e-05 - trainbatchsize: 4 - evalbatchsize: 4 - gradientaccumulationsteps: 4 - totaltrainbatchsize: 16 - lrschedulertype: linear - lrschedulerwarmupratio: 0.1 - numepochs: 1 - mixedprecisiontraining: Native AMP - Transformers 4.28.0 - Pytorch 2.0.0+cu118 - Tokenizers 0.13.3
Model · Audio classification
Audio Spectrogram Transformer (AST) model fine-tuned on AudioSet. It was introduced in the paper AST: Audio Spectrogram Transformer by Gong et al. and first released in this repository. Disclaimer: The team releasing Audio Spectrogram Transformer did not write a model card for this model so this model card has been written by the Hugging Face team. The Audio Spectrogram Transformer is equivalent to ViT, but applied on audio. Audio is first turned into an image (as a spectrogram), after which a Vision Transformer is applied. The model gets state-of-the-art results on several audio classification benchmarks. You can use the raw model for classifying audio into one of the AudioSet classes. See…
This model has been pushed to the Hub using the PytorchModelHubMixin integration: Unified automatic quality assessment for speech, music, and sound. Paper arXiv / MetaAI. Blogpost ai.meta.com This repository requires Python 3.9 and Pytorch 2.2 or greater. To install, you can clone this repo and run: if you only want to predict aesthetic scores from certain timestamp and save it as input.jsonl If you haven't downloade the checkpoint, the script will try to download it automatically. Otherwise, you can provide the path by --ckpt /path/to/checkpoint.pt If you have SLURM, run the following command Please adjust CPU & GPU settings using --slurm-gpu, --slurm-cpu depending on your nodes. 3. Output…
This is the official repository for the paper "MuQ: Self-Supervised Music Representation Learning with Mel Residual Vector Quantization". For more detailed information, we strongly recommend referring to https://github.com/tencent-ailab/MuQ and the paper). In this repo, the following models are released: - MuQ(see this link): A large music foundation model pre-trained via Self-Supervised Learning (SSL), achieving SOTA in various MIR tasks. - MuQ-MuLan(see this link): A music-text joint embedding model trained via contrastive learning, supporting both English and Chinese texts. To begin with, please use pip to install the official muq lib, and ensure that your python>=3.8: To extract music…
The pre-trained model is this one - facebook/hubert-large-ls960-ft The DUSHA dataset used can be found here Fine-tuned in Google Colab using Pro account with A100 GPU Freezed all layers exept projector, classifier and all 24 HubertEncoderLayerStableLayerNorm layers Used half of the train dataset - 2 epochs - train batch size = 8 - eval batch size = 8 - gradient accumulation steps = 4 - learning rate = 5e-5 without warm up and decay Achieved - accuracy = 0.86 - balanced = 0.76 - macro f1 score = 0.81 on test set, improving accucary and f1 score compared to dataset baseline
Wav2Vec2: Self-Supervised Learning for Speech Recognition: https://arxiv.org/pdf/2006.11477 male female Common-Voice-Gender-Detection is designed for: Speech Analytics – Assist in analyzing speaker demographics in call centers or customer service recordings. Conversational AI Personalization – Adjust tone or dialogue based on gender detection for more personalized voice assistants. Voice Dataset Curation – Automatically tag or filter voice datasets by speaker gender for better dataset management. Research Applications – Enable linguistic and acoustic research involving gender-specific speech patterns. Multimedia Content Tagging – Automate metadata generation for gender identification in…
The model expects a raw audio signal as input and outputs predictions for age in a range of approximately 0...1 (0...100 years) and gender expressing the probababilty for being child, female, or male. In addition, it also provides the pooled states of the last transformer layer. The model was created by fine-tuning Wav2Vec2-Large-Robust Timit and For this version of the model we only trained the first six transformer layers. An ONNX export of the model is available from Further details are given in the associated paper and tutorial.
GGUF quantizations of the CED family (Consistent Ensemble Distillation, Xiaomi) - SOTA-tier audio-tagging models that classify everyday sounds (baby cry, footsteps, glass breaking, alarms, dog bark,...) into the 527-class AudioSet ontology. These files run with ced.cpp, a standalone C++/ggml port (no Python, no PyTorch at inference), and with LocalAI via the ced backend. Converted from the mispeech/ced- checkpoints (Apache-2.0). CED is a plain AST/DeiT Vision Transformer over a log-mel spectrogram; the port is numerically equal to the PyTorch reference. One self-contained GGUF per size + quant (config, 527 labels, and the mel filterbank/window are all embedded). Pick by your accuracy/size…
open-vakgyata is an open-source language identification model capable of detecting and classifying indian languages from speech inputs. Specification If you use this model in your research or application, please consider citing the model and its base source
This is a spoken language recognition model trained on the VoxLingua107 dataset using SpeechBrain. The model uses the ECAPA-TDNN architecture that has previously been used for speaker recognition. However, it uses more fully connected hidden layers after the embedding layer, and cross-entropy loss was used for training. We observed that this improved the performance of extracted utterance embeddings for downstream tasks. The system is trained with recordings sampled at 16kHz (single channel). The code will automatically normalize your audio (i.e., resampling + mono channel selection) when calling classifyfile if needed. The model can classify a speech utterance according to the language…
Abstract: The recognition of accented speech still remains a dominant problem in Automatic Speech Recognition (ASR) systems. We approach the classification of accented English speech through the Emphasized Channel Attention, Propagation and Aggregation Time Delay Neural Network (ECAPA-TDNN) architecture which has been shown to perform well on a variety of speech tasks. Three models are proposed: one trained from scratch, another two models (one using data augmentation and a baseline model) fine-tuned from the checkpoints of speechbrain/spkrec-ecapa-voxceleb (VoxCeleb). Our results show that the model fine-tuned with data augmentation yield the best results. Most of the misclassifications…
The development log of our Music Audio Pre-training (m-a-p) model family: - 02/06/2023: arxiv pre-print and training codes released. - 17/03/2023: we release two advanced music understanding models, MERT-v1-95M and MERT-v1-330M, trained with new paradigm and dataset. They outperform the previous models and can better generalize to more tasks. - 14/03/2023: we retrained the MERT-v0 model with open-source-only music dataset MERT-v0-public - 29/12/2022: a music understanding model MERT-v0 trained with MLM paradigm, which performs better at downstream tasks. - 29/10/2022: a pre-trained MIR model music2vec trained with BYOL paradigm. Here is a table for quick model pick-up: The m-a-p models…
The development log of our Music Audio Pre-training (m-a-p) model family: - 02/06/2023: arxiv pre-print and training codes released. - 17/03/2023: we release two advanced music understanding models, MERT-v1-95M and MERT-v1-330M, trained with new paradigm and dataset. They outperform the previous models and can better generalize to more tasks. - 14/03/2023: we retrained the MERT-v0 model with open-source-only music dataset MERT-v0-public - 29/12/2022: a music understanding model MERT-v0 trained with MLM paradigm, which performs better at downstream tasks. - 29/10/2022: a pre-trained MIR model music2vec trained with BYOL paradigm. Here is a table for quick model pick-up: The m-a-p models…
This model combines the SpeechBrain ECAPA-TDNN speaker embedding model with an SVM classifier to predict speaker gender from audio input. The model was trained and evaluated on the VoxCeleb2, Mozilla Common Voice v10.0, and TIMIT datasets - Mozilla Common Voice v10.0 English validated test set: 92.3% accuracy The model was trained on VoxCeleb2 dataset: - Converted to WAV format, single channel, 16kHz sampling rate, 256 kp/s bitrate - Applied SileroVAD for voice activity detection, taking the first voiced segment You can install the package directly from GitHub: - Model was trained on celebrity voices from YouTube interviews - Performance may vary on different audio qualities or recording…
This checkpoint is a model fine-tuned for speech language identification (LID) and part of Facebook's Massive Multilingual Speech project. This checkpoint is based on the Wav2Vec2 architecture and classifies raw audio input to a probability distribution over 1024 output classes (each class representing a language). The checkpoint consists of 1 billion parameters and has been fine-tuned from facebook/mms-1b on 1024 languages. This MMS checkpoint can be used with Transformers to identify the spoken language of an audio. It can recognize the following 1024 languages. Let's look at a simple example. First, we install transformers and some other libraries Note: In order to use MMS you need to…
This is a ported version of The base model is wav2vec2-base, which is pretrained on 16kHz sampled speech audio. When using the model make sure that your speech input is also sampled at 16Khz. For more information refer to SUPERB: Speech processing Universal PERformance Benchmark Emotion Recognition (ER) predicts an emotion class for each utterance. The most widely used ER dataset IEMOCAP is adopted, and we follow the conventional evaluation protocol: we drop the unbalanced emotion classes to leave the final four classes with a similar amount of data points and cross-validate on five folds of the standard splits. For the original model's training and evaluation instructions refer to the You…
This is the official repository for the paper "MuQ: Self-Supervised Music Representation Learning with Mel Residual Vector Quantization". For more detailed information, we strongly recommend referring to https://github.com/tencent-ailab/MuQ and the paper). In this repo, the following models are released: - MuQ(see this link): A large music foundation model pre-trained via Self-Supervised Learning (SSL), achieving SOTA in various MIR tasks. - MuQ-MuLan(see this link): A music-text joint embedding model trained via contrastive learning, supporting both English and Chinese texts. To begin with, please use pip to install the official muq lib, and ensure that your python>=3.8: Using MuQ-MuLan to…
This checkpoint is a model fine-tuned for speech language identification (LID) and part of Facebook's Massive Multilingual Speech project. This checkpoint is based on the Wav2Vec2 architecture and classifies raw audio input to a probability distribution over 126 output classes (each class representing a language). The checkpoint consists of 1 billion parameters and has been fine-tuned from facebook/mms-1b on 126 languages. This MMS checkpoint can be used with Transformers to identify the spoken language of an audio. It can recognize the following 126 languages. Let's look at a simple example. First, we install transformers and some other libraries Note: In order to use MMS you need to have…
This repo contains the inference code to use pretrained human voice gender classifier. - You could also try Huggingface online demo. First, clone the original github repository and install the packages via pip. For those who need pretrained weights, please download it in here State-of-the-art speaker verification model already produces good representation of the speaker's gender. I used the pretrained ECAPA-TDNN from TaoRuijie's repository, added one linear layer to make two-class classifier, and finetuned the model with the VoxCeleb2 dev set. The model achieved 98.7% accuracy on the VoxCeleb1 identification test split. I would like to note the training dataset I've used for this model…
This repository provides all the necessary tools to perform emotion recognition with a fine-tuned wav2vec2 (base) model using SpeechBrain. It is trained on IEMOCAP training data. For a better experience, we encourage you to learn more about SpeechBrain. The model performance on IEMOCAP test set is: This system is composed of an wav2vec2 model. It is a combination of convolutional and residual blocks. The embeddings are extracted using attentive statistical pooling. The system is trained with Additive Margin Softmax Loss. Speaker Verification is performed using cosine distance between speaker embeddings. The system is trained with recordings sampled at 16kHz (single channel). The code will…
This model is a fine-tuned version of facebook/wav2vec2-base on the None dataset. It achieves the following results on the evaluation set: The following hyperparameters were used during training: - learningrate: 5e-05 - trainbatchsize: 32 - evalbatchsize: 32 - lrschedulertype: linear - numepochs: 2 - Transformers 4.28.0 - Pytorch 2.0.0 - Datasets 2.12.0 - Tokenizers 0.13.3
This checkpoint is a model fine-tuned for speech language identification (LID) and part of Facebook's Massive Multilingual Speech project. This checkpoint is based on the Wav2Vec2 architecture and classifies raw audio input to a probability distribution over 256 output classes (each class representing a language). The checkpoint consists of 1 billion parameters and has been fine-tuned from facebook/mms-1b on 256 languages. This MMS checkpoint can be used with Transformers to identify the spoken language of an audio. It can recognize the following 256 languages. Let's look at a simple example. First, we install transformers and some other libraries Note: In order to use MMS you need to have…
This model is a fine tuning for the deepfake audio classification task. It achieves the following results on its evalutation data: It achieves the following results on ASVspoof2019 evaluation subset
MLX-compatible weights for WeSpeaker ResNet34-LM, converted from the pyannote speaker embedding model with BatchNorm fused into Conv2d. WeSpeaker ResNet34-LM is a speaker embedding model (~6.6M params) that produces 256-dimensional L2-normalized speaker embeddings from audio. Trained on VoxCeleb for speaker verification and diarization. BatchNorm is fused into Conv2d at conversion time — no BN layers in the MLX model. Part of speech-swift. Converts the original pyannote/wespeaker-voxceleb-resnet34-LM checkpoint using a custom unpickler (no pyannote.audio dependency required). Key transformations: - Fuse BatchNorm into Conv2d: wfused = w × γ/√(σ²+ε), bfused = β − μ×γ/√(σ²+ε) - Transpose…
Model · Audio classification
The model is a fine-tuned version of jonatasgrosman/wav2vec2-large-xlsr-53-english for a Speech Emotion Recognition (SER) task. The dataset used to fine-tune the original pre-trained model is the RAVDESS dataset. This dataset provides 1440 samples of recordings from actors performing on 8 different emotions in English, which are: It achieves the following results on the evaluation set: The following hyperparameters were used during training: - learningrate: 0.0001 - trainbatchsize: 4 - evalbatchsize: 4 - gradientaccumulationsteps: 2 - totaltrainbatchsize: 8 - lrschedulertype: linear - numepochs: 3 - mixedprecisiontraining: Native AMP Any doubt, contact me on Twitter. - Transformers 4.8.2…
This model is a fine-tuned version of facebook/wav2vec2-xls-r-300m for voicemail detection. It is trained on a dataset of call recordings to distinguish between voicemail greetings and live human responses. This model builds on wav2vec2-xls-r-300m, a self-supervised speech model trained on large-scale multilingual data. We fine-tuned it on the first two seconds of a call. - Automated voicemail detection in AI-powered call assistants. - Filtering voicemail responses in customer service and sales call automation. - Only trianed on the English language. - Assumes the voicemail track is isolated and contains no audio from the caller. - Designed for the first two seconds of audio when calling a…
Speech emotion recognition for Russian over seven classes: anger, disgust, enthusiasm, fear, happiness, neutral, sadness. Fine-tuned from jonatasgrosman/expw2v2truwavlms363 on Aniemore/resd. Audio resampled to 16 kHz mono, clips capped at 12 s, normalized per utterance, padding masked. UA is macro-averaged recall, WA is accuracy, F1 is macro-averaged. All three test sets went through the same harness, so the rows are comparable to each other. The RESD split matches fold 1 of EmoBox bit for bit. The top entry there is WavLM-large at WA 56.47 / UA 55.87 / F1 55.82. These numbers are higher, but the training protocol differs — EmoBox freezes the encoder and trains a probe, this is a full…
CED are simple ViT-Transformer-based models for audio tagging, achieving sota performance on Audioset. Notable differences from other available models include: 1. Simplification for finetuning: Batchnormalization of Mel-Spectrograms. During finetuning one does not need to first compute mean/variance over the dataset, which is common for AST. 1. Support for variable length inputs. Most other models use a static time-frequency position embedding, which hinders the model's generalization to segments shorter than 10s. Many previous transformers simply pad their input to 10s in order to avoid the performance impact, which in turn slows down training/inference drastically. 1. Training/Inference…
The recent surge in AI-generated songs presents exciting possibilities and challenges. These innovations necessitate the ability to distinguish between human-composed and synthetic songs to safeguard artistic integrity and protect human musical artistry. Existing research and datasets in fake song detection only focus on singing voice deepfake detection (SVDD), where the vocals are AI-generated but the instrumental music is sourced from real songs. However, these approaches are inadequate for detecting contemporary end-to-end artificial songs where all components (vocals, music, lyrics, and style) could be AI-generated. Additionally, existing datasets lack music-lyrics diversity…
https://huggingface.co/MIT/ast-finetuned-audioset-10-10-0.4593 with ONNX weights to be compatible with Transformers.js. If you haven't already, you can install the Transformers.js JavaScript library from NPM using: Example: Perform audio classification with Xenova/ast-finetuned-audioset-10-10-0.4593 and return top 4 results. Note: Having a separate repo for ONNX weights is intended to be a temporary solution until WebML gains more traction. If you would like to make your models web-ready, we recommend converting to ONNX using Optimum and structuring your repo like this one (with ONNX weights located in a subfolder named onnx).
This checkpoint is a model fine-tuned for speech language identification (LID) and part of Facebook's Massive Multilingual Speech project. This checkpoint is based on the Wav2Vec2 architecture and classifies raw audio input to a probability distribution over 4017 output classes (each class representing a language). The checkpoint consists of 1 billion parameters and has been fine-tuned from facebook/mms-1b on 4017 languages. This MMS checkpoint can be used with Transformers to identify the spoken language of an audio. It can recognize the following 4017 languages. Let's look at a simple example. First, we install transformers and some other libraries Note: In order to use MMS you need to…
MLX port of snakers4/silero-vad tag v6.2.1 for voice activity detection on Apple Silicon. Measured with speech-swift release tests on Apple Silicon using a 20 s 16 kHz speech fixture, 625 streaming chunks. Parity against the matching CoreML v6.2.1 export: The exported safetensors were also checked tensor-by-tensor against the upstream v6.2.1 JIT state dict after conversion; the maximum absolute tensor difference was 0. Converted from snakers4/silero-vad tag v6.2.1. The upstream project is MIT licensed. - speech-swift - Apple SDK - Docs - install and CLI docs - soniqo.audio - website - blog - blog
This is a ported version of The base model is wav2vec2-base, which is pretrained on 16kHz sampled speech audio. When using the model make sure that your speech input is also sampled at 16Khz. For more information refer to SUPERB: Speech processing Universal PERformance Benchmark Keyword Spotting (KS) detects preregistered keywords by classifying utterances into a predefined set of words. The task is usually performed on-device for the fast response time. Thus, accuracy, model size, and inference time are all crucial. SUPERB uses the widely used Speech Commands dataset v1.0 for the task. The dataset consists of ten classes of keywords, a class for silence, and an unknown class to include the…
This checkpoint is a model fine-tuned for speech language identification (LID) and part of Facebook's Massive Multilingual Speech project. This checkpoint is based on the Wav2Vec2 architecture and classifies raw audio input to a probability distribution over 512 output classes (each class representing a language). The checkpoint consists of 1 billion parameters and has been fine-tuned from facebook/mms-1b on 512 languages. This MMS checkpoint can be used with Transformers to identify the spoken language of an audio. It can recognize the following 512 languages. Let's look at a simple example. First, we install transformers and some other libraries Note: In order to use MMS you need to have…
Model · Audio classification
This project leverages the Whisper model to recognize emotions in speech. The goal is to classify audio recordings into different emotional categories, such as Happy, Sad, Surprised, and etc. The dataset used for training and evaluation is sourced from multiple datasets, including: The dataset contains recordings labeled with various emotions. Below is the distribution of the emotions in the dataset: This distribution reflects the balance of emotions in the dataset, with some emotions having more samples than others. Excluded the "calm" emotion during training due to its underrepresentation. The model used is the Whisper Large V3 model, fine-tuned for audio classification tasks: I map the…
This model is a fine-tuned version of mo-thecreator/wav2vec2-base-finetuned on the None dataset. It achieves the following results on the evaluation set: The following hyperparameters were used during training: - learningrate: 3e-05 - trainbatchsize: 8 - evalbatchsize: 8 - gradientaccumulationsteps: 4 - totaltrainbatchsize: 32 - lrschedulertype: linear - lrschedulerwarmupratio: 0.1 - numepochs: 5 - Transformers 4.39.3 - Pytorch 2.1.2 - Datasets 2.18.0 - Tokenizers 0.15.2 - mo-thecreator
Fine-tuned Wav2Vec2 model for detecting AI-generated speech. Determines if audio was spoken by a human or created by AI text-to-speech/voice cloning software. Fine-tuned Wav2Vec2 transformer for binary audio classification (real vs AI-generated speech). Trained to distinguish authentic human speech from synthetic audio generated by AI text-to-speech and voice cloning services including: Note: This model uses transfer learning from a base model already trained for deepfake detection. Fast convergence is expected due to task similarity and TTS engine overlap with the base model's training data. The model outputs logits (raw, unnormalized scores) for two classes: Apply softmax to convert raw…
This model is a fine-tuned version of motheecreator/Deepfake-audio-detection on the audiofolder dataset. It achieves the following results on the evaluation set: The following hyperparameters were used during training: - learningrate: 3e-05 - trainbatchsize: 32 - evalbatchsize: 32 - gradientaccumulationsteps: 4 - totaltrainbatchsize: 128 - lrschedulertype: cosine - lrschedulerwarmupratio: 0.1 - numepochs: 5 - Transformers 4.41.2 - Pytorch 2.1.2 - Datasets 2.19.2 - Tokenizers 0.19.1
DistilWav2Vec2 Adult/Child Speech Classifier is an audio classification model based on the wav2vec 2.0 architecture. This model is a distilled version of wav2vec2-adult-child-cls on a private adult/child speech classification dataset. This model was trained using HuggingFace's PyTorch framework. All training was done on a Tesla P100, provided by Kaggle. Training metrics were logged via Tensorboard. The model achieves the following results on evaluation: The following hyperparameters were used during training: - learningrate: 3e-05 - trainbatchsize: 32 - evalbatchsize: 32 - seed: 42 - gradientaccumulationsteps: 4 - totaltrainbatchsize: 128 - optimizer: Adam with betas=(0.9,0.999) and…
Speech emotion recognition for Russian over seven classes: anger, disgust, enthusiasm, fear, happiness, neutral, sadness. Fine-tuned from jonatasgrosman/wav2vec2-large-xlsr-53-russian on Aniemore/resd. Audio resampled to 16 kHz mono, clips capped at 12 s, normalized per utterance, padding masked. UA is macro-averaged recall, WA is accuracy, F1 is macro-averaged. All three test sets went through the same harness, so the rows are comparable to each other. The RESD split matches fold 1 of EmoBox bit for bit. The top entry there is WavLM-large at WA 56.47 / UA 55.87 / F1 55.82. These numbers are higher, but the training protocol differs — EmoBox freezes the encoder and trains a probe, this is a…
Fine-tuned facebook/wav2vec2-base for audio classification of single drum/percussion sounds into 10 classes. - clap, conga, crash, cymbal, hat, kick, ride, rim, snare, tom - Trained on short, single-hit drum sounds. Performance may drop on long mixes, multiple overlapping sounds, or very different recording conditions.
CED are simple ViT-Transformer-based models for audio tagging, achieving sota performance on Audioset. Notable differences from other available models include: 1. Simplification for finetuning: Batchnormalization of Mel-Spectrograms. During finetuning one does not need to first compute mean/variance over the dataset, which is common for AST. 1. Support for variable length inputs. Most other models use a static time-frequency position embedding, which hinders the model's generalization to segments shorter than 10s. Many previous transformers simply pad their input to 10s in order to avoid the performance impact, which in turn slows down training/inference drastically. 1. Training/Inference…
Dasheng (Deep Audio-Signal Holistic Embeddings), or “大声” ("great sound"), is a general-purpose audio encoder trained on a large-scale self-supervised learning task. Dasheng is designed to capture rich audio information across various domains, including speech, music, and environmental sounds. The model is trained on 272,356 hours of diverse audio data with 1.2 billion parameters, and exhibits significant performance gains on the HEAR benchmark. Dasheng outperforms previous works on CREMA-D, LibriCount, Speech Commands, VoxLingua, and competes well in music and environmental sound classification tasks. examplefinetuneesc50.ipynb demonstrates how to train a linear head on the ESC-50 dataset…
Model · Audio classification
Audio Spectrogram Transformer (AST) model fine-tuned on AudioSet. It was introduced in the paper AST: Audio Spectrogram Transformer by Gong et al. and first released in this repository. Disclaimer: The team releasing Audio Spectrogram Transformer did not write a model card for this model so this model card has been written by the Hugging Face team. The Audio Spectrogram Transformer is equivalent to ViT, but applied on audio. Audio is first turned into an image (as a spectrogram), after which a Vision Transformer is applied. The model gets state-of-the-art results on several audio classification benchmarks. You can use the raw model for classifying audio into one of the AudioSet classes. See…
This model is a fine-tuned version of facebook/wav2vec2-base-960h for Speech Emotion Recognition (SER). It has been trained using a Frozen Feature Extractor strategy to preserve the model's acoustic understanding while adapting to emotion detection. This approach ensures stable performance and prevents "Catastrophic Forgetting," achieving nearly 80% accuracy on the validation set. Update: The "Calm" and "Neutral" classes have been merged to improve classification consistency, resulting in 7 distinct emotion classes. The model was trained on a combined dataset of ~12,000 audio files from: The model classifies audio into one of the following emotions: 1. Angry 2. Disgust 3. Fear 4. Happy 5.…
This is a ported version of The base model is hubert-base-ls960, which is pretrained on 16kHz sampled speech audio. When using the model make sure that your speech input is also sampled at 16Khz. For more information refer to SUPERB: Speech processing Universal PERformance Benchmark Emotion Recognition (ER) predicts an emotion class for each utterance. The most widely used ER dataset IEMOCAP is adopted, and we follow the conventional evaluation protocol: we drop the unbalanced emotion classes to leave the final four classes with a similar amount of data points and cross-validate on five folds of the standard splits. For the original model's training and evaluation instructions refer to the…
Audio Spectrogram Transformer (AST) model fine-tuned on Speech Commands v2. It was introduced in the paper AST: Audio Spectrogram Transformer by Gong et al. and first released in this repository. Disclaimer: The team releasing Audio Spectrogram Transformer did not write a model card for this model so this model card has been written by the Hugging Face team. The Audio Spectrogram Transformer is equivalent to ViT, but applied on audio. Audio is first turned into an image (as a spectrogram), after which a Vision Transformer is applied. The model gets state-of-the-art results on several audio classification benchmarks. You can use the raw model for classifying audio into one of the Speech…
This repository provides all the necessary tools to extract speaker embeddings with a pretrained TDNN model using SpeechBrain. The system is trained on Voxceleb 1+ Voxceleb2 training data. For a better experience, we encourage you to learn more about SpeechBrain. The given model performance on Voxceleb1-test set (Cleaned) is: This system is composed of a TDNN model coupled with statistical pooling. The system is trained with Categorical Cross-Entropy Loss. First of all, please install SpeechBrain with the following command: Please notice that we encourage you to read our tutorials and learn more about The system is trained with recordings sampled at 16kHz (single channel). The code will…
This is a ported version of The base model is hubert-large-ll60k, which is pretrained on 16kHz sampled speech audio. When using the model make sure that your speech input is also sampled at 16Khz. For more information refer to SUPERB: Speech processing Universal PERformance Benchmark Emotion Recognition (ER) predicts an emotion class for each utterance. The most widely used ER dataset IEMOCAP is adopted, and we follow the conventional evaluation protocol: we drop the unbalanced emotion classes to leave the final four classes with a similar amount of data points and cross-validate on five folds of the standard splits. For the original model's training and evaluation instructions refer to the…
Model · Audio classification
MAEST is a family of Transformer models based on PASST and focused on music analysis applications. The MAEST models are also available for inference in the Essentia library and for inference and training in the official repository. You can try the MAEST interactive demo on replicate. MAEST is a music audio representation model pre-trained on the task of music style classification. According to the evaluation reported in the original paper, it reports good performance in several downstream music analysis tasks. The MAEST models can make predictions for a taxonomy of 400 music styles derived from the public metadata of Discogs. The MAEST models have reported good performance in downstream…
This is an ONNX version of prithivMLmods/Common-Voice-Gender-Detection. It was automatically converted and uploaded using this space. Wav2Vec2: Self-Supervised Learning for Speech Recognition: https://arxiv.org/pdf/2006.11477 Common-Voice-Gender-Detection is designed for: Speech Analytics – Assist in analyzing speaker demographics in call centers or customer service recordings. Conversational AI Personalization – Adjust tone or dialogue based on gender detection for more personalized voice assistants. Voice Dataset Curation – Automatically tag or filter voice datasets by speaker gender for better dataset management. Research Applications – Enable linguistic and acoustic research involving…
This model includes the implementation of dimensional emotion classification described in Vox-Profile: A Speech Foundation Model Benchmark for Characterizing Diverse Speaker and Speech Traits (https://arxiv.org/pdf/2505.14648) The training pipeline used is also the top-performing solution (SAILER) in INTERSPEECH 2025—Speech Emotion Challenge (https://lab-msp.com/MSP-PodcastCompetition/IS2025/). Note that we did not use the transcript compared to our official challenge submission system, and we created a speech-only system to make the model simple but still effective. We use the MSP-Podcast data to train this model, noting that the model might be sensitive to content information when making…
4-bit quantized version of Qwen/Qwen3-ForcedAligner-0.6B for Apple Silicon inference via MLX. Predicts word-level timestamps for audio+text pairs in a single non-autoregressive forward pass. Unlike ASR (autoregressive, token-by-token), the forced aligner runs the entire sequence in one forward pass through the decoder. The classify head predicts a timestamp class (0–4999) at each token position, which maps to time via classindex × 80ms. This model is designed for use with speech-swift: Text decoder (attention projections, MLP, embeddings) quantized to 4-bit using group quantization (groupsize=64). Audio encoder and classify head kept as float16 for accuracy.
The model was trained on MSP-Podcast for the Odyssey 2024 Emotion Recognition competition baseline This particular model is the multi-attributed based model which predict arousal, dominance and valence in a range of approximately 0...1. CCC based on Test3 and Development sets of the Odyssey Competition
This model includes the implementation of categorical emotion classification described in Vox-Profile: A Speech Foundation Model Benchmark for Characterizing Diverse Speaker and Speech Traits (https://arxiv.org/pdf/2505.14648) The training pipeline used is also the top-performing solution (SAILER) in INTERSPEECH 2025—Speech Emotion Challenge (https://lab-msp.com/MSP-PodcastCompetition/IS2025/). Note that we did not use all the augmentation and did not use the transcript compared to our official challenge submission system, but we created a speech-only system to make the model simple but still effective. We use the MSP-Podcast data to train this model, noting that the model might be…
This model is a fine-tuned version of ntu-spml/distilhubert on the GTZAN dataset. It achieves the following results on the evaluation set: - evalloss: 0.8970 - evalmodelpreparationtime: 0.0018 - evalaccuracy: 0.87 - evalruntime: 443.4759 - evalsamplespersecond: 0.225 - evalstepspersecond: 0.029 The following hyperparameters were used during training: - learningrate: 5e-05 - trainbatchsize: 8 - evalbatchsize: 8 - lrschedulertype: linear - lrschedulerwarmupsteps: 100 - numepochs: 10 - mixedprecisiontraining: Native AMP - labelsmoothingfactor: 0.1 - Transformers 5.16.1 - Pytorch 2.11.0+cpu - Datasets 2.19.0 - Tokenizers 0.23.1
By monthly downloads reported by the Hugging Face Hub: clap-htsat-fused (8.2M); wav2vec2-large-robust-24-ft-age-gender (2.5M); wav2vec2-large-robust-12-ft-emotion-msp-dim (804.8k).
