GGUF quantizations of the CED family (Consistent Ensemble Distillation, Xiaomi) - SOTA-tier audio-tagging models that classify everyday sounds (baby cry, footsteps, glass breaking, alarms, dog bark,...) into the 527-class AudioSet ontology. These files run with ced.cpp, a standalone C++/ggml port (no Python, no PyTorch at inference), and with LocalAI via the ced backend. Converted from the mispeech/ced- checkpoints (Apache-2.0). CED is a plain AST/DeiT Vision Transformer over a log-mel spectrogram; the port is numerically equal to the PyTorch reference. One self-contained GGUF per size + quant (config, 527 labels, and the mel filterbank/window are all embedded). Pick by your accuracy/size…
Open-weight model · Audio classification
spkrec-xvect-voxceleb
by SpeechBrain speechbrain/spkrec-xvect-voxceleb
This repository provides all the necessary tools to extract speaker embeddings with a pretrained TDNN model using SpeechBrain. The system is trained on Voxceleb 1+ Voxceleb2 training data.
Parameters—
Context—
Weights32.7 MB
Licenseapache-2.0
AccessOpen weights
Monthly Downloads7.2k
Model Card
By SpeechBrain, published under apache-2.0, revision 56895a2df401.
This repository provides all the necessary tools to extract speaker embeddings with a pretrained TDNN model using SpeechBrain. The system is trained on Voxceleb 1+ Voxceleb2 training data. For a better experience, we encourage you to learn more about SpeechBrain. The given model performance on Voxceleb1-test set (Cleaned) is: This system is composed of a TDNN model coupled with statistical pooling. The system is trained with Categorical Cross-Entropy Loss. First of all, please install SpeechBrain with the following command: Please notice that we encourage you to read our tutorials and learn more about The system is trained with recordings sampled at 16kHz (single channel). The code will…
Read SpeechBrain's full model card
Speaker Verification with xvector embeddings on Voxceleb
This repository provides all the necessary tools to extract speaker embeddings with a pretrained TDNN model using SpeechBrain. The system is trained on Voxceleb 1+ Voxceleb2 training data.
For a better experience, we encourage you to learn more about SpeechBrain. The given model performance on Voxceleb1-test set (Cleaned) is:
| Release | EER(%) |
|---|---|
| 05-03-21 | 3.2 |
Pipeline description
This system is composed of a TDNN model coupled with statistical pooling. The system is trained with Categorical Cross-Entropy Loss.
Install SpeechBrain
First of all, please install SpeechBrain with the following command:
pip install speechbrain
Please notice that we encourage you to read our tutorials and learn more about SpeechBrain.
Compute your speaker embeddings
import torchaudio
from speechbrain.inference.speaker import EncoderClassifier
classifier = EncoderClassifier.from_hparams(source="speechbrain/spkrec-xvect-voxceleb", savedir="pretrained_models/spkrec-xvect-voxceleb")
signal, fs = torchaudio.load('tests/samples/ASR/spk1_snt1.wav')
embeddings = classifier.encode_batch(signal)
The system is trained with recordings sampled at 16kHz (single channel). The code will automatically normalize your audio (i.e., resampling + mono channel selection) when calling classify_file if needed. Make sure your input tensor is compliant with the expected sampling rate if you use encode_batch and classify_batch.
Inference on GPU
To perform inference on the GPU, add run_opts={"device":"cuda"} when calling the from_hparams method.
Training
The model was trained with SpeechBrain (aa018540). To train it from scratch follows these steps: 1. Clone SpeechBrain:
git clone https://github.com/speechbrain/speechbrain/
- Install it:
cd speechbrain
pip install -r requirements.txt
pip install -e .
- Run Training:
cd recipes/VoxCeleb/SpeakerRec/
python train_speaker_embeddings.py hparams/train_x_vectors.yaml --data_folder=your_data_folder
You can find our training results (models, logs, etc) here.
Limitations
The SpeechBrain team does not provide any warranty on the performance achieved by this model when used on other datasets.
Referencing xvectors
```@inproceedings{DBLP:conf/odyssey/SnyderGMSPK18, author = {David Snyder and Daniel Garcia{-}Romero and Alan McCree and Gregory Sell and Daniel Povey and Sanjeev Khudanpur}, title = {Spoken Language Recognition using X-vectors}, booktitle = {Odyssey 2018}, pages = {105--111}, year = {2018}, }
# **Citing SpeechBrain**
Please, cite SpeechBrain if you use it for your research or business.
```bibtex
@misc{speechbrain,
title={{SpeechBrain}: A General-Purpose Speech Toolkit},
author={Mirco Ravanelli and Titouan Parcollet and Peter Plantinga and Aku Rouhe and Samuele Cornell and Loren Lugosch and Cem Subakan and Nauman Dawalatabad and Abdelwahab Heba and Jianyuan Zhong and Ju-Chieh Chou and Sung-Lin Yeh and Szu-Wei Fu and Chien-Feng Liao and Elena Rastorgueva and François Grondin and William Aris and Hwidong Na and Yan Gao and Renato De Mori and Yoshua Bengio},
year={2021},
eprint={2106.04624},
archivePrefix={arXiv},
primaryClass={eess.AS},
note={arXiv:2106.04624}
}
Identity and Version
- Repository
- speechbrain/spkrec-xvect-voxceleb
- Publisher
- SpeechBrain
- Task
- Audio classification
- Modality
- Audio
- Library
- speechbrain
- Parameters
- Not stated by the source
- Languages
- en
- Revision
- 56895a2df401be4150a159f3a1c653f00051d477
- First published
- 2022-03-02
- Last updated
- 2024-02-25
Files and Weights
9 files, 33.0 MB in total. The weights are 3 files totalling 32.7 MB in ckpt.
Weights3 files · 32.7 MB
Configuration2 files · 2.1 KB
Documentation1 file · 4.3 KB
Other2 files · 233.0 KB
Repository1 file · 858 B
Every file
| File | Type | Size | SHA-256 |
|---|---|---|---|
| classifier.ckpt | Weights | 15.9 MB | e84a7cf53ea5 |
| embedding_model.ckpt | Weights | 16.9 MB | 9d96cafa0ede |
| mean_var_norm_emb.ckpt | Weights | 3.2 KB | d061fe599653 |
| config.json | Configuration | 50 B | — |
| hyperparams.yaml | Configuration | 2.0 KB | — |
| README.md | Documentation | 4.3 KB | — |
| example1.wav | Other | 104.4 KB | — |
| label_encoder.txt | Other | 128.6 KB | — |
| .gitattributes | Repository | 858 B | — |
License and Download
- License
- apache-2.0
- Access
- Open weights, no gate
- Download size
- 32.7 MB
Download from SpeechBrain
Released by SpeechBrain through its official repository on Hugging Face. Read the license.
Built From
- Described by arXiv:2106.04624
- Trained on (disclosed) voxceleb
Memory Requirements
| Precision | Weights in memory |
|---|---|
| As published | 32.7 MB |
Weights only, from the published parameter count; the key-value cache and runtime add to this.
Questions About spkrec-xvect-voxceleb
Can I use spkrec-xvect-voxceleb commercially?
Yes. spkrec-xvect-voxceleb is released under Apache License 2.0. The Apache License 2.0 is a permissive open-source license. It permits commercial use, modification and redistribution. It requires keeping the license and copyright notices and any NOTICE file, stating significant changes, and it includes an express patent grant from contributors.
Similar Models
This is a spoken language recognition model trained on the VoxLingua107 dataset using SpeechBrain. The model uses the ECAPA-TDNN architecture that has previously been used for speaker recognition. However, it uses more fully connected hidden layers after the embedding layer, and cross-entropy loss was used for training. We observed that this improved the performance of extracted utterance embeddings for downstream tasks. The system is trained with recordings sampled at 16kHz (single channel). The code will automatically normalize your audio (i.e., resampling + mono channel selection) when calling classifyfile if needed. The model can classify a speech utterance according to the language…
Abstract: The recognition of accented speech still remains a dominant problem in Automatic Speech Recognition (ASR) systems. We approach the classification of accented English speech through the Emphasized Channel Attention, Propagation and Aggregation Time Delay Neural Network (ECAPA-TDNN) architecture which has been shown to perform well on a variety of speech tasks. Three models are proposed: one trained from scratch, another two models (one using data augmentation and a baseline model) fine-tuned from the checkpoints of speechbrain/spkrec-ecapa-voxceleb (VoxCeleb). Our results show that the model fine-tuned with data augmentation yield the best results. Most of the misclassifications…
The development log of our Music Audio Pre-training (m-a-p) model family: - 02/06/2023: arxiv pre-print and training codes released. - 17/03/2023: we release two advanced music understanding models, MERT-v1-95M and MERT-v1-330M, trained with new paradigm and dataset. They outperform the previous models and can better generalize to more tasks. - 14/03/2023: we retrained the MERT-v0 model with open-source-only music dataset MERT-v0-public - 29/12/2022: a music understanding model MERT-v0 trained with MLM paradigm, which performs better at downstream tasks. - 29/10/2022: a pre-trained MIR model music2vec trained with BYOL paradigm. Here is a table for quick model pick-up: The m-a-p models…
The development log of our Music Audio Pre-training (m-a-p) model family: - 02/06/2023: arxiv pre-print and training codes released. - 17/03/2023: we release two advanced music understanding models, MERT-v1-95M and MERT-v1-330M, trained with new paradigm and dataset. They outperform the previous models and can better generalize to more tasks. - 14/03/2023: we retrained the MERT-v0 model with open-source-only music dataset MERT-v0-public - 29/12/2022: a music understanding model MERT-v0 trained with MLM paradigm, which performs better at downstream tasks. - 29/10/2022: a pre-trained MIR model music2vec trained with BYOL paradigm. Here is a table for quick model pick-up: The m-a-p models…
This model combines the SpeechBrain ECAPA-TDNN speaker embedding model with an SVM classifier to predict speaker gender from audio input. The model was trained and evaluated on the VoxCeleb2, Mozilla Common Voice v10.0, and TIMIT datasets - Mozilla Common Voice v10.0 English validated test set: 92.3% accuracy The model was trained on VoxCeleb2 dataset: - Converted to WAV format, single channel, 16kHz sampling rate, 256 kp/s bitrate - Applied SileroVAD for voice activity detection, taking the first voiced segment You can install the package directly from GitHub: - Model was trained on celebrity voices from YouTube interviews - Performance may vary on different audio qualities or recording…