wav2vec-vm-finetune

Model · Audio classification

wav2vec2-large-xlsr-53-gender-recognition-librispeech

Alef Iury Siqueira Ferreira

This model is a fine-tuned version of facebook/wav2vec2-xls-r-300m on Librispeech-clean-100 for gender recognition. It achieves the following results on the evaluation set: The Librispeech-clean-100 dataset was used to train the model, with 70% of the data used for training, 10% for validation, and 20% for testing. The following hyperparameters were used during training: - learningrate: 3e-05 - trainbatchsize: 4 - evalbatchsize: 4 - gradientaccumulationsteps: 4 - totaltrainbatchsize: 16 - lrschedulertype: linear - lrschedulerwarmupratio: 0.1 - numepochs: 1 - mixedprecisiontraining: Native AMP - Transformers 4.28.0 - Pytorch 2.0.0+cu118 - Tokenizers 0.13.3

This model is a fine tuning for the deepfake audio classification task. It achieves the following results on its evalutation data: It achieves the following results on ASVspoof2019 evaluation subset

Model · Audio classification

wav2vec2-deepfake-voice-detector

Gary Stafford

Fine-tuned Wav2Vec2 model for detecting AI-generated speech. Determines if audio was spoken by a human or created by AI text-to-speech/voice cloning software. Fine-tuned Wav2Vec2 transformer for binary audio classification (real vs AI-generated speech). Trained to distinguish authentic human speech from synthetic audio generated by AI text-to-speech and voice cloning services including: Note: This model uses transfer learning from a base model already trained for deepfake detection. Fast convergence is expected due to task similarity and TTS engine overlap with the base model's training data. The model outputs logits (raw, unnormalized scores) for two classes: Apply softmax to convert raw…

The pre-trained model is this one - facebook/hubert-large-ls960-ft The DUSHA dataset used can be found here Fine-tuned in Google Colab using Pro account with A100 GPU Freezed all layers exept projector, classifier and all 24 HubertEncoderLayerStableLayerNorm layers Used half of the train dataset - 2 epochs - train batch size = 8 - eval batch size = 8 - gradient accumulation steps = 4 - learning rate = 5e-5 without warm up and decay Achieved - accuracy = 0.86 - balanced = 0.76 - macro f1 score = 0.81 on test set, improving accucary and f1 score compared to dataset baseline

Model · Audio classification

wav2vec2-xlsr-53-russian-emotion-recognition

Aniemore

Speech emotion recognition for Russian over seven classes: anger, disgust, enthusiasm, fear, happiness, neutral, sadness. Fine-tuned from jonatasgrosman/wav2vec2-large-xlsr-53-russian on Aniemore/resd. Audio resampled to 16 kHz mono, clips capped at 12 s, normalized per utterance, padding masked. UA is macro-averaged recall, WA is accuracy, F1 is macro-averaged. All three test sets went through the same harness, so the rows are comparable to each other. The RESD split matches fold 1 of EmoBox bit for bit. The top entry there is WavLM-large at WA 56.47 / UA 55.87 / F1 55.82. These numbers are higher, but the training protocol differs — EmoBox freezes the encoder and trains a probe, this is a…

Model · Audio classification

wav2vec2-lg-xlsr-en-speech-emotion-recognition

Enrique Hernández Calabrés

The model is a fine-tuned version of jonatasgrosman/wav2vec2-large-xlsr-53-english for a Speech Emotion Recognition (SER) task. The dataset used to fine-tune the original pre-trained model is the RAVDESS dataset. This dataset provides 1440 samples of recordings from actors performing on 8 different emotions in English, which are: It achieves the following results on the evaluation set: The following hyperparameters were used during training: - learningrate: 0.0001 - trainbatchsize: 4 - evalbatchsize: 4 - gradientaccumulationsteps: 2 - totaltrainbatchsize: 8 - lrschedulertype: linear - numepochs: 3 - mixedprecisiontraining: Native AMP Any doubt, contact me on Twitter. - Transformers 4.8.2…