GGUF quantizations of the CED family (Consistent Ensemble Distillation, Xiaomi) - SOTA-tier audio-tagging models that classify everyday sounds (baby cry, footsteps, glass breaking, alarms, dog bark,...) into the 527-class AudioSet ontology. These files run with ced.cpp, a standalone C++/ggml port (no Python, no PyTorch at inference), and with LocalAI via the ced backend. Converted from the mispeech/ced- checkpoints (Apache-2.0). CED is a plain AST/DeiT Vision Transformer over a log-mel spectrogram; the port is numerically equal to the PyTorch reference. One self-contained GGUF per size + quant (config, 527 labels, and the mel filterbank/window are all embedded). Pick by your accuracy/size…
Open-weight model · Audio classification
sonics-spectttra-alpha-120s
by Awsaf awsaf49/sonics-spectttra-alpha-120s
The recent surge in AI-generated songs presents exciting possibilities and challenges. These innovations necessitate the ability to distinguish between human-composed and synthetic songs to safeguard artistic integrity and protect human musical artistry.
Parameters—
Context—
Weights75.3 MB
Licensemit
AccessOpen weights
Monthly Downloads18.4k
Model Card
By Awsaf, published under mit, revision 094b32a55450.
The recent surge in AI-generated songs presents exciting possibilities and challenges. These innovations necessitate the ability to distinguish between human-composed and synthetic songs to safeguard artistic integrity and protect human musical artistry. Existing research and datasets in fake song detection only focus on singing voice deepfake detection (SVDD), where the vocals are AI-generated but the instrumental music is sourced from real songs. However, these approaches are inadequate for detecting contemporary end-to-end artificial songs where all components (vocals, music, lyrics, and style) could be AI-generated. Additionally, existing datasets lack music-lyrics diversity…
Read Awsaf's full model card
SONICS: Synthetic Or Not - Identifying Counterfeit Songs
ICLR 2025 [Poster]
Abstract
The recent surge in AI-generated songs presents exciting possibilities and challenges. These innovations necessitate the ability to distinguish between human-composed and synthetic songs to safeguard artistic integrity and protect human musical artistry. Existing research and datasets in fake song detection only focus on singing voice deepfake detection (SVDD), where the vocals are AI-generated but the instrumental music is sourced from real songs. However, these approaches are inadequate for detecting contemporary end-to-end artificial songs where all components (vocals, music, lyrics, and style) could be AI-generated. Additionally, existing datasets lack music-lyrics diversity, long-duration songs, and open-access fake songs. To address these gaps, we introduce SONICS, a novel dataset for end-to-end Synthetic Song Detection (SSD), comprising over 97k songs (4,751 hours) with over 49k synthetic songs from popular platforms like Suno and Udio. Furthermore, we highlight the importance of modeling long-range temporal dependencies in songs for effective authenticity detection, an aspect entirely overlooked in existing methods. To utilize long-range patterns, we introduce SpecTTTra, a novel architecture that significantly improves time and memory efficiency over conventional CNN and Transformer-based models. For long songs, our top-performing variant outperforms ViT by 8% in F1 score, is 38% faster, and uses 26% less memory, while also surpassing ConvNeXt with a 1% F1 score gain, 20% speed boost, and 67% memory reduction.
Links
Model Performance
| Model Name | HF Link | Variant | Duration | f_clip | t_clip | F1 | Sensitivity | Specificity | Speed (A/S) | FLOPs (G) | Mem. (GB) | # Act. (M) | # Param. (M) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
sonics-spectttra-alpha-5s |
HF | SpecTTTra-α | 5s | 1 | 3 | 0.78 | 0.69 | 0.94 | 148 | 2.9 | 0.5 | 6 | 17 |
sonics-spectttra-beta-5s |
HF | SpecTTTra-β | 5s | 3 | 5 | 0.78 | 0.69 | 0.94 | 152 | 1.1 | 0.2 | 5 | 17 |
sonics-spectttra-gamma-5s |
HF | SpecTTTra-γ | 5s | 5 | 7 | 0.76 | 0.66 | 0.92 | 154 | 0.7 | 0.1 | 2 | 17 |
sonics-spectttra-alpha-120s |
HF | SpecTTTra-α | 120s | 1 | 3 | 0.97 | 0.96 | 0.99 | 47 | 23.7 | 3.9 | 50 | 19 |
sonics-spectttra-beta-120s |
HF | SpecTTTra-β | 120s | 3 | 5 | 0.92 | 0.86 | 0.99 | 80 | 14.0 | 2.3 | 29 | 21 |
sonics-spectttra-gamma-120s |
HF | SpecTTTra-γ | 120s | 5 | 7 | 0.88 | 0.79 | 0.99 | 97 | 10.1 | 1.6 | 20 | 24 |
Model Architecture
- Base Model: SpectTTTra (Spectro-Temporal Tokens Transformer)
- Embedding Dimension: 384
- Number of Heads: 6
- Number of Layers: 12
- MLP Ratio: 2.67
Audio Processing
- Sample Rate: 16kHz
- FFT Size: 2048
- Hop Length: 512
- Mel Bands: 128
- Frequency Range: 20Hz - 8kHz
- Normalization: Mean-std normalization
Usage
# Install from GitHub
!pip install git+https://github.com/awsaf49/sonics.git
# Load model
from sonics import HFAudioClassifier
model = HFAudioClassifier.from_pretrained("awsaf49/sonics-spectttra-alpha-120s")
Citation
@inproceedings{rahman2024sonics,
title={SONICS: Synthetic Or Not - Identifying Counterfeit Songs},
author={Rahman, Md Awsafur and Hakim, Zaber Ibn Abdul and Sarker, Najibul Haque and Paul, Bishmoy and Fattah, Shaikh Anowarul},
booktitle={International Conference on Learning Representations (ICLR)},
year={2025},
}
Identity and Version
- Repository
- awsaf49/sonics-spectttra-alpha-120s
- Publisher
- Awsaf
- Task
- Audio classification
- Modality
- Audio
- Library
- Not stated by the source
- Parameters
- Not stated by the source
- Languages
- en
- Revision
- 094b32a5545098a71c113f6ae9d5c55310564268
- First published
- 2025-02-23
- Last updated
- 2025-03-01
Files and Weights
4 files, 75.3 MB in total. The weights are 1 file totalling 75.3 MB in bin.
Weights1 file · 75.3 MB
Configuration1 file · 1.9 KB
Documentation1 file · 7.5 KB
Repository1 file · 1.5 KB
Every file
| File | Type | Size | SHA-256 |
|---|---|---|---|
| pytorch_model.bin | Weights | 75.3 MB | 082df7e38704 |
| config.json | Configuration | 1.9 KB | — |
| README.md | Documentation | 7.5 KB | — |
| .gitattributes | Repository | 1.5 KB | — |
License and Download
- License
- mit
- Access
- Open weights, no gate
- Download size
- 75.3 MB
Download from Awsaf
Released by Awsaf through its official repository on Hugging Face. Read the license.
Built From
- Described by arXiv:2408.14080
- Trained on (disclosed) awsaf49/sonics
Memory Requirements
| Precision | Weights in memory |
|---|---|
| As published | 75.3 MB |
Weights only, from the published parameter count; the key-value cache and runtime add to this.
Questions About sonics-spectttra-alpha-120s
Can I use sonics-spectttra-alpha-120s commercially?
Yes. sonics-spectttra-alpha-120s is released under MIT License. The MIT License is a short permissive license. It permits commercial use, modification and redistribution, provided the copyright notice and permission notice are included.
Similar Models
This is a spoken language recognition model trained on the VoxLingua107 dataset using SpeechBrain. The model uses the ECAPA-TDNN architecture that has previously been used for speaker recognition. However, it uses more fully connected hidden layers after the embedding layer, and cross-entropy loss was used for training. We observed that this improved the performance of extracted utterance embeddings for downstream tasks. The system is trained with recordings sampled at 16kHz (single channel). The code will automatically normalize your audio (i.e., resampling + mono channel selection) when calling classifyfile if needed. The model can classify a speech utterance according to the language…
Abstract: The recognition of accented speech still remains a dominant problem in Automatic Speech Recognition (ASR) systems. We approach the classification of accented English speech through the Emphasized Channel Attention, Propagation and Aggregation Time Delay Neural Network (ECAPA-TDNN) architecture which has been shown to perform well on a variety of speech tasks. Three models are proposed: one trained from scratch, another two models (one using data augmentation and a baseline model) fine-tuned from the checkpoints of speechbrain/spkrec-ecapa-voxceleb (VoxCeleb). Our results show that the model fine-tuned with data augmentation yield the best results. Most of the misclassifications…
The development log of our Music Audio Pre-training (m-a-p) model family: - 02/06/2023: arxiv pre-print and training codes released. - 17/03/2023: we release two advanced music understanding models, MERT-v1-95M and MERT-v1-330M, trained with new paradigm and dataset. They outperform the previous models and can better generalize to more tasks. - 14/03/2023: we retrained the MERT-v0 model with open-source-only music dataset MERT-v0-public - 29/12/2022: a music understanding model MERT-v0 trained with MLM paradigm, which performs better at downstream tasks. - 29/10/2022: a pre-trained MIR model music2vec trained with BYOL paradigm. Here is a table for quick model pick-up: The m-a-p models…
The development log of our Music Audio Pre-training (m-a-p) model family: - 02/06/2023: arxiv pre-print and training codes released. - 17/03/2023: we release two advanced music understanding models, MERT-v1-95M and MERT-v1-330M, trained with new paradigm and dataset. They outperform the previous models and can better generalize to more tasks. - 14/03/2023: we retrained the MERT-v0 model with open-source-only music dataset MERT-v0-public - 29/12/2022: a music understanding model MERT-v0 trained with MLM paradigm, which performs better at downstream tasks. - 29/10/2022: a pre-trained MIR model music2vec trained with BYOL paradigm. Here is a table for quick model pick-up: The m-a-p models…
This model combines the SpeechBrain ECAPA-TDNN speaker embedding model with an SVM classifier to predict speaker gender from audio input. The model was trained and evaluated on the VoxCeleb2, Mozilla Common Voice v10.0, and TIMIT datasets - Mozilla Common Voice v10.0 English validated test set: 92.3% accuracy The model was trained on VoxCeleb2 dataset: - Converted to WAV format, single channel, 16kHz sampling rate, 256 kp/s bitrate - Applied SileroVAD for voice activity detection, taking the first voiced segment You can install the package directly from GitHub: - Model was trained on celebrity voices from YouTube interviews - Performance may vary on different audio qualities or recording…