vocos-mel-24khz

Model Card

By Charactr Inc., published under mit, revision 0feb3fdd929b.

Paper [[abs]](https://arxiv.org/abs/2306.00814) [[pdf]](https://arxiv.org/pdf/2306.00814.pdf) Vocos is a fast neural vocoder designed to synthesize audio waveforms from acoustic features. Trained using a Generative Adversarial Network (GAN) objective, Vocos can generate waveforms in a single forward pass. Unlike other typical GAN-based vocoders, Vocos does not model audio samples in the time domain. Instead, it generates spectral coefficients, facilitating rapid audio reconstruction through inverse Fourier transform. To use Vocos only in inference mode, install it using: If you wish to train the model, install it with additional dependencies: If this code contributes to your research…

pip install vocos

pip install vocos[train]

import torch

from vocos import Vocos

vocos = Vocos.from_pretrained("charactr/vocos-mel-24khz")

mel = torch.randn(1, 100, 256)  # B, C, T
audio = vocos.decode(mel)

import torchaudio

y, sr = torchaudio.load(YOUR_AUDIO_FILE)
if y.size(0) > 1:  # mix to mono
    y = y.mean(dim=0, keepdim=True)
y = torchaudio.functional.resample(y, orig_freq=sr, new_freq=24000)
y_hat = vocos(y)

@article{siuzdak2023vocos,
  title={Vocos: Closing the gap between time-domain and Fourier-based neural vocoders for high-quality audio synthesis},
  author={Siuzdak, Hubert},
  journal={arXiv preprint arXiv:2306.00814},
  year={2023}
}

Identity and Version

Repository

charactr/vocos-mel-24khz

Publisher

Charactr Inc.

Task

Not stated by the source

Modality

Other

Library

Not stated by the source

Parameters

Not stated by the source

Languages

Not stated by the source

Revision

0feb3fdd929bcd6649e0e7c5a688cf7dd012ef21

First published

2023-06-11

Last updated

2023-10-17

Files and Weights

4 files, 54.4 MB in total. The weights are 1 file totalling 54.4 MB in bin.

License and Download

License

mit

Access

Open weights, no gate

Download size

54.4 MB

Built From

• Described by arXiv:2306.00814

Memory Requirements

Weights only, from the published parameter count; the key-value cache and runtime add to this.