cml-tts

Dataset Card for CML-TTS

Table of Contents

• Dataset Description
• Dataset Summary
• Supported Tasks
• Languages
• How to use
• Dataset Structure
• Data Instances
• Data Fields
• Data Splits
• Data Statistics
• Dataset Creation
• Curation Rationale
• Source Data
• Annotations
• Personal and Sensitive Information
• Considerations for Using the Data
• Social Impact of Dataset
• Discussion of Biases
• Other Known Limitations
• Additional Information
• Dataset Curators
• Licensing Information
• Citation Information
• Contributions

Dataset Description

• Homepage: MultiLingual LibriSpeech ASR corpus
• Repository: CML-TTS-Dataset
• Paper: CML-TTS A Multilingual Dataset for Speech Synthesis in Low-Resource Languages

Dataset Summary

CML-TTS is a recursive acronym for CML-Multi-Lingual-TTS, a Text-to-Speech (TTS) dataset developed at the Center of Excellence in Artificial Intelligence (CEIA) of the Federal University of Goias (UFG). CML-TTS is a dataset comprising audiobooks sourced from the public domain books of Project Gutenberg, read by volunteers from the LibriVox project. The dataset includes recordings in Dutch, German, French, Italian, Polish, Portuguese, and Spanish, all at a sampling rate of 24kHz.

The data archives were restructured from the original ones from OpenSLR to make it easier to stream.

Supported Tasks

• text-to-speech, text-to-audio: The dataset can also be used to train a model for Text-To-Speech (TTS).

Languages

The dataset includes recordings in Dutch, German, French, Italian, Polish, Portuguese, and Spanish, all at a sampling rate of 24kHz.

How to use

The datasets library allows you to load and pre-process your dataset in pure Python, at scale. The dataset can be downloaded and prepared in one call to your local drive by using the load_dataset function.

For example, to download the German config, simply specify the corresponding language config name (i.e., "german" for German):

from datasets import load_dataset

mls = load_dataset("ylacombe/cml-tts", "german", split="train")

Using the datasets library, you can also stream the dataset on-the-fly by adding a streaming=True argument to the load_dataset function call. Loading a dataset in streaming mode loads individual samples of the dataset at a time, rather than downloading the entire dataset to disk.

from datasets import load_dataset

mls = load_dataset("ylacombe/cml-tts", "german", split="train", streaming=True)

print(next(iter(mls)))

Bonus

You can create a PyTorch dataloader directly with your own datasets (local/streamed).

Local:

from datasets import load_dataset
from torch.utils.data.sampler import BatchSampler, RandomSampler

mls = load_dataset("ylacombe/cml-tts", "german", split="train")
batch_sampler = BatchSampler(RandomSampler(mls), batch_size=32, drop_last=False)
dataloader = DataLoader(mls, batch_sampler=batch_sampler)

Streaming:

from datasets import load_dataset
from torch.utils.data import DataLoader

mls = load_dataset("ylacombe/cml-tts", "german", split="train", streaming=True)
dataloader = DataLoader(mls, batch_size=32)

To find out more about loading and preparing audio datasets, head over to hf.co/blog/audio-datasets.

Dataset Structure

Data Instances

A typical data point comprises the path to the audio file, usually called file and its transcription, called text. Some additional information about the speaker and the passage which contains the transcription is provided.

{'audio': {'path': '6892_8912_000729.wav', 'array': array([-1.52587891e-...7344e-05]), 'sampling_rate': 24000}, 'wav_filesize': 601964, 'text': 'Proszę pana, tu pano... zdziwiony', 'transcript_wav2vec': 'proszę pana tu panow... zdziwiony', 'levenshtein': 0.96045197740113, 'duration': 13.648979591836737, 'num_words': 29, 'speaker_id': 6892}

Data Fields

• audio: A dictionary containing the audio filename, the decoded audio array, and the sampling rate. Note that when accessing the audio column: dataset[0]["audio"] the audio file is automatically decoded and resampled to dataset.features["audio"].sampling_rate. Decoding and resampling of a large number of audio files might take a significant amount of time. Thus it is important to first query the sample index before the "audio" column, i.e. dataset[0]["audio"] should always be preferred over dataset["audio"][0].

• text: the transcription of the audio file.

• speaker_id: unique id of the speaker. The same speaker id can be found for multiple data samples.

• transcript_wav2vec: the transcription of the audio file using the wav2vec model. Has been used to curate the dataset.

• wav_filesize: The size of the audio waveform file. Has been used to curate the dataset.

• levenshtein: The Levenshtein distance between the wav2vec transcription and the original transcription. Has been used to curate the dataset.

• duration: The duration of the audio in seconds.

• num_words: The number of words of the transcription.

Data Splits

Data Statistics

Dataset Creation

Curation Rationale

[Needs More Information]

Source Data

Initial Data Collection and Normalization

[Needs More Information]

Who are the source language producers?

[Needs More Information]

Annotations

Annotation process

[Needs More Information]

Who are the annotators?

[Needs More Information]

Personal and Sensitive Information

The dataset consists of people who have donated their voice online. You agree to not attempt to determine the identity of speakers in this dataset.

Considerations for Using the Data

Social Impact of Dataset

[More Information Needed]

Discussion of Biases

[More Information Needed]

Other Known Limitations

[Needs More Information]

Additional Information

Dataset Curators

[Needs More Information]

Licensing Information

Public Domain, Creative Commons Attribution 4.0 International Public License (CC-BY-4.0)

Citation Information

@misc{oliveira2023cmltts,
      title={CML-TTS A Multilingual Dataset for Speech Synthesis in Low-Resource Languages}, 
      author={Frederico S. Oliveira and Edresson Casanova and Arnaldo Cândido Júnior and Anderson S. Soares and Arlindo R. Galvão Filho},
      year={2023},
      eprint={2306.10097},
      archivePrefix={arXiv},
      primaryClass={eess.AS}
}

Contributions

Thanks to @ylacombe for adding this dataset.