Dataset Card for Annotated LibriTTS-R

This dataset is an annotated version of a filtered LibriTTS-R [1].

LibriTTS-R [1] is a sound quality improved version of the LibriTTS corpus which is a multi-speaker English corpus of approximately 960 hours of read English speech at 24kHz sampling rate, published in 2019.

In the text_description column, it provides natural language annotations on the characteristics of speakers and utterances, that have been generated using the Data-Speech repository.

This dataset was used alongside its original version LibriTTS-R and the English subset of MLS to train [Parler-TTS Mini v1 and Large v1. A training recipe is available in the Parler-TTS library.

Motivation

This dataset is a reproduction of work from the paper Natural language guidance of high-fidelity text-to-speech with synthetic annotations by Dan Lyth and Simon King, from Stability AI and Edinburgh University respectively. It was designed to train the Parler-TTS Mini v1 and Large v1 models.

Contrarily to other TTS models, Parler-TTS is a fully open-source release. All of the datasets, pre-processing, training code and weights are released publicly under permissive license, enabling the community to build on our work and develop their own powerful TTS models. Parler-TTS was released alongside:

• The Parler-TTS repository - you can train and fine-tuned your own version of the model.
• The Data-Speech repository - a suite of utility scripts designed to annotate speech datasets.
• The Parler-TTS organization - where you can find the annotated datasets as well as the future checkpoints.

Usage

Here is an example on how to oad the clean config with only the train.clean.360 split.

from datasets import load_dataset

load_dataset("parler-tts/libritts-r-filtered-speaker-descriptions", "clean", split="train.clean.100")

Streaming is also supported.

from datasets import load_dataset

load_dataset("parler-tts/libritts-r-filtered-speaker-descriptions", "clean", streaming=True)

Note: This dataset doesn't actually keep track of the audio column of the original version. You can merge it back to the original dataset using this script from Parler-TTS or, even better, get inspiration from the training script of Parler-TTS, that efficiently process multiple annotated datasets.

Dataset Description

• License: CC BY 4.0

Dataset Sources

• Homepage: https://www.openslr.org/141/
• Paper: https://arxiv.org/abs/2305.18802

Citation

@ARTICLE{Koizumi2023-hs,
  title         = "{LibriTTS-R}: A restored multi-speaker text-to-speech corpus",
  author        = "Koizumi, Yuma and Zen, Heiga and Karita, Shigeki and Ding,
                   Yifan and Yatabe, Kohei and Morioka, Nobuyuki and Bacchiani,
                   Michiel and Zhang, Yu and Han, Wei and Bapna, Ankur",
  abstract      = "This paper introduces a new speech dataset called
                   ``LibriTTS-R'' designed for text-to-speech (TTS) use. It is
                   derived by applying speech restoration to the LibriTTS
                   corpus, which consists of 585 hours of speech data at 24 kHz
                   sampling rate from 2,456 speakers and the corresponding
                   texts. The constituent samples of LibriTTS-R are identical
                   to those of LibriTTS, with only the sound quality improved.
                   Experimental results show that the LibriTTS-R ground-truth
                   samples showed significantly improved sound quality compared
                   to those in LibriTTS. In addition, neural end-to-end TTS
                   trained with LibriTTS-R achieved speech naturalness on par
                   with that of the ground-truth samples. The corpus is freely
                   available for download from
                   \textbackslashurl\{http://www.openslr.org/141/\}.",
  month         =  may,
  year          =  2023,
  copyright     = "http://creativecommons.org/licenses/by-nc-nd/4.0/",
  archivePrefix = "arXiv",
  primaryClass  = "eess.AS",
  eprint        = "2305.18802"
}

@misc{kawamura2024librittspcorpusspeakingstyle,
      title={LibriTTS-P: A Corpus with Speaking Style and Speaker Identity Prompts for Text-to-Speech and Style Captioning}, 
      author={Masaya Kawamura and Ryuichi Yamamoto and Yuma Shirahata and Takuya Hasumi and Kentaro Tachibana},
      year={2024},
      eprint={2406.07969},
      archivePrefix={arXiv},
      primaryClass={eess.AS},
      url={https://arxiv.org/abs/2406.07969}, 
}

@misc{lacombe-etal-2024-dataspeech,
  author = {Yoach Lacombe and Vaibhav Srivastav and Sanchit Gandhi},
  title = {Data-Speech},
  year = {2024},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://github.com/ylacombe/dataspeech}}
}

@misc{lyth2024natural,
      title={Natural language guidance of high-fidelity text-to-speech with synthetic annotations},
      author={Dan Lyth and Simon King},
      year={2024},
      eprint={2402.01912},
      archivePrefix={arXiv},
      primaryClass={cs.SD}
}