tsinghua-ee/QualiSpeech

Dataset

QualiSpeech: A Speech Quality Assessment Dataset with Natural Language Reasoning and Descriptions

25

110 commits

12 linked in READMEs

updated Aug 4, 2025

See the code

README

QualiSpeech: A Speech Quality Assessment Dataset with Natural Language Reasoning and Descriptions

QualiSpeech is a comprehensive English-language speech quality assessment dataset designed to go beyond traditional numerical scores. It introduces detailed natural language comments with reasoning, capturing low-level speech perception aspects such as noise, distortion, continuity, speed, naturalness, listening effort, and overall quality.

🌟 Key Features

  • 11 annotated aspects including 7 numerical scores and 4 specific descriptions (e.g., noise type and time, distortion type and time, unnatural pauses, vocal characteristics).
  • Natural language descriptions capturing contextual and logical insights for overall quality reasoning.
  • Over 15,000 speech samples from diverse sources including synthetic (e.g., BVCC, recent TTS models) and real speech (e.g., NISQA, GigaSpeech).
  • QualiSpeech Benchmark for evaluating low-level speech perception in auditory large language models (LLMs).

📁 Dataset Structure

Each sample in the dataset contains:

- audio_path: path/to/audio.wav
- scores:
    - noise: 4
    - distortion: 3
    - speed: 3
    - continuity: 5
    - naturalness: 3
    - listening_effort: 5
    - overall: 3
- descriptions:
    - noise_description: "Outdoor music noise, 0–3s"
    - distortion_description: "None"
    - unnatural_pause: "None"
    - feeling_of_voice: "A young man’s gentle voice with a peaceful tone"
- natural_language_description: |
    The speech sample presents a gentle and peaceful tone...

🔽 Download Instructions

Due to licensing restrictions on the Blizzard Challenge data (the data may NOT be redistributed), please first download the required BVCC data following the provided scripts:

bash download_bvcc.sh   # or download manually
bash merge_data.sh      # to construct the final QualiSpeech dataset

📄 References of resources & models used

Resources:

Acoustic models:

Noise:

Synthesized text:

Speaker for zero-shot TTS:

📄 License:

Creative Commons AttributionNonCommercial-ShareAlike 4.0

📚 Citation

If you use QualiSpeech in your work, please cite:

@inproceedings{wang2025qualispeech,
  title={QualiSpeech: A Speech Quality Assessment Dataset with Natural Language Reasoning and Descriptions},
  author={Siyin Wang and Wenyi Yu and Xianzhao Chen and Xiaohai Tian and Jun Zhang and Lu Lu and Yu Tsao and Junichi Yamagishi and Yuxuan Wang and Chao Zhang},
  year={2025},
  booktitle={Proc. ACL},
  address={Vienna}
}

Contributors

Siyin519

103 commits

WA
wangsiyin

6 commits

nielsr

1 commits

tsinghua-ee/QualiSpeech

Dataset

QualiSpeech: A Speech Quality Assessment Dataset with Natural Language Reasoning and Descriptions

25

110 commits

12 linked in READMEs

updated Aug 4, 2025

See the code

README

QualiSpeech: A Speech Quality Assessment Dataset with Natural Language Reasoning and Descriptions

QualiSpeech is a comprehensive English-language speech quality assessment dataset designed to go beyond traditional numerical scores. It introduces detailed natural language comments with reasoning, capturing low-level speech perception aspects such as noise, distortion, continuity, speed, naturalness, listening effort, and overall quality.

🌟 Key Features

  • 11 annotated aspects including 7 numerical scores and 4 specific descriptions (e.g., noise type and time, distortion type and time, unnatural pauses, vocal characteristics).
  • Natural language descriptions capturing contextual and logical insights for overall quality reasoning.
  • Over 15,000 speech samples from diverse sources including synthetic (e.g., BVCC, recent TTS models) and real speech (e.g., NISQA, GigaSpeech).
  • QualiSpeech Benchmark for evaluating low-level speech perception in auditory large language models (LLMs).

📁 Dataset Structure

Each sample in the dataset contains:

- audio_path: path/to/audio.wav
- scores:
    - noise: 4
    - distortion: 3
    - speed: 3
    - continuity: 5
    - naturalness: 3
    - listening_effort: 5
    - overall: 3
- descriptions:
    - noise_description: "Outdoor music noise, 0–3s"
    - distortion_description: "None"
    - unnatural_pause: "None"
    - feeling_of_voice: "A young man’s gentle voice with a peaceful tone"
- natural_language_description: |
    The speech sample presents a gentle and peaceful tone...

🔽 Download Instructions

Due to licensing restrictions on the Blizzard Challenge data (the data may NOT be redistributed), please first download the required BVCC data following the provided scripts:

bash download_bvcc.sh   # or download manually
bash merge_data.sh      # to construct the final QualiSpeech dataset

📄 References of resources & models used

Resources:

Acoustic models:

Noise:

Synthesized text:

Speaker for zero-shot TTS:

📄 License:

Creative Commons AttributionNonCommercial-ShareAlike 4.0

📚 Citation

If you use QualiSpeech in your work, please cite:

@inproceedings{wang2025qualispeech,
  title={QualiSpeech: A Speech Quality Assessment Dataset with Natural Language Reasoning and Descriptions},
  author={Siyin Wang and Wenyi Yu and Xianzhao Chen and Xiaohai Tian and Jun Zhang and Lu Lu and Yu Tsao and Junichi Yamagishi and Yuxuan Wang and Chao Zhang},
  year={2025},
  booktitle={Proc. ACL},
  address={Vienna}
}

Contributors

Siyin519

103 commits

WA
wangsiyin

6 commits

nielsr

1 commits