Dataset Card for Earnings 22

Table of Contents

• Dataset Description
  • Dataset Summary
  • Supported Tasks and Leaderboards
  • Languages
• Dataset Structure
  • Data Instances
  • Data Fields
  • Data Splits

• Additional Information

• Contributions

Dataset Description

• Repository: revdotcom Speech Datasets
• Paper: Earnings-22: A Practical Benchmark for Accents in the Wild
• Point of Contact: Miguel Del Rio Fernandez

Dataset Summary

Earnings-22 provides a free-to-use benchmark of real-world, accented audio to bridge academic and industrial research. This dataset contains 125 files totalling roughly 119 hours of English language earnings calls from global countries. This dataset provides the full audios, transcripts, and accompanying metadata such as ticker symbol, headquarters country, and our defined "Language Region".

Supported Tasks and Leaderboards

The dataset is intended to be used to evaluate Automatic Speech Recognition (ASR) models. The model is presented with an long audio file, ranging from several minutes to tens of minutes, and asked to transcribe the audio file to written text. The most common evaluation metric is the word error rate (WER), averaged over the 125 audio files.

Languages

The audio is in English, with speakers from seven different langauge regions and a total of 27 unique countries. As such, there is large diversity in the speakers and accents.

Dataset Structure

Configurations

The Earnings-22 dataset comes in two forms:

• full: contains the full audio recordings as single long audio files. Intended for evaluation ASR systems on long-form audio files.
• chunked: contains the audio recordings chunked into smaller audio files of maximum 20-seconds. The audio recordings are chunked on punctuation by computing the start/end timestamps for each segment using the Wav2Vec2 model. Intended for evaluation ASR systems on short-form audio files.

Data Instances

A typical data point comprises the audio input, denoted by the key audio, and its transcription, denoted by `transcription. Some additional information about the speaker, accent and passage which contains the transcription is provided as metadata:

{'audio': {'path': '/fsx/sanchit/speech-datasets/earnings22/media/4468679.mp3',
  'array': array([ 0.00000000e+00, -3.36748518e-09, -3.54287222e-09, ...,
          4.77626486e-07, -7.80206960e-07, -8.02787653e-07]),
  'sampling_rate': 16000},
 'file_id': '4468679',
 'ticker_symbol': 'PAM',
 'country_by_ticker': 'Argentina',
 'un_defined': 'Latin America and Caribbean',
 'major_dialect_family': 'Other',
 'language_family': 'Spanish/Portuguese',
 'file_length': '3300',
 'sampling_rate': '16000',
 'transcription': "Good morning ladies and gentlemen, and thank you for waiting. I'm Margarita Chun from IR, and we would like to welcome everyone to Pampa Energia's Third Quarter 2021 Results Video Conference...

Data Fields

• audio: A dictionary containing the path to the downloaded audio file, the decoded audio array, and the sampling rate. Note that when accessing the audio column: dataset[0]["audio"] the audio file is automatically decoded and resampled to dataset.features["audio"].sampling_rate. Decoding and resampling of a large number of audio files might take a significant amount of time. Thus it is important to first query the sample index before the "audio" column, i.e. dataset[0]["audio"] should always be preferred over dataset["audio"][0].

• file_id: unique id of the data sample.

• ticker_symbol: ticker symbol of the company from which the earning call was taken.

• country_by_ticker: country to which the ticker symbol belongs (i.e. where the company is registered).

• un_defined: UN defined language region.

• major_dialect_family: the large-span (major) dialect family to which the country belongs.

• language_family: the Earnings-22 assigned language family. One of seven possible values: African, Asian, English, Germanic, Other Romance, Slavic, Spanish / Portuguese.

• file_length: length of the audio in seconds.

• sampling_rate: sampling rate at which the audio data was saved.

• transcription: the target transcription of the audio file.

Data Splits

The Earnings-22 dataset is intended to be used as a test-only split for evaluating ASR systems. As such, only one split is provided: the test split.

Citation Information

@misc{delrio2022earnings22,
      title={"Earnings-22: A Practical Benchmark for Accents in the Wild"}, 
      author={Miguel Del Rio and Peter Ha and Quinten McNamara and Corey Miller and Shipra Chandra},
      year={2022},
      eprint={2203.15591},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}

Contributions

Thanks to @sanchit-gandhi for adding this dataset.