Dataset Description

• Repository: https://github.com/DBD-research-group/BirdSet
• Paper: BirdSet
• Point of Contact: Lukas Rauch

BirdSet

Deep learning (DL) has greatly advanced audio classification, yet the field is limited by the scarcity of large-scale benchmark datasets that have propelled progress in other domains. While AudioSet aims to bridge this gap as a universal-domain dataset, its restricted accessibility and lack of diverse real-world evaluation use cases challenge its role as the only resource. Additionally, to maximize the potential of cost-effective and minimal-invasive passive acoustic monitoring (PAM), models must analyze bird vocalizations across a wide range of species and environmental conditions. Therefore, we introduce BirdSet, a large-scale benchmark dataset for audio classification focusing on avian bioacoustics. BirdSet surpasses AudioSet with over 6,800 recording hours (17% increase) from nearly 10,000 classes (18x) for training and more than 400 hours (7x) across eight strongly labeled evaluation datasets. It serves as a versatile resource for use cases such as multi-label classification, covariate shift or self-supervised learning.

• Complementary Code: Repository
• Complementary Paper: ArXiv Link

Note that Birdset supports datasets<=3.6.0 version, we are looking into updates to support the newest version.

Datasets

• We assemble a training dataset for each test dataset that is a subset of a complete Xeno-Canto (XC) snapshot. We extract all recordings that have vocalizations of the bird species appearing in the test dataset.
• The focal training datasets or soundscape test datasets components can be individually accessed using the identifiers NAME_xc and NAME_scape, respectively (e.g., HSN_xc for the focal part and HSN_scape for the soundscape).
• We use the .ogg format for every recording and a sampling rate of 32 kHz.
• Each sample in the training dataset is a recording that may contain more than one vocalization of the corresponding bird species.
• Each recording in the training datasets has a unique recordist and the corresponding license from XC. We omit all recordings from XC that are CC-ND.
• The bird species are translated to ebird_codes
• Snapshot date of XC: 03/10/2024

Each dataset (except for XCM and XCL that only feature Train) comes with a dataset dictionary that features Train, Test_5s, and Test:

Train

• Exclusively using focal audio data as a subset from XCL with quality ratings A, B, C and excluding all recordings that are CC-ND.
• Each dataset is tailored for specific target species identified in the corresponding test soundscape files.
• We transform the scientific names of the birds into the corresponding ebird_code label.
• We offer detected events and corresponding cluster assignments to identify bird sounds in each recording.
• We provide the full recordings from XC. These can generate multiple samples from a single instance.

Test_5s

• Task: Processed to multilabel classification ("ebird_code_multilabel").
• Only soundscape data from Zenodo formatted acoording to the Kaggle evaluation scheme.
• Each recording is segmented into 5-second intervals where each ground truth bird vocalization is assigned to.
• This contains segments without any labels which results in a [0] vector.

Test

• Only soundscape data sourced from Zenodo.
• Each sample points to the complete soundscape file where the strong label with bounding boxes appears.
• This dataset does automatically have samples with recordings that do not contain bird calls.

How to

• We recommend to first explore the readme in our repository
• Additionally, you can refer to the Intro notebook
• The BirdSet Code package simplfies the data processing steps
• For multi-label evaluation with a segment-based evaluation use the test_5s column for testing.

We provide a very short example where no additional code is required. We load the first 5 seconds to quickly create an examplary training dataset. We recommend to start with HSN. It is a medium size dataset with a low number of overlaps within a segment.

from datasets import Audio

dataset = load_dataset("DBD-research-group/BirdSet", "HSN")

# slice example
dataset["train"] = dataset["train"].select(range(500))

# the dataset comes without an automatic Audio casting, this has to be enabled via huggingface
# this means that each time a sample is called, it is decoded (which may take a while if done for the complete dataset)
# in BirdSet, this is all done on-the-fly during training and testing (since the dataset size would be too big if mapping and saving it only once)
dataset = dataset.cast_column("audio", Audio(sampling_rate=32_000))

# extract the first five seconds of each sample in training (not utilizing event detection)
# a custom decoding with soundfile, stating start and end would be more efficient (see BirdSet Code)
def map_first_five(sample):
    max_length = 160_000 # 32_000hz*5sec
    sample["audio"]["array"] =  sample["audio"]["array"][:max_length]
    return sample

# train is now available as an array that can be transformed into a spectrogram for example 
train = dataset["train"].map(map_first_five, batch_size=1000, num_proc=2)

# the test_5s dataset is already divided into 5-second chunks where each sample can have zero, one or multiple bird vocalizations (ebird_code labels)
test = dataset["test_5s"]

Changelog

2025.09.10

• Updated dataset description and citation in BirdSet.py
• Now works with datasets<4.0.0
  • load_dataset(..., cache_dir="/path/to/custom/cache") almost works like in previous datasets<3.0.0 version. With the different that archive files are downloaded to HF_HOME/HF_DATASETS environment variable path but extracted file are save in cache_dir if specified. During extraction archive files are still deleted during extraction, see previous update/issue, to save disk space.

2024.12.06

• The data download size descrepancy has been solved.
• TL;DR: During the extraction process, unnecessary archives are now removed immediately. This reduces the required disk space by half, now aligning it with the table below.
• Note: If you downloaded the data between this and last update and don't want to update, you can use the following revision=b0c14a03571a7d73d56b12c4b1db81952c4f7e64:

from datasets import load_dataset
ds = load_dataset("DBD-research-group/BirdSet", "HSN", trust_remote_code=True, revision="b0c14a03571a7d73d56b12c4b1db81952c4f7e64")

2024.11.27

• Additional bird taxonomy metadata, including "Genus," "Species Group," and "Order," is provided using the 2021 eBird taxonomy, consistent with the taxonomy used for the 'ebird_code' data. These metadata fields follow the same format and encoding as 'ebird_code' and 'ebird_code_multilabel'. See below for an updated explanation of the metadata.
• If you don't require the additional taxonomy at the moment and prefer to avoid re-downloading all files, you can specify the previous revision directly in load_dataset as follows:

from datasets import load_dataset
ds = load_dataset("DBD-research-group/BirdSet", "HSN", trust_remote_code=True, revision="629b54c06874b6d2fa886e1c0d73146c975612d0")

Metadata

Example Metadata Train

{'audio': {'path': '.ogg',
  'array': array([ 0.0008485 ,  0.00128899, -0.00317163, ...,  0.00228528,
          0.00270796, -0.00120562]),
  'sampling_rate': 32000},
 'filepath': '.ogg',
 'start_time': None,
 'end_time': None,
 'low_freq': None,
 'high_freq': None,
 'ebird_code': 0,
 'ebird_code_multilabel': [0],
 'ebird_code_secondary': ['plaant1', 'blfnun1', 'butwoo1', 'whtdov', 'undtin1', 'gryhaw3'],
 'call_type': 'song',
 'sex': 'uncertain',
 'lat': -16.0538,
 'long': -49.604,
 'length': 46,
 'microphone': 'focal',
 'license': '//creativecommons.org/licenses/by-nc/4.0/',
 'source': 'xenocanto',
 'local_time': '18:37',
 'detected_events': [[0.736, 1.824],
  [9.936, 10.944],
  [13.872, 15.552],
  [19.552, 20.752],
  [24.816, 25.968],
  [26.528, 32.16],
  [36.112, 37.808],
  [37.792, 38.88],
  [40.048, 40.8],
  [44.432, 45.616]],
 'event_cluster': [0, 0, 0, 0, 0, -1, 0, 0, -1, 0],
 'peaks': [14.76479119037789, 41.16993396760847],
 'quality': 'A',
 'recordist': '...'
 'genus': 26,
 'species_group': 7,
 'order': 5,
 'genus_multilabel': None,
 'species_group_multilabel': None,
 'order_multilabel': None}}

Example Metadata Test5s

{'audio': {'path': '.ogg',
  'array': array([-0.67190468, -0.9638235 , -0.99569213, ..., -0.01262935,
         -0.01533066, -0.0141047 ]),
  'sampling_rate': 32000},
 'filepath': '.ogg',
 'start_time': 0.0,
 'end_time': 5.0,
 'low_freq': 0,
 'high_freq': 3098,
 'ebird_code': None,
 'ebird_code_multilabel': [1, 10],
 'ebird_code_secondary': None,
 'call_type': None,
 'sex': None,
 'lat': 5.59,
 'long': -75.85,
 'length': None,
 'microphone': 'Soundscape',
 'license': 'Creative Commons Attribution 4.0 International Public License',
 'source': 'https://zenodo.org/record/7525349',
 'local_time': '4:30:29',
 'detected_events': None,
 'event_cluster': None,
 'peaks': None,
 'quality': None,
 'recordist': None,
 'genus': None,
 'species_group': None,
 'order': None,
 'genus_multilabel': [26],
 'species_group_multilabel': [7],
 'order_multilabel': [5]}}

Citation Information

@misc{rauch2025birdsetlargescaledatasetaudio,
      title={BirdSet: A Large-Scale Dataset for Audio Classification in Avian Bioacoustics}, 
      author={Lukas Rauch and Raphael Schwinger and Moritz Wirth and René Heinrich and Denis Huseljic and Marek Herde and Jonas Lange and Stefan Kahl and Bernhard Sick and Sven Tomforde and Christoph Scholz},
      year={2025},
      eprint={2403.10380},
      archivePrefix={arXiv},
      primaryClass={cs.SD},
      url={https://arxiv.org/abs/2403.10380}, 
}

Licensing

• Researchers shall use this dataset only for non-commercial research and educational purposes.
• Each train recording in BirdSet taken from Xeno-Canto has its own CC license. Please refer to the metadata file to view the license for each recording.
• We exclude all recordings with a ND licenses. Every recording is NC.
• Each test dataset is licensed under CC BY 4.0.
• POW as validation dataset is licensed under CC0 1.0.

We have diligently selected and composed the contents of this dataset. Despite our careful review, if you believe that any content violates licensing agreements or infringes on intellectual property rights, please contact us immediately. Upon notification, we will promptly investigate the issue and remove the implicated data from our dataset if necessary. Users are responsible for ensuring that their use of the dataset complies with all licenses, applicable laws, regulations, and ethical guidelines. We make no representations or warranties of any kind and accept no responsibility in the case of violations.