Dataset Card for CanadianInvertebrates-ML

Alternative names: InvertebratesCanada-ML, CanInv-ML, CanInv-1M, Canada-1.5M

This dataset is used in the paper BarcodeBERT: Transformers for Biodiversity Analysis.

Paper Abstract

In the global challenge of understanding and characterizing biodiversity, short species-specific genomic sequences known as DNA barcodes play a critical role, enabling fine-grained comparisons among organisms within the same kingdom of life. Although machine learning algorithms specifically designed for the analysis of DNA barcodes are becoming more popular, most existing methodologies rely on generic supervised training algorithms. We introduce BarcodeBERT, a family of models tailored to biodiversity analysis and trained exclusively on data from a reference library of 1.5M invertebrate DNA barcodes. We compared the performance of BarcodeBERT on taxonomic identification tasks against a spectrum of machine learning approaches including supervised training of classical neural architectures and fine-tuning of general DNA foundation models. Our self-supervised pretraining strategies on domain-specific data outperform fine-tuned foundation models, especially in identification tasks involving lower taxa such as genera and species. We also compared BarcodeBERT with BLAST, one of the most widely used bioinformatics tools for sequence searching, and found that our method matched BLAST's performance in species-level classification while being 55 times faster. Our analysis of masking and tokenization strategies also provides practical guidance for building customized DNA language models, emphasizing the importance of aligning model training strategies with dataset characteristics and domain knowledge. The code repository is available at this https URL .

Overview

CanadianInvertebrates-ML is a machine learning-ready dataset derived from the raw DNA barcodes published in deWaard et. al, 2019. The data is specifically designed and curated for different machine learning tasks in biodiversity analysis: species classification, genus identification of novel species, and BIN reconstruction.

Citation

If you make use of this dataset and/or its code repository, please cite the following paper:

@misc{arias2025barcodeberttransformersbiodiversityanalysis,
      title={BarcodeBERT: Transformers for Biodiversity Analysis}, 
      author={Pablo Millan~Arias and Niousha Sadjadi and Monireh Safari and ZeMing Gong and Austin T. Wang and Joakim Bruslund Haurum and Iuliia Zarubiieva and Dirk Steinke and Lila Kari and Angel X. Chang and Scott C. Lowe and Graham W. Taylor},
      year={2025},
      eprint={2311.02401},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/2311.02401}, 
}

Dataset content

Each specimen contains the following annotations:

• Indexing fields
  • processid: A unique number assigned by BOLD (International Barcode of Life Consortium).
  • sampleid: A unique identifier given by the collector.
• Taxonomic labels
  • phylum, class, order, family, genus, species: Hierarchical taxonomic classification
• Genetic information
  • dna_bin: Barcode Index Number
  • dna_barcode: DNA barcode sequence
• Split and localization
  • split: Data partition label (e.g., pretrain, train, test, val, test_unseen)

Sample Usage

To use this dataset with the BarcodeBERT model, you can follow the example from the associated GitHub repository:

from transformers import AutoTokenizer, AutoModel

# Load the tokenizer
tokenizer = AutoTokenizer.from_pretrained(
    "bioscan-ml/BarcodeBERT", trust_remote_code=True
)

# Load the model
model = AutoModel.from_pretrained("bioscan-ml/BarcodeBERT", trust_remote_code=True)

# Sample sequence
dna_seq = "ACGCGCTGACGCATCAGCATACGA"

# Tokenize
input_seq = tokenizer(dna_seq, return_tensors="pt")["input_ids"]

# Pass through the model
output = model(input_seq.unsqueeze(0))["hidden_states"][-1]

# Compute Global Average Pooling
features = output.mean(1)

BIOSCAN Datasets for PyTorch

CanadianInvertebrates-ML is being incorporated into the BIOSCAN Datasets for PyTorch project, which provides a standardized way to access and utilize this dataset in PyTorch-based machine learning workflows. You can see the work in progress in this pull request.

Dataset Sources

• GitHub: https://github.com/bioscan-ml/BarcodeBERT
• Zenodo: https://doi.org/10.5281/zenodo.15650124
• Paper: https://arxiv.org/abs/2311.02401