OMG_prot50: An Open MetaGenomic Protein Dataset

The OMG_prot50 dataset is a protein-only dataset, created by clustering the Open MetaGenomic dataset (OMG) at 50% sequence identity.

MMseqs2 linclust (Steinegger and Söding 2018) was used to cluster all 4.2B protein sequences from the OMG dataset, resulting in 207M protein sequences. Sequences were clustered at 50% sequence id and 90% sequence coverage, and singleton clusters were removed.

See https://github.com/TattaBio/OMG for details.

Use

import datasets
ds = datasets.load_dataset('tattabio/OMG_prot50')

To preview the dataset without downloading, load in streaming mode:

import datasets
ds = datasets.load_dataset('tattabio/OMG_prot50', streaming=True)['train']
print(next(iter(ds)))

Citation

BibTeX:

@article{Cornman2024,
  title = {The OMG dataset: An Open MetaGenomic corpus for mixed-modality genomic language modeling},
  url = {https://www.biorxiv.org/content/early/2024/08/17/2024.08.14.607850},
  DOI = {10.1101/2024.08.14.607850},
  publisher = {Cold Spring Harbor Laboratory},
  author = {Cornman, Andre and West-Roberts, Jacob and Camargo, Antonio Pedro and Roux, Simon and Beracochea, Martin and Mirdita, Milot and Ovchinnikov, Sergey and Hwang, Yunha},
  year = {2024},
}