Dataset Card for DUVEL

Dataset Description

• Homepage: https://huggingface.co/datasets/cnachteg/DUVEL/
• Repository: https://github.com/cnachteg/DUVEL
• Paper: TBA
• Point of Contact: Charlotte Nachtegael - Charlotte.Nachtegael@ulb.be

Dataset Summary

This dataset was created to identity oligogenic variant combinations, i.e. relation between several genes and their mutations, causing genetic diseases in scientific articles written in english. At the moment, it contains only digenic variant combinations, i.e. relations between two genes and at least two variants. The dataset is intended for binary relation extraction where the entities are masked within the text.

Supported Task

The dataset can be used to train a model for text-classification (as the relation extraction task is here considered as a classification task). Success on this task is typically measured by achieving a high F1-score.

The BiomedBERT-large (https://huggingface.co/microsoft/BiomedNLP-BiomedBERT-large-uncased-abstract) currently achieves the best performance with the following F1-score of 0.8371, with a precision of 0.8506 and a recall of 0.8239.

Languages

The dataset consists in text extracted from scientific articles written in english (en).

Dataset Structure

Data Instances

Each instance describes the two genes and two variants composing the potential digenic variant combination, as well as the fragment of text with the masked entities, the PubMed Central identifier of the article and the label of the instance (i.e., if the fragment of text contains a valid digenic variant combination or not, respectively 1 and 0).

{
  'sentence': 'Two unrelated KS patients had heterozygous NELF mutations and mutation in a second gene: NELF/@GENE$ (@VARIANT$; p.Ala253Thr of @GENE$ and c.488_490delGTT; p.Cys163del of KAL1) and NELF/TACR3 (c. 1160-13C>T of NELF and c.824G>A; @VARIANT$ of TACR3).',
  'pmcid': 3888818,
  'gene1': 'KAL1;55445',
  'gene2': 'NELF;10648',
  'variant1': 'c.757G>A;tmVar:c|SUB|G|757|A;HGVS:c.757G>A;VariantGroup:3;CorrespondingGene:26012;RS#:142726563;CA#:5370407',
  'variant2': 'p.Trp275X;tmVar:p|SUB|W|275|X;HGVS:p.W275X;VariantGroup:1;CorrespondingGene:6870;RS#:144292455;CA#:144871',
  'label': 0
}

Data Fields

• sentence: string, text containing the entities masked with either @GENE$ for the gene type or @VARIANT$ for the mutation type. The text can be either single or cross-sentence, but no longer than 256 tokens according to the BiomedBERT tokenizer (see BiomedBERT).
• pmcid: int, PubMed Central identifier of the article from which the text was extracted (https://www.ncbi.nlm.nih.gov/pmc/)
• gene1: string, first gene mention as it appears in the text and internal identifier.
• gene2: string, second gene mention as it appears in the text and internal identifier.
• variant1: string, first variant mention as it appears in the text, with its normalized form, HGVS form (https://varnomen.hgvs.org/), gene where it occurs, and eventually variation identifier is available.
• variant2: string, second variant mention as it appears in the text, with its normalized form, HGVS form (https://varnomen.hgvs.org/), gene where it occurs, and eventually variation identifier is available.
• label: int, class of the instance, 0 if there is no relation between the entities, 1 if there is.

Data Splits

Dataset is split between train, dev and test sets. Splitting has been done with a stratified split based on the labels in order to maintain a similar distribution (around 9.4% of positive class).

Dataset Creation

Curation Rationale

The curation of oligogenic variant combinations requires high expertise and time, while the number of genetic studies have increased across the years, especially with the apparition of the next-generation sequencing technologies. This dataset aims to support such curation by extracting potential candidates directly from the text.

Source Data

Initial Data Collection and Normalization

Scientific articles containing oligogenic variant combinations potentially causing genetic diseases were retrieved from OLIDA, the OLIgogenic diseases DAtabase. Articles were filtered to keep only those containing at least one digenic variant combination, i.e. combination between two genes and at least one variant in each gene. The articles were then pre-annotated with the help of PubTator API (https://www.ncbi.nlm.nih.gov/research/pubtator/api.html) to obtain the full text of the articles with the genes and variants identified.

Fragment of texts to annotate were created by extracting all the text (both single and cross-sentence) containing two different gene and two different variant mentions with a maximum length of 256 tokens, as tokenized by the BiomedBERT tokenizer (see BiomedBERT). Text containing tables or incomplete sentences were excluded during the annotation process.

Who are the source language producers?

The dataset is machine-generated, as the full annotated text of the article is retrieved from the PubTator API and then the relevant text containing two genes and two variants are generated through python scripts.

Annotations

The annotation was done with the ALAMBIC platform, with an Active Learning (AL) setting (see Nachtegael 2023).

Annotation process

1500 samples were randomly selected to be labelled, with 1000 samples for the test set and 500 as seed for the AL process. 9 iterations of AL selection of 500 samples with the Margin Sampling strategy was conducted with BiomedBERT as the model used for the selection (see BiomedBERT), samples subsequently annotated. The annotation limit was initially set at 6000 samples, but was exceeded due to several restarts of the process due to exclusion of invalid instances.

The annotator had access to the genes and variants, the PMCID of the article the text was extracted from and the text with the masked entities. One out of three possible classes is given to each fragment of text :

• 0 for the absence of a digenic variant combination relation in the text.
• 1 for the presence of a digenic variant combination relation. The genes and the variants need to be relating to each other for there to be a valid relation. If the entities are involved in an alleged digenic relation according to OLIDA, but the syntactic aspects of the text showed no clear relation between the entities, then the text contains no relation. The combination needs to be carried by at least one individual.
• -1 if the fragment of text is not valid. The text can be deemed as invalid if one of the entities is not a valid entity, i.e. not a valid gene name or mutation, or the text contains an unfinished sentence or invalid sentence, i.e. with part of the text being a table. Invalid gene name and mutation comprised : (a) error in the annotation, e.g. P05, a patient denomination, which was annotated as a gene name or the cell line HEK293 which was annotated as variant; (b) genes in species not human; (c) Isoforms denominations of proteins and (d) gene products. Tables were excluded as it is not considered as comprehensive text without the notion of their structure. To be used, they would need to be parsed in order to convey this structure, which is not rendered in free text.

Only instances from the positive and the negative classes (labels of 0 and 1) are included in the final data set, all the invalid instances are excluded from further use as they do not fill our quality standards.

It must be noted that while the articles were filtered for those containing digenic variant combinations, it is possible to also find oligogenic variant combinations involving more than two genes and/or two variants. In that case, a subset of those variant combinations, i.e. two gene-variant pairs which are connected in the text and are part of the variant combination, were considered as a valid digenic variant combinations and classified them as class 1.

Who are the annotators?

Annotation was done by Charlotte Nachtegael, one of the author and curator of OLIDA, with a substantial background in genetics and molecular biology.

Personal and Sensitive Information

None.

Considerations for Using the Data

Social Impact of Dataset

The dataset should help to the curation of complex genetic diseases, contributing to the research of such medical problems. It should not, at the moment, but used exclusively for support of the curation and not as the curation iteself of oligogenic/digenic variant combinations.

Discussion of Biases

Some diseases are more studied/known as oligogenic, thus the variants and genes could be biased towards those gene panels more well-known. Moreover, some articles are more represented in the dataset than others because they had more genes and/or variants in the text than others.

The named entity recognition step was also done automatically, so it could be possible that some entities were not recognized and thus ignored when creating the candidates. When errors were encountered during the annotation process, the candidates were excluded from the dataset.

Other Known Limitations

None.

Additional Information

Dataset Curators

This work was supported by the Service Public de Wallonie Recherche by DIGITALWALLONIA4.AI [2010235—ARIAC]

• Charlotte Nachtegael, Université Libre de Bruxelles, Belgium

Licensing Information

This dataset is under the Creative Commons Attribution Non Commercial Share Alike 4.0 license.

Citation Information

TBA

@article{DUVEL_2024,
  author    = {},
  title     = {},
  journal   = {},
  year      = {2024}
}

Contributions

Thanks to Barbara Gravel and Sofia Papadimitriou for their initial work with OLIDA. Thanks to Jacopo De Stefani, Anthony Cnudde and Tom Lenaerts for their help with the experimental design and writing of the paper for DUVEL.