Dataset Card for "turkish-nlp-suite/turkish-wikiNER"

Dataset Description

• Repository: Turkish-WikiNER
• Paper: ACL link
• Dataset: Turkish-WikiNER
• Domain: Wiki
• Number of Labels: 18

Dataset Summary

Turkish NER dataset from Wikipedia sentences. 20.000 sentences are sampled and re-annotated from Kuzgunlar NER dataset. Annotations are done by Co-one. Many thanks to them for their contributions. This dataset is also used in our brand new spaCy Turkish packages.

Dataset Instances

An instance of this dataset looks as follows:

{
"tokens": ["Çekimler", "5", "Temmuz", "2005", "tarihinde", "Reebok", "Stadyum", ",", "Bolton", ",", "İngiltere'de", "yapılmıştır", "."],
"tags": [O", "B-DATE", "I-DATE", "I-DATE", "O", "B-FAC", "I-FAC", "O", "B-GPE", "O", "B-GPE", "O", "O"]
}

or even better:

Labels

• CARDINAL
• DATE
• EVENT
• FAC
• GPE
• LANGUAGE
• LAW
• LOC
• MONEY
• NORP
• ORDINAL
• ORG
• PERCENT
• PERSON
• PRODUCT
• QUANTITY
• TIME
• TITLE
• WORK_OF_ART

Tagset

This dataset uses IOB tagging with the 39 labels above stored as Sequence(ClassLabel).

from datasets import load_dataset
ds = load_dataset("turkish-nlp-suite/turkish-wikiNER")
id2label = ds["train"].features["tags"].feature.names

Data Split

Benchmarking

BERTurk achieved the following result on this dataset:

              precision    recall  f1-score   support

    CARDINAL       0.75      0.71      0.73       240
        DATE       0.75      0.79      0.77       343
       EVENT       0.54      0.72      0.62        95
         FAC       0.60      0.67      0.63        46
         GPE       0.81      0.80      0.80       566
    LANGUAGE       0.56      0.69      0.62        29
         LAW       0.00      0.00      0.00         6
         LOC       0.55      0.70      0.61        50
       MONEY       0.00      0.00      0.00         5
        NORP       0.64      0.80      0.71       212
     ORDINAL       0.78      0.84      0.81       104
         ORG       0.81      0.66      0.73       250
     PERCENT       0.00      0.00      0.00         4
      PERSON       0.90      0.92      0.91       654
     PRODUCT       0.67      0.49      0.57        61
    QUANTITY       0.51      0.58      0.55        36
        TIME       0.00      0.00      0.00         4
       TITLE       0.50      0.35      0.41       153
 WORK_OF_ART       0.72      0.78      0.75       158

   micro avg       0.76      0.76      0.76      3016
   macro avg       0.53      0.55      0.54      3016
weighted avg       0.75      0.76      0.76      3016

Citation

This work is supported by Google Developer Experts Program. Part of Duygu 2022 Fall-Winter collection, "Turkish NLP with Duygu"/ "Duygu'yla Türkçe NLP". All rights reserved. If you'd like to use this dataset in your own work, please kindly cite A Diverse Set of Freely Available Linguistic Resources for Turkish :

@inproceedings{altinok-2023-diverse,
    title = "A Diverse Set of Freely Available Linguistic Resources for {T}urkish",
    author = "Altinok, Duygu",
    booktitle = "Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2023",
    address = "Toronto, Canada",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2023.acl-long.768",
    pages = "13739--13750",
    abstract = "This study presents a diverse set of freely available linguistic resources for Turkish natural language processing, including corpora, pretrained models and education material. Although Turkish is spoken by a sizeable population of over 80 million people, Turkish linguistic resources for natural language processing remain scarce. In this study, we provide corpora to allow practitioners to build their own applications and pretrained models that would assist industry researchers in creating quick prototypes. The provided corpora include named entity recognition datasets of diverse genres, including Wikipedia articles and supplement products customer reviews. In addition, crawling e-commerce and movie reviews websites, we compiled several sentiment analysis datasets of different genres. Our linguistic resources for Turkish also include pretrained spaCy language models. To the best of our knowledge, our models are the first spaCy models trained for the Turkish language. Finally, we provide various types of education material, such as video tutorials and code examples, that can support the interested audience on practicing Turkish NLP. The advantages of our linguistic resources are three-fold: they are freely available, they are first of their kind, and they are easy to use in a broad range of implementations. Along with a thorough description of the resource creation process, we also explain the position of our resources in the Turkish NLP world.",
}