Dataset Card for "FiNER-ORD"

Table of Contents

• Dataset Description
  • Dataset Summary
  • Supported Tasks and Leaderboards
  • Languages
• Dataset Structure
  • Data Instances
  • Data Fields
• Dataset Creation and Annotation
• Additional Information
  • Licensing Information
  • Citation Information
  • Contact Information

Dataset Description

• Homepage: https://github.com/gtfintechlab/FiNER
• Repository: https://github.com/gtfintechlab/FiNER
• Paper: Arxiv Link
• Point of Contact: Agam A. Shah
• Size of train dataset file: 1.08 MB
• Size of validation dataset file: 135 KB
• Size of test dataset file: 336 KB

Dataset Summary

The FiNER-Open Research Dataset (FiNER-ORD) consists of a manually annotated dataset of financial news articles (in English) collected from webz.io. In total, there are 47851 news articles available in this data at the point of writing this paper. Each news article is available in the form of a JSON document with various metadata information like the source of the article, publication date, author of the article, and the title of the article. For the manual annotation of named entities in financial news, we randomly sampled 220 documents from the entire set of news articles. We observed that some articles were empty in our sample, so after filtering the empty documents, we were left with a total of 201 articles.
We use Doccano, an open-source annotation tool, to ingest the raw dataset and manually label person (PER), location (LOC), and organization (ORG) entities. For our experiments, we use the manually labeled FiNER-ORD to benchmark model performance. Thus, we make a train, validation, and test split of FiNER-ORD. To avoid biased results, manual annotation is performed by annotators who have no knowledge about the labeling functions for the weak supervision framework. The train and validation sets are annotated by two separate annotators and validated by a third annotator. The test dataset is annotated by another annotator. We present a manual annotation guide in the Appendix of the paper detailing the procedures used to create the manually annotated FiNER-ORD.

After manual annotation, the news articles are split into sentences. We then tokenize each sentence, employing a script to tokenize multi-token entities into separate tokens (e.g. PER_B denotes the beginning token of a person (PER) entity and PER_I represents intermediate PER tokens). We exclude white spaces when tokenizing multi-token entities.

For more details check information in paper

Supported Tasks and Leaderboards

More Information Needed

Languages

• It is a monolingual English dataset

Dataset Structure

Data Instances

FiNER-ORD

• Size of train dataset file: 1.08 MB
• Size of validation dataset file: 135 KB
• Size of test dataset file: 336 KB

Data Fields

The data fields are the same among all splits.

conll2003

• doc_idx: Document ID (int)
• sent_idx: Sentence ID within each document (int)
• gold_token: Token (string)
• gold_label: a list of classification labels (int). Full tagset with indices:

{'O': 0, 'PER_B': 1, 'PER_I': 2, 'LOC_B': 3, 'LOC_I': 4, 'ORG_B': 5, 'ORG_I': 6}

Dataset Creation and Annotation

Information in paper

Additional Information

This dataset is also available in the IOB format described in the CoNLL 2003 NER shared task paper (tner/conll2003 format). You can find this alternative dataset at: gtfintechlab/finer-ord-bio.

Licensing Information

Information in paper

Citation Information

@article{shah2024finerordfinancialnamedentity,
  title={FiNER-ORD: Financial Named Entity Recognition Open Research Dataset},
  author={Agam Shah and Abhinav Gullapalli and Ruchit Vithani and Michael Galarnyk and Sudheer Chava},
  journal={arXiv preprint arXiv:2302.11157},
  year={2024}
}

Contact Information

Please contact Agam Shah (ashah482[at]gatech[dot]edu) or Ruchit Vithani (rvithani6[at]gatech[dot]edu) about any FiNER-related issues and questions.
GitHub: @shahagam4, @ruchit2801 Website: https://shahagam4.github.io/