Tasks

1. Sentence Recognition
   • One-Step, receives sequence of tokens as input and label each token as B-BACKGROUND, I-BACKGROUND, B-QUESTION, I-QUESTION, B-IGNORE, or I-IGNORE.
   • Two-Step
     • Boundary Detection, receives sequence of tokens as input and label each token as B (the beginning of a sentence) or I (not the beginning of a sentence).
     • Sentence Classification, receives sequence of sentences as input and label each sentence as BACKGROUND, QUESTION, or IGNORE.
2. Medical Entity Recognition, receives sequence of tokens as input and label each token as B-DISEASE, I-DISEASE, B-DRUG, I-DRUG, B-SYMPTOM, I-SYMPTOM, B-TREATMENT, I-TREATMENT, or OO.
3. Keyphrases Extraction, receives sequence of tokens as input and label each token as BK, IK, or OO.
4. Multi-Task Learning for Medical Entity Recognition and Keyphrases Extraction.

BERT

1. IndoDistilBERT, set distil_bert = True when initialize task object to use it.
2. IndoBERT by IndoLEM, set distil_bert = False and indonlu = False when initialize task object to use it.
3. IndoBERT by IndoNLU (Base Version), set distil_bert = False and indonlu = True, and base_version = True when initialize task object to use it.
4. IndoBERT by IndoNLU (Large Version), set distil_bert = False and indonlu = True, and base_version = False when initialize task object to use it.

Models

There are three models that can be used for one-step sentence recognition, boundary detection, medical entity recognition, and keyphrases extraction:

1. BERT
2. BERT-BiLSTMs
3. BERT-BiLSTMs-CRF

For multi-task learning, you can use regular BERT-BiLSTMs or use medical entity as keyphrases extraction's feature

Post-Processing

When a token is split into word-pieces during the tokenization process, there are four rules that can be used to decide the final label:

1. Set label prediction for the first word-piece as final label
2. Set label prediction for the last word-piece as final label
3. Set majority label prediction among word-pieces as final label. If two or more labels have same count, choose the label that appears first.
4. Use following rules to decide the final label:
   • If there is one or more B-<...> label, choose B-<...> that appears first as final label.
   • If there is no B-<...> label, choose I-<...> with the most occurences as final label.
   • If there is more than one I-<...> label with the most occurences, choose I-<...> label that appears first as final label.
   • If there is no B-<...> or I-<...>, label, choose OO as final label.

To use one of those rules, pass corresponding rule number (1, 2, 3, or 4) as argument for version parameter when calling task object's post_evaluate function.

Notes

For a more detailed guide, take a look at example.ipynb file.