Hybrid Transformer-RNN Model for Classification of Indonesian Regional Language

Author: Muhammad Fadhil Al-Amal

Description

This repository contains the code and data for the research paper "Hybrid Transformer-RNN Model for Classification of Indonesian Regional Language". The project focuses on developing a hybrid model that combines the strengths of Transformer-based models (NusaBERT) with Recurrent Neural Networks (RNNs) to improve the classification of various Indonesian regional languages. The goal is to provide a more inclusive and effective approach to Natural Language Processing for underrepresented languages.

This research has been published in the 2023 3rd International Conference on Information System & Information Technology (ICISIT).

Publication: https://ieeexplore.ieee.org/abstract/document/11389462/

Repository Structure

The repository is organized as follows:

• data/: Contains all data files, including CSVs, Excel files, and logs.
• docs/: Contains all documentation files, including papers, presentations, and diagrams.
• experiments/: Contains experiment-related files, such as configurations and results.
• images/: Contains all image files.
• models/: Contains the machine learning models.
• notebooks/: Contains the Jupyter notebook for experimentation.
• src/: Contains all Python source code files.

Installation

1. Clone the repository:

   git clone <repository-url>
   

2. Install the required dependencies:

   pip install -r requirements.txt
   

Usage

To run the experiments, you can use the main.py script in the src directory. You can specify the configuration file for the experiment you want to run.

python src/main.py --config experiments/config/nusabert-bigru-16.yaml

Datasets

The datasets used in this research are from the NusaX collection and are included in the data directory. The languages covered are:

• Batak (btk)
• Betawi (bew)
• Buginese (bug)
• Javanese (jav)
• Madurese (mad)
• Makassarese (mak)
• Minangkabau (min)
• Musi (mui)
• Rejang (rej)
• Sundanese (sun)

The tasks include:

• Topic Classification (nusaparagraph_topic)
• Emotion Classification (nusaparagraph_emot)
• Rhetoric Classification (nusaparagraph_rhetoric)

Models

The following models are implemented in the models directory:

• BERTBiGRUClassifier
• BERTBiLSTMClassifier
• BERTPrototypicalClassifier
• BERTBiGRUPrototypicalClassifier

Results

The results of the experiments can be found in the experiments/results directory.