sanadbawab0/TranslateBasha

Jupyter Notebook

0

25 commits

updated May 17, 2024

See the code

README

Translate Basha

This project aims to enhance communication across the Arabic-speaking world by translating various colloquial Arabic dialects into Modern Standard Arabic (MSA) by fine-tuning araT5 model. We collected diverse linguistic data to train the model. The project culminates in a web application built with the Django framework, providing an accessible platform for users to input colloquial Arabic and receive translations in MSA.

Dataset

The dataset used for this project was collected from the Transformer_NMT_AD repository. This dataset includes a variety of colloquial Arabic dialects and their corresponding translations in Modern Standard Arabic, providing a robust foundation for training and evaluating our model.

notebooks/

This directory contains all the Jupyter notebooks related to training and testing the model.

'notebooks/training/'

This subfolder includes the notebooks used for training the model.

'notebooks/testing/'

This subfolder contains:

  • Evaluation Notebook: A single notebook that was used to evaluate the models.
  • test data Folder: A collection of test data used for evaluating the models.

Fine-Tuned Models

Gulf Dialect

Model Link: Gulf Fine-Tuned Model

Levantine Dialect

Model Link: Levantine Fine-Tuned Model

Iraqi Dialect

Model Link: Iraqi Fine-Tuned Model

North African Dialect

Model Link: North African Fine-Tuned Model

Nile Basin Dialect

Model Link: Nile Basin Fine-Tuned Model

Features

  • Translation: Translation from colloquial Arabic to Modern Standard Arabic (MSA).
  • Tashkeel Addition: Harakat (diacritical marks) for improved readability and pronunciation.
  • Grammatical Error Correction (GEC): To correct grammatical errors, enhancing the accuracy and fluency of translated sentences.
  • Dialect Classifier: Automatically identifies the dialect of the input text if the user forgets to select it, ensuring accurate translation.
  • Feedback Mechanism: Allows users to provide feedback for continuous improvement.
  • Saving Translations: Users can save translated sentences for future reference.
  • Translation History: Stores a history of translated sentences for easy reference.

Screenshots

App Screenshot

Run Locally

Clone the project

  git clone https://github.com/sanadbawab0/TranslateBasha.git

Install dependencies

  pip install -r requirements.txt

Start the server

  python manage.py runserver

sanadbawab0/TranslateBasha

Jupyter Notebook

0

25 commits

updated May 17, 2024

See the code

README

Translate Basha

This project aims to enhance communication across the Arabic-speaking world by translating various colloquial Arabic dialects into Modern Standard Arabic (MSA) by fine-tuning araT5 model. We collected diverse linguistic data to train the model. The project culminates in a web application built with the Django framework, providing an accessible platform for users to input colloquial Arabic and receive translations in MSA.

Dataset

The dataset used for this project was collected from the Transformer_NMT_AD repository. This dataset includes a variety of colloquial Arabic dialects and their corresponding translations in Modern Standard Arabic, providing a robust foundation for training and evaluating our model.

notebooks/

This directory contains all the Jupyter notebooks related to training and testing the model.

'notebooks/training/'

This subfolder includes the notebooks used for training the model.

'notebooks/testing/'

This subfolder contains:

  • Evaluation Notebook: A single notebook that was used to evaluate the models.
  • test data Folder: A collection of test data used for evaluating the models.

Fine-Tuned Models

Gulf Dialect

Model Link: Gulf Fine-Tuned Model

Levantine Dialect

Model Link: Levantine Fine-Tuned Model

Iraqi Dialect

Model Link: Iraqi Fine-Tuned Model

North African Dialect

Model Link: North African Fine-Tuned Model

Nile Basin Dialect

Model Link: Nile Basin Fine-Tuned Model

Features

  • Translation: Translation from colloquial Arabic to Modern Standard Arabic (MSA).
  • Tashkeel Addition: Harakat (diacritical marks) for improved readability and pronunciation.
  • Grammatical Error Correction (GEC): To correct grammatical errors, enhancing the accuracy and fluency of translated sentences.
  • Dialect Classifier: Automatically identifies the dialect of the input text if the user forgets to select it, ensuring accurate translation.
  • Feedback Mechanism: Allows users to provide feedback for continuous improvement.
  • Saving Translations: Users can save translated sentences for future reference.
  • Translation History: Stores a history of translated sentences for easy reference.

Screenshots

App Screenshot

Run Locally

Clone the project

  git clone https://github.com/sanadbawab0/TranslateBasha.git

Install dependencies

  pip install -r requirements.txt

Start the server

  python manage.py runserver