This repository contains the code and datasets for detecting AI-generated texts (AIGTs) and human-written texts (HWTs) in Arabic. Our focus is on addressing the challenges posed by diacritics in Arabic text using various transformer-based models, including AraELECTRA, AraBERT, XLM-R, and mBERT. The model architecture is adaptable to any language, although our primary focus is on Arabic due to the unique challenges it presents.
The Dataset directory includes the datasets used in our experiments. These datasets are prepared with Training, Validation, and Testing splits to facilitate model training and evaluation:
Our models are designed to handle the complexities of diacritics in Arabic text, but the architecture is flexible enough to be adapted for other languages. We use transformer-based pre-trained models and fine-tune them on our custom datasets to enhance their ability to distinguish between AIGTs and HWTs. Additionally, we have another code branch called Diacritics-free, which implements a dediacritization filter as a preprocessing step for the dataset. This branch is focused on improving model performance by removing diacritics before training and evaluation.
All narrative results, logs, and performance metrics are included in the Experiments Details and Detection Models directory. Each experiment’s results and the corresponding models are provided in the Results.txt file, which includes links to download the model checkpoints and their outputs.
To set up the environment, install the required packages using the requirements.txt file:
pip install -r requirements.txt
To run the training and evaluation process, ensure that your dataset directory contains Training.csv, Validation.csv, and Testing.csv files.
For inference, use the Evaluator.py module. This module is dedicated to evaluating model performance and requires the model checkpoint files, which can be downloaded from the Experiments Details and Detection Models directory.
Train the Model:
python main.py
Evaluate the Model:
Ensure the model_checkpoints folder is available in the project directory and contains the best_model.pt file of the model that matches the config.json file's model_name. Then run:
python Evaluator.py
This repository contains the code and datasets for detecting AI-generated texts (AIGTs) and human-written texts (HWTs) in Arabic. Our focus is on addressing the challenges posed by diacritics in Arabic text using various transformer-based models, including AraELECTRA, AraBERT, XLM-R, and mBERT. The model architecture is adaptable to any language, although our primary focus is on Arabic due to the unique challenges it presents.
The Dataset directory includes the datasets used in our experiments. These datasets are prepared with Training, Validation, and Testing splits to facilitate model training and evaluation:
Our models are designed to handle the complexities of diacritics in Arabic text, but the architecture is flexible enough to be adapted for other languages. We use transformer-based pre-trained models and fine-tune them on our custom datasets to enhance their ability to distinguish between AIGTs and HWTs. Additionally, we have another code branch called Diacritics-free, which implements a dediacritization filter as a preprocessing step for the dataset. This branch is focused on improving model performance by removing diacritics before training and evaluation.
All narrative results, logs, and performance metrics are included in the Experiments Details and Detection Models directory. Each experiment’s results and the corresponding models are provided in the Results.txt file, which includes links to download the model checkpoints and their outputs.
To set up the environment, install the required packages using the requirements.txt file:
pip install -r requirements.txt
To run the training and evaluation process, ensure that your dataset directory contains Training.csv, Validation.csv, and Testing.csv files.
For inference, use the Evaluator.py module. This module is dedicated to evaluating model performance and requires the model checkpoint files, which can be downloaded from the Experiments Details and Detection Models directory.
Train the Model:
python main.py
Evaluate the Model:
Ensure the model_checkpoints folder is available in the project directory and contains the best_model.pt file of the model that matches the config.json file's model_name. Then run:
python Evaluator.py