MTAGEC: Multi-Task Arabic Grammatical Error Correction with Synthetic Data

MTAGEC is a unified framework for Arabic Grammatical Error Correction (AGEC) that performs:

• Error correction
• Error-type classification
• Explanation (rationale), and evidence word extraction

Trained on ExplAGEC: the largest synthetic dataset for explainable AGEC, with 21.8M sentence pairs and 25 error types.

MTAGEC architecture: Arabic multi-task transformer for correction, error-type classification, and explanation

Features

• AEEG: Used to generate synthetic error generation.
• Unified Transformer: Joint sequence generation for correction + explanation.
• Benchmarks: ExplAGEC, QALB-2014, and QALB-2015.

Quick Start

1. Clone the repo

git clone https://github.com/Zainabobied/MTAGEC.git
cd MTAGEC

2. Install dependencies

pip install -r requirements.txt

3. Generate Synthetic Data

Step 1: Download the seed corpora

• Arabic Wikipedia (2019-02-01)
• OSIAN corpus

Step 2: Preprocess the corpora

• Combine the two datasets into a single CSV file.
• Each line in the file should contain one Arabic sentence.

Step 3: Generate the synthetic ExplAGEC dataset

python scripts/generate_dataset.py --input data/raw/seed_data.csv --output data/explagec/

Also, the dataset is available for download from Google Drive:

• Train (29.5 GB)
• Valudation (7.4 GB)
• Test (1.6 MB)

4. Download required models

# Download AraBERT and AraT5v2 models
python scripts/setup_models.py

This script will download the following models:

• AraBERT v2 - A BERT-based model for Arabic language understanding
• AraT5v2-base-1024 - A T5-based model for Arabic sequence-to-sequence tasks

The models will be saved to models/pretrained/ directory. You can specify a different directory using the --output-dir argument.

5. Train the model

# Train MTAGEC model
python scripts/train_mtagec.py --config config/config.yaml

6. Evaluate the model

# Evaluate on ExplAGEC test set
python scripts/evaluate.py --config config/config.yaml --model_path models/checkpoints/mtagec_transformer/checkpoint-epoch-X --explagec

Project Structure

MTAGEC/
├── config/                 # Configuration files
│   └── config.yaml        
├── data/                   # Data directory
│   ├── train.json/          
│   ├── val.json/        
│   └── test.json/                
├── models/                 # Model implementations
│   ├── checkpoints/       
│   └── mtagec_transformer.py 
├── scripts/                # Scripts for data processing and training
│   ├── evaluate.py         # Evaluation script
│   ├── generate_dataset.py # Synthetic data generation
│   ├── train_mtagec.py     # Model training
│   └── utils.py            # Utility functions
├── README.md               # This README
└── requirements.txt        # Python dependencies

Data

ExplAGEC Dataset

The ExplAGEC dataset is generated using the Arabic Explainable Error Generator (AEEG), which introduces 25 types of linguistic errors:

1. Orthographic errors: Alif/Ya confusion, Hamza confusion, Ta/Ha confusion, etc.
2. Morphological errors: Definite article misuse, Gender agreement, Number agreement, etc.
3. Semantic errors: Confusion in conjunction, Incorrect word choice
4. Punctuation errors: Punctuation replacement, deletion, insertion
5. Merge/Split errors: Word merging, Extra space

Example of synthetic data generated by AEEG:

{
  "source": ["ذهب", "الولد", "الى", "المدرسة", "بسرعة", "."],
  "target": ["ذهب", "الولد", "إلى", "المدرسة", "بسرعة", "."],
  "evidence_index": [[2, 2]],
  "correction_index": [[2, 2]],
  "error_type": ["خطأ في التمييز بين الألف والياء"],
  "error_classification": ["OA"],
  "predicted_parsing_order": [
    {
      "0": 3,
      "1": 3,
      "2": 1,
      "3": 3,
      "4": 3,
      "5": 3,
      "6": 2,
      "7": 3,
      "8": 3
    }
  ]
}

In this example:

• The original correct sentence is "يعتبر التعليم أساس تقدم المجتمعات." (Education is the foundation of societal progress)
• AEEG introduced a character deletion error (OM) by removing the letter 'ا' from the word "المجتمعات" (societies)
• The error is explained in Arabic: "خطأ في حذف حرف: تم حذف 'ا'" (Error in character deletion: 'ا' was deleted)
• The evidence points to token index 5 (the affected word)

Dependencies

Below are the required dependencies for running MTAGEC:

torch>=1.10.0
transformers>=4.18.0
datasets>=2.0.0
pyarabic>=0.6.15
arabert>=1.0.0
numpy>=1.20.0
scikit-learn>=1.0.0
tqdm>=4.62.0
pyyaml>=6.0
matplotlib>=3.5.0
seaborn>=0.11.2
pandas>=1.3.0
sacrebleu>=2.0.0
sentencepiece>=0.1.96  

Synthetic Data Generation

The Arabic Explainable Error Generator (AEEG) systematically introduces 25 types of linguistic errors based on the error taxonomy from Belkebir & Habash (2021) and Alfaifi (2014), covering morphological, syntactic, and orthographic errors. The AEEG algorithm follows these steps:

1. For each input sentence, it tokenizes the text and calculates the expected number of errors based on sentence length and a configurable error rate.
2. It samples error types using a dynamic weighted probability distribution that ensures balanced representation of all error categories.
3. For each error, it applies transformations specific to the error type (e.g., character substitution, word merging, gender agreement changes).
4. It generates explanations in Arabic describing each error and identifies evidence tokens that justify the correction.
5. The output includes the original sentence, modified sentence with errors, error type classifications, and contextual evidence.

This process creates the ExplAGEC dataset with 21.8 million sentence pairs, featuring aligned error-type classifications and contextual evidence to address Arabic's linguistic complexities.

Model

MTAGEC is based on Transformer encoder-decoder architecture with a unified softmax layer that combines:

1. Vocabulary tokens: For generating corrected text
2. Pointer indices: For extracting evidence from the input
3. Error type labels: For classifying the type of error

The model supports different training configurations:

• Baseline: Standard GEC (correction only)
• Infusion: GEC with explanation as auxiliary input
• Explanation-only: Error type classification and evidence extraction
• Self-Rationalization: Joint correction and explanation generation

Citation

@article{mahmoud2023mtagec,
  title={MTAGEC: Multi-Task Arabic Grammatical Error Correction as a sequence generation with Synthetic data},
  author={Zeinab Mahmoud, Marco Zappatore, Natalia Kryvinska, Mohammed Abdalsalm, Ali Alfatemi, Aiman Solyman},
  journal={Journal of King Saud University-Computer and Information Sciences},
  year={2025}
}