Neural Arabic text diacritization
See the code
This repository contains the models, dataset, helpers, and systems' comparison for our paper on Arabic Text Diacritization:
"Neural Arabic Text Diacritization: State of the Art Results and a Novel Approach for Machine Translation", Ali Fadel, Ibraheem Tuffaha, Bara' Al-Jawarneh and Mahmoud Al-Ayyoub, EMNLP-IJCNLP 2019.
These folders each contain the generated dataset, system output and DER/WER statistics used to compare our system with each of the three systems.
requirements.txt file
pip install -r requirements.txtTo predict the diacritized text using any model provided in this repository the script predict.py can be used, example:
python predict.py --input-file-path sample_input \
--model-type rnn \
--model-number 3 \
--model-size small \
--model-average 20 \
--output-file-path sample_output
The previous line will diacritize the text inside sample_input file using the rnn model that have number 3 trained on the small dataset (without extra training dataset) after averaging the last 20 epochs and writes the diacritized text on sample_output.
The allowed option are:
Before training any FFNN model you need to prepare the dataset using prepare_feed_forward_data.py script. After that to train any FFNN model you can use the model.ipynb notebooks that exists under models/ffnn_models/*/
There is no need to prepare any data to train RNN models, to train any RNN model you can use the model.ipynb notebooks that exists under models/rnn_models/*/
Note that the RNN models use CuDNNLSTM layers which should run on GPU, to train the models or predict output from them using CPU only you can use regular LSTM layers. Moreover, all RNN models checkpoints exist under models/rnn_models/*/*/ use CuDNNLSTM layers, so the checkpoints should be loaded on GPU, but under model/rnn_models/*/*/lstm/ you can find the same checkpoints with same weights and structure but with regular LSTM layers used instead of CuDNNLSTM layers.
Basic RNN model structure
All results reported below are on (Fadel et al., 2019) test set (best results shown in bold).
There are three feed-forward neural network models, the following table show the results of each of them:
| DER/WER | With case ending | Without case ending | With case ending | Without case ending |
|---|---|---|---|---|
| Including no diacritic | Excluding no diacritic | |||
| Basic Model | 9.33%/25.93% | 6.58%/13.89% | 10.85%/25.39% | 7.51%/13.53% |
| 100-Hot Model | 6.57%/20.21% | 4.83%/11.14% | 7.75%/19.83% | 5.62%/10.93% |
| Embeddings Model | 5.52%/17.12% | 4.06%/9.38% | 6.44%/16.63% | 4.67%/9.10% |
DER/WER statistics for FFNN models
There are three recurrent neural network models, each of them was trained twice with and without the extra training dataset. The following tables show the results of each of them:
| DER/WER | With case ending | Without case ending | With case ending | Without case ending |
|---|---|---|---|---|
| Including no diacritic | Excluding no diacritic | |||
| Basic Model | 2.68%/7.91% | 2.19%/4.79% | 3.09%/7.61% | 2.51%/4.66% |
| CRF Model | 2.67%/7.73% | 2.19%/4.69% | 3.08%/7.46% | 2.52%/4.60% |
| Normalized Model | 2.60%/7.69% | 2.11%/4.57% | 3.00%/7.39% | 2.42%/4.44% |
DER/WER statistics for RNN models without training on extra training dataset
| DER/WER | With case ending | Without case ending | With case ending | Without case ending |
|---|---|---|---|---|
| Including no diacritic | Excluding no diacritic | |||
| Basic Model | 1.72%/5.16% | 1.37%/2.98% | 1.99%/4.96% | 1.59%/2.92% |
| CRF Model | 1.84%/5.42% | 1.47%/3.17% | 2.13%/5.22% | 1.69%/3.09% |
| Normalized Model | 1.69%/5.09% | 1.34%/2.91% | 1.95%/4.89% | 1.54%/2.83% |
DER/WER statistics for RNN models with training on extra training dataset
The following figure shows the validation DER of each model while training reported every 5 epochs.
RNN models validation DER while training
The project is available as open source under the terms of the MIT License.
198 followers · starred Jul 2022
Neural Arabic text diacritization
See the code
This repository contains the models, dataset, helpers, and systems' comparison for our paper on Arabic Text Diacritization:
"Neural Arabic Text Diacritization: State of the Art Results and a Novel Approach for Machine Translation", Ali Fadel, Ibraheem Tuffaha, Bara' Al-Jawarneh and Mahmoud Al-Ayyoub, EMNLP-IJCNLP 2019.
These folders each contain the generated dataset, system output and DER/WER statistics used to compare our system with each of the three systems.
requirements.txt file
pip install -r requirements.txtTo predict the diacritized text using any model provided in this repository the script predict.py can be used, example:
python predict.py --input-file-path sample_input \
--model-type rnn \
--model-number 3 \
--model-size small \
--model-average 20 \
--output-file-path sample_output
The previous line will diacritize the text inside sample_input file using the rnn model that have number 3 trained on the small dataset (without extra training dataset) after averaging the last 20 epochs and writes the diacritized text on sample_output.
The allowed option are:
Before training any FFNN model you need to prepare the dataset using prepare_feed_forward_data.py script. After that to train any FFNN model you can use the model.ipynb notebooks that exists under models/ffnn_models/*/
There is no need to prepare any data to train RNN models, to train any RNN model you can use the model.ipynb notebooks that exists under models/rnn_models/*/
Note that the RNN models use CuDNNLSTM layers which should run on GPU, to train the models or predict output from them using CPU only you can use regular LSTM layers. Moreover, all RNN models checkpoints exist under models/rnn_models/*/*/ use CuDNNLSTM layers, so the checkpoints should be loaded on GPU, but under model/rnn_models/*/*/lstm/ you can find the same checkpoints with same weights and structure but with regular LSTM layers used instead of CuDNNLSTM layers.
Basic RNN model structure
All results reported below are on (Fadel et al., 2019) test set (best results shown in bold).
There are three feed-forward neural network models, the following table show the results of each of them:
| DER/WER | With case ending | Without case ending | With case ending | Without case ending |
|---|---|---|---|---|
| Including no diacritic | Excluding no diacritic | |||
| Basic Model | 9.33%/25.93% | 6.58%/13.89% | 10.85%/25.39% | 7.51%/13.53% |
| 100-Hot Model | 6.57%/20.21% | 4.83%/11.14% | 7.75%/19.83% | 5.62%/10.93% |
| Embeddings Model | 5.52%/17.12% | 4.06%/9.38% | 6.44%/16.63% | 4.67%/9.10% |
DER/WER statistics for FFNN models
There are three recurrent neural network models, each of them was trained twice with and without the extra training dataset. The following tables show the results of each of them:
| DER/WER | With case ending | Without case ending | With case ending | Without case ending |
|---|---|---|---|---|
| Including no diacritic | Excluding no diacritic | |||
| Basic Model | 2.68%/7.91% | 2.19%/4.79% | 3.09%/7.61% | 2.51%/4.66% |
| CRF Model | 2.67%/7.73% | 2.19%/4.69% | 3.08%/7.46% | 2.52%/4.60% |
| Normalized Model | 2.60%/7.69% | 2.11%/4.57% | 3.00%/7.39% | 2.42%/4.44% |
DER/WER statistics for RNN models without training on extra training dataset
| DER/WER | With case ending | Without case ending | With case ending | Without case ending |
|---|---|---|---|---|
| Including no diacritic | Excluding no diacritic | |||
| Basic Model | 1.72%/5.16% | 1.37%/2.98% | 1.99%/4.96% | 1.59%/2.92% |
| CRF Model | 1.84%/5.42% | 1.47%/3.17% | 2.13%/5.22% | 1.69%/3.09% |
| Normalized Model | 1.69%/5.09% | 1.34%/2.91% | 1.95%/4.89% | 1.54%/2.83% |
DER/WER statistics for RNN models with training on extra training dataset
The following figure shows the validation DER of each model while training reported every 5 epochs.
RNN models validation DER while training
The project is available as open source under the terms of the MIT License.
198 followers · starred Jul 2022