🏷️ Plain and gentle language for more comfortable co-existence: Plain Japanese Standard

This project is conducted for my bachelor thesis in the winter semester 2022/2023.

📌 About the project

The aim of this project is to conduct classification and translation experiments on Japanese Simplified Corpus SNOW T15: やさしい日本語コーパス.

The goal of the two tasks are to answer the questions:

1. "Can a language model tell the readability of a sentence?", i.e. binary classification of original (0) or simplified (1)
2. "Can a language model learn to simplify a sentence as a translation between the same language?", i.e. simplification as a form of translation between the same language.

🏃 Getting started

Prerequisites

For this project, Python 3.10.9 is used. Use requirements.txt to install necessary libraries. e.g. run pip install -r requirements.txt
(⚠️ As requirements.txt is created using the conda environment I used locally for this project, it includes a large number of libraries)

Fine-tuned models

A fine-tuned model is available for each task. Fine-tuned Models

• fine-tuned roberta base model for classification: roberta_base.zip
• fine-tuned bart base model for translation: bart_base.zip ⚠️ this data is very large (1GB)

📂 Directory structure

|--dataset_analysis
|  |--criteria.py
|  |--create_vocab.py
|  |--data_analysis.py
|  |--data_analysis.ipynb
|  |--resources
|  |--analysed
|--classification
|  |--create_data_classification.py
|  |--train_classify.py
|  |--param_tune.py
|  |--train_param_tune.py
|  |--resources
|      |--df_test2.csv
|      |--df_train2.csv
|      |--df_valid2.csv
|--translation
|  |--create_data_translate.py
|  |--train_translate.py
|  |--test_translate.py
|  |--evaluate_simplified.py
|  |--resources
|      |--train_test_valid_dataset
|      |--sample_generated.txt
|  |--scores
|      |--base_scores.csv
|      |--large_scores.csv
|      |--summary_scores.ipynb
|--.gitignore
|--README.md
|--index.xml

Some important directories and files are:

• analysis: code of dataset analysis and results (the results themselves are in analyzed and data_analysis_results.ipynb contains the visualisation).
• classification: code of classification experiment.
• translation: code of translation experiment. score contains results.
• resources: dataset for classification and translation experiments. Both same dataset, but processed differently. df_*2.csv is named like this because they are datasets that don't contain duplicates. (See create_data_*.py for further details.)
• README.me: this file

📋 Usage

Use GPU to run all the scripts, except for the evaluation of generated simplification (evaluate_simplified.py). Activate the neccessary virtual environment for the metric.

Classification

For fine-tuning or test models:

• run python train_classify.py with the appropriate parameters

e.g.
python train_classify.py --mode train --checkpoint ku-nlp/roberta-base-japanese-char-wwm --output models/roberta_base to train a roberta base model and save the model in models/roberta_base.

python train_classify.py --mode test --checkpoint /home/students/udaka/bachelorarbeit/classification/models/roberta_base --tokenizer ku-nlp/roberta-base-japanese-char-wwm to test a fine-tuned roberta base model using bert base tokenizer. (replace checkpoint path with the path in which you save fine-tuned models. See Fine-tuned models section for available fine-tuned models)

For conducting hyperparameter tuning:

• run python param_tune.py with the appropriate parameters

e.g. python param_tune.py --checkpoint ku-nlp/roberta-base-japanese-char-wwm to do hyperparameter tuning on a roberta base model.

Translation

For fine-tuning or test models:

• run python train_translate.py with the appropriate parameters

e.g.
python train_translate.py --checkpoint Formzu/bart-base-japanese --output models/bart_base to train a bart base model and save the model in models/bart_base.

python test_translate.py --checkpoint /home/students/udaka/bachelorarbeit/translation/models/bart_base --tokenizer Formzu/bart-large-japanese --output bart_base.txt to test a bart base model and output the generated test in bart_base.txt. (replace checkpoint path with the path in which you save fine-tuned models. See Fine-tuned models section for available fine-tuned models)

For evaluating the quality of generated simplification: * run `python evaluate_simplified.py` with the appropriate parameters

e.g. python evaluate_simplified.py --generated bart_base.txt to evaluate bart_base.txt.

📛 Author

• Maya Udaka

[email protected]

🔗 References

Dataset

• 言語商会: SNOW T15:やさしい日本語コーパス
• huggingface: Datasets: snow_simplified_japanese_corpus

Models

classification

• BERT base Japanese model
• RoBERTa base Japanese model
• LUKE base Japanese model

translation

• BART base Japanese model
• BART large Japanese model

Metrics

• BLEU
• BERTscore
• SARI

Others

• huggingface: classification demo
• huggingface: translation demo
• mecab-python3