Multilingual-NLP

Create virtual enviroment

python -m venv .env
source .env/bin/activate   # Linus/MacOS
venv\Scripts\activate      # Windows

Install

pip install transformers protobuf seqeval accelerate sentencepiece tiktoken
pip install "datasets<3.0.0"

Run

1. python models.py
2. Full train (3 epochs, auto-save 30 mins)
   python train.py
3. Only train NER head
   python train.py --freeze_encoder --epochs 5
4. Custom hyperparams
   python train.py --epochs 5 --batch_size 32 --lr_mmbert 1e-5 --lr_mt5 2e-5
5. Resume training after crash
   python train.py --resume_mmbert ./checkpoints/mmBERT-small/mmBERT-small_best.pt