Indonesian Hate Speech Classification — Experiments & Results

A fun little experiment comparing different approaches to detect hate speech in Bahasa Indonesia. Just trying out a few combos and seeing what sticks.

Basically, I wanted to see how these things compare against each other:

• Fine-tuning a transformer vs. using it just for embeddings
• Light vs. heavy text preprocessing — does cleaning more actually help?
• Indonesian-specific models vs. multilingual ones

Dataset

Labels used: 0 = Non_HS and 1 = HS (hate speech).

Dataset source: Indonesian hate speech superset

---

Models I Tested

1. IndoBERT fine-tuning — light preprocessing
2. IndoBERT fine-tuning — heavy preprocessing
3. SVM + IndoBERT embeddings — light preprocessing
4. SVM + IndoBERT embeddings — heavy preprocessing
5. XLM-RoBERTa — light preprocessing
6. IndoBERTweet — light preprocessing

---

Preprocessing Approaches

Light preprocessing — minimal cleaning, keeping things as natural as possible:

• Lowercasing
• Remove URLs
• Remove mentions
• Basic punctuation cleanup

Heavy preprocessing — more aggressive, almost "traditional NLP" style:

• Remove numbers
• Remove stopwords
• Stemming
• Additional normalization

---

Results

*XLM-RoBERTa was trained with a smaller batch size + gradient accumulation because my GPU ran out of memory. So the result might not be perfectly comparable.

---

What I Learned

1. IndoBERT wins

Fine-tuning IndoBERT with light preprocessing gave the best numbers overall. Not super surprising — it was pretrained specifically on Indonesian text, so it naturally picks up on local language patterns, slang, and context better than multilingual models.

2. Heavy preprocessing actually hurts transformers

Normally in classic NLP, more cleaning helps. But transformers really depend on context. Stripping out stopwords and stemming words removes a lot of subtle signals the model would've used. Less is more here.

3. SVM + embeddings is solid, but not a replacement

Using IndoBERT just as an embedding extractor and throwing an classic SVM on top still got decent results (~82% accuracy). It's a lighter approach and could work well in minimum-resource setups — but you do lose a few percent in performance compared to full fine-tuning.

4. Multilingual models are close

XLM-RoBERTa performed really well! But it still fell slightly short of IndoBERT. Makes sense — when a model is trained on 100+ languages, it can't go as deep on any single one.

---

TL;DR — Best Approach

Fine-tune IndoBERT with light preprocessing.

---

How to Run It Yourself

1. Clone the repo

git clone https://github.com/your-repo/indo-hate-speech-classification
cd indo-hate-speech-classification

2. Set up a Python environment

python -m venv .venv
source .venv/bin/activate

3. Install dependencies

pip install -r requirements.txt

Main libraries used: transformers, datasets, torch, scikit-learn, pandas

4. Start training

python app/train.py

Trained artifacts will be saved to:

• Fine-tuning: artifacts/model/
• SVM + embedding: artifacts/svm/svm.joblib

Want to swap models? Just change the model constant in app/train.py and you're good to go.

4. Run prediction CLI

python app/prediction_cli.py

Run this to if you're curious your sentence counts as hate speech or not. It loads the saved model and lets you type a sentence to test it out.

---

Hardware I Used

• GPU: RTX 3060 (6GB VRAM)
• RAM: 24GB
• CPU: Intel i7

Nothing crazy — this should run fine on any mid-range gaming PC with a decent GPU.

---

What's Next?

A few things I want to try when I get the time:

• Hyperparameter tuning — haven't touched this at all yet
• Class weighting to deal with the imbalanced dataset
• Data augmentation for the minority class
• Testing on larger Indonesian language models

---

License

MIT — do whatever you want with it. :D