Vietnamese Social Media Hate Speech Detection

A deep learning-based system for detecting and classifying hate speech in Vietnamese social media content.

Features

• PhoBERT-based Classification: Utilizes pre-trained Vietnamese BERT model for state-of-the-art performance
• Multiple Model Support: Includes both transformer-based and traditional ML approaches
• Vietnamese Text Preprocessing: Custom preprocessing pipeline for Vietnamese language
• Comprehensive Evaluation: Multiple metrics including accuracy, precision, recall, F1-score
• API Ready: FastAPI endpoint for production deployment
• Easy to Use: Simple training and inference interfaces

Project Structure

vn-hate-speech-detection/
├── data/                      # Data directory
│   ├── raw/                  # Raw datasets
│   ├── processed/            # Preprocessed data
│   └── sample/               # Sample data for testing
├── models/                    # Saved model checkpoints
├── notebooks/                 # Jupyter notebooks for exploration
├── src/
│   ├── data/                 # Data loading and preprocessing
│   ├── models/               # Model architectures
│   ├── training/             # Training utilities
│   ├── evaluation/           # Evaluation metrics
│   ├── inference/            # Prediction interface
│   └── utils/                # Utility functions
├── scripts/                   # Training and evaluation scripts
├── config/                    # Configuration files
├── requirements.txt          # Python dependencies
└── README.md                 # This file

Installation

1. Clone the repository:

git clone https://github.com/DHPh/vn-hate-speech-detection.git
cd vn-hate-speech-detection

2. Create a virtual environment:

python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

3. Install dependencies:

pip install -r requirements.txt

4. Download the ViHSD dataset:

python scripts/download_dataset.py --output_dir data/raw --prepare

5. (Optional) Download Vietnamese language resources:

python -m pip install underthesea

Quick Start

Data Preparation

Download the ViHSD dataset:

python scripts/download_dataset.py --prepare

Or use it in your code:

from src.data.dataset import HateSpeechDataset
from src.data.preprocessor import VietnamesePreprocessor

# Load and preprocess data
preprocessor = VietnamesePreprocessor()
dataset = HateSpeechDataset(data_path="data/processed/vihsd_processed.csv", preprocessor=preprocessor)

Training a Model

python scripts/train.py --data data/processed/vihsd_processed.csv --model phobert --epochs 10 --batch_size 16

Making Predictions

from src.inference.predictor import HateSpeechPredictor

predictor = HateSpeechPredictor(model_path="models/best_model.pth")
text = "Your Vietnamese text here"
result = predictor.predict(text)
print(f"Label: {result['label']}, Confidence: {result['confidence']:.2f}")

Running the API

python src/inference/api.py

Then access the API at http://localhost:8000/docs

Dataset

This project uses the ViHSD (Vietnamese Hate Speech Detection) dataset from HuggingFace.

• Dataset: sonlam1102/vihsd
• Description: Vietnamese hate speech detection dataset from social media
• Format: CSV with free_text (text content) and label (0: normal, 1: hate speech)

Download Dataset

python scripts/download_dataset.py --output_dir data/raw --prepare

This will:

1. Download the ViHSD dataset from HuggingFace
2. Save train and test splits
3. Prepare the data for training (standardized format)

Custom Dataset Format

You can also use your own CSV dataset with columns:

• text: Vietnamese text content
• label: Binary label (0: normal, 1: hate speech) or multi-class labels

Example:

text,label
"Văn bản tiếng Việt bình thường",0
"Văn bản chứa nội dung xúc phạm",1

Model Architecture

PhoBERT Classifier

• Pre-trained Vietnamese BERT (PhoBERT-base)
• Fine-tuned classification head
• Dropout for regularization
• Support for binary and multi-class classification

Traditional ML Baseline

• TF-IDF vectorization
• Support Vector Machine (SVM)
• Logistic Regression
• Random Forest

Configuration

Edit config/config.yaml to customize:

• Model hyperparameters
• Training settings
• Data paths
• Preprocessing options

Evaluation Metrics

• Accuracy
• Precision
• Recall
• F1-Score
• Confusion Matrix
• ROC-AUC (for binary classification)

Results

Results will be saved in:

• models/: Trained model checkpoints
• results/: Evaluation metrics and plots
• logs/: Training logs

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

License

MIT License

Citation

If you use this project, please cite:

@software{vn_hate_speech_detection,
  title={Vietnamese Social Media Hate Speech Detection},
  author={Your Name},
  year={2025}
}

Acknowledgments

• PhoBERT - Vietnamese BERT model
• VnCoreNLP - Vietnamese NLP toolkit
• Transformers - HuggingFace transformers library