Political Sentiment Analysis of Tamil X Comments

Overview

This project implements a multiclass sentiment analysis system for Tamil political comments from X (Twitter) using transformer-based models. The system classifies Tamil text into seven sentiment categories while handling challenges like morphological richness, agglutinative structure, code-mixing, slang, and sarcasm.

Comparison of different transformer-based base models for Tamil sentiment analysis

Performance comparison of machine learning models used as classification heads

Overall model performance metrics across different approaches

Training progress and convergence analysis

Features

• Transformer-Based Architecture: Utilizes IndicBERT for robust Tamil text representation
• Hybrid Classification Strategy: Combines transformer embeddings with Random Forest classifier
• Advanced Text Preprocessing: Handles Tamil morphology, emoji retention, and code-mixed text
• Multiclass Sentiment Classification: Seven sentiment categories for comprehensive analysis
• Benchmark Performance: Achieved macro F1-score of 0.3122 (improved) and ranked 14th in NAACL

Dataset Classes

The system classifies Tamil political tweets into seven sentiment categories:

1. Opinionated - Subjective views based on personal beliefs
2. Substantiated - Remarks backed by factual data or references
3. Neutral - Objective or fact-based statements
4. Negative - Critical or unfavorable remarks
5. Sarcastic - Comments using irony or sarcasm
6. None of the above - Tweets that don't fit any category
7. Positive - Supportive statements toward political leaders or policies

Architecture

Model Implementations

The project includes implementations for multiple transformer models:

• DistilBERT: Lightweight multilingual model (66M parameters)
• IndicBERT: Specialized for Indic languages including Tamil (470M parameters)
• TamilBERT: Tamil-specific model (389M parameters)
• XLM-RoBERTa: Cross-lingual multilingual model (278M parameters)

Performance Comparison

Machine Learning Classifiers

Traditional ML models tested with extracted features:

Implementation Details

Data Preprocessing

# Tamil text preprocessing pipeline
def preprocess(text):
    text = re.sub(r"#\S+|@\S+|https?:\/\/\S+", "", text)  # Remove hashtags, mentions, URLs
    text = re.sub(r"[^அ-ஹ0-9\s]", "", text)  # Keep only Tamil script and numbers
    text = re.sub(r"[\n\r\t]+", " ", text)  # Normalize whitespace
    text = re.sub(r"\s+", " ", text).strip()  # Remove extra spaces
    text = re.sub(r"(.)\1{2,}", r"\1\1", text)  # Reduce character repetitions
    return text

Model Architecture

class PSAModel(nn.Module):
    def __init__(self, num_classes=7):
        super(PSAModel, self).__init__()
        self.bert = AutoModelForMaskedLM.from_pretrained("ai4bharat/IndicBERTv2-MLM-Sam-TLM")
        self.final_fc = nn.Linear(in_features=250000, out_features=num_classes, bias=True)

    def forward(self, x, mask=None):
        x = self.bert(x, attention_mask=mask).logits
        x = torch.mean(x, dim=1)
        x = self.final_fc(x)
        return x

Training Configuration

• Loss Function: CrossEntropyLoss
• Optimizer: Adam (LR: 1e-6, Betas: (0.9, 0.999))
• Scheduler: ReduceLROnPlateau (Factor: 0.1, Patience: 4 epochs)
• Batch Size: 4
• Sequence Length: 512 tokens
• Evaluation Metric: Macro F1-score

Project Structure

PoliticalSA/
├── Distil BERT/           # DistilBERT implementation
├── Indic BERT/            # IndicBERT implementation (best performing)
├── Tamil Bert/            # TamilBERT implementation
├── XLM-Roberta/          # XLM-RoBERTa implementation
├── Machine Learning Models/  # Traditional ML approaches
├── Trainer/              # Training utilities and logging
├── Graphs/               # Performance visualization plots
└── final_test.csv        # Feature vectors for ML models

Results

The hybrid approach combining IndicBERT feature extraction with Random Forest classification achieved the best performance:

• Initial F1-score: 0.2759 (14th rank in NAACL benchmark)
• Improved F1-score: 0.3122 (post-competition optimization)
• Best Base Model: IndicBERT (specialized for Indic languages)
• Best Classifier: Random Forest (on extracted features)

Usage

Training a Model

cd "Indic BERT"
python training_model.py

Feature Extraction

jupyter notebook vector_generation.ipynb

Running ML Models

cd "Machine Learning Models"
jupyter notebook other_models.ipynb

Key Innovations

1. Language-Specific Preprocessing: Tailored for Tamil morphology and script characteristics
2. Hybrid Architecture: Combines deep learning feature extraction with traditional ML classification
3. Comprehensive Model Comparison: Systematic evaluation of multiple transformer architectures
4. Tamil-Focused Approach: Addresses unique challenges of Tamil political discourse analysis

Technical Challenges Addressed

• Morphological Complexity: Tamil's agglutinative nature
• Code-Mixing: Mixed Tamil-English text common in social media
• Sarcasm Detection: Implicit sentiment expressions
• Limited Labeled Data: Efficient learning from constrained datasets
• Class Imbalance: Handling uneven distribution across sentiment categories