README
russian-reviews-sentiment-analysis
I just wanted to try out :hugs: Huggingface's new Trainer API and get up-to-date with recent torchtext releases, also I've stumbled upon on an interesting Russian reviews dataset. So I decided to perform sentiment classification, gradually increasing the complexity of classifiers and compare their performance.
Data
You can see dataset overview in the original repository.
Models
I chose TF-IDF and SVM as a baseline model, CNN (because it was used in the original article) and DeepPavlov's RuBERT (rubert-base-cased-conversational), because I just wanted to try it out. Also, creators of the original dataset have provided pre-trained embeddings, so I decided to use them also.
Results
| Model | Precision (macro) | Recall (macro) | F1-score (macro) |
|---|---|---|---|
| MNB (article) | 74.47 | 73.79 | 73.90 |
| CNN (without emoticons) (article) | 74.71 | 74.54 | 74.31 |
| CNN (with emoticons) (article) | 75.63 | 75.31 | 75.45 |
| TF-IDF + SVM (mine) | 74.44 | 74.46 | 74.45 |
| CNN (mine) | 75.51 | 74.99 | 75.19 |
| RuBERT (rubert-base-cased-conversational) (mine) | 78.28 | 77.68 | 77.91 |
References
- Smetanin, S., & Komarov, M. (2019, July). Sentiment Analysis of Product Reviews in Russian using Convolutional Neural Networks. In 2019 IEEE 21st Conference on Business Informatics (CBI) (Vol. 1, pp. 482-486). IEEE.
Contributors
12 commits
Similar repos
Languages
Python
100.0%
README
russian-reviews-sentiment-analysis
I just wanted to try out :hugs: Huggingface's new Trainer API and get up-to-date with recent torchtext releases, also I've stumbled upon on an interesting Russian reviews dataset. So I decided to perform sentiment classification, gradually increasing the complexity of classifiers and compare their performance.
Data
You can see dataset overview in the original repository.
Models
I chose TF-IDF and SVM as a baseline model, CNN (because it was used in the original article) and DeepPavlov's RuBERT (rubert-base-cased-conversational), because I just wanted to try it out. Also, creators of the original dataset have provided pre-trained embeddings, so I decided to use them also.
Results
| Model | Precision (macro) | Recall (macro) | F1-score (macro) |
|---|---|---|---|
| MNB (article) | 74.47 | 73.79 | 73.90 |
| CNN (without emoticons) (article) | 74.71 | 74.54 | 74.31 |
| CNN (with emoticons) (article) | 75.63 | 75.31 | 75.45 |
| TF-IDF + SVM (mine) | 74.44 | 74.46 | 74.45 |
| CNN (mine) | 75.51 | 74.99 | 75.19 |
| RuBERT (rubert-base-cased-conversational) (mine) | 78.28 | 77.68 | 77.91 |
References
- Smetanin, S., & Komarov, M. (2019, July). Sentiment Analysis of Product Reviews in Russian using Convolutional Neural Networks. In 2019 IEEE 21st Conference on Business Informatics (CBI) (Vol. 1, pp. 482-486). IEEE.
Similar repos
Contributors
12 commits
Languages
Python
100.0%