Learning to Rank and Classify Romanian Text: Embedding Strategies and Neural Architectures

As large language models continue to rise, observing how well they understand semantics and how do they reason remains an open challenge. In this paper, we evaluate and compare several Romanian-capable LLMs by analyzing their text embeddings on two distinct tasks: sentiment understanding using LAROSeDa dataset and logical reasoning on RoARC Challenge. We extract embeddings from Llmic 3B, mGPT 1.3B, Romanian Llama2 7B, Llama 3.1 8B, Gemma 3 1B using 3 pooling strategies, then train lightweight neural networks to perform classification or ranking. For RoARC we apply different embedding strategies to get the most out of the representations. We experimented various models and chose the best results for this paper.

Results presenation

.

How to run?

conda create -n dnn_env python=3.12.3 -y conda activate dnn_env

pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu124 pip install transformers datasets accelerate pip install protobuf sentencepiece