Multilingual Emotions (small, 141M): the 28 GoEmotions labels in many languages

A multi-label emotion classifier with the 28 labels of Google's GoEmotions (admiration, amusement, anger, ... , surprise, neutral). It works on text in English and 35 other languages. It is a drop-in multilingual alternative to English-only GoEmotions models such as SamLowe/roberta-base-go_emotions: the labels are the same and it uses the same sigmoid multi-label output. Built on mmBERT-small, Apache-2.0. ONNX files for CPU and the browser (transformers.js) are included. A larger, more accurate version is available as multilingual-emotions-base. Try it in the browser.

• Each label gets an independent probability. A text can have several emotions, or only neutral.
• thresholds.json has one threshold per label, tuned on the GoEmotions validation set. Use it instead of 0.5, especially for rare labels (grief, pride, relief, nervousness). See Usage.
• For the 6 Ekman emotions (anger, disgust, fear, joy, sadness, surprise), group the labels with the GoEmotions paper's mapping and take the maximum per group (ekman_mapping.json, code below).
• onnx/model_quantized.onnx (int8 embeddings, 268 MB) agrees with fp32 on 98.7% of 448 test texts (all 28 labels thresholded at 0.5).

Usage

import json
from huggingface_hub import hf_hub_download
from transformers import pipeline

clf = pipeline("text-classification", model="Horizon-Labs/multilingual-emotions-small", top_k=None)
thr = json.load(open(hf_hub_download("Horizon-Labs/multilingual-emotions-small", "thresholds.json")))
out = clf(["Thank you so much, this made my day!", "Tengo miedo de que no lleguemos a tiempo."])
print([[x["label"] for x in o if x["score"] >= thr[x["label"]]] for o in out])
# e.g. [['excitement', 'gratitude', 'joy'], ['fear']]

# 6 Ekman emotions: max score over each group
ekman = json.load(open(hf_hub_download("Horizon-Labs/multilingual-emotions-small", "ekman_mapping.json")))
scores = {x["label"]: x["score"] for x in out[1]}
print({e: max(scores[l] for l in ls) for e, ls in ekman.items()})

transformers.js:

import { pipeline } from "@huggingface/transformers";
const clf = await pipeline("text-classification", "Horizon-Labs/multilingual-emotions-small", { dtype: "q8" });
console.log(await clf("Je suis tellement fier de toi !", { top_k: 5 }));

Evaluation

The benchmarks were used only for evaluation, and training texts that also occur in them were removed.

• GoEmotions test (English Reddit comments, 5,427, 28 labels). Macro-F1 over the 28 labels, at threshold 0.5 and with per-label thresholds tuned on the GoEmotions validation set. Only models with the 28 GoEmotions labels are scored.
• BRIGHTER (brighter-dataset/BRIGHTER-emotion-categories, CC-BY-4.0): human-labelled texts written in 28 languages (not translations), with 6 emotions and multiple labels per text; up to 1,500 test texts per language. Every model's labels are grouped into the 6 emotions (GoEmotions labels via the GoEmotions paper's Ekman mapping; for other label sets, by name). Each model gets one threshold per language and emotion, tuned on that language's BRIGHTER dev set. The score is macro-F1 over the annotated emotions, averaged over languages. "4 emotions" = anger, fear, joy and sadness only, the set every compared model covers.

• The table shows the released checkpoint. Means over two training seeds: small GoEmotions tuned .499, BRIGHTER 6 emotions .405, 4 emotions .434; base GoEmotions tuned .520, BRIGHTER 6 emotions .424, 4 emotions .454.
• Models ahead of this one: GoEmotions (tuned): roberta-base-go_emotions, multilingual_go_emotions; BRIGHTER 6 emotions: none; BRIGHTER 4 emotions: xlm-emo-t. Most BRIGHTER languages are low-resource African and Asian languages that are not in our training data; GoEmotions texts are Reddit comments, while BRIGHTER includes tweets, news comments and other sources.

Per BRIGHTER language (macro-F1):

Weakest GoEmotions labels (test F1, tuned thresholds): grief 0.20, relief 0.21, disappointment 0.29, realization 0.31, annoyance 0.36, excitement 0.36. Several of these are rare, and GoEmotions annotators often disagree on such labels.

Training

• Data: the GoEmotions training set (43,410 English Reddit comments with human labels; Apache-2.0), plus translations of it into 35 languages made with Qwen3.8-27B (Apache-2.0), published as Horizon-Labs/go-emotions-multilingual. Each language gets its own random sample of 12,000 comments, and the human labels are copied to the translation (442,991 training examples in total). The languages are Afrikaans, Arabic, Bengali, Chinese (Simplified), Czech, Danish, Dutch, Finnish, French, German, Greek, Hausa, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Marathi, Norwegian, Persian, Polish, Portuguese (Brazilian), Romanian, Russian, Spanish, Swahili, Swedish, Tatar, Thai, Turkish, Ukrainian, Urdu, Vietnamese.
• Translations that failed to parse, looked like refusals, had an implausible length or were copied unchanged were dropped (about 1.2%). 3% of the source comments (in every language) were held out for validation.
• Model: mmBERT-small with 28 sigmoid outputs and binary cross-entropy, 1.5 epochs, learning rate 5e-5, max length 256 tokens. Epochs and learning rate were chosen on the GoEmotions validation set and BRIGHTER dev sets, never the test sets. The checkpoint was chosen by macro-F1 on held-out training comments.
• Code: code/ in this repository.

Limitations

• The labels come from English Reddit comments. Translations keep the labels but can shift nuance, and the model has seen little native (non-translated) emotional text outside English. BRIGHTER results show it is much weaker in languages it was not trained on.
• The labels are subjective and inter-annotator agreement on GoEmotions is modest, so F1 around 0.5 is typical for this task. Rare labels are unreliable.
• It classifies the emotion expressed in the text, not the writer's actual state. Don't use it to make decisions about individuals.