Global MMLU-Lite — Human Translated

Global MMLU-Lite is a multilingual evaluation benchmark for LLMs covering 18 languages. This dataset extends it with professional human translations for three additional low-resource languages that are not in the original: Chichewa (nya), Māori (mri), and Inuktitut (iku).

Released as part of the BYOL: Bring Your Own Language Into LLMs project (paper).

What's New

The original Global MMLU-Lite by Cohere covers 18 languages: Arabic, Bengali, Chinese, Burmese, Welsh, German, English, Spanish, French, Hindi, Indonesian, Italian, Japanese, Korean, Portuguese, Albanian, Swahili, and Yoruba.

This dataset adds 3 new languages with professional human translations (not machine-translated):

Usage

from datasets import load_dataset

# Same API as the original CohereForAI/Global-MMLU-Lite
ds = load_dataset("ai-for-good-lab/Global-MMLU-Lite", "mri", split="test")
print(ds[0])

Format

All subsets share the same 17-column schema as the original dataset, including sample_id, subject, subject_category, question, option_a–option_d, answer, and annotation metadata fields.

Note: Annotation metadata (e.g., required_knowledge, cultural_sensitivity_label) for Māori and Inuktitut was populated from the Chichewa annotations by matching on sample_id, since all three languages translate the same English source questions.

License

This dataset is released under Apache 2.0, consistent with the original Global MMLU-Lite license.

Citation

@article{zamir2026byolbringlanguagellms,
    title={BYOL: Bring Your Own Language Into LLMs},
    author={Syed Waqas Zamir and Wassim Hamidouche and Boulbaba Ben Amor and Luana Marotti and Inbal Becker-Reshef and Juan Lavista Ferres},
    year={2026},
    journal={arXiv:2601.10804},
    url={https://arxiv.org/abs/2601.10804},
}

Acknowledgments

We thank the Government of Nunavut for providing the professional human translations of Global MMLU-Lite into Inuktitut.

This dataset builds on Global MMLU-Lite by Cohere Labs.