Mu-SHROOM: The Multilingual Shared-task on Hallucinations and Related Observable Overgeneration Mistakes

This is the official repository for MuSHROOM @ SemEval 2025.

Contents:

• unlabelled data (unsplit, unsorted), in data/
• annotation interface (Django project) in mushroom-annot-interface/
• eval interface (Django project) in mushroom_eval_interface/
• published data (per split) in splits/
• useful scripts in participant_kit/
• backups of the SQLITE DB for the interfaces in backups/

Acknowledge us ;)

If you use these resources, please cite us:

@misc{vazquez-etal-2025-mu-shroom,
author={Ra'ul V'azquez and Timothee Mickus and Elaine Zosa and Teemu Vahtola and J"org Tiedemann and Aman Sinha and Vincent Segonne and Fernando S'anchez-Vega and Alessandro Raganato and Jindřich Libovický and Jussi Karlgren and Shaoxiong Ji and Jindřich Helcl and Liane Guillou and Ona de Gibert and Jaione Bengoetxea and Joseph Attieh and Marianna Apidianaki},
title={Sem{E}val-2025 {T}ask 3: {Mu-SHROOM}, the Multilingual Shared-task on Hallucinations and Related Observable Overgeneration Mistakes},
year={2025},
url={https://helsinki-nlp.github.io/shroom/}
}