V-WSD

This repository is for the paper UAlberta at SemEval-2023 Task 1: Context Augmentation and Translation for Multilingual Visual Word Sense Disambiguation. In Proceedings of the 17th International Workshop on Semantic Evaluation (SemEval-2023), pages 2043–2051, Toronto, Canada. Association for Computational Linguistics.

disambiguate.py

Use this script to disambiguate target words in context.

Installation:

• Install pytorch following instructions from the official website.
• Install dependencies bash consec/setup.sh
• Download the ConSec "consec_wngt_best.ckpt" checkpoint from here
• Move "consec_wngt_best.ckpt" to consec/checkpoints
• Run disambigute.py in the following way PYTHONPATH=$(PWD)/consec python3 disambiguate.py --data_path data/trial.data.txt --output consec.tsv where --data_path is the SE23 dataset path and --output is the path where you would like to save the output

Authors

• Michael Ogezi - ogezi@ualberta.ca

BibTex

@inproceedings{ogezi-etal-2023-ualberta,
    title = "{UA}lberta at {S}em{E}val-2023 Task 1: Context Augmentation and Translation for Multilingual Visual Word Sense Disambiguation",
    author = "Ogezi, Michael  and
      Hauer, Bradley  and
      Omarov, Talgat  and
      Shi, Ning  and
      Kondrak, Grzegorz",
    editor = {Ojha, Atul Kr.  and
      Do{\u{g}}ru{\"o}z, A. Seza  and
      Da San Martino, Giovanni  and
      Tayyar Madabushi, Harish  and
      Kumar, Ritesh  and
      Sartori, Elisa},
    booktitle = "Proceedings of the 17th International Workshop on Semantic Evaluation (SemEval-2023)",
    month = jul,
    year = "2023",
    address = "Toronto, Canada",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2023.semeval-1.281",
    doi = "10.18653/v1/2023.semeval-1.281",
    pages = "2043--2051",
    abstract = "We describe the systems of the University of Alberta team for the SemEval-2023 Visual Word Sense Disambiguation (V-WSD) Task. We present a novel algorithm that leverages glosses retrieved from BabelNet, in combination with text and image encoders. Furthermore, we compare language-specific encoders against the application of English encoders to translated texts. As the contexts given in the task datasets are extremely short, we also experiment with augmenting these contexts with descriptions generated by a language model. This yields substantial improvements in accuracy. We describe and evaluate additional V-WSD methods which use image generation and text-conditioned image segmentation. Some of our experimental results exceed those of our official submissions on the test set. Our code is publicly available at \url{https://github.com/UAlberta-NLP/v-wsd}.",
}